Re: [Qemu-devel] [PATCH] qemu-img: Saner printing of large file sizes

Re: [Qemu-devel] [PATCH] qemu-img: Saner printing of large file sizes

[Vladimir Sementsov-Ogievskiy]

30.03.2019 18:07, Eric Blake wrote:
> Disk sizes close to INT64_MAX cause overflow, for some pretty
> ridiculous output:
> 
>    $ ./nbdkit -U - memory size=$((2**63 - 512)) --run 'qemu-img info $nbd'
>    image: nbd+unix://?socket=/tmp/nbdkitHSAzNz/socket
>    file format: raw
>    virtual size: -8388607T (9223372036854775296 bytes)
>    disk size: unavailable
> 
> But there's no reason to have two separate implementations of integer
> to human-readable abbreviation, where one has overflow and stops at
> 'T', while the other avoids overflow and goes all the way to 'E'. With
> this patch, the output now claims 8EiB instead of -8388607T, which
> really is the correct rounding of largest file size supported by qemu
> (we could go 511 bytes larger if we used byte-accurate sizing instead
> of rounding up to the next sector boundary, but that wouldn't change
> the human-readable result).
> 
> Reported-by: Richard W.M. Jones <rjones@redhat.com>
> Signed-off-by: Eric Blake <eblake@redhat.com>

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH] qemu-img: Saner printing of large file sizes

[Kevin Wolf]

Am 30.03.2019 um 16:07 hat Eric Blake geschrieben:
> Disk sizes close to INT64_MAX cause overflow, for some pretty
> ridiculous output:
> 
>   $ ./nbdkit -U - memory size=$((2**63 - 512)) --run 'qemu-img info $nbd'
>   image: nbd+unix://?socket=/tmp/nbdkitHSAzNz/socket
>   file format: raw
>   virtual size: -8388607T (9223372036854775296 bytes)
>   disk size: unavailable
> 
> But there's no reason to have two separate implementations of integer
> to human-readable abbreviation, where one has overflow and stops at
> 'T', while the other avoids overflow and goes all the way to 'E'. With
> this patch, the output now claims 8EiB instead of -8388607T, which
> really is the correct rounding of largest file size supported by qemu
> (we could go 511 bytes larger if we used byte-accurate sizing instead
> of rounding up to the next sector boundary, but that wouldn't change
> the human-readable result).
> 
> Reported-by: Richard W.M. Jones <rjones@redhat.com>
> Signed-off-by: Eric Blake <eblake@redhat.com>

This is quite obviously a bug fix for some cases. This suggests that we
want it in 4.0.

It is also an output change for other cases, like going from "8M" to
"8 MiB". We probably can't tell for sure whether some tools expect the
spelling "8M" (even if this is supposed to be the human interface and
tools should be using JSON) or feed the change back to qemu-img or
qemu-io (which accept "8M", but not "8 MiB" as sizes in most places).
This suggests that we shouldn't make this change as late as -rc2.

So what is the conclusion?

Kevin

Re: [Qemu-devel] [PATCH] qemu-img: Saner printing of large file sizes

[Daniel P. Berrangé]

On Mon, Apr 01, 2019 at 03:04:16PM +0200, Kevin Wolf wrote:
> Am 30.03.2019 um 16:07 hat Eric Blake geschrieben:
> > Disk sizes close to INT64_MAX cause overflow, for some pretty
> > ridiculous output:
> > 
> >   $ ./nbdkit -U - memory size=$((2**63 - 512)) --run 'qemu-img info $nbd'
> >   image: nbd+unix://?socket=/tmp/nbdkitHSAzNz/socket
> >   file format: raw
> >   virtual size: -8388607T (9223372036854775296 bytes)
> >   disk size: unavailable
> > 
> > But there's no reason to have two separate implementations of integer
> > to human-readable abbreviation, where one has overflow and stops at
> > 'T', while the other avoids overflow and goes all the way to 'E'. With
> > this patch, the output now claims 8EiB instead of -8388607T, which
> > really is the correct rounding of largest file size supported by qemu
> > (we could go 511 bytes larger if we used byte-accurate sizing instead
> > of rounding up to the next sector boundary, but that wouldn't change
> > the human-readable result).
> > 
> > Reported-by: Richard W.M. Jones <rjones@redhat.com>
> > Signed-off-by: Eric Blake <eblake@redhat.com>
> 
> This is quite obviously a bug fix for some cases. This suggests that we
> want it in 4.0.
> 
> It is also an output change for other cases, like going from "8M" to
> "8 MiB". We probably can't tell for sure whether some tools expect the
> spelling "8M" (even if this is supposed to be the human interface and
> tools should be using JSON) or feed the change back to qemu-img or
> qemu-io (which accept "8M", but not "8 MiB" as sizes in most places).
> This suggests that we shouldn't make this change as late as -rc2.

If it breaks our own tests, then it is possible to break other tools
too.

> So what is the conclusion?

The safe option is to do the minimal fix for the existing code and look at
the refactoring in the next dev cycle.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH] qemu-img: Saner printing of large file sizes

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1722 bytes --]

On 4/1/19 8:16 AM, Daniel P. Berrangé wrote:
> On Mon, Apr 01, 2019 at 03:04:16PM +0200, Kevin Wolf wrote:
>> Am 30.03.2019 um 16:07 hat Eric Blake geschrieben:
>>> Disk sizes close to INT64_MAX cause overflow, for some pretty
>>> ridiculous output:
>>>
>>>   $ ./nbdkit -U - memory size=$((2**63 - 512)) --run 'qemu-img info $nbd'
>>>   image: nbd+unix://?socket=/tmp/nbdkitHSAzNz/socket
>>>   file format: raw
>>>   virtual size: -8388607T (9223372036854775296 bytes)
>>>   disk size: unavailable
>>>

>> This is quite obviously a bug fix for some cases. This suggests that we
>> want it in 4.0.
>>
>> It is also an output change for other cases, like going from "8M" to
>> "8 MiB". We probably can't tell for sure whether some tools expect the
>> spelling "8M" (even if this is supposed to be the human interface and
>> tools should be using JSON) or feed the change back to qemu-img or
>> qemu-io (which accept "8M", but not "8 MiB" as sizes in most places).
>> This suggests that we shouldn't make this change as late as -rc2.
> 
> If it breaks our own tests, then it is possible to break other tools
> too.
> 
>> So what is the conclusion?
> 
> The safe option is to do the minimal fix for the existing code and look at
> the refactoring in the next dev cycle.

It's not worth fixing the second implementation to match the first in
terms of support for sizes > T with no overflow, but with the existing
format of 8.0M vs. 8 MiB.  I'm happy to defer this patch to 4.1; the bug
is not new to 4.0, so living with it for one more release doesn't hurt.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Qemu-devel] [PATCH] qemu-img: Saner printing of large file sizes

[Kevin Wolf]

Am 30.03.2019 um 16:07 hat Eric Blake geschrieben:
> Disk sizes close to INT64_MAX cause overflow, for some pretty
> ridiculous output:
> 
>   $ ./nbdkit -U - memory size=$((2**63 - 512)) --run 'qemu-img info $nbd'
>   image: nbd+unix://?socket=/tmp/nbdkitHSAzNz/socket
>   file format: raw
>   virtual size: -8388607T (9223372036854775296 bytes)
>   disk size: unavailable
> 
> But there's no reason to have two separate implementations of integer
> to human-readable abbreviation, where one has overflow and stops at
> 'T', while the other avoids overflow and goes all the way to 'E'. With
> this patch, the output now claims 8EiB instead of -8388607T, which
> really is the correct rounding of largest file size supported by qemu
> (we could go 511 bytes larger if we used byte-accurate sizing instead
> of rounding up to the next sector boundary, but that wouldn't change
> the human-readable result).
> 
> Reported-by: Richard W.M. Jones <rjones@redhat.com>
> Signed-off-by: Eric Blake <eblake@redhat.com>

Thanks, applied to the block-next branch for 4.1.

Kevin

Re: [Qemu-devel] [PATCH] qemu-img: Saner printing of large file sizes

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1316 bytes --]

On 4/1/19 9:21 AM, Kevin Wolf wrote:
> Am 30.03.2019 um 16:07 hat Eric Blake geschrieben:
>> Disk sizes close to INT64_MAX cause overflow, for some pretty
>> ridiculous output:
>>
>>   $ ./nbdkit -U - memory size=$((2**63 - 512)) --run 'qemu-img info $nbd'
>>   image: nbd+unix://?socket=/tmp/nbdkitHSAzNz/socket
>>   file format: raw
>>   virtual size: -8388607T (9223372036854775296 bytes)
>>   disk size: unavailable
>>
>> But there's no reason to have two separate implementations of integer
>> to human-readable abbreviation, where one has overflow and stops at
>> 'T', while the other avoids overflow and goes all the way to 'E'. With
>> this patch, the output now claims 8EiB instead of -8388607T, which
>> really is the correct rounding of largest file size supported by qemu
>> (we could go 511 bytes larger if we used byte-accurate sizing instead
>> of rounding up to the next sector boundary, but that wouldn't change
>> the human-readable result).
>>
>> Reported-by: Richard W.M. Jones <rjones@redhat.com>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
> 
> Thanks, applied to the block-next branch for 4.1.

You'll need v2. Coming up shortly.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]