Re: [Qemu-devel] [PATCH 0/3] info blockstats (block-qcow2): show highest allocated offset (bytes)

[Shahar Frank <sfrank@redhat.com>]

Uri Lublin wrote:
> Anthony Liguori wrote:
>> Uri Lublin wrote:
>>> From: Uri Lublin <uril@redhat.com>
>>>
>>> This patchset let the user know the highest allocated byte of qcow2 
>>> images.
>>> Actually it's the first unallocated byte after the highest byte written,
>>> cluster-size aligned.
>>>
>>> The highest allocated byte gives a maximal limit (easy to calculate)
>>> to the number of bytes allocated for that image, and may hint how 
>>> many more allocations can be done before we reach end-of-file (end of 
>>> host block device).
>>> Although there may be many free blocks below that number (allocated 
>>> and freed)
>>> the file system can not deallocate those blocks, and they have to be 
>>> reused
>>> by qemu. Also note that due to fragmentation those free blocks may not
>>> be used on next allocations.
>>>
>>> It can be useful for truncation of backing file images (ftruncate).
>>> Also it may be useful for defragmentation later (although we'll need
>>> the number of free blocks as well).
>>
>> I'm having trouble seeing the utility of this as it seems to be not 
>> really reliable.  Surely, after a lot of work, you'll have one block 
>> far at the end of the file, no?  I don't see how knowing this location 
>> helps practically speaking.  Can you explain a little more about what 
>> you want to use this functionality for?
>>
> 
> Currently, qcow2 images can only grow, never shrink.
> The main usage would be to trigger an appropriate operation when a 
> threshold is reached. The threshold and operation are defined by a 
> management application.
> Basically we can do one of the following:
> 1. Defragment the qcow2 image (simplest way is to qemu-img convert it, 
> the best is to do it online if possible).
> 2. Allocate more space (especially when using LVM)
> 
> I plan on adding another "blockstat" that shows the number of free 
> bytes/blocks/clusters for a qcow2 image. This would make it easier to 
> choose the appropriate operation above.
> 

As Uri wrote this patch is part of a patch set that will include also 
free clusters statistics. Together these new statistics will enable an 
image repository system to quickly estimate how "fragmented" is a given 
image, so it can decide if to perform expensive 
cleaning/shrinking/defragmenting processes on that image.

This highest allocated byte is also critical to let you run qcow2 images 
over raw devices- it give you a good estimate how much space you still 
have for the image to expand. This is a bit pessimistic statistic, but 
due the much too simplistic allocation mechanism within qcow2 and the 
recent multiple clusters allocations optimizations, this may be also a 
pretty realistic figure.
Note that if you run qcow2 over expandable volumes such as LVM volumes, 
you can use this figure by monitoring process that will expand the 
volume once a threshold is passed.

Shahar
> Thanks,
>     Uri.
> 
>