Re: [Qemu-devel] [PATCH v2 RFC 0/8] block: persistent dirty bitmaps

[Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>]

On 12.06.2015 13:36, Stefan Hajnoczi wrote:
> On Fri, Jun 12, 2015 at 12:58:35PM +0300, Denis V. Lunev wrote:
>> On 11/06/15 23:06, Stefan Hajnoczi wrote:
>>> The load/store API is not scalable when bitmaps are 1 MB or larger.
>>>
>>> For example, a 500 GB disk image with 64 KB granularity requires a 1 MB
>>> bitmap.  If a guest has several disk images of this size, then multiple
>>> megabytes must be read to start the guest and written out to shut down
>>> the guest.
>>>
>>> By comparison, the L1 table for the 500 GB disk image is less than 8 KB.
>>>
>>> I think something like qcow2-cache.c or metabitmaps should be used to
>>> lazily read/write persistent bitmaps.  That way only small portions need
>>> to be read/written at a time.
>>>
>>> Stefan
>> for the first iteration we could open the image, start tracking,
>> read bitmap as one entity in the background and or read
>> and collected data.
>>
>> partial read could be done in the next step
> Making bitmap load/store fully lazy will require changes to the
> load/store API, so it's worth thinking about a little upfront.
> Otherwise there will be a lot of code churn when the fully lazy patches
> are posted.  As a reviewer it's in my interest to only spend time
> reviewing the final version instead of code that gets thrown out :-),
> but I understand.
>
> If you can make the read lazy to some extent that's a good start.
That way we can improve load performance, but what about store?

I see two solutions:
1) meta bitmaps (already mentioned)
2) Always (optionally?) have two bitmaps instead one: backing, which 
should be equal to the bitmap, already stored to the image, and active 
delta. This can be used instead of meta bitmaps in migration too.

difference:
with meta bitmaps we have double time overhead for writing to the bitmap 
(which is more often operations as I think),
with second approach we have double overhead for read from the bitmap 
(but for backup, we can *or* these two bitmaps once, and it can be done 
fast, using the power of HBitmap). And of course double ram overhead..

-- 
Best regards,
Vladimir
* now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.