Re: [Qemu-devel] [PATCH 9/9] block-migration: add named dirty bitmaps migration

[John Snow <jsnow@redhat.com>]

On 01/13/2015 04:16 AM, Vladimir Sementsov-Ogievskiy wrote:
>
> On 12.01.2015 17:42, Paolo Bonzini wrote:
>> On 12/01/2015 15:20, Vladimir Sementsov-Ogievskiy wrote:
>>> On 09.01.2015 01:36, Paolo Bonzini wrote:
>>>> The bitmaps are transmitted many times in their entirety, but only the
>>>> last copy actually means something. The others are lost. This means
>>>> you should use the non-live interface (register_savevm). This will
>>>> simplify the code a lot.
>>> But what if the dirty bitmap is really big?
>>>
>>> For example, for 5 Pb drive the bitmap with granularity 65536 will be of
>>> 2 Gb size. So, shouldn't it be migrated iteratively for live migration?
>> But your code is not attempting to do that.  It is not attempting to
>> track the dirtiness of the dirty bitmap, so to speak.
>>
>> For such a huge storage, in any case, I suspect the solution is to not
>> use QEMU's live operations and, instead, operate at the storage level.
>>
>> Paolo
>
> On 12.01.2015 22:09, Paolo Bonzini wrote:
>> On 12/01/2015 18:31, John Snow wrote:
>>>> How do you track which parts of the disk have been acted upon (e.g.
>>>> mirrored in the case of the drive-mirror command), so that they have
>>>> become 0?
>>> Hm... so your question here is, "What happens if the named bitmap you
>>> are transferring  gets reset to 0 during migration? Will anybody track
>>> this change?"
>>>
>>> As far as I know, the only "consumer" of _named_ BdrvDirtyBitmaps is
>>> block/backup.c, and we do not support it for mirror (yet?)
>> Right.
>>
>>> So can a block-backup job be running WHILE we migrate?
>> I think so.
>>
>> Paolo
>
> There is no dirty bitmap for dirty bitmap in my solution, actually block
> migration dirty bitmap is used for this. It provides tracking of setting
> bitmap bits to 1 (when corresponding blocks are written), and if the
> bitmap changes only is this way (no other block-jobs are working with
> the migrated bitmap) - then the bitmap will be migrated iteratively as
> it is.
>
> Such approach doesn't provide tracking of resetting the bitmap to 0, and
> such changes are not migrated. So if the bitmap is in use by any
> block-job, it may have additional set bits after migration.
>
> If we want to migrate more precisely we need separate dirty bitmap for
> every migrated bitmap. And tracking of this dirty-dirty bitmap should be
> injected into BdrvDirtyBitmap interface.
>
> But do we really need precise migration of the bitmap being in use by
> some block-job (backup for example)? In this case we should "migrate"
> the backup process to, migrate working block-job.. Is it possible and is
> it planned to be implemented?
>
> Best regards,
> Vladimir
>
>
>

This is my question, too. The approach presented here will at least work 
correctly, as far as I can tell. The only problem may arise if we try to 
migrate while doing an incremental backup which will lead to too many 
dirty bits being migrated, which is just an inefficiency.

I'm not convinced it's important ...
Stefan, any thoughts?