Re: [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>]

23.08.2017 11:59, Vladimir Sementsov-Ogievskiy wrote:
> 22.08.2017 22:07, John Snow wrote:
>> Well, we knew we'd want this sooner or later. I've got some pings
>> downstream over whether or not we support persistent bitmaps for
>> non-qcow2 formats.
>>
>> Currently: no, we don't.
>>
>> We tried two different ideas for storage agnostic bitmap persistence:
>>
>>
>> (1) Using qcow2 as a storage container format (similar to VM save
>> states) for information not associated with that particular drive.
>>
>> This didn't go over so well; we decided it was too messy logistically to
>> manage data in any meaningful sense in a file that didn't share any
>> semantic relationship to the qcow2 in question.
>>
>> Well, "We" decided is more like "Kevin and Max" decided. They are right,
>> though.
>>
>>
>> (2) Using a new proto-wrapper format that Fam Zheng designed that serves
>> as a simple top-layer redirect for metadata that allows us to associate
>> raw bitmap data with an arbitrary backing image.
>>
>> This also didn't go over so well, the desire for a "new format" was
>> pretty thin, even if it was only a pseudo-format.
>>
>>
>> We'd still like to be able to use bitmaps with non-qcow2 formats
>> however, and people are going to keep asking. So here's a third thought:
>>
>>
>> (3) Add either a new flag that turns qcow2's backing file into a full
>> R/W backing file, or add a new extension to qcow2 entirely (bypassing
>> the traditional backing file mechanism to avoid confusion for older
>> tooling) that adds a new read-write backing file field.
>>
>> This RW backing file field will be used for all reads AND writes; the
>> qcow2 in question becomes a metadata container on top of the BDS chain.
>> We can re-use Vladimir's bitmap persistence extension to save bitmaps in
>> a qcow2 shell.
>>
>> The qcow2 becomes effectively a metadata cache for a new (essentially)
>> filter node that handles features such as bitmaps. This could also be
>> used to provide allocation map data for RAW files and other goodies down
>> the road.
>>
>> Hopefully this achieves our desire to not create new formats AND our
>> desire to concentrate features (and debugging, testing, etc) into qcow2,
>> while allowing users to "have bitmaps with raw files."
>>
>> Of course, in this scenario, users now have two files: a qcow2 wrapper
>> and the actual raw file in question; but regardless of how we were going
>> to solve this, a raw file necessitates an external file of some sort,
>> else we give up the idea that it was a raw file.
>>
>>
>> Sound good?
>
> Looks interesting. It is a clean reusing of qcow2-bitmaps without any 
> modifications to them.
>
> Should there be some problems with internal snapshots and other things?
>
>
>
Hm. looks like that this backing file should not only receive all reads 
and writes, but almost everything ->bdrv_ handlers, except bitmap 
related of course. This doesn't seems simple to implement. Especially if 
imaging some not-raw feature-full format under this thin qcow2 layer.. 
Or we can restrict this RW backing file to be raw-only?


-- 
Best regards,
Vladimir