[Qemu-devel] External COW format for raw images

[Qemu-devel] External COW format for raw images

[Robert Wang]

As you known, raw image is very popular,but the raw image format does
NOT support Copy-On-Write,a raw image file can NOT be used as a copy
destination, then image streaming/Live Block Copy will NOT work.

To fix this, we need to add a new block driver raw-cow to QEMU. If
finished, we can use qemu-img like this:
qemu-img create -f raw-cow -o backing_file=ubuntu.img,raw_file=my_vm.img
my_vm.raw-cow

1) ubuntu.img is the backing file, my_vm.img is a raw file,
my_vm.raw-cow stores a COW bitmap related to my_vm.img.

2) If the entire COW bitmap is set to dirty flag then we can get all
information from my_vm.img and can ignore ubuntu.img and my_vm.raw-cow
from now.

To implement this, I think I can follow these steps:
1) Add a new member to BlockDriverState struct:
char raw_file[1024];
This member will track raw_file parameter related to raw-cow file from
command line.

2) 	* Create a new file block/raw-cow.c. It will be much more like the
mixture of block/cow.c and block/raw.c.

So I will change some functions in cow.c and raw.c to none-static, then
raw-cow.c can re-use them. When read operation occurs, determine whether
dirty flag in raw-cow image is set. If true, read directly from the raw
file. After write operation, set related dirty flag in raw-cow image.
And other functions might also be modified.

	* Of course, format_name member of BlockDriver struct will be "raw-cow".
And in order to keep relationship with raw file( like my_vm.img) ,
raw_cow_header struct should be
struct raw_cow_header {
uint32_t magic;
uint32_t version;
char backing_file[1024];
char raw_file[1024];/* added*/
int32_t mtime;
uint64_t size;
uint32_t sectorsize;
};
	* Struct raw_cow_create_options should be one member plus based on
cow_create_options:
{
.name = BLOCK_OPT_RAW_FILE,
.type = OPT_STRING,
.help = "Raw file name"
},

3) Add bdrv_get_raw_filename in img_info function of qemu-img.c. In
bdrv_get_raw_filename, if the format of the image file is "raw-cow",
print the related raw file.

Do you think my approach is right?
Thank you.

Re: [Qemu-devel] External COW format for raw images

[Stefan Hajnoczi]

2011/7/19 Robert Wang <wdongxu@linux.vnet.ibm.com>:
> 2)      * Create a new file block/raw-cow.c. It will be much more like the
> mixture of block/cow.c and block/raw.c.
>
> So I will change some functions in cow.c and raw.c to none-static, then
> raw-cow.c can re-use them. When read operation occurs, determine whether
> dirty flag in raw-cow image is set. If true, read directly from the raw
> file. After write operation, set related dirty flag in raw-cow image.
> And other functions might also be modified.

The block/cow.c driver is inefficient because it does I/O for each
bitmap set/test operation.  I think doing this more efficiently means
basically rewriting the bitmap code to keep a writethrough bitmap in
memory.

Regarding the file header, the msize is not really useful - there is
no interface to read it and no feature makes use of msize.  The
sector_size field could also be dropped.  The true sector size does
from the underlying storage that contains this image file.  Especially
in the cache=none (O_DIRECT) case we need to honor the underlying
sector size and I'm not sure the sector_size field helps.

Stefan

Re: [Qemu-devel] External COW format for raw images

[Kevin Wolf]

Am 19.07.2011 11:25, schrieb Robert Wang:
> As you known, raw image is very popular,but the raw image format does
> NOT support Copy-On-Write,a raw image file can NOT be used as a copy
> destination, then image streaming/Live Block Copy will NOT work.
> 
> To fix this, we need to add a new block driver raw-cow to QEMU. If
> finished, we can use qemu-img like this:
> qemu-img create -f raw-cow -o backing_file=ubuntu.img,raw_file=my_vm.img
> my_vm.raw-cow

Just one comment for the start: This is not only useful for raw (while
certainly being the most important case), but also for every other image
format for which qemu doesn't support backing files.

This means that we should look for a better name than raw-cow. I know,
it was me who introduced this, but only for lack of a better name.

> 1) ubuntu.img is the backing file, my_vm.img is a raw file,
> my_vm.raw-cow stores a COW bitmap related to my_vm.img.
> 
> 2) If the entire COW bitmap is set to dirty flag then we can get all
> information from my_vm.img and can ignore ubuntu.img and my_vm.raw-cow
> from now.
> 
> To implement this, I think I can follow these steps:
> 1) Add a new member to BlockDriverState struct:
> char raw_file[1024];
> This member will track raw_file parameter related to raw-cow file from
> command line.

Can't this be private to the COW driver? It certainly will have a
BDRVRawCowState or something like that.

> 2) 	* Create a new file block/raw-cow.c. It will be much more like the
> mixture of block/cow.c and block/raw.c.
> 
> So I will change some functions in cow.c and raw.c to none-static, then
> raw-cow.c can re-use them.

I think it's better to keep drivers cleanly separated. If we really need
to share code, we should provide some sort of a library that is used by
both, but I doubt that it's required in this case.

What the driver should probably do, is to open the raw file internally
and keep a BlockDriverState of the raw file in its private structure.
For all accesses to the raw file, use the "official" interfaces of the
raw driver, like bdrv_aio_readv/writev.

>  When read operation occurs, determine whether
> dirty flag in raw-cow image is set. If true, read directly from the raw
> file. After write operation, set related dirty flag in raw-cow image.
> And other functions might also be modified.
> 
> 	* Of course, format_name member of BlockDriver struct will be "raw-cow".
> And in order to keep relationship with raw file( like my_vm.img) ,
> raw_cow_header struct should be
> struct raw_cow_header {
> uint32_t magic;
> uint32_t version;
> char backing_file[1024];
> char raw_file[1024];/* added*/
> int32_t mtime;
> uint64_t size;
> uint32_t sectorsize;

I don't think any of mtime, size and sectorsize are necessary. They will
just be taken from the raw file (if needed at all).

> };
> 	* Struct raw_cow_create_options should be one member plus based on
> cow_create_options:
> {
> .name = BLOCK_OPT_RAW_FILE,
> .type = OPT_STRING,
> .help = "Raw file name"
> },
> 
> 3) Add bdrv_get_raw_filename in img_info function of qemu-img.c. In
> bdrv_get_raw_filename, if the format of the image file is "raw-cow",
> print the related raw file.

Hm... Won't be implemented by any other driver, but I guess it makes
some sense to provide this information.

Kevin

Re: [Qemu-devel] External COW format for raw images

[Anthony Liguori]

On 07/19/2011 04:25 AM, Robert Wang wrote:
> As you known, raw image is very popular,but the raw image format does
> NOT support Copy-On-Write,a raw image file can NOT be used as a copy
> destination, then image streaming/Live Block Copy will NOT work.
> 
> To fix this, we need to add a new block driver raw-cow to QEMU. If
> finished, we can use qemu-img like this:
> qemu-img create -f raw-cow -o backing_file=ubuntu.img,raw_file=my_vm.img
> my_vm.raw-cow
> 
> 1) ubuntu.img is the backing file, my_vm.img is a raw file,
> my_vm.raw-cow stores a COW bitmap related to my_vm.img.
> 
> 2) If the entire COW bitmap is set to dirty flag then we can get all
> information from my_vm.img and can ignore ubuntu.img and my_vm.raw-cow
> from now.
> 
> To implement this, I think I can follow these steps:
> 1) Add a new member to BlockDriverState struct:
> char raw_file[1024];
> This member will track raw_file parameter related to raw-cow file from
> command line.
> 
> 2) 	* Create a new file block/raw-cow.c. It will be much more like the
> mixture of block/cow.c and block/raw.c.
> 
> So I will change some functions in cow.c and raw.c to none-static, then
> raw-cow.c can re-use them. When read operation occurs, determine whether
> dirty flag in raw-cow image is set. If true, read directly from the raw
> file. After write operation, set related dirty flag in raw-cow image.
> And other functions might also be modified.
> 
> 	* Of course, format_name member of BlockDriver struct will be "raw-cow".
> And in order to keep relationship with raw file( like my_vm.img) ,
> raw_cow_header struct should be
> struct raw_cow_header {
> uint32_t magic;
> uint32_t version;
> char backing_file[1024];
> char raw_file[1024];/* added*/
> int32_t mtime;
> uint64_t size;
> uint32_t sectorsize;
> };

I'd suggest that doing an image format is the wrong approach here.  Why
not just have a image format where you can pass it the location of a
bitmap?  That let's you compose arbitrarily complex backing file chains
and avoids the introduce of a new bitmap.

The bitmap format is also useful for implementing things like dirty
tracking.

Regards,

Anthony Liguori

> 	* Struct raw_cow_create_options should be one member plus based on
> cow_create_options:
> {
> .name = BLOCK_OPT_RAW_FILE,
> .type = OPT_STRING,
> .help = "Raw file name"
> },
> 
> 3) Add bdrv_get_raw_filename in img_info function of qemu-img.c. In
> bdrv_get_raw_filename, if the format of the image file is "raw-cow",
> print the related raw file.
> 
> Do you think my approach is right?
> Thank you.
> 
> 

Re: [Qemu-devel] External COW format for raw images

[Frediano Ziglio]

2011/7/19 Robert Wang <wdongxu@linux.vnet.ibm.com>:
> As you known, raw image is very popular,but the raw image format does
> NOT support Copy-On-Write,a raw image file can NOT be used as a copy
> destination, then image streaming/Live Block Copy will NOT work.
>
> To fix this, we need to add a new block driver raw-cow to QEMU. If
> finished, we can use qemu-img like this:
> qemu-img create -f raw-cow -o backing_file=ubuntu.img,raw_file=my_vm.img
> my_vm.raw-cow
>
> 1) ubuntu.img is the backing file, my_vm.img is a raw file,
> my_vm.raw-cow stores a COW bitmap related to my_vm.img.
>
> 2) If the entire COW bitmap is set to dirty flag then we can get all
> information from my_vm.img and can ignore ubuntu.img and my_vm.raw-cow
> from now.
>
> To implement this, I think I can follow these steps:
> 1) Add a new member to BlockDriverState struct:
> char raw_file[1024];
> This member will track raw_file parameter related to raw-cow file from
> command line.
>
> 2)      * Create a new file block/raw-cow.c. It will be much more like the
> mixture of block/cow.c and block/raw.c.
>
> So I will change some functions in cow.c and raw.c to none-static, then
> raw-cow.c can re-use them. When read operation occurs, determine whether
> dirty flag in raw-cow image is set. If true, read directly from the raw
> file. After write operation, set related dirty flag in raw-cow image.
> And other functions might also be modified.
>
>        * Of course, format_name member of BlockDriver struct will be "raw-cow".
> And in order to keep relationship with raw file( like my_vm.img) ,
> raw_cow_header struct should be
> struct raw_cow_header {
> uint32_t magic;
> uint32_t version;
> char backing_file[1024];
> char raw_file[1024];/* added*/
> int32_t mtime;
> uint64_t size;
> uint32_t sectorsize;
> };
>        * Struct raw_cow_create_options should be one member plus based on
> cow_create_options:
> {
> .name = BLOCK_OPT_RAW_FILE,
> .type = OPT_STRING,
> .help = "Raw file name"
> },
>
> 3) Add bdrv_get_raw_filename in img_info function of qemu-img.c. In
> bdrv_get_raw_filename, if the format of the image file is "raw-cow",
> print the related raw file.
>
> Do you think my approach is right?
> Thank you.
>

I don't understand if you mean just a way to track clusters/sectors
changed or a new way to implement snapshotting, something that writing
data just store original to cow like like:


normal backfile

is allocated on image
  write to image
else
  allocate on image
  copy from backing to image
  write to image (patch before previous write)

cow-raw (inverse backfile)

is allocated on image
  write to backing
else
  allocate on image
  copy from backing to image
  write to backing


that is


is not allocated on image
  allocate on image
  copy from backing to image
is normal backing
  write to image
else
  write to backing


Frediano

Re: [Qemu-devel] External COW format for raw images

[Robert Wang]

On Tue, 2011-07-19 at 16:39 +0200, Frediano Ziglio wrote:
> 2011/7/19 Robert Wang <wdongxu@linux.vnet.ibm.com>:
> > As you known, raw image is very popular,but the raw image format does
> > NOT support Copy-On-Write,a raw image file can NOT be used as a copy
> > destination, then image streaming/Live Block Copy will NOT work.
> >
> > To fix this, we need to add a new block driver raw-cow to QEMU. If
> > finished, we can use qemu-img like this:
> > qemu-img create -f raw-cow -o backing_file=ubuntu.img,raw_file=my_vm.img
> > my_vm.raw-cow
> >
> > 1) ubuntu.img is the backing file, my_vm.img is a raw file,
> > my_vm.raw-cow stores a COW bitmap related to my_vm.img.
> >
> > 2) If the entire COW bitmap is set to dirty flag then we can get all
> > information from my_vm.img and can ignore ubuntu.img and my_vm.raw-cow
> > from now.
> >
> > To implement this, I think I can follow these steps:
> > 1) Add a new member to BlockDriverState struct:
> > char raw_file[1024];
> > This member will track raw_file parameter related to raw-cow file from
> > command line.
> >
> > 2)      * Create a new file block/raw-cow.c. It will be much more like the
> > mixture of block/cow.c and block/raw.c.
> >
> > So I will change some functions in cow.c and raw.c to none-static, then
> > raw-cow.c can re-use them. When read operation occurs, determine whether
> > dirty flag in raw-cow image is set. If true, read directly from the raw
> > file. After write operation, set related dirty flag in raw-cow image.
> > And other functions might also be modified.
> >
> >        * Of course, format_name member of BlockDriver struct will be "raw-cow".
> > And in order to keep relationship with raw file( like my_vm.img) ,
> > raw_cow_header struct should be
> > struct raw_cow_header {
> > uint32_t magic;
> > uint32_t version;
> > char backing_file[1024];
> > char raw_file[1024];/* added*/
> > int32_t mtime;
> > uint64_t size;
> > uint32_t sectorsize;
> > };
> >        * Struct raw_cow_create_options should be one member plus based on
> > cow_create_options:
> > {
> > .name = BLOCK_OPT_RAW_FILE,
> > .type = OPT_STRING,
> > .help = "Raw file name"
> > },
> >
> > 3) Add bdrv_get_raw_filename in img_info function of qemu-img.c. In
> > bdrv_get_raw_filename, if the format of the image file is "raw-cow",
> > print the related raw file.
> >
> > Do you think my approach is right?
> > Thank you.
> >
> 
> I don't understand if you mean just a way to track clusters/sectors
> changed or a new way to implement snapshotting, something that writing
> data just store original to cow like like:
> 
> 
> normal backfile
> 
> is allocated on image
>   write to image
> else
>   allocate on image
>   copy from backing to image
>   write to image (patch before previous write)
> 
> cow-raw (inverse backfile)
> 
> is allocated on image
>   write to backing
> else
>   allocate on image
>   copy from backing to image
>   write to backing
> 
> 
> that is
> 
> 
> is not allocated on image
>   allocate on image
>   copy from backing to image
> is normal backing
>   write to image
> else
>   write to backing
> 
> 
> Frediano
> 
Frediano, it mainly is to track clusters/sectors changes, not a new way
to implement snapshotting.

Re: [Qemu-devel] External COW format for raw images

[Stefan Hajnoczi]

2011/7/19 Anthony Liguori <anthony@codemonkey.ws>:
> On 07/19/2011 04:25 AM, Robert Wang wrote:
>> As you known, raw image is very popular,but the raw image format does
>> NOT support Copy-On-Write,a raw image file can NOT be used as a copy
>> destination, then image streaming/Live Block Copy will NOT work.
>>
>> To fix this, we need to add a new block driver raw-cow to QEMU. If
>> finished, we can use qemu-img like this:
>> qemu-img create -f raw-cow -o backing_file=ubuntu.img,raw_file=my_vm.img
>> my_vm.raw-cow
>>
>> 1) ubuntu.img is the backing file, my_vm.img is a raw file,
>> my_vm.raw-cow stores a COW bitmap related to my_vm.img.
>>
>> 2) If the entire COW bitmap is set to dirty flag then we can get all
>> information from my_vm.img and can ignore ubuntu.img and my_vm.raw-cow
>> from now.
>>
>> To implement this, I think I can follow these steps:
>> 1) Add a new member to BlockDriverState struct:
>> char raw_file[1024];
>> This member will track raw_file parameter related to raw-cow file from
>> command line.
>>
>> 2)    * Create a new file block/raw-cow.c. It will be much more like the
>> mixture of block/cow.c and block/raw.c.
>>
>> So I will change some functions in cow.c and raw.c to none-static, then
>> raw-cow.c can re-use them. When read operation occurs, determine whether
>> dirty flag in raw-cow image is set. If true, read directly from the raw
>> file. After write operation, set related dirty flag in raw-cow image.
>> And other functions might also be modified.
>>
>>       * Of course, format_name member of BlockDriver struct will be "raw-cow".
>> And in order to keep relationship with raw file( like my_vm.img) ,
>> raw_cow_header struct should be
>> struct raw_cow_header {
>> uint32_t magic;
>> uint32_t version;
>> char backing_file[1024];
>> char raw_file[1024];/* added*/
>> int32_t mtime;
>> uint64_t size;
>> uint32_t sectorsize;
>> };
>
> I'd suggest that doing an image format is the wrong approach here.  Why
> not just have a image format where you can pass it the location of a
> bitmap?  That let's you compose arbitrarily complex backing file chains
> and avoids the introduce of a new bitmap.
>
> The bitmap format is also useful for implementing things like dirty
> tracking.

Are you describing something like -drive
file=bitmap:raw.img:backing.img:dirty.bmap?

Stefan

Re: [Qemu-devel] External COW format for raw images

[Marcelo Tosatti]

On Wed, Jul 20, 2011 at 09:35:05AM +0100, Stefan Hajnoczi wrote:
> 2011/7/19 Anthony Liguori <anthony@codemonkey.ws>:
> > On 07/19/2011 04:25 AM, Robert Wang wrote:
> >> As you known, raw image is very popular,but the raw image format does
> >> NOT support Copy-On-Write,a raw image file can NOT be used as a copy
> >> destination, then image streaming/Live Block Copy will NOT work.
> >>
> >> To fix this, we need to add a new block driver raw-cow to QEMU. If
> >> finished, we can use qemu-img like this:
> >> qemu-img create -f raw-cow -o backing_file=ubuntu.img,raw_file=my_vm.img
> >> my_vm.raw-cow
> >>
> >> 1) ubuntu.img is the backing file, my_vm.img is a raw file,
> >> my_vm.raw-cow stores a COW bitmap related to my_vm.img.
> >>
> >> 2) If the entire COW bitmap is set to dirty flag then we can get all
> >> information from my_vm.img and can ignore ubuntu.img and my_vm.raw-cow
> >> from now.
> >>
> >> To implement this, I think I can follow these steps:
> >> 1) Add a new member to BlockDriverState struct:
> >> char raw_file[1024];
> >> This member will track raw_file parameter related to raw-cow file from
> >> command line.
> >>
> >> 2)    * Create a new file block/raw-cow.c. It will be much more like the
> >> mixture of block/cow.c and block/raw.c.
> >>
> >> So I will change some functions in cow.c and raw.c to none-static, then
> >> raw-cow.c can re-use them. When read operation occurs, determine whether
> >> dirty flag in raw-cow image is set. If true, read directly from the raw
> >> file. After write operation, set related dirty flag in raw-cow image.
> >> And other functions might also be modified.
> >>
> >>       * Of course, format_name member of BlockDriver struct will be "raw-cow".
> >> And in order to keep relationship with raw file( like my_vm.img) ,
> >> raw_cow_header struct should be
> >> struct raw_cow_header {
> >> uint32_t magic;
> >> uint32_t version;
> >> char backing_file[1024];
> >> char raw_file[1024];/* added*/
> >> int32_t mtime;
> >> uint64_t size;
> >> uint32_t sectorsize;
> >> };
> >
> > I'd suggest that doing an image format is the wrong approach here.  Why
> > not just have a image format where you can pass it the location of a
> > bitmap?  That let's you compose arbitrarily complex backing file chains
> > and avoids the introduce of a new bitmap.

Its possible to implement backing file chains in any case, no need for 
separate bitmap on-disk.

> > The bitmap format is also useful for implementing things like dirty
> > tracking.
> 
> Are you describing something like -drive
> file=bitmap:raw.img:backing.img:dirty.bmap?
> 
> Stefan

The bitmap (whether its separate or part of the image) is intimately
related to the raw file, and the relation is specific indicating
allocated status.

Perhaps what Anthony is suggesting is an interface for on-disk bitmap
access, with caching?

Re: [Qemu-devel] External COW format for raw images

[Stefan Hajnoczi]

On Wed, Jul 20, 2011 at 4:57 PM, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> On Wed, Jul 20, 2011 at 09:35:05AM +0100, Stefan Hajnoczi wrote:
>> 2011/7/19 Anthony Liguori <anthony@codemonkey.ws>:
>> > On 07/19/2011 04:25 AM, Robert Wang wrote:
>> >> As you known, raw image is very popular,but the raw image format does
>> >> NOT support Copy-On-Write,a raw image file can NOT be used as a copy
>> >> destination, then image streaming/Live Block Copy will NOT work.
>> >>
>> >> To fix this, we need to add a new block driver raw-cow to QEMU. If
>> >> finished, we can use qemu-img like this:
>> >> qemu-img create -f raw-cow -o backing_file=ubuntu.img,raw_file=my_vm.img
>> >> my_vm.raw-cow
>> >>
>> >> 1) ubuntu.img is the backing file, my_vm.img is a raw file,
>> >> my_vm.raw-cow stores a COW bitmap related to my_vm.img.
>> >>
>> >> 2) If the entire COW bitmap is set to dirty flag then we can get all
>> >> information from my_vm.img and can ignore ubuntu.img and my_vm.raw-cow
>> >> from now.
>> >>
>> >> To implement this, I think I can follow these steps:
>> >> 1) Add a new member to BlockDriverState struct:
>> >> char raw_file[1024];
>> >> This member will track raw_file parameter related to raw-cow file from
>> >> command line.
>> >>
>> >> 2)    * Create a new file block/raw-cow.c. It will be much more like the
>> >> mixture of block/cow.c and block/raw.c.
>> >>
>> >> So I will change some functions in cow.c and raw.c to none-static, then
>> >> raw-cow.c can re-use them. When read operation occurs, determine whether
>> >> dirty flag in raw-cow image is set. If true, read directly from the raw
>> >> file. After write operation, set related dirty flag in raw-cow image.
>> >> And other functions might also be modified.
>> >>
>> >>       * Of course, format_name member of BlockDriver struct will be "raw-cow".
>> >> And in order to keep relationship with raw file( like my_vm.img) ,
>> >> raw_cow_header struct should be
>> >> struct raw_cow_header {
>> >> uint32_t magic;
>> >> uint32_t version;
>> >> char backing_file[1024];
>> >> char raw_file[1024];/* added*/
>> >> int32_t mtime;
>> >> uint64_t size;
>> >> uint32_t sectorsize;
>> >> };
>> >
>> > I'd suggest that doing an image format is the wrong approach here.  Why
>> > not just have a image format where you can pass it the location of a
>> > bitmap?  That let's you compose arbitrarily complex backing file chains
>> > and avoids the introduce of a new bitmap.
>
> Its possible to implement backing file chains in any case, no need for
> separate bitmap on-disk.
>
>> > The bitmap format is also useful for implementing things like dirty
>> > tracking.
>>
>> Are you describing something like -drive
>> file=bitmap:raw.img:backing.img:dirty.bmap?
>>
>> Stefan
>
> The bitmap (whether its separate or part of the image) is intimately
> related to the raw file, and the relation is specific indicating
> allocated status.
>
> Perhaps what Anthony is suggesting is an interface for on-disk bitmap
> access, with caching?

Anthony, could you elaborate what you meant?

Stefan