From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59025) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZYHLt-0000ZT-Iu for qemu-devel@nongnu.org; Sat, 05 Sep 2015 13:34:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZYHLp-0000cS-3T for qemu-devel@nongnu.org; Sat, 05 Sep 2015 13:34:09 -0400 Received: from relay.parallels.com ([195.214.232.42]:54565) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZYHLo-0000bo-OE for qemu-devel@nongnu.org; Sat, 05 Sep 2015 13:34:05 -0400 Message-ID: <55EB277D.9030700@virtuozzo.com> Date: Sat, 5 Sep 2015 20:33:49 +0300 From: Vladimir Sementsov-Ogievskiy MIME-Version: 1.0 References: <1441471439-6157-1-git-send-email-vsementsov@virtuozzo.com> <1441471439-6157-4-git-send-email-vsementsov@virtuozzo.com> In-Reply-To: <1441471439-6157-4-git-send-email-vsementsov@virtuozzo.com> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH 03/17] spec: add qcow2-dirty-bitmaps specification List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: kwolf@redhat.com, stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com On 05.09.2015 19:43, Vladimir Sementsov-Ogievskiy wrote: > Persistent dirty bitmaps will be saved into qcow2 files. It may be used > as 'internal' bitmaps (for qcow2 drives) or as 'external' bitmaps for > other drives (there may be qcow2 file with zero disk size but with > several dirty bitmaps for other drives). > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > docs/specs/qcow2.txt | 127 ++++++++++++++++++++++++++++++++++++++++++++++++++- > 1 file changed, 126 insertions(+), 1 deletion(-) > > diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt > index 121dfc8..5fc0365 100644 > --- a/docs/specs/qcow2.txt > +++ b/docs/specs/qcow2.txt > @@ -103,7 +103,13 @@ in the description of a field. > write to an image with unknown auto-clear features if it > clears the respective bits from this field first. > > - Bits 0-63: Reserved (set to 0) > + Bit 0: Dirty bitmaps bit. If this bit is set then > + there is a _consistent_ Dirty bitmaps extension > + in the image. If it is not set, but there is a > + Dirty bitmaps extension, its data should be > + considered as inconsistent. > + > + Bits 1-63: Reserved (set to 0) > > 96 - 99: refcount_order > Describes the width of a reference count block entry (width > @@ -123,6 +129,7 @@ be stored. Each extension has a structure like the following: > 0x00000000 - End of the header extension area > 0xE2792ACA - Backing file format name > 0x6803f857 - Feature name table > + 0x23852875 - Dirty bitmaps > other - Unknown header extension, can be safely > ignored > > @@ -166,6 +173,24 @@ the header extension data. Each entry look like this: > terminated if it has full length) > > > +== Dirty bitmaps == > + > +Dirty bitmaps is an optional header extension. It provides an ability to store > +dirty bitmaps in a qcow2 image. The fields are: > + > + 0 - 3: nb_dirty_bitmaps > + The number of dirty bitmaps contained in the image. Valid > + values: 0 - 65535. > + > + 4 - 7: dirty_bitmap_directory_size > + Size of the Dirty Bitmap Directory in bytes. Valid values: > + 0 - 67108864 (= 1024 * nb_dirty_bitmaps). > + > + 8 - 15: dirty_bitmap_directory_offset > + Offset into the image file at which the Dirty Bitmap > + Directory starts. Must be aligned to a cluster boundary. > + > + > == Host cluster management == > > qcow2 manages the allocation of host clusters by maintaining a reference count > @@ -360,3 +385,103 @@ Snapshot table entry: > > variable: Padding to round up the snapshot table entry size to the > next multiple of 8. > + > + > +== Dirty bitmaps == > + > +The feature supports storing dirty bitmaps in a qcow2 image. > + > +=== Cluster mapping === > + > +Dirty bitmaps are stored using a ONE-level structure for the mapping of > +bitmaps to host clusters. It is called Dirty Bitmap Table. > + > +The Dirty Bitmap Table has a variable size (stored in the Dirty Bitmap > +Directory Entry) and may use multiple clusters, however it must be contiguous > +in the image file. > + > +Given an offset (in bytes) into the bitmap, the offset into the image file can > +be obtained as follows: > + > + byte_offset = > + dirty_bitmap_table[offset / cluster_size] + (offset % cluster_size) > + > +Taking into accout the granularity of the bitmap, an offset in bits into the > +image file can be obtained like this: > + > + bit_offset = > + byte_offset(bit_nr / granularity / 8) * 8 + (bit_nr / granularity) % 8 > + > +Here bit_nr is a number of "virtual" bit of the bitmap, which is covered by > +"physical" bit with number (bit_nr / granularity). > + > +Dirty Bitmap Table entry: > + > + Bit 0 - 8: Reserved > + > + 9 - 55: Bits 9-55 of host cluster offset. Must be aligned to a > + cluster boundary. If the offset is 0, the cluster is > + unallocated, and should be read as all zeros. > + > + 56 - 63: Reserved > + > +=== Dirty Bitmap Directory === > + > +Each dirty bitmap, saved in the image is described in the Dirty Bitmap > +Directory entry. Dirty Bitmap Directory is a contiguous area in the image file, > +whose starting offset and length are given by the header extension fields > +dirty_bitmap_directory_offset and dirty_bitmap_directory_size. The entries of > +the bitmap directory have variable length, depending on the length of the > +bitmap name. > + > +Dirty Bitmap Directory Entry: > + > + Byte 0 - 7: dirty_bitmap_table_offset > + Offset into the image file at which the Dirty Bitmap Table > + for the bitmap starts. Must be aligned to a cluster > + boundary. > + > + 8 - 15: nb_virtual_bits > + Number of "virtual" bits in the bitmap. Number of > + "physical" bits would be: > + (nb_virtual_bits + granularity - 1) / granularity > + > + 16 - 19: dirty_bitmap_table_size > + Number of entries in the Dirty Bitmap Table of the bitmap. > + Valid values: 0 - 0x8000000. > + Also, (dirty_bitmap_table_size * cluster_size) should not > + be greater than 0x20000000 (512 MB) > + > + 20 - 23: granularity_bits > + Granularity bits. Valid values are: 0 - 63. > + > + Granularity is calculated as > + granularity = 1 << granularity_bits > + > + Granularity of the bitmap is how many "virtual" bits > + accounts for one "physical" bit. > + > + 24 - 27: flags > + Bit > + 0: in_use > + The bitmap is in use and may be inconsistent. > + > + 1: self > + The bitmap is a dirty bitmap for the containing image. > + > + 2: auto > + The bitmap should be autoloaded as block dirty bitmap. > + Only available if bit 1 (self) is set. > + > + 3: read_only > + The bitmap should not be rewritten. > + > + Bits 4 - 31 are reserved. Is this appropriate as field, reserved for future extensiion? Or we need an additional one? Do we need scheme like with snapshots? (somthing like field 'additional_area_size', and additional offset of this size after the name) > + > + 28 - 29: name_size > + Size of the bitmap name. Valid values: 0 - 1023. > + > + variable: The name of the bitmap (not null terminated). > + > + variable: Padding to round up the Dirty Bitmap Directory Entry size to > + the next multiple of 8. -- Best regards, Vladimir * now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.