From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51969) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z2bhf-0008NK-5I for qemu-devel@nongnu.org; Wed, 10 Jun 2015 04:49:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z2bhb-0006Zm-Uw for qemu-devel@nongnu.org; Wed, 10 Jun 2015 04:49:43 -0400 Received: from relay.parallels.com ([195.214.232.42]:44282) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z2bhb-0006Zb-HR for qemu-devel@nongnu.org; Wed, 10 Jun 2015 04:49:39 -0400 Message-ID: <5577FA11.3080009@virtuozzo.com> Date: Wed, 10 Jun 2015 11:49:21 +0300 From: Vladimir Sementsov-Ogievskiy MIME-Version: 1.0 References: <1433776886-27239-1-git-send-email-vsementsov@virtuozzo.com> <1433776886-27239-2-git-send-email-vsementsov@virtuozzo.com> <20150609170325.GI3181@stefanha-thinkpad.redhat.com> <5577F312.9040502@virtuozzo.com> In-Reply-To: <5577F312.9040502@virtuozzo.com> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH 1/8] spec: add qcow2-dirty-bitmaps specification List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: kwolf@redhat.com, qemu-devel@nongnu.org, Vladimir Sementsov-Ogievskiy , stefanha@redhat.com, den@openvz.org, pbonzini@redhat.com, jsnow@redhat.com On 10.06.2015 11:19, Vladimir Sementsov-Ogievskiy wrote: > On 09.06.2015 20:03, Stefan Hajnoczi wrote: >> On Mon, Jun 08, 2015 at 06:21:19PM +0300, Vladimir >> Sementsov-Ogievskiy wrote: >>> From: Vladimir Sementsov-Ogievskiy >>> >>> Persistent dirty bitmaps will be saved into qcow2 files. It may be used >>> as 'internal' bitmaps (for qcow2 drives) or as 'external' bitmaps for >>> other drives (there may be qcow2 file with zero disk size but with >>> several dirty bitmaps for other drives). >>> >>> Signed-off-by: Vladimir Sementsov-Ogievskiy >>> --- >>> docs/specs/qcow2.txt | 66 >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> 1 file changed, 66 insertions(+) >>> >>> diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt >>> index 121dfc8..0fffba2 100644 >>> --- a/docs/specs/qcow2.txt >>> +++ b/docs/specs/qcow2.txt >>> @@ -123,6 +123,7 @@ be stored. Each extension has a structure like >>> the following: >>> 0x00000000 - End of the header extension area >>> 0xE2792ACA - Backing file format name >>> 0x6803f857 - Feature name table >>> + 0x23852875 - Dirty bitmaps >>> other - Unknown header extension, can >>> be safely >>> ignored >>> @@ -166,6 +167,19 @@ the header extension data. Each entry look >>> like this: >>> terminated if it has full length) >>> +== Dirty bitmaps == >>> + >>> +Dirty bitmaps is an optional header extension. It provides a >>> possibility of >>> +storing dirty bitmaps in qcow2 image. The fields are: >>> + >>> + 0 - 3: nb_dirty_bitmaps >>> + Number of dirty bitmaps contained in the image >> Is there a maximum? > hmm. any proposals for this? >> >>> + >>> + 4 - 11: dirty_bitmaps_offset >>> + Offset into the image file at which the dirty >>> bitmaps table >>> + starts. Must be aligned to a cluster boundary. >> The autoclear feature bit is undocumented. >> >>> == Host cluster management == >>> qcow2 manages the allocation of host clusters by maintaining a >>> reference count >>> @@ -360,3 +374,55 @@ Snapshot table entry: >>> variable: Padding to round up the snapshot table entry >>> size to the >>> next multiple of 8. >>> + >>> + >>> +== Dirty bitmaps == >>> + >>> +The feature supports storing several dirty bitmaps in the qcow2 file. >>> + >>> +=== Cluster mapping === >>> + >>> +Dirty bitmaps are stored using a ONE-level structure for the >>> mapping of >>> +bitmaps to host clusters. There is only an L1 table. >>> + >>> +The L1 table has a variable size (stored in the Bitmap table entry) >>> and may >>> +use multiple clusters, however it must be contiguous in the image >>> file. >> The use of "L1 table" could be confusing. The refcount metadata uses >> "refcount table" and "refcount block" to describe a one-level table. > I agree. Hmm.. dirty bitmaps table? ok? oh, no, bad idea. dirty bitmaps table is other thing)) >> >>> + >>> +Given an offset into the bitmap, the offset into the image file can be >>> +obtained as follows: >>> + >>> + offset = l1_table[offset / cluster_size] + (offset % cluster_size) >> It might help to add granularity to this formula. >> >> Instead of "offset", "bit_number" or "bitnr" might be clearer since >> "offset" means something different in other parts of the document. > Hmm. In my opinion, the bitmap here is stored as raw data. And > granularity is an additional parameter (for deserializing this data). > So, it is an offset in bytes for this data. The format is not for > accessing bitmap bits, it's only for loading the whole bitmap one time. >> >>> + >>> +L1 table entry: >>> + >>> + Bit 0 - 61: Standard cluster descriptor >>> + >>> + 62 - 63: Reserved >> Do you really want to use the standard cluster descriptor with it's zero >> bit? >> >> Since bitmaps don't honor backing files there doesn't seem much point in >> using the zero bit, things are simpler if just bits 9-55 are contain the >> host cluster offset and 0 means the cluster is unallocated. >> >> By honoring the zero bit there are three states: >> 1. Zero bit set, read zeroes >> 2. Zero bit not set, host cluster offset != 0, bits valid >> 3. Zero bit not set, host cluster offset == 0, unallocated >> >> State 1 is not useful. >> >>> +=== Bitmap table === >>> + >>> +A directory of all bitmaps is stored in the bitmap table, a >>> contiguous area in >>> +the image file, whose starting offset and length are given by the >>> header fields >>> +dirty_bitmaps_offset and nb_dirty_bitmaps. The entries of the >>> bitmap table have >>> +variable length, depending on the length of name and extra data. >>> + >>> +Bitmap table entry: >>> + >>> + Byte 0 - 7: Offset into the image file at which the L1 >>> table for the >>> + bitmap starts. Must be aligned to a cluster >>> boundary. >>> + >>> + 8 - 11: Number of entries in the L1 table of the bitmap >>> + >>> + 12 - 15: Bitmap granularity in bytes >>> + >>> + 16 - 23: Bitmap size in sectors >>> + >>> + 24 - 25: Size of the bitmap name >>> + >>> + variable: The name of the bitmap (not null terminated) >>> + >>> + variable: Padding to round up the bitmap table entry size >>> to the >>> + next multiple of 8. >>> + >>> +The fields "size", "granularity" and "name" are corresponding with >>> the fields >>> +in struct BdrvDirtyBitmap. >> Referring to the internals of a C struct in QEMU is not appropriate for >> a file format specification. Please document the fields fully including >> their constraints, minimums, maximums, etc. > > -- Best regards, Vladimir * now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.