From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38713) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xrc1j-0001rS-G7 for qemu-devel@nongnu.org; Thu, 20 Nov 2014 19:24:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Xrc1e-0007p6-2h for qemu-devel@nongnu.org; Thu, 20 Nov 2014 19:24:43 -0500 Received: from mx1.redhat.com ([209.132.183.28]:51204) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xrc1d-0007p2-Qz for qemu-devel@nongnu.org; Thu, 20 Nov 2014 19:24:38 -0500 Message-ID: <546E8643.6080904@redhat.com> Date: Thu, 20 Nov 2014 17:24:35 -0700 From: Eric Blake MIME-Version: 1.0 References: <546A88DD.10006@redhat.com> <1416479664-3414-1-git-send-email-vsementsov@parallels.com> In-Reply-To: <1416479664-3414-1-git-send-email-vsementsov@parallels.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="5DBoWlQhUS15uK1bu9c4kMi978eh43Rhp" Subject: Re: [Qemu-devel] [PATCH v2] persistent dirty bitmap: add QDB file spec. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, jsnow@redhat.com Cc: famz@redhat.com, den@parallels.com, stefanha@redhat.com This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --5DBoWlQhUS15uK1bu9c4kMi978eh43Rhp Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 11/20/2014 03:34 AM, Vladimir Sementsov-Ogievskiy wrote: > QDB file is for storing dirty bitmap. The specification is based on > qcow2 specification. >=20 > Saving several bitmaps is necessary when server shutdowns during > backup. In this case 2 tables for each disk are available. One > collected for a previous period and one active. Though this feature > is discussable. >=20 > Big endian format and Standard Cluster Descriptor are used to simplify > integration with qcow2, to support internal bitmaps for qcow2 in future= =2E >=20 > The idea is that the same procedure writing the data to QDB file could > do the same for QCOW2. The only difference is cluster refcount table. > Should we use it here or not is still questionable. >=20 > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > docs/specs/qdb.txt | 132 +++++++++++++++++++++++++++++++++++++++++++++= ++++++++ > 1 file changed, 132 insertions(+) > create mode 100644 docs/specs/qdb.txt No comment on whether the approach itself makes sense - just a high-level review of this document in isolation. >=20 > diff --git a/docs/specs/qdb.txt b/docs/specs/qdb.txt > new file mode 100644 > index 0000000..d570a69 > --- /dev/null > +++ b/docs/specs/qdb.txt > @@ -0,0 +1,132 @@ > +=3D=3D General =3D=3D Missing a copyright notice. Yeah, you've got a lot of bad examples in this directory (in docs/* in general), but there ARE a few of the newer files that are starting to buck the trend and use a copyright/license blu= rb. > + > +"QDB" means "Qemu Dirty Bitmaps". QDB file can store several dirty bit= maps. > +QDB file is organized in units of constant size, which are called clus= ters. > + > +All numbers in QDB are stored in Big Endian byte order. > + > +=3D=3D Header =3D=3D > + > +The first cluster of a QDB image contains the file header: > + > + Byte 0 - 3: magic > + QDB magic string ("QDB\0") > + > + 4 - 7: version > + Version number (valid value is 1) > + > + 8 - 11: cluster_bits > + Number of bits that are used for addressing an off= set > + within a cluster (1 << cluster_bits is the cluster= size). > + Must not be less than 9 (i.e. 512 byte clusters). Is there a maximum? > + > + 12 - 15: nb_bitmaps > + Number of bitmaps contained in the file > + > + 16 - 23: bitmaps_offset > + Offset into the QDB file at which the bitmap table= starts. > + Must be aligned to a cluster boundary. > + > + 24 - 27: header_length > + Length of the header structure in bytes. does that include the length of all extensions? Should we enforce a maximum header length of one cluster? > + > +Like in qcow2, directly after the image header, optional sections call= ed header extensions can > +be stored. Each extension has a structure like the following: > + > + Byte 0 - 3: Header extension type: > + 0x00000000 - End of the header extension area > + other - Unknown header extension, can be = safely > + ignored > + > + 4 - 7: Length of the header extension data > + > + 8 - n: Header extension data > + > + n - m: Padding to round up the header extension size to t= he next > + multiple of 8. > + > +Unless stated otherwise, each header extension type shall appear at mo= st once > +in the same image. I like how qcow2 v3 has a header extension for listing the name of each header extension, for nicer error messages. Also, I think that declaring all unknown extensions as ignorable may be dangerous, since you lack a capability bitmask. Maybe it would be wise to copy the qcow2 v3 capabilities (including flags for ignorable vs. mandatory support of given features, where a client can sanely decide what to do if it does not recognize a feature). > + > + 26 - 27: Size of the bitmap name > + > + 36 - 39: Size of extra data in the table entry (used for fu= ture > + extensions of the format) > + > + variable: Extra data for future extensions. Unknown fields m= ust be > + ignored. This block is width 0 if bytes 36-39 is 0? How are extensions identified? Are they required to be done like overall file headers, with an id, length, and then variable data, so that it is possible to scan to the end of each unknown extension to see if the next extension is known? This is where capability bits in the overall header may make more sense. > + > + variable: Name of the bitmap (not null terminated) The length of this block is determined by bytes 26-27? > + > + variable: Padding to round up the bitmap table entry size to= the > + next multiple of 8. > + > +The fields "size", "granularity", "enabled" and "name" are correspondi= ng with > +the fields in struct BdrvDirtyBitmap. --=20 Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --5DBoWlQhUS15uK1bu9c4kMi978eh43Rhp Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg iQEcBAEBCAAGBQJUboZDAAoJEKeha0olJ0NqbrMIAIbi/ZzxtI0oV4Zc0FFPvhbT IUynzRLg3bI8sPgUKkz8ZRd+KRcvI8HADf+SCM23CsrljQ2sIIQ4s6ddyL/1HIKy DrZ3CugMm/jahP0k7pwk5/ZG2nAiipREgu5r2J2gvhl4xj3iFEdlLmEBFUCoBKOp Un5GEf/umJbF1Z3Y4VB2+GsDIIFzZ6Zz49tab3rqqMJ9qCp/8aSRiKY9zh9XPzqg JWs3K60scdghOs+wc5mHa5qwjr8Ifsmsx8Tl9cGkwZEDPi68wYTGWwOlmOkGcZdl euoWGMlRNABM0KfmxYwmOPdyku+2+0LeQCAhYc6TVPGJ75uWA+HsOdEZ0a06jLg= =3OaF -----END PGP SIGNATURE----- --5DBoWlQhUS15uK1bu9c4kMi978eh43Rhp--