From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:56359) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UXmKA-0006T5-Tr for qemu-devel@nongnu.org; Thu, 02 May 2013 01:45:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UXmK9-0007dA-Lz for qemu-devel@nongnu.org; Thu, 02 May 2013 01:44:58 -0400 Received: from e38.co.us.ibm.com ([32.97.110.159]:51290) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UXmK9-0007cp-DO for qemu-devel@nongnu.org; Thu, 02 May 2013 01:44:57 -0400 Received: from /spool/local by e38.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 1 May 2013 23:44:55 -0600 Received: from d03relay03.boulder.ibm.com (d03relay03.boulder.ibm.com [9.17.195.228]) by d03dlp01.boulder.ibm.com (Postfix) with ESMTP id ED1681FF001D for ; Wed, 1 May 2013 23:39:48 -0600 (MDT) Received: from d03av06.boulder.ibm.com (d03av06.boulder.ibm.com [9.17.195.245]) by d03relay03.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r425iq3l103026 for ; Wed, 1 May 2013 23:44:53 -0600 Received: from d03av06.boulder.ibm.com (loopback [127.0.0.1]) by d03av06.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r425llCh023622 for ; Wed, 1 May 2013 23:47:47 -0600 Message-ID: <5181FD52.9080100@linux.vnet.ibm.com> Date: Thu, 02 May 2013 13:44:50 +0800 From: Dong Xu Wang MIME-Version: 1.0 References: <1365581513-3475-1-git-send-email-wdongxu@linux.vnet.ibm.com> <1365581513-3475-2-git-send-email-wdongxu@linux.vnet.ibm.com> <517B0382.3080104@redhat.com> In-Reply-To: <517B0382.3080104@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH V18 1/6] docs: document for add-cow file format List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Eric Blake Cc: kwolf@redhat.com, wdongxu@cn.ibm.com, qemu-devel@nongnu.org, stefanha@redhat.com On 2013/4/27 6:45, Eric Blake wrote: > On 04/10/2013 02:11 AM, Dong Xu Wang wrote: >> Document for add-cow format, the usage and spec of add-cow are >> introduced. >> >> Signed-off-by: Dong Xu Wang >> --- >> V17->V18: >> 1) remove version field. >> 2) header size is maximum value and cluster size value. >> 3) fix type. >> docs/specs/add-cow.txt | 165 +++++++++++++++++++++++++++++++++++++++++++++++++ >> 1 file changed, 165 insertions(+) >> create mode 100644 docs/specs/add-cow.txt >> >> diff --git a/docs/specs/add-cow.txt b/docs/specs/add-cow.txt >> new file mode 100644 >> index 0000000..151028b >> --- /dev/null >> +++ b/docs/specs/add-cow.txt >> @@ -0,0 +1,165 @@ >> +== General == > > No copyright notice? Not necessarily your fault, since many other files > in this directory suffer from the same problem. > Yep, all documents in docs/specs have no copyright, so I omit it. >> + >> +The raw file format does not support backing files or copy on write >> +feature. The add-cow image format makes it possible to use backing >> +files with a image by keeping a separate .add-cow metadata file. >> +Once all clusters have been written into the image it is safe to >> +discard the .add-cow and backing files, then we can use the image >> +directly. >> + >> +An example usage of add-cow would look like: >> +(ubuntu.img is a disk image which has an installed OS.) >> + 1) Create a image, such as raw format, with the same size of >> + ubuntu.img: >> + qemu-img create -f raw test.raw 8G >> + 2) Create an add-cow image which will store dirty bitmap >> + qemu-img create -f add-cow test.add-cow \ >> + -o backing_file=ubuntu.img,image_file=test.raw >> + 3) Run qemu with add-cow image >> + qemu -drive if=virtio,file=test.add-cow >> + >> +test.raw may be larger than ubuntu.img, in that case, the size of >> +test.add-cow will be calculated from the size of test.raw. >> + >> +image_fmt can be omitted, in that case image_fmt is assumed to be >> +"raw". backing_fmt can also be omitted, add-cow should do a probe >> +operation and determine what the backing file's format is. > > In general, probing a raw file is a security hole (we just plugged a CVE > with NBD probing); you probably ought to mention that it is recommended > to always specify the format for any raw file, so that probing doesn't > misinterpret the contents of the file as some other format. > Okay, will mention. >> + >> +=Specification= >> + >> +The file format looks like this: >> + >> + +---------------+-------------------------------+ >> + | Header | COW bitmap | >> + +---------------+-------------------------------+ >> + >> +All numbers in add-cow are stored in Little Endian byte order. >> + >> +== Header == >> + >> +The Header is included in the first bytes: >> +(HEADER_SIZE is defined in 40-43 bytes.) >> + Byte 0 - 3: magic >> + add-cow magic string ("ACOW"). > > Probably ought to mention that this magic string is in the ASCII > encoding (those characters map to different bytes on EBCDIC, although I > doubt qemu will ever really been ported to EBCDIC) > Okay. >> + >> + 4 - 7: backing file name offset >> + Offset in the add-cow file at which the backing >> + file name is stored (NB: The string is not >> + lNUL-terminated). > > s/lNUL/NUL/ > Okay. >> + If backing file name does NOT exist, this field >> + will be 0. Must be between 76 and [HEADER_SIZE >> + - 2](a file name must be at least 1 byte). >> + > >> + >> + 40 - 43: HEADER_SIZE >> + The header field is variable-sized. This field >> + indicates how many bytes will be used to store >> + add-cow header. By default, it is maximum value >> + of 4096 and cluster size value. > > Should it be required to be a multiple of 4096, for efficient alignment > of clusters? > I think I can make cluster size be multiple of 4096.. >> + >> + 44 - 59: backing file format >> + Format of backing file. It will be filled with >> + 0 if backing file name offset is 0. If backing >> + file name offset is non-empty, it must be >> + non-empty. It is coded in free-form ASCII, and >> + is not NUL-terminated. Zero padded on the right. > > Requiring this to be non-empty if a backing file is named contradicts > your earlier statement that backing format is probed (I actually like > mandating the backing format, though). I think logic can be: 1) if "-o backing_fmt = raw" then do not probe. 2) else even if "-o backing_fmt = $fmt" is used, also perform a probe operation and write backing_fmt in add_cow headers. > >> + >> + 60 - 75: image file format >> + Format of image file. It must be non-empty. It >> + is coded in free-form ASCII, and is not >> + NUL-terminated. Zero padded on the right. > > Again, requiring a format contradicts the earlier statement about format > probing. > > How does this compare with Paolo's efforts to design a persistent bitmap > for drive-mirror/block-backup use? >