From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48527) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XHdLt-00047I-1v for qemu-devel@nongnu.org; Wed, 13 Aug 2014 14:32:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XHdLl-0001zx-Di for qemu-devel@nongnu.org; Wed, 13 Aug 2014 14:32:48 -0400 Received: from mx1.redhat.com ([209.132.183.28]:18544) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XHdLl-0001zk-66 for qemu-devel@nongnu.org; Wed, 13 Aug 2014 14:32:41 -0400 Date: Wed, 13 Aug 2014 20:32:36 +0200 From: Kevin Wolf Message-ID: <20140813183236.GA2469@noname.redhat.com> References: <20140813155415.GE3701@noname.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] disk image: self-organized format or raw file List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Xingbo Wu Cc: qemu-devel@nongnu.org Am 13.08.2014 um 18:38 hat Xingbo Wu geschrieben: > On Wed, Aug 13, 2014 at 11:54 AM, Kevin Wolf wrote: > > Am 12.08.2014 um 01:38 hat =E5=90=B4=E5=85=B4=E5=8D=9A geschrieben: > >> Hello, > >> > >> The introduction in the wiki page present several advantages of qc= ow2 [1]. > >> But I'm a little confused. I really appreciate if any one can give m= e some help > >> on this :). > >> > >> (1) Currently the raw format doesn't support COW. In other words, a= raw image > >> cannot have a backing file. COW depends on the mapping table on whic= h we it > >> knows whether each block/cluster is present (has been modified) in t= he current > >> image file. Modern file-systems like xfs/ext4/etc. provide extent/bl= ock > >> allocation information to user-level. Like what 'filefrag' does with= ioctl > >> 'FIBMAP' and 'FIEMAP'. I guess the raw file driver (maybe block/raw-= posix.c) > >> may obtain correct 'present information about blocks. However this i= nformation > >> may be limited to be aligned with file allocation unit size. Maybe i= t's just > >> because a raw file has no space to store the "backing file name"? I = don't think > >> this could hinder the useful feature. > >> > >> (2) As most popular filesystems support delay-allocation/on-demand = allocation/ > >> holes, whatever, a raw image is also thin provisioned as other forma= ts. It > >> doesn't consume much disk space by storing useless zeros. However, I= don't know > >> if there is any concern on whether fragmented extents would become a= burden of > >> the host filesystem. > >> > >> (3) For compression and encryption, I'm not an export on these topi= cs at all > >> but I think these features may not be vital to a image format as bot= h guest/ > >> host's filesystem can also provide similar functionality. > >> > >> (4) I don't have too much understanding on how snapshot works but I= think > >> theoretically it would be using the techniques no more than that use= d in COW > >> and backing file. > >> > >> After all these thoughts, I still found no reason to not using a 'ra= w' file > >> image (engineering efforts in Qemu should not count as we don't ask = for more > >> features from outside world). > >> I would be very sorry if my ignorance wasted your time. > > > > Even if it did work (that it's problematic is already discussed in ot= her > > subthreads) what advantage would you get from using an extended raw > > driver compared to simply using qcow2, which supports all of this tod= ay? > > > > Kevin >=20 >=20 > I read several messages from this thread: "[RFC] qed: Add QEMU > Enhanced Disk format". To my understanding, if the new format can be > acceptable to the community: > It needs to retain all the key features provided by qcow2, > especially for compression, encryption, and internal snapshot, as > mentioned in that thread. > And, needless to say, it must run faster. >=20 > Yes I agree it's at least a subset of the homework one need to do > before selling the new format to the community. So your goal is improved performance? Why do you think that a raw driver with backing file support would run much faster than qcow2? It would have to solve the same problems, like doing efficient COW. > Thanks and another question: > What's the magic that makes QED runs faster than QCOW2? During cluster allocation (which is the real critical part), QED is a lot slower than today's qcow2. And by that I mean not just a few percent, but like half the performance. After that, when accessing already allocated data, both perform similar. Mailing list discussions of four years ago don't reflect accurately how qemu works today. The main trick of QED was to introduce a dirty flag, which allowed to call fdatasync() less often because it was okay for image metadata to become inconsistent. After a crash, you have to repair the image then. qcow2 supports the same with lazy_refcounts=3Don, but it's really only useful in rare cases, mostly with cache=3Dwritethrough. > In some simple > parallel IO tests QED can run a magnitude faster than QCOW2. I saw > differences on simple/complex metadata organization, and coroutine/aio > (however "bdrv_co_"s finally call "bdrv_aio_"s via "_em". If you can > provide some insight on this I would be really appreciate. Today, everything is internally coroutine operations, so every request goes through bdrv_co_do_preadv/pwritev. The aio_* versions are just wrappers around it for callers and block drivers that prefer a callback based interface. Kevin