Re: [Qemu-devel] [RFC] Propose the Fast Virtual Disk (FVD) image format that outperforms QCOW2 by 249%

[Anthony Liguori <anthony@codemonkey.ws>]

Hi Chunqiang,

On 01/04/2011 03:44 PM, Chunqiang Tang wrote:
> Dear QEMU Community Members,
>
> Happy new year! We would like to contribute a new year gift to the
> community.
>
> As the community considers the next-generation image formats for QEMU,
> hopefully we really challenge ourselves hard enough to find the right
> solution for the long term, rather than just a convenient solution for the
> short term, because an image format has long-term impacts and is hard to
> change once released.  In this spirit, we would like to argue that QCOW2
> and QED’s use of a two-level lookup table as the basis for implementing
> all features is a fundamental obstacle for achieving high performance.
> Accordingly, we advocate the newly developed Fast Virtual Disk (FVD) image
> format for adoption in the QEMU mainline. FVD achieves the performance of
> a RAW image running on a raw partition, while providing the rich features
> of compact image, copy-on-write, copy-on-read, and adaptive prefetching.
> FVD is extensible and can accommodate additional features. Experiments
> show that the throughput of FVD is 249% higher than that of QCOW2 when
> using the PostMark benchmark to create files.
>
> FVD came out of the work done at IBM T.J. Watson Research Center, when
> studying virtual disk related issues during the development of the IBM
> Cloud (http://www.ibm.com/services/us/igs/cloud-development/). At IBM
> internally, FVD (a.k.a. ODS) has been widely demonstrated since June 2010.
> Recently, the FVD technical papers were completed and the source code was
> cleared for external release. Now we finally can share FVD with the
> community, and seek your valuable feedback and contributions. All related
> information is available at
> https://researcher.ibm.com/researcher/view_project.php?id=1852 , including
> a high-level overview of FVD, the source code, and the technical papers.
>
> The FVD patch also includes a fully automated testing framework that
> exercises QEMU block device drivers under stress load and extreme race
> conditions. Currently (as of January 2011), QCOW2 cannot pass the
> automated test. The symptom is that QCOW2 attempts to read beyond the end
> of the base image. QCOW2 experts please take a look at this "potential"
> bug.
>    

For any feature to be seriously considered for inclusion in QEMU, 
patches need to be posted to the mailing list against the latest git 
tree.  That's a pre-requisite for any real discussion.

There's a tremendous amount of desire to avoid further fragmentation of 
image formats.  Based on my limited understanding, I think FVD shares a 
lot in common with the COW format (block/cow.c).

But I think most of the advantages you mention could be considered as 
additions to either qcow2 or qed.  At any rate, the right way to have 
that discussion is in the form of patches on the ML.

Regards,

Anthony Liguori

> Best Regards,
> Chunqiang Tang
>
> Homepage: http://www.research.ibm.com/people/c/ctang
>
>