From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:57564) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Tiox3-0000sE-6F for qemu-devel@nongnu.org; Wed, 12 Dec 2012 11:14:35 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Tiowv-0000sh-2x for qemu-devel@nongnu.org; Wed, 12 Dec 2012 11:14:29 -0500 Received: from nodalink.pck.nerim.net ([62.212.105.220]:51656 helo=paradis.irqsave.net) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Tiowu-0000r6-Lk for qemu-devel@nongnu.org; Wed, 12 Dec 2012 11:14:21 -0500 Date: Wed, 12 Dec 2012 17:14:28 +0100 From: =?iso-8859-1?Q?Beno=EEt?= Canet Message-ID: <20121212161428.GC3478@irqsave.net> References: <1353935123-24199-1-git-send-email-benoit@irqsave.net> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <1353935123-24199-1-git-send-email-benoit@irqsave.net> Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [RFC V3 00/24] QCOW2 deduplication List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: kwolf@redhat.com, stefanha@redhat.com Hi Stefan, I have a few questions 1) overlapping sequential sub-cluster writes The current code pass most of the tests and behave well with a 4KB cluste= r sized ext3 volume on the deduplicated image. But less than cluster size sequentials writes are troublesome. They fail with xfstest. The problem is that the lock is released twice so that coherency is not garanteed when two sub cluster size write are done on the same area. (a deduplication attempt is done while the first write is yet not on disk= ) My understanding is that a wait_for_overlapping_cluster_write function ca= lled before the writev loop in order to serialize such writes would solve the = problem. What do you this of this idea ? 2) Internal snapshot I don't fully understand if the current deduplication implementation is compatible with internal snapshots. If not could it be done on a latter patchset ? Beno=EEt > Le Monday 26 Nov 2012 =E0 14:04:59 (+0100), Beno=EEt Canet a =E9crit : > This patchset is the first working version of the QCOW2 deduplication. >=20 > Images must be created with "-o dedup=3Don" in order to activate the > deduplication in the image. >=20 >=20 > Since v2: make it work barely > replace kernel red black trees by gtree. >=20 > Beno=EEt Canet (24): > qcow2: Add deduplication to the qcow2 specification. > qcow2: Add deduplication structures and fields. > qcow2: Add qcow2_dedup_read_missing_and_concatenate > qcow2: Make update_cluster_refcount public. > qcow2: Create a way to link to l2 tables in dedup. > qcow2: Add qcow2_dedup and related functions. > qcow2: Add qcow2_dedup_write_new_hashes. > qcow2: Implement qcow2_compute_cluster_hash. > qcow2: Extract qcow2_dedup_grow_table > qcow2: create function to load deduplication hashes at startup. > qcow2: Load and save deduplication table header extension. > qcow2: Extract qcow2_do_table_init. > qcow2: Add qcow2_dedup_init and qcow2_dedup_close. > qcow2: Extract qcow2_add_feature and qcow2_remove_feature. > block: Add dedup image create option. > qcow2: Allow creation of images using deduplication. > qcow2: Behave correctly when refcount reach 0 or 2^16. > qcow2: Integrate deduplication in qcow2_co_writev loop. > qcow2: Add verification of dedup table. > qcow2: Adapt checking of QCOW_OFLAG_COPIED for dedup. > qcow2: Add check_dedup_l2 in order to check l2 of dedup table. > qcow2: Do not overwrite existing entries with QCOW_OFLAG_COPIED. > qcow2: init and cleanup deduplication. > qemu-iotests: Filter dedup=3Don/off so existing tests don't break. >=20 > Makefile | 3 + > Makefile.target | 2 +- > block/Makefile.objs | 1 + > block/qcow2-cluster.c | 115 ++++-- > block/qcow2-dedup.c | 914 ++++++++++++++++++++++++++++++++++= ++++++++ > block/qcow2-refcount.c | 154 +++++-- > block/qcow2.c | 267 ++++++++++-- > block/qcow2.h | 89 +++- > block_int.h | 1 + > docs/specs/qcow2.txt | 33 +- > tests/qemu-iotests/common.rc | 3 +- > 11 files changed, 1480 insertions(+), 102 deletions(-) > create mode 100644 block/qcow2-dedup.c >=20 > --=20 > 1.7.10.4 >=20