From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59306) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UpfnW-0002pc-5K for qemu-devel@nongnu.org; Thu, 20 Jun 2013 10:25:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UpfnP-0000IN-Ao for qemu-devel@nongnu.org; Thu, 20 Jun 2013 10:25:14 -0400 Received: from nodalink.pck.nerim.net ([62.212.105.220]:37511 helo=paradis.irqsave.net) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UpfnP-0000E6-1E for qemu-devel@nongnu.org; Thu, 20 Jun 2013 10:25:07 -0400 From: =?UTF-8?q?Beno=C3=AEt=20Canet?= Date: Thu, 20 Jun 2013 16:26:08 +0200 Message-Id: <1371738392-9594-1-git-send-email-benoit@irqsave.net> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: [Qemu-devel] [RFC V8 00/24] QCOW2 deduplication core functionality List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: kwolf@redhat.com, =?UTF-8?q?Beno=C3=AEt=20Canet?= , stefanha@redhat.com Hello, This is the new patchset of the QCOW2 deduplication feature using an on d= isk key value store modelled after the SILT one. This is more a millestone reached post than a for review post. Performance is slow because the code is issuing a lot of read and write t= o the SSD in a sequential ways. The next revision will focus on parallelising the IOs. Kevin: the first patch add the description of the QCOW2 journal structure= . The code make it easy to have multiple journal in QCOW2. Do you think it could fit others QCOW2 usages. Patches describing the others data structure are comming. Best regards Beno=C3=AEt Beno=C3=AEt Canet (24): qcow2: Add journal specification. qcow2: Add deduplication structures and fields. qcow2: Add journal. qcow2: Create the log store. qcow2: Add the hash store. qcow2: Add the deduplication store. qcow2: Add qcow2_dedup_read_missing_and_concatenate qcow2: Create a way to link to l2 tables when deduplicating. qcow2: Make qcow2_update_cluster_refcount public. qcow2: Add qcow2_dedup and related functions qcow2: Add qcow2_dedup_store_new_hashes. qcow2: Do allocate on rewrite on the dedup case. qcow2: Implement qcow2_compute_cluster_hash. qcow2: Load and save deduplication table header extension. qcow2: Extract qcow2_set_incompat_feature and qcow2_clear_incompat_feature. block: Add qcow2_dedup format and image creation code. qcow2: Drop hash for a given cluster when dedup makes refcount > 2^16/2. qcow2: Remove hash when cluster is deleted. qcow2: Integrate deduplication in qcow2_co_writev loop. qcow2: Serialize write requests when deduplication is activated. qcow2: Integrate SKEIN hash algorithm in deduplication. qcow2: Add qcow2_dedup_init and qcow2_dedup_close. qcow2: Enable the deduplication feature. qcow2: Enable deduplication tests block/Makefile.objs | 1 + block/qcow2-cluster.c | 13 +- block/qcow2-dedup.c | 778 +++++++++++++++++++++++++ block/qcow2-hash-store.c | 802 ++++++++++++++++++++++++++ block/qcow2-journal.c | 587 +++++++++++++++++++ block/qcow2-log-store.c | 1281 ++++++++++++++++++++++++++++++++++++= ++++++ block/qcow2-refcount.c | 20 +- block/qcow2-store.c | 771 +++++++++++++++++++++++++ block/qcow2.c | 431 +++++++++++++- block/qcow2.h | 357 +++++++++++- configure | 121 +++- docs/specs/qcow2.txt | 42 ++ include/block/block_int.h | 1 + tests/qemu-iotests/common | 6 + tests/qemu-iotests/common.rc | 3 +- 15 files changed, 5144 insertions(+), 70 deletions(-) create mode 100644 block/qcow2-dedup.c create mode 100644 block/qcow2-hash-store.c create mode 100644 block/qcow2-journal.c create mode 100644 block/qcow2-log-store.c create mode 100644 block/qcow2-store.c --=20 1.7.10.4