From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48405) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UugKK-0007dv-MQ for qemu-devel@nongnu.org; Thu, 04 Jul 2013 05:59:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UugKI-0004fp-Iu for qemu-devel@nongnu.org; Thu, 04 Jul 2013 05:59:48 -0400 Received: from nodalink.pck.nerim.net ([62.212.105.220]:39553 helo=paradis.irqsave.net) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UugKI-0004fB-7X for qemu-devel@nongnu.org; Thu, 04 Jul 2013 05:59:46 -0400 Date: Thu, 4 Jul 2013 12:01:28 +0200 From: =?iso-8859-1?Q?Beno=EEt?= Canet Message-ID: <20130704100128.GA5429@irqsave.net> References: <1371738392-9594-1-git-send-email-benoit@irqsave.net> <1371738392-9594-2-git-send-email-benoit@irqsave.net> <20130702144224.GF9870@stefanha-thinkpad.redhat.com> <20130702212355.GB4985@irqsave.net> <20130703081235.GD16585@stefanha-thinkpad.muc.redhat.com> <20130703125327.GC5482@irqsave.net> <20130704071300.GA2584@stefanha-thinkpad.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <20130704071300.GA2584@stefanha-thinkpad.redhat.com> Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [RFC V8 01/24] qcow2: Add journal specification. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: =?iso-8859-1?Q?Beno=EEt?= Canet , kwolf@redhat.com, qemu-devel@nongnu.org > Simple is good. Even for deduplication alone, I think data integrity i= s > critical - otherwise we risk stale dedup metadata pointing to clusters > that are unallocated or do not contain the right data. So the journal > will probably need to follow techniques for commits/checksums. I agree that checksums are missing for the dedup. Maybe we could even use some kind of error correcting code instead of a c= hecksum. Concerning data integrity the events that the deduplication code cannot l= oose are hash deletions because they mark a previously inserted hash as obsole= te. The problem with a commit/flush mechanism on hash deletion is that it wil= l slow down the store insertion speed and also create some extra SSD wear out. To solve this I considered the fact that the dedup metadata as a whole is disposable. So I implemented a "dedup dirty" bit. When QEMU stop the journal is flushed and the dirty bit is cleared. When QEMU start and the dirty bit is set a crash is detected and _all_ th= e deduplication metadata is dropped. The QCOW2 data integrity won't suffer only the dedup ratio will be lower. As you said once on irc crashes don't happen often. Beno=EEt