From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47215) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XSCXa-0003KS-Ad for qemu-devel@nongnu.org; Thu, 11 Sep 2014 18:08:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XSCXR-0003lK-90 for qemu-devel@nongnu.org; Thu, 11 Sep 2014 18:08:34 -0400 Received: from mail-wi0-x22e.google.com ([2a00:1450:400c:c05::22e]:57183) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XSCXQ-0003k3-UU for qemu-devel@nongnu.org; Thu, 11 Sep 2014 18:08:25 -0400 Received: by mail-wi0-f174.google.com with SMTP id n3so1752642wiv.13 for ; Thu, 11 Sep 2014 15:08:23 -0700 (PDT) Message-ID: <54121D54.2040903@gmail.com> Date: Fri, 12 Sep 2014 00:08:20 +0200 From: Walid Nouri MIME-Version: 1.0 References: <53DBE726.4050102@gmail.com> <1406947532.2680.11.camel@usa> <53E0AA60.9030404@gmail.com> <1407376929.21497.2.camel@usa> <53E60F34.1070607@gmail.com> <1407587152.24027.5.camel@usa> <53E8FBBD.7050703@gmail.com> <53E92470.60806@linux.vnet.ibm.com> <53F07B73.60407@redhat.com> <54107187.8040706@gmail.com> <20140911174407.GP2353@work-vm> In-Reply-To: <20140911174407.GP2353@work-vm> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Dr. David Alan Gilbert" Cc: kwolf@redhat.com, eddie.dong@intel.com, qemu-devel@nongnu.org, "Michael R. Hines" , stefanha@redhat.com, Paolo Bonzini , yanghy@cn.fujitsu.com Am 11.09.2014 19:44, schrieb Dr. David Alan Gilbert: >> For keeping the complete system state consistent on the secondary system >> there must be a possibility for MC to commit/discard block device state >> changes. In normal operation the mirrored block device state changes (block >> buffer) are committed to disk when the complete checkpoint is committed. In >> case of a crash of the primary system while transferring a checkpoint the >> data in the block buffer corresponding to the failed Checkpoint must be >> discarded. > > I think for COLO there's a requirement that the secondary can do reads/writes > in parallel with the primary, and the secondary can discard those reads/writes > - and that doesn't happen in MC (Yang or Eddie should be able to confirm that). > >> The storage architecture should be ???shared nothing??? so that no shared >> storage is required and primary/secondary can have separate block device >> images. I admit that my formulation was unintentionally a bit ambiguous :) I should have written that a shared storage should not be mandatory. I'm comming from an SMB environment and (redundant) shared storage systems are still not usual in small companies :) I looked for a storage agnostic approach which allows the number of system components to be as low as possible and still get redundancy and fault tolerance. > > MC/COLO with shared storage still needs some stuff like this; but it's subtely > different. They still need to be able to buffer/release modifications > to the shared storage; if any of this code can also be used in the > shared-storage configurations it would be good. The proposed approach with block filter and the commit/discard protocol should be storage agnostic and will also work in a shared storage environment, but only with distinct images (because of the protocol). In case of a shared storage and a common image used by the primary and secondary another storage protocol must be used. It's not commit/discard but commit/rollback The primary still sends asynchronously the block state changes. The secondary buffers block device state changes but doesn't apply them in normal operation. When the next checkpoint is complete the secondary clears the buffer and forgets about the old block state data. If the primary fails the secondary must rollback the common image with the block state data corresponding to the actual checkpoint. Otherwise the state of the image and rest of the system state on the secondary will not be in sync. When there is no block state data corresponding to the actual checkpoint, then there is nothing to do on the storage for the secondary :) There is a little danger in this though. When the secondary fails during rollback, the common image will be left in an inconsistent state. I think this risk cannot be avoided when using a common image. But this unfortunate situation can also happen in other scenarios. Sharing a common immage with this protocol will lead to a longer fail over time in case of existing block device state data for the actual checkpoint. The secondary must initiate the rollback and wait until all blocks of the actual checkpoint are commited to the common immage before taking over the active role. Walid