From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeff Garzik Subject: Re: [PATCH v1] CLD replication (WIP) Date: Fri, 31 Jul 2009 14:38:30 -0400 Message-ID: <4A733A26.2080801@garzik.org> References: <20090731104031.GA21249@havoc.gtf.org> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: Sender: hail-devel-owner@vger.kernel.org List-ID: Content-Type: text/plain; charset="us-ascii"; format="flowed" To: Sage Weil Cc: hail-devel@vger.kernel.org Sage Weil wrote: > Hi Jeff, > > Do you still plan to replace bdb (and it's replication) with a something > based on paxos? I'm considering replacing the Ceph monitors (which > currently implement paxos, but in a very ceph-specific way) with cld if it > can meet the basic requirements. > > What I'd kind of like to see is a clean implementation of a paxos > library--one that leaves out message transport and storage--to build a > replicated write-ahead log. And then a separate library for handling the > database/namespace served up by cld (be it regular files, bdb, whatever) > that leaves replication up to paxos. It looks like Google ended up doing > something similar with Chubby (see > http://labs.google.com/papers/paxos_made_live.html). > > Does this sound like the direction you guys are heading in? You mean something like http://linux.yyz.us/misc/paxreg.c ? :) Yes, a straight PAXOS implementation is definitely in the plans, for similar reasons as Google describes: CLD just doesn't need full db4 replicated transactions, when a PAXOS replicated, write-ahead logging database would work just fine. That said, the urgency of this task is lowered, because current db4 (after much trial, tribulation and debugging) did eventually start supporting PAXOS. Some of the Google hints paper hint at this, noting that Sleepycat/Oracle "eventually" fixed all the problems -- but by then it was too late, Google had moved on and done their own db. Most importantly, from the view of a CLD client (libcldc user), CLD will provide the necessary guarantees today. When CLD switches to native PAXOS, the CLD client API will not change at all. So, the switchover should be transparent from the client's point of view. Jeff