Re: [PATCH v1] CLD replication (WIP)

[Jeff Garzik <jeff@garzik.org>]

Sage Weil wrote:
> Hi Jeff,
> 
> Do you still plan to replace bdb (and it's replication) with a something 
> based on paxos?  I'm considering replacing the Ceph monitors (which 
> currently implement paxos, but in a very ceph-specific way) with cld if it 
> can meet the basic requirements.
> 
> What I'd kind of like to see is a clean implementation of a paxos 
> library--one that leaves out message transport and storage--to build a 
> replicated write-ahead log.  And then a separate library for handling the 
> database/namespace served up by cld (be it regular files, bdb, whatever) 
> that leaves replication up to paxos.  It looks like Google ended up doing 
> something similar with Chubby (see 
> http://labs.google.com/papers/paxos_made_live.html).
> 
> Does this sound like the direction you guys are heading in?

You mean something like http://linux.yyz.us/misc/paxreg.c  ?   :)

Yes, a straight PAXOS implementation is definitely in the plans, for 
similar reasons as Google describes:  CLD just doesn't need full db4 
replicated transactions, when a PAXOS replicated, write-ahead logging 
database would work just fine.

That said, the urgency of this task is lowered, because current db4 
(after much trial, tribulation and debugging) did eventually start 
supporting PAXOS.  Some of the Google hints paper hint at this, noting 
that Sleepycat/Oracle "eventually" fixed all the problems -- but by then 
it was too late, Google had moved on and done their own db.

Most importantly, from the view of a CLD client (libcldc user), CLD will 
provide the necessary guarantees today.  When CLD switches to native 
PAXOS, the CLD client API will not change at all.  So, the switchover 
should be transparent from the client's point of view.

	Jeff