[Cluster-devel] New cluster mirror log implementation (RFC)

[Cluster-devel] New cluster mirror log implementation (RFC)

[Jonathan E Brassow]

I've started on the rewrite for the cluster mirror log (can be found in 
CVS head in cluster/cmirror-kernel).  The purpose is to bring it inline 
with the new CMAN/OpenAIS framework.  Since this framework is now in 
user-space, the cluster mirror log server implementation will exist in 
user-space.

I've only worked on the client side (kernel) code so far.  I plan to 
use netlink to communicate requests from the client (kernel) to the 
server (user-space).  The server will be responsible for coordinating 
cluster events, responding to client request, handling disk commits, 
and notifying the kernel of any log device related failures.

The server will listen to requests from the client (netlink) and send 
them to the rest of the cluster via OpenAIS.  The results will be 
replicated on each machine (running the server) with the server that 
has the lowest cluster id responsible for committing the data to the 
log device.

There are 17 types of client functions - most of which will need some 
communication with the server.  The input and output data varies.  I 
don't want to create unique structures to pass between the client and 
server for each type of request, so right now I plan on passing generic 
data.  The server will interpret the data based on the request type.  
The part I don't like about this plan is it requires both the client 
and server to be able to interpret the data correctly without the help 
of a header file...  I can keep them in sync and check for version 
mismatches during the log creation phase easily enough, but perhaps 
there are better ideas.

Now that there are source files in-place, I'll try to submit patches to 
this list before committing them.  When I start work on the server 
side, I'll probably do an initial commit and post patches thereafter 
(unless people would prefer I submit the opening files here first).

  brassow

For those that don't know:  Logging implementations are modular and use 
the APIs found in linux/drivers/md/dm-log.h.  Primarily, the logging 
implementations are used by device-mapper mirroring.  Right now, there 
are only single machine logging implementations in the kernel.  Red Hat 
has a cluster logging implementation for RHEL4 (cvs co -r RHEL4 
cluster/cmirror-kernel), but since the upstream cluster infrastructure 
is based on userspace CMAN/OpenAIS, that implementation will likely not 
live outside of RHEL4.

[Cluster-devel] New cluster mirror log implementation (RFC)

[Steven Dake]

On Tue, 2006-06-20 at 21:17 -0500, Jonathan E Brassow wrote:
> I've started on the rewrite for the cluster mirror log (can be found in 
> CVS head in cluster/cmirror-kernel).  The purpose is to bring it inline 
> with the new CMAN/OpenAIS framework.  Since this framework is now in 
> user-space, the cluster mirror log server implementation will exist in 
> user-space.
> 
> I've only worked on the client side (kernel) code so far.  I plan to 
> use netlink to communicate requests from the client (kernel) to the 
> server (user-space).  The server will be responsible for coordinating 
> cluster events, responding to client request, handling disk commits, 
> and notifying the kernel of any log device related failures.
> 
> The server will listen to requests from the client (netlink) and send 
> them to the rest of the cluster via OpenAIS.  The results will be 
> replicated on each machine (running the server) with the server that 
> has the lowest cluster id responsible for committing the data to the 
> log device.
> 
> There are 17 types of client functions - most of which will need some 
> communication with the server.  The input and output data varies.  I 
> don't want to create unique structures to pass between the client and 
> server for each type of request, so right now I plan on passing generic 
> data.  The server will interpret the data based on the request type.  
> The part I don't like about this plan is it requires both the client 
> and server to be able to interpret the data correctly without the help 
> of a header file...  I can keep them in sync and check for version 
> mismatches during the log creation phase easily enough, but perhaps 
> there are better ideas.
> 
> Now that there are source files in-place, I'll try to submit patches to 
> this list before committing them.  When I start work on the server 
> side, I'll probably do an initial commit and post patches thereafter 
> (unless people would prefer I submit the opening files here first).
> 
>   brassow
> 
> For those that don't know:  Logging implementations are modular and use 
> the APIs found in linux/drivers/md/dm-log.h.  Primarily, the logging 
> implementations are used by device-mapper mirroring.  Right now, there 
> are only single machine logging implementations in the kernel.  Red Hat 
> has a cluster logging implementation for RHEL4 (cvs co -r RHEL4 
> cluster/cmirror-kernel), but since the upstream cluster infrastructure 
> is based on userspace CMAN/OpenAIS, that implementation will likely not 
> live outside of RHEL4.
> 


Jon,

When your ready to start on the server side let me or Patrick know.  I
would suggest creating an openais component for this purpose.  These are
pretty simple to write and take about 1-2 weeks if the component is
simple.  With 17 separate messages, your implementation could take a bit
longer.  Fortunately openais handles all of the abstractions of messages
and services, but does require sending the messages as a binary data
structure blob within the server itself.  Because each message has a
separate identifier, it is easy to handle versions using a scheme where
new message types are handled by an entirely new message handler while
the remaining message handlers stay intact.

Fortunately Patrick has done all of the heavy lifting of figuring out
how to get all the header files correct to make your own components. :)

Regards
-steve

[Cluster-devel] New cluster mirror log implementation (RFC)

[Lon Hohberger]

On Tue, 2006-06-20 at 20:57 -0700, Steven Dake wrote:
> On Tue, 2006-06-20 at 21:17 -0500, Jonathan E Brassow wrote:

> When your ready to start on the server side let me or Patrick know.  I
> would suggest creating an openais component for this purpose.  

It looks like that's a natural fit.

-- Lon