Re: [ANNOUNCE] Minneapolis Cluster Summit, July 29-30

[James Bottomley <James.Bottomley@SteelEye.com>]

On Sat, 2004-07-10 at 11:04, David Teigland wrote:
> The "it" refers to gfs.  This means gfs doesn't make a lot of sense and isn't
> very practical without it.  I'm not the one to speculate on what gfs would
> become otherwise, others would do that better.

This is what you actually said:

> It simply makes most sense to put cman in the kernel for
> what we're doing with it.

I interpret that to mean you think cman (your cluster manager) should be
in the kernel.  Is this incorrect?

> 
> > You also face two other additional hurdles:
> > 
> > 1) GFS today uses a user space DLM.  What critical problems does this have
> > that you suddenly need to move it all into the kernel?
> 
> GFS does not use a user space dlm today.  GFS uses the client-server gulm lock
> manager for which the client (gfs) side runs in the kernel and the gulm server
> runs in userspace on some other node.  People have naturally been averse to
> using servers like this with gfs for a long time and we've finally created the
> serverless dlm (a la VMS clusters).  For many people this is the only option
> that makes gfs interesting; it's also what the opengfs group was doing.

OK, whatever you choose to call it, the previous lock manager used by
gfs was userspace.

OK, so why is a kernel based DLM the only option that makes GFS
interesting?  What are the concrete advantages you achieve with a kernel
based DLM that you don't get with a user space one?  There are plenty of
symmetric serverless userspace DLM implementations that follow the old
VMS (and even updated by Oracle) spec.

Steve Dake has already given a pretty compelling list of why you
shouldn't put the DLM and clustering in the kernel, what is the more
compelling list of reasons why it should be?

> This is a revealing discussion.  We've worked hard to make gfs's lock manager
> independent from gfs itself so it could be useful to others and make gfs less
> monolithic.  We could have left it embedded within the file system itself --
> that's what most other cluster file systems do.  If we'd done that we would
> have avoided this objection altogether but with an inferior design.  The fact
> that there's an independent lock manager to point at and question illustrates
> our success.  The same goes for the cluster manager.  (We could, of course, do
> some simple glueing together and make a monlithic system again :-)

I'm not questioning your goal, merely your in-kernel implementation.
Sharing is good.  However things which are shared don't automatically
have to be in-kernel.

> > 2) We have numerous other clustering products for Linux, none of which (well
> > except the Veritas one) has any requirement at all on having pieces in the
> > kernel.  If all the others operate in user space, why does yours need to be
> > in the kernel?
> 
> If you want gfs in user space you don't want gfs; you want something different.

I didn't say GFS, I said "cluster products".  That's the DLM and CMAN
pieces of your architecture.

Once you can convince us that CMAN et al should be in the kernel, the
next stage of the discussion would be the API.  Several groups (like
GGL, SAF and OCF) have done API work for clusters.  They were mostly
careful to select APIs that avoided mandating cluster policy.  You seem
to have chosen a particular policy (voting quorate) to implement. 
Again, that's a red flag.  Policy should not be in the kernel;  if we
all agree there should be in-kernel APIs for clustering then they should
be sufficiently abstracted to support all current cluster policies.

James