[Cluster-devel] CMAN/DLM without SCTP

[Cluster-devel] CMAN/DLM without SCTP

[Pratik Mehta]

Hi,
I am trying to use a cluster with Pacemaker + CMAN on CentOS 6.4. The
application that runs on the cluster includes a userspace SCTP stack.
However CMAN loads dlm which loads the Linux kernel sctp module, which
interferes with the userspace SCTP.

I do not have any GFS/locking requirements on this cluster. I use a 2 node
cluster to failover a bunch of IP addresses.

I tried setting DLM_CONTROLD_OPTS="-r 0" and blacklisting sctp module.
Didn't help since dlm depends on sctp and cman loads dlm.

In /etc/init.d/cman:
errmsg=$( modprobe dlm 2>&1 ) || return 1
errmsg=$( modprobe lock_dlm 2>&1 ) || true

As a hack: I modified the cman service to not load dlm and lock_dlm. The
cluster seems working. However I do not know what additional functionality
was broken when I disabled these. Though a hack: I am eager to know if
someone sees an issue of running a production cluster with this.

Another options is to introduce a compile time flag in the DLM kernel
module to not use SCTP.

Is there any other way?

Thanks in advance
Pratik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/cluster-devel/attachments/20140218/4fe97014/attachment.htm>

[Cluster-devel] CMAN/DLM without SCTP

[David Teigland]

On Tue, Feb 18, 2014 at 07:03:44PM +0530, Pratik Mehta wrote:
> Hi,
> I am trying to use a cluster with Pacemaker + CMAN on CentOS 6.4. The
> application that runs on the cluster includes a userspace SCTP stack.
> However CMAN loads dlm which loads the Linux kernel sctp module, which
> interferes with the userspace SCTP.
> 
> I do not have any GFS/locking requirements on this cluster. I use a 2 node
> cluster to failover a bunch of IP addresses.
> 
> I tried setting DLM_CONTROLD_OPTS="-r 0" and blacklisting sctp module.
> Didn't help since dlm depends on sctp and cman loads dlm.
> 
> In /etc/init.d/cman:
> errmsg=$( modprobe dlm 2>&1 ) || return 1
> errmsg=$( modprobe lock_dlm 2>&1 ) || true
> 
> As a hack: I modified the cman service to not load dlm and lock_dlm. The
> cluster seems working. However I do not know what additional functionality
> was broken when I disabled these. Though a hack: I am eager to know if
> someone sees an issue of running a production cluster with this.

That's a fine solution.  You might also be able to use
'service cman start quorum'.  The cman init script could probably
use some sysconfig option to either disable dlm/gfs2/etc or to
tell it to quit after the quorum step.

> Another options is to introduce a compile time flag in the DLM kernel
> module to not use SCTP.

Not long ago it was possible to avoid loading sctp, but people kept adding
sctp symbols and I didn't have to time to try to keep them out.  It would
be nice if that could be corrected again.

Dave

[Cluster-devel] CMAN/DLM without SCTP

[Pratik Mehta]

On Wed, Feb 19, 2014 at 11:43 PM, David Teigland <teigland@redhat.com>
wrote:
>
>
> That's a fine solution.  You might also be able to use
> 'service cman start quorum'.

Apart from DLM, wouldn't this prevent fenced from starting? Trying this
caused cman status to be non-zero (fenced is stopped), causing pacemaker
start to force a "cman start".

>
> The cman init script could probably
> use some sysconfig option to either disable dlm/gfs2/etc or to
> tell it to quit after the quorum step.

I can help do this. Is there an equivalent example I can look at? Would you
envision this to be exported as a exclude-functionality type of config or
more as a breakpoint=foo (linear/procedural exclusion)?

>
>
> Not long ago it was possible to avoid loading sctp, but people kept adding
> sctp symbols and I didn't have to time to try to keep them out.  It would
> be nice if that could be corrected again.
>

Any pointers on which release/timeline can I go look back at source from? I
can take a shot at replicating that.

Thanks again
Pratik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/cluster-devel/attachments/20140226/cf25db55/attachment.htm>

[Cluster-devel] CMAN/DLM without SCTP

[David Teigland]

On Wed, Feb 26, 2014 at 04:52:14PM +0530, Pratik Mehta wrote:
> On Wed, Feb 19, 2014 at 11:43 PM, David Teigland <teigland@redhat.com>
> wrote:
> >
> >
> > That's a fine solution.  You might also be able to use
> > 'service cman start quorum'.
> 
> Apart from DLM, wouldn't this prevent fenced from starting? Trying this
> caused cman status to be non-zero (fenced is stopped), causing pacemaker
> start to force a "cman start".

If you don't need dlm or gfs, then you shouldn't need anything after the
quorum breakpoint.

It may be possible to insert your own step after the cman init script that
stops or removes daemons and/or modules you don't need.

> > The cman init script could probably
> > use some sysconfig option to either disable dlm/gfs2/etc or to
> > tell it to quit after the quorum step.
> 
> I can help do this. Is there an equivalent example I can look at? Would you
> envision this to be exported as a exclude-functionality type of config or
> more as a breakpoint=foo (linear/procedural exclusion)?

I'd probably use a config option for each start step.  I'm not entirely
sure if it would be taken into the cluster source tree at this point.

> > Not long ago it was possible to avoid loading sctp, but people kept adding
> > sctp symbols and I didn't have to time to try to keep them out.  It would
> > be nice if that could be corrected again.
> >
> 
> Any pointers on which release/timeline can I go look back at source from? I
> can take a shot at replicating that.

Hm, if you're using RHEL6 it would require working on the RHEL6 kernel,
which is going to be different from upstream in the area of SCTP.  And
again, it's unlikely such a patch would be taken for RHEL6 at this point.

Dave