[Cluster-devel] spectator setting in cluster.conf

[Fabio Massimo Di Nitto <fabbione@ubuntu.com>]

Hi David,

David Teigland wrote:
> I believe that all cluster.conf settings currently map to a specific
> setting in one specific subsystem.  We now have a proposal to add an
> abstract "spectator" setting that would be translated into multiple
> lower-level, subsystem-specific settings.  It's not clear what the
> cluster.conf syntax would look like, but it would need to be per-node.

I did a couple of tests and:

<clusternode name="node1" nodeid="1" votes="1">
 <spectator/>
</clusternode>

seems to be the sanest one and the query would look like:

/cluster/clusternodes/clusternode[@name=\"%s\"]/spectator"

no ccs_get error and empty str (looking at general ccs_get sintax from cmanccs.c).

If we want to use this sintax:

<clusternode name="node1" nodeid="1" votes="1" spectator="..">

then we need to add a value and parse it. At that point you might as well
set votes to 0 directly.

So I would personally prefer the first one.

> 2. /etc/init.d/cman would skip the 'fence_tool join' step for nodes
>    with this setting.  Because gfs is guaranteed to never write to the
>    fs, we can allow the node to mount without joining the fence domain.
>    If the spectator node fails, it would not be fenced.
> 
>    The standard, low-level mechanism for doing this (without the new
>    setting in cluster.conf) is to add a new option to the init-script's
>    config file /etc/sysconfig/cman.

This one will be tricky because we cannot ask ccsd for this setting via
ccs_tools before ccs is talking to cman and that won't happen until cman is
started. There is a little risk of a race condition where we will need to wait
for cman/ccsd to talk before we can decide if we need to execute
"fence_tool join" or not. It is a small window but still there.
This problem will not exists once we kill ccsd.

> 3. service_cman.lcrso (the cman openais plugin) would read this and
>    assign the node 0 votes, so its membership wouldn't affect quorum.
> 
>    The standard way to do this would be to set votes="0" in the
>    <clusternode/> entry.

I have a test patch for it if the assumption is that:

- spectator is more important than votes (spectator will override votes settings)
- spectator setting cannot be changed by anything other than the cluster.conf
(read below)

> In each case, the existence of the lower-level setting would override the
> effect of the abstract spectator setting.

What kind of lower-level settings are you thinking about? env vars? In cman,
votes are checked at the very beginning of the startup process. There isn't much
other than env vars to override.

Also note that the same env vars should be set across the whole cluster to have
an real effect. At that point I believe it would be easier to just update the
config to remove/add spectator and recalculate quorum and votes.

Cheers
Fabio

-- 
I'm going to make him an offer he can't refuse.