[Cluster-devel] fence_scsi - Configuration file

[Cluster-devel] fence_scsi - Configuration file

[Jan Friesse]

I was working on fence_scsi configuration file and will be very happy
for opinions of larger audience for syntax/semantic.

Syntax:
Basically it's little improved corosync.conf so example:

# Comment line

# ^^^ white lines ignored

# Definition of device
/dev/sda

# Can be more
/dev/sdb
/dev/sdc

# Configuration section -> everything what doesn't start with /dev/
global {
  key1: value
  key2: value
}

# Device definition + per device configuration
/dev/sdd {
  key1: value
}

### CUT HERE ####

This syntax should give us very flexible background for future
additions. On the other hand, it allows user to overwrite vgs
auto_detection by simple list of device.

Now something about semantic (what is more important).

What I think is good to have is support for hierarchy of options
processed in this way:
1) Hardcoded defaults in fence_scsi - Used in case nobody will say
something different
2) global device options - Used for all devices listed (and maybe auto
detected, will be discussed later)
3) Per device configuration options

So I will try to give example for APTPL flag. This is typical flag,
where we want to be by default enabled or disabled (really don't care
and should be point of discussion) and we want give user ability to
change it and maybe change it for some devices differently. So for
example, user has:
- First array - supports APTPL
/dev/sda
/dev/sdb
- Second array - Doesn't support APTPL
/dev/sdc

and default hardcoded in fence_scsi is to NOT use APTPL:

In my point of view, user can use:

### Cut HERE
global {
  aptpl: on
}

/dev/sda
/dev/sdb
/dev/sdc {
  aptpl: off
}
### CUT HERE

OR

### Cut HERE
/dev/sda {
  aptpl: on
}
/dev/sdb {
  aptpl: on
}
# CUT HERE


Next, in my design is global variable auto_detect. In case, auto_detect
is on, we are using OLD method (vgs scanning) and use per device
options. So another example. Now user has only one array, supporting
APTPL and default is to NOT use APTPL. User DON'T want to enter devices,
so can use:
### CUT HERE
global {
  auto_detect: on
  aptpl: on
}
### CUT HERE

Thats more or less everything about design, any comments highly welcomed.

Regards,
  Honza

[Cluster-devel] fence_scsi - Configuration file

[Ryan O'Hara]

On Thu, Jul 30, 2009 at 04:08:44PM +0200, Jan Friesse wrote:
> I was working on fence_scsi configuration file and will be very happy
> for opinions of larger audience for syntax/semantic.
> 
> Syntax:
> Basically it's little improved corosync.conf so example:
> 
> # Comment line
> 
> # ^^^ white lines ignored
> 
> # Definition of device
> /dev/sda
> 
> # Can be more
> /dev/sdb
> /dev/sdc
> 
> # Configuration section -> everything what doesn't start with /dev/
> global {
>   key1: value
>   key2: value
> }
> 
> # Device definition + per device configuration
> /dev/sdd {
>   key1: value
> }
> 
> ### CUT HERE ####
> 
> This syntax should give us very flexible background for future
> additions. On the other hand, it allows user to overwrite vgs
> auto_detection by simple list of device.

Right. I'd also like to point out that this config file will need to
be manually replicated to all node in the cluster if a user wants to
explicitly list which devices to use.

> Now something about semantic (what is more important).
> 
> What I think is good to have is support for hierarchy of options
> processed in this way:
> 1) Hardcoded defaults in fence_scsi - Used in case nobody will say
> something different
> 2) global device options - Used for all devices listed (and maybe auto
> detected, will be discussed later)
> 3) Per device configuration options
>
> So I will try to give example for APTPL flag. This is typical flag,
> where we want to be by default enabled or disabled (really don't care
> and should be point of discussion) and we want give user ability to
> change it and maybe change it for some devices differently. So for
> example, user has:
> - First array - supports APTPL
> /dev/sda
> /dev/sdb
> - Second array - Doesn't support APTPL
> /dev/sdc
> 
> and default hardcoded in fence_scsi is to NOT use APTPL:
> 
> In my point of view, user can use:
> 
> ### Cut HERE
> global {
>   aptpl: on
> }
> 
> /dev/sda
> /dev/sdb
> /dev/sdc {
>   aptpl: off
> }
> ### CUT HERE
> 
> OR
> 
> ### Cut HERE
> /dev/sda {
>   aptpl: on
> }
> /dev/sdb {
>   aptpl: on
> }
> # CUT HERE

I'm ok with this, but I'm not sure that having 2 arrays, one that
supports APTPL and one that does not, would be useful. Only in the
case where the array that did support it lost power ... but the one
that did not support it remained on ... would you see it being
useful. Still, if people was to do this I suppose it is ok.
 
> Next, in my design is global variable auto_detect. In case, auto_detect
> is on, we are using OLD method (vgs scanning) and use per device
> options. So another example. Now user has only one array, supporting
> APTPL and default is to NOT use APTPL. User DON'T want to enter devices,
> so can use:
> ### CUT HERE
> global {
>   auto_detect: on
>   aptpl: on
> }
> ### CUT HERE

What happens if we list devices and have auto_detect set to 'on'? Will
we ignore the devices? With this auto_detect parameter, it seems that
it will have to be explicitly set to 'off' and devices will have to be
listed if we want to avoid auto-detect. Also, what happens if I set
auto_detect to 'off' and I don't list any devices?

Ryan

[Cluster-devel] fence_scsi - Configuration file

[Jan Friesse]

Ryan O'Hara napsal(a):
> On Thu, Jul 30, 2009 at 04:08:44PM +0200, Jan Friesse wrote:
>> I was working on fence_scsi configuration file and will be very happy
>> for opinions of larger audience for syntax/semantic.
>>
>> Syntax:
>> Basically it's little improved corosync.conf so example:
>>
>> # Comment line
>>
>> # ^^^ white lines ignored
>>
>> # Definition of device
>> /dev/sda
>>
>> # Can be more
>> /dev/sdb
>> /dev/sdc
>>
>> # Configuration section -> everything what doesn't start with /dev/
>> global {
>>   key1: value
>>   key2: value
>> }
>>
>> # Device definition + per device configuration
>> /dev/sdd {
>>   key1: value
>> }
>>
>> ### CUT HERE ####
>>
>> This syntax should give us very flexible background for future
>> additions. On the other hand, it allows user to overwrite vgs
>> auto_detection by simple list of device.
> 
> Right. I'd also like to point out that this config file will need to
> be manually replicated to all node in the cluster if a user wants to
> explicitly list which devices to use.
> 
>> Now something about semantic (what is more important).
>>
>> What I think is good to have is support for hierarchy of options
>> processed in this way:
>> 1) Hardcoded defaults in fence_scsi - Used in case nobody will say
>> something different
>> 2) global device options - Used for all devices listed (and maybe auto
>> detected, will be discussed later)
>> 3) Per device configuration options
>>
>> So I will try to give example for APTPL flag. This is typical flag,
>> where we want to be by default enabled or disabled (really don't care
>> and should be point of discussion) and we want give user ability to
>> change it and maybe change it for some devices differently. So for
>> example, user has:
>> - First array - supports APTPL
>> /dev/sda
>> /dev/sdb
>> - Second array - Doesn't support APTPL
>> /dev/sdc
>>
>> and default hardcoded in fence_scsi is to NOT use APTPL:
>>
>> In my point of view, user can use:
>>
>> ### Cut HERE
>> global {
>>   aptpl: on
>> }
>>
>> /dev/sda
>> /dev/sdb
>> /dev/sdc {
>>   aptpl: off
>> }
>> ### CUT HERE
>>
>> OR
>>
>> ### Cut HERE
>> /dev/sda {
>>   aptpl: on
>> }
>> /dev/sdb {
>>   aptpl: on
>> }
>> # CUT HERE
> 
> I'm ok with this, but I'm not sure that having 2 arrays, one that
> supports APTPL and one that does not, would be useful. Only in the
> case where the array that did support it lost power ... but the one
> that did not support it remained on ... would you see it being
> useful. Still, if people was to do this I suppose it is ok.
>  

It really doesn't make much sense. But I'm really not sure what 
customers will want and this idea can be one of them.

>> Next, in my design is global variable auto_detect. In case, auto_detect
>> is on, we are using OLD method (vgs scanning) and use per device
>> options. So another example. Now user has only one array, supporting
>> APTPL and default is to NOT use APTPL. User DON'T want to enter devices,
>> so can use:
>> ### CUT HERE
>> global {
>>   auto_detect: on
>>   aptpl: on
>> }
>> ### CUT HERE
> 
> What happens if we list devices and have auto_detect set to 'on'? Will
> we ignore the devices? With this auto_detect parameter, it seems that
> it will have to be explicitly set to 'off' and devices will have to be
> listed if we want to avoid auto-detect. Also, what happens if I set
> auto_detect to 'off' and I don't list any devices?
> 

autodetect set to "on" can have one of these two consequences:
- Ignore listed device - read only global sections for parameters
- Ignore listed device but use per device parameters in case we will 
find this device by old method - This is what is implemented now
- Use merge of listed devices and autodetected device

autodetect is by default off. When we not have config file, it will 
become on. When no devices is listed -> we turn it on

I hope nobody will ever set autodetect to off and don't list any device, 
but we can do two things:
- Ignore this flag and use vgs (this is what code does now)
- Do what user want -> no device fencing


> Ryan

(I'm moving discussion to more "visible" place ;) )

Regards,
   Honza

[Cluster-devel] fence_scsi - Configuration file

[Ryan O'Hara]

On Mon, Aug 03, 2009 at 05:33:51PM +0200, Jan Friesse wrote:
> Ryan O'Hara napsal(a):
>> What happens if we list devices and have auto_detect set to 'on'? Will
>> we ignore the devices? With this auto_detect parameter, it seems that
>> it will have to be explicitly set to 'off' and devices will have to be
>> listed if we want to avoid auto-detect. Also, what happens if I set
>> auto_detect to 'off' and I don't list any devices?
>>
>
> autodetect set to "on" can have one of these two consequences:
> - Ignore listed device - read only global sections for parameters

^^ I think this is the most logical thing to do.

> - Ignore listed device but use per device parameters in case we will  
> find this device by old method - This is what is implemented now
> - Use merge of listed devices and autodetected device
>
> autodetect is by default off. When we not have config file, it will  
> become on. When no devices is listed -> we turn it on

Shouldn't auto_detect be default on? We want fence_scsi to work like
it does now unless it is configured otherwise. I was thinking ...

- If not config file exists, auto_detect is on.
- If a config file does exists and auto_detect is not defined to be
  off, auto_detect should default to on ... regardless of whether or
  not any devices are listed.

> I hope nobody will ever set autodetect to off and don't list any device,  
> but we can do two things:
> - Ignore this flag and use vgs (this is what code does now)
> - Do what user want -> no device fencing

If this evern happens, the node will not register with any
devices. There are two possible outcomes:

1. If a reservation already exists (being held by another node/key),
the node in question won't have write access to the device(s) since it
didn't register with anything.
2. If a reservation does not already exist, nothing changes and the
node is completely unprotected and there is no way to fence it.

I suppose there is a third case where the node is already registered,
so doing nothing is valid, but this is just pure luck.

[Cluster-devel] fence_scsi - Configuration file

[Jan Friesse]

Ryan,

Ryan O'Hara wrote:
> On Mon, Aug 03, 2009 at 05:33:51PM +0200, Jan Friesse wrote:
>> Ryan O'Hara napsal(a):
>>> What happens if we list devices and have auto_detect set to 'on'? Will
>>> we ignore the devices? With this auto_detect parameter, it seems that
>>> it will have to be explicitly set to 'off' and devices will have to be
>>> listed if we want to avoid auto-detect. Also, what happens if I set
>>> auto_detect to 'off' and I don't list any devices?
>>>
>> autodetect set to "on" can have one of these two consequences:
>> - Ignore listed device - read only global sections for parameters
> 
> ^^ I think this is the most logical thing to do.

I think first or second are more or less same. It's matter of line with
%device_params_list = ();. Maybe we can make this configurable...

> 
>> - Ignore listed device but use per device parameters in case we will  
>> find this device by old method - This is what is implemented now
>> - Use merge of listed devices and autodetected device
>>
>> autodetect is by default off. When we not have config file, it will  
>> become on. When no devices is listed -> we turn it on
> 
> Shouldn't auto_detect be default on? We want fence_scsi to work like
> it does now unless it is configured otherwise. I was thinking ...
> 
> - If not config file exists, auto_detect is on.
> - If a config file does exists and auto_detect is not defined to be
>   off, auto_detect should default to on ... regardless of whether or
>   not any devices are listed.
> 

- Interesting idea... I'm little fear about BZs like "fence_scsi ignores
my devices listed in config file, ..."

>> I hope nobody will ever set autodetect to off and don't list any device,  
>> but we can do two things:
>> - Ignore this flag and use vgs (this is what code does now)
>> - Do what user want -> no device fencing
> 
> If this evern happens, the node will not register with any
> devices. There are two possible outcomes:
> 
> 1. If a reservation already exists (being held by another node/key),
> the node in question won't have write access to the device(s) since it
> didn't register with anything.
> 2. If a reservation does not already exist, nothing changes and the
> node is completely unprotected and there is no way to fence it.
> 
> I suppose there is a third case where the node is already registered,
> so doing nothing is valid, but this is just pure luck.
> 

I'm really not sure what to do in this case and turning on autodetection
seems to me like a not so bad idea.

Anyway, it looks like we are still standing on one place, so what about
just do this:
- Remove per device options
- Remove all flags and config sections (it looks like they are not needed)
- Let it be as a simple list with [/dev/device\n]* (+ comments)

?

Syntax is ready for future bigger things and parsing code will remain
same -> future additions should be simple. I will just remove (or
ignore) some variables. Thats simple

Regards,
  Honza