Re: [Drbd-dev] [RFC] (CRM and) DRBD (0.8) states and transistions, recovery strategies

[Philipp Reisner <philipp.reisner@linbit.com>]

[-- Attachment #1: Type: text/plain, Size: 2125 bytes --]

On Wednesday 29 September 2004 19:07, Lars Ellenberg wrote:
> / 2004-09-29 14:58:47 +0200
>
> \ Philipp Reisner:
> > > [...]
> > >
> > > >     Currently this covers only the states, and outlines the
> > > > transitions. It should help to define the actions to be taken on
> > > > every possible "input" to the DRBD internal "state machine".
> > >
> > > While reading through this giant e-mail I lost my confidence that it
> > > could be a good idea to have a "central" state switching function in
> > > DRBD, but of course I will see what this discussions gives...
> >
> > Thought about this a bit more... and came to the conclusion that it
> > would be a good idea. What do you think of this skeleton -
> > pseude code (it compiles actually).
>
> I think it probably should come out more like a real state machine,
> with a defined set of possible INPUTS,
> a defined set of states (which should not have the same detail depth as
> the actual drbd internal state set with all its different attributes),
> a set of actions, and a defined state[INPUT] => action => newstate
> matrix.
>
> maybe that is overkill.

Currently I want to go the way that was outlined with the new_st.c
skeleton.

Regarding: only worker should do state changes.
  We have quite a lot of inputs that are asynchronous by their nature.
  E.g. Disk fails. It does not make any sense to synchronize an advance
  in the disk-state state-machine with anything.

While it makes a lot of sense to synchronize changes to the node
state machine. 

At first I drew a directed graph of the cstates we have in 
drbd-0.7 (see cstates-7.ps)

You will immediately realize that the differentiation between
Unconfigured and StandAllone is a leftover from drbd-0.6

Then I drew directed graphs of the "state machines" as I see
them for drbd-0.8

conn-states-8.ps, disk-states-8.ps, node-states-8.ps (has 2 pages)

PS: The program is graphviz

-Philipp
-- 
: Dipl-Ing Philipp Reisner                      Tel +43-1-8178292-50 :
: LINBIT Information Technologies GmbH          Fax +43-1-8178292-82 :
: Schönbrunnerstr 244, 1120 Vienna, Austria    http://www.linbit.com :

[-- Attachment #2: conn-states-8.dot --]
[-- Type: text/plain, Size: 833 bytes --]

digraph conn_states {
	StandAllone  -> WFConnection   [ label = "ioctl_set_net()" ]
	WFConnection -> Unconnected    [ label = "unable to bind()" ]
	WFConnection -> WFReportParams [ label = "in connect() after accept" ]
	WFReportParams -> StandAllone  [ label = "checks in receive_param()" ]
	WFReportParams -> Connected    [ label = "in receive_param()" ]
	WFReportParams -> WFBitMapS    [ label = "sync_handshake()" ]
	WFReportParams -> WFBitMapT    [ label = "sync_handshake()" ]
	WFBitMapS -> SyncSource        [ label = "receive_bitmap()" ]
	WFBitMapT -> SyncTarget        [ label = "receive_bitmap()" ]
	SyncSource -> Connected
	SyncTarget -> Connected
	SyncSource -> PausedSyncS
	SyncTarget -> PausedSyncT
	PausedSyncS -> SyncSource
	PausedSyncT -> SyncTarget
	Connected   -> WFConnection    [ label = "* on network error" ]
}

[-- Attachment #3: disk-states-8.dot --]
[-- Type: text/plain, Size: 913 bytes --]

digraph disk_states {
	Diskless -> Inconsistent       [ label = "ioctl_set_disk()" ]
	Diskless -> Consistent         [ label = "ioctl_set_disk()" ]
	Diskless -> Outdated           [ label = "ioctl_set_disk()" ]
	Consistent -> Outdated         [ label = "receive_param()" ]
	Consistent -> UpToDate         [ label = "receive_param()" ]
	Consistent -> Inconsistent     [ label = "start resync" ]
	Outdated   -> Inconsistent     [ label = "start resync" ]
	UpToDate   -> Inconsistent     [ label = "ioctl_replicate" ]
	Inconsistent -> UpToDate       [ label = "resync completed" ]
	Consistent -> Failed           [ label = "io completion error" ]
	Outdated   -> Failed           [ label = "io completion error" ]
	UpToDate   -> Failed           [ label = "io completion error" ]
	Inconsistent -> Failed         [ label = "io completion error" ]
	Failed -> Diskless             [ label = "sending notify to peer" ]
}

[-- Attachment #4: disk-states-8.ps --]
[-- Type: application/postscript, Size: 12540 bytes --]

[-- Attachment #5: node-states-8.dot --]
[-- Type: text/plain, Size: 540 bytes --]

digraph node_states {
	Secondary -> Primary           [ label = "ioctl_set_state()" ]
	Primary   -> Secondary 	       [ label = "ioctl_set_state()" ]
}

digraph peer_states {
	Secondary -> Primary           [ label = "recv state packet" ]
	Primary   -> Secondary 	       [ label = "recv state packet" ]
	Primary   -> Unknown 	       [ label = "connection lost" ]
	Secondary  -> Unknown  	       [ label = "connection lost" ]
	Unknown   -> Primary           [ label = "connected" ]
	Unknown   -> Secondary         [ label = "connected" ]
}


[-- Attachment #6: node-states-8.ps --]
[-- Type: application/postscript, Size: 9774 bytes --]

[-- Attachment #7: cstates-7.dot --]
[-- Type: text/plain, Size: 1318 bytes --]

digraph cstate {
	Unconfigured -> StandAllone    [ label = "ioctl_set_disk()" ]
	StandAllone  -> Unconnected    [ label = "ioctl_set_net()" ]
	Unconfigured -> Unconnected    [ label = "ioctl_set_net()" ]
	Unconnected  -> WFConnection   [ label = "connect()[1]" ]
	WFConnection -> Unconnected    [ label = "unable to bind()" ]
	WFConnection -> WFReportParams [ label = "in connect() after accept" ]
	WFReportParams -> StandAllone  [ label = "checks in receive_param()" ]
	WFReportParams -> Connected    [ label = "in receive_param()" ]
	WFReportParams -> WFBitMapS    [ label = "sync_handshake()" ]
	WFReportParams -> WFBitMapT    [ label = "sync_handshake()" ]
	WFBitMapS -> SyncSource        [ label = "receive_bitmap()" ]
	WFBitMapT -> SyncTarget        [ label = "receive_bitmap()" ]
	SyncSource -> Connected
	SyncTarget -> Connected
	SyncSource -> PausedSyncS
	SyncTarget -> PausedSyncT
	PausedSyncS -> SyncSource
	PausedSyncT -> SyncTarget
	Connected   -> BrokenPipe       [ label = "* recv error" ]
	BrokenPipe  -> WFConnection     [ label = "connect()[1]" ]
	Connected   -> NetworkFailure   [ label = "* set by asender()" ]
	NetworkFailure -> WFConnection  [ label = "connect()[1]" ]
	Connected   -> Timeout          [ label = "* set drbd_send()" ]
	Timeout     -> WFConnection     [ label = "connect()[1]" ]
}

[-- Attachment #8: conn-states-8.ps --]
[-- Type: application/postscript, Size: 13380 bytes --]

[-- Attachment #9: cstates-7.ps --]
[-- Type: application/postscript, Size: 17743 bytes --]