Re: [PATCH v4 00/11] Witness protocol support for transparent failover

[Paulo Alcantara <palcantara@suse.de>]

Aurélien Aptel <aaptel@suse.de> writes:

> I've added a "failover" test group to the buildbot that mounts a
> "regular" (non-scaleout) cluster and switches the fileserver to another
> cluster node live and it looks like it's working: you can keep on using
> the mount.
>
> In non-scale-out, the file server has its own virtual IP that both node
> share. So when you "move" the fileserver to a different node, it doesn't
> actually change IP. After doing that we realized that this actually
> works already without -o witness since it's reconnecting to the same IP.
>
> Now we need to add a scale-out cluster fileserver in buildbot where,
> IIUC (please correct me Samuel) the fileserver is actually using the
> node IP instead of this virtual-IP shared by nodes. So that when we move
> the fileserver, it actually changes its IP address and we can test this
> properly.
>
> As for the code, I'm not an expert on reconnection but it looks for
> merging I think. It doesn't handle multichannel but multchannel doesn't
> handle reconnection well anyway. There is an issue which pops up in
> other parts of the code as well.
>
> If you run a command too quickly after the transition, they will fail
> with EIO so it's not completely failing over but I think there can be
> the same issue with DFS (Paulo, any ideas/comments?)  which is why we do
> 2 times ls and we ignore the result of the first in the DFS tests.
>
> the dfs test code:
>
>     def io_reco_test(unc, opts, cwd, expected):
>         try:
>             lsdir = '.'
>             cddir = os.path.join(ARGS.mnt, cwd)
>             info(("TEST: mount {unc} , cd {cddir} , ls {lsdir}, expect:[{expect}]\n"+
>                   "      disconnect {cddir} , ls#1 {lsdir} (fail here is ok),  ls#2 (fail here NOT ok)").format(
>                       unc=unc, cddir=cddir, lsdir=lsdir, expect=" ".join(['"%s"'%x for x in expected])
>                   ))

For soft mounts, it is OK ignoring the first ls.  But for hard mounts,
we shouldn't ignore the first ls as it must retry forever until failover
is done.