Re: blacklisted DS with pnfs

[Boaz Harrosh <bharrosh@panasas.com>]

On 01/25/2012 11:56 AM, Tigran Mkrtchyan wrote:
> Hi,
> 
> we have observed that in some situations ( probably network glitches )
> the pnfs client blacklisted one of the data servers:
> 
> NFS: data server 83a95099 connection error -12. Deviceid [22000000000]
> marked out of use.
> 
> As a result, data server can't be used by this client anymore.
> 
> Is there a way to let client to forget about data server?
> Some magic in /proc ?
> 
> This is SL6.2 (RHEL 6.2):
> # uname -a
> Linux p3-wgs13 2.6.32-220.2.1.el6.x86_64 #1 SMP Thu Dec 22 11:15:52
> CST 2011 x86_64 x86_64 x86_64 GNU/Linux
> #
> 

Look in the source code, I think there is a RECALL that the server
can do to trash the all device cache. or one of the devices.

What happens is that the device is marked with error but is in 
cache so is not re-fetched.

wait let me look ....

I found it! The server sends a NOTIFY_DEVICEID4_CHANGE. The
client will remove the deviceid from cache and unmount if needed.
Next layout with that deviceid will re-establish the connection and
will put a new clean entry in the dev cache.

[If you decide to enhance pynfs to send a NOTIFY_DEVICEID4_CHANGE as an admin
 tool. That would be interesting]

> Regards,
>    Tigran.

Cheers
Boaz