Re: Performance Diagnosis

[Trond Myklebust <trond.myklebust@fys.uio.no>]

On Tue, 2008-07-15 at 15:55 -0400, Peter Staubach wrote:
> It seems to me that as long as we don't shut down a connection
> which is actively being used for an outstanding request, then
> we shouldn't have any larger problems with the duplicate caches
> on servers than we do now.
> 
> We can do this easily enough by reference counting the connection
> state and then only closing connections which are not being
> referenced.

Agreed.

> A gain would be that we could reduce the numbers of connections on
> active clients if we could disassociate a connection with a
> particular mounted file system.  As long as we can achieve maximum
> network bandwidth through a single connection, then we don't need
> more than one connection per server.

Isn't that pretty much the norm today anyway? The only call to
rpc_create() that I can find is made when creating the nfs_client
structure. All other NFS-related rpc connections are created as clones
of the above shared structure, and thus share the same rpc_xprt.

I'm not sure that we want to share connections in the cases where we
can't share the same nfs_client, since that usually means that RPC level
parameters such as timeout values, NFS protocol versions differ.

> We could handle the case where the client was talking to more
> servers than it had connection space for by forcibly, but safely
> closing connections to servers and then using the space for a
> new connection to a server.  We could do this in the connection
> manager by checking to see if there was an available connection
> which was not marked as in the process of being closed.  If so,
> then it just enters the fray as needing a connection and am
> working like all of the others.
> 
> The algorithm could look something like:
> 
> top:
>     Look for a connection to the right server which is not marked
>         as being closed.
>     If one was found, then increment its reference count and
>        return it.
>     Attempt to create a new connect,
>     If this works, then increment its reference count and
>        return it.
>     Find a connection to be closed, either one not being currently
>        used or via some heuristic like round-robin.
>     If this connection is not actively being used, then close it
>        and go to top.
>     Mark the connection as being closed, wait until it is closed,
>        and then go to top.

Actually, what you really want to do is look at whether or not any of
the rpc slots are in use or not. If they aren't, then you are free to
close the connection, if not, go to the next.

Unfortunately, you still can't get rid of the 2 minute TIME_WAIT state
in the case of a TCP connection, so I'm not sure how useful this will
turn out to be...

Cheers
  Trond