Re: Performance Diagnosis

[Peter Staubach <staubach@redhat.com>]

Trond Myklebust wrote:
> On Tue, 2008-07-15 at 15:55 -0400, Peter Staubach wrote:
>   
>> It seems to me that as long as we don't shut down a connection
>> which is actively being used for an outstanding request, then
>> we shouldn't have any larger problems with the duplicate caches
>> on servers than we do now.
>>
>> We can do this easily enough by reference counting the connection
>> state and then only closing connections which are not being
>> referenced.
>>     
>
> Agreed.
>
>   
>> A gain would be that we could reduce the numbers of connections on
>> active clients if we could disassociate a connection with a
>> particular mounted file system.  As long as we can achieve maximum
>> network bandwidth through a single connection, then we don't need
>> more than one connection per server.
>>     
>
> Isn't that pretty much the norm today anyway? The only call to
> rpc_create() that I can find is made when creating the nfs_client
> structure. All other NFS-related rpc connections are created as clones
> of the above shared structure, and thus share the same rpc_xprt.
>
>   

Well, it is the norm for the shared superblock situation, yes.


> I'm not sure that we want to share connections in the cases where we
> can't share the same nfs_client, since that usually means that RPC level
> parameters such as timeout values, NFS protocol versions differ.
>
>   

I think the TCP connection can be managed independent of these
things.

>> We could handle the case where the client was talking to more
>> servers than it had connection space for by forcibly, but safely
>> closing connections to servers and then using the space for a
>> new connection to a server.  We could do this in the connection
>> manager by checking to see if there was an available connection
>> which was not marked as in the process of being closed.  If so,
>> then it just enters the fray as needing a connection and am
>> working like all of the others.
>>
>> The algorithm could look something like:
>>
>> top:
>>     Look for a connection to the right server which is not marked
>>         as being closed.
>>     If one was found, then increment its reference count and
>>        return it.
>>     Attempt to create a new connect,
>>     If this works, then increment its reference count and
>>        return it.
>>     Find a connection to be closed, either one not being currently
>>        used or via some heuristic like round-robin.
>>     If this connection is not actively being used, then close it
>>        and go to top.
>>     Mark the connection as being closed, wait until it is closed,
>>        and then go to top.
>>     
>
> Actually, what you really want to do is look at whether or not any of
> the rpc slots are in use or not. If they aren't, then you are free to
> close the connection, if not, go to the next.
>
>   

I think that we would still need to be able to handle the
situation where we needed a connection and all connections
appeared to be in use.  I think that the ability to force
a connection to become unused would be required.


> Unfortunately, you still can't get rid of the 2 minute TIME_WAIT state
> in the case of a TCP connection, so I'm not sure how useful this will
> turn out to be...

Well, there is that...  :-)

It sure seems like there has to be some way of dealing with that
thing.

    Thanx...

       ps