Re: NFS dentry caching mechanism

[Peter Staubach <staubach@redhat.com>]

Trond Myklebust wrote:

>
>Note that the strong cache validation your describe also relies heavily
>on the mtime accuracy on the server. A typical exported ext3 or reiserfs
>filesystem will still fail for the distributed make case since it has an
>mtime resolution of 1 second.
>
>  
>

True.  I don't believe that the client is the right place to work around
such a problem with a server though.  We should get the server fixed, or
in this case, any relevant local file systems fixed so that they support
the required semantics.  There are lots of NFS servers in the world which
do not have this issue.

I know that fixing/changing these file systems will be difficult, but
as they are, they don't make for very good NFS server service.

>>>Operations such as RMDIR and unlink() do have a race, but in the case
>>>where you have one client creating a directory and another client
>>>destroying it, there will always be a race unless you have some method
>>>of synchronisation between the processes on the clients.
>>>
>>>There is a potential caching race if you try to open the file, but that
>>>is (as I said previously) quite intentional: it is done for scalability
>>>reasons.
>>>
>>> 
>>>
>>>      
>>>
>>I don't think that I understand this last paragraph.  Does this mean that
>>the consistency was purposefully relaxed in order to increase performance?
>>    
>>
>
>Not performance as such, but scalability. Both the server and network
>suffer in the case where you have nfsroot clients flooding the system
>with GETATTR requests in order to revalidate negative dentries. Consider
>for instance at all the little shared libraries, config files, and other
>junk that a typical GNOME or KDE desktop login involves, and you'll know
>what I mean. The difference between negative dentry caching and not is
>very significant in those cases, and so we were seeing some nasty
>network floods in the early 2.4 series when it was briefly turned off.
>
>I am basically very wary of increasing the number of GETATTR calls:
>we're already seeing a large number of HPC sites complaining about the
>scalability problems those cause on their servers, and asking for a
>reduction in the number of unnecessary revalidations (particularly so
>for 2.6 kernels).
>

I agree completely.  It in the decision making for "unnecessary" where
the issue lie.  It is very easy to go too far and relax the consistency
too much or go too far in the other direction and end up with the extra
over the wire round trips for little or no gain.

    Thanx...

       ps


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs