Re: NFSv4 and client caching to local disk?

[Skottie Miller <skottie@anim.dreamworks.com>]

Bogdan Costescu wrote:
> On Mon, 31 Mar 2003, Jake Gold wrote:
> 
>>I have a FAS940 with a cluster of Linux machines mounting _read-only_
>>volumes over NFSv3. My situation is the same as James Pearson's....those
>>machines read large, rarely modified files, where memory caching doesn't
>>help a whole lot. It would be very nice to be able to have my
>>under-worked local disks take some of the load off the filer.
> 
> How about doing it in the application itself ? Often application has more 
> knowledge about what it needs than the OS. So in a case like the one that 
> you described I would copy the rarely modified file(s) to the local disk 
> and just before accessing them in any way check to see if they were 
> modified. If this is true read-only you don't even need to check this, but 
> otherwise you could use some rsync-style transfer to get only the parts 
> that were modified (assuming that new data is not completely different)... 
> Modification can be detected from the file attributes, md5sum or something 
> else.

We do the caching to compute-node local disk, via the applications when we can.
But two things often make that difficult:  (1) the working set of the cached data
is larger than the compute-note local disk, and (2) we don't own all the applications.

So, I complement the compute-node local disk caches (and sometimes replace them)
with a series of NetCache c2100 dNFS boxes.  The origin data is on 960's,
which serve up their data to the user workstations, and to the caches.
The renderfarm compute nodes get almost all their high-traffic read-only
data off the caches.  This maintains good interactive performance for the
user workstations and keeps the renderfarm fed.

 -skottie


-- 

Scott Miller                 | Animation Technology
work: skottie@dreamworks.com | Dreamworks Feature Animation
life: skottie@pobox.com



-------------------------------------------------------
This SF.net email is sponsored by: ValueWeb: 
Dedicated Hosting for just $79/mo with 500 GB of bandwidth! 
No other company gives more support or power for your dedicated server
http://click.atdmt.com/AFF/go/sdnxxaff00300020aff/direct/01/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs