Re: nfsd tuning - please help me! (Alan Powell)

Re: nfsd tuning - please help me! (Alan Powell)

[Heflin, Roger A.]

>  Date: Fri, 14 Feb 2003 09:44:53 -0800 (PST)
> From: Alan Powell <>
> Subject: Re: [NFS] nfsd tuning - please help me!
> To: Steve Dickson <SteveD@RedHat.com>, nfs@lists.sourceforge.net
>=20
> Unfortunately, we've tried all that already. So given
> that we are not hardware/network constrained, does all
> this mean that the Linux kernel NFS runs into
> performance issues beyond 100 file reads/sec?
>=20
>=20
	I have been able to get closer to 10-20MBytes per second with
	linux nfs.  The netapps will do around 4-5 times that though at
	a higher cost.  And you can get it out of linux, by putting more
	cheap smaller servers to obtain the same rate.

	What are you underlying disks?  You could still be hardware
	constrained depending on what your underlying disks are,
	and what you underlying disk controller is.   Both can have
	issues.

	I have machines that are servicing around 2500 8k reads
	per second and seem to work fine, though mine may break
	down to fewer larger reads.

	Other things that will get you in trouble is having lots of files
	in a single directory (in the several thousand range will hurt
	pretty bad), also check to make sure you aren't accumulating lots
	of .nfs* files in the directies in question, I had a situation where
	there where lots of files being messed with (read and write) and
	lots of these files accumulated and pretty much brought
	performance to its knees.  The solution was to run a cron job
	to clean up the .nfs* files.  The .nfs files are created when you
	are reading a file that is being deleted by another process at
	the same time, the .nfs* stays around to service the reader,
	and does not always go away (this is on all NFS implementations
	I have seen).

	Do a ls -ld dirname and see the size of the directories, and include it
	in the next message if one of the above don't pan out.

							Roger

				=09


-------------------------------------------------------
This SF.NET email is sponsored by: FREE  SSL Guide from Thawte
are you planning your Web Server Security? Click here to get a FREE
Thawte SSL guide and find the answers to all your  SSL security issues.
http://ads.sourceforge.net/cgi-bin/redirect.pl?thaw0026en
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Re: nfsd tuning - please help me! (Alan Powell)

[Skottie Miller]

This is slightly OT, but worth noting.

Heflin, Roger A. wrote:
 
> 	I have been able to get closer to 10-20MBytes per second with
> 	linux nfs.  The netapps will do around 4-5 times that though at
> 	a higher cost.  And you can get it out of linux, by putting more
> 	cheap smaller servers to obtain the same rate.

15 - 20 times that; my netapp 960's sustain 280 - 330 Mbytes/sec of read traffic
when feeding the renderfarm.  That's one of the many reasons we're using NetApp
and not Linux NFS servers, in an otherwise all-linux shop.

  -skottie


-- 

Scott Miller                 | Animation Technology
work: skottie@dreamworks.com | Dreamworks Feature Animation
life: skottie@pobox.com



-------------------------------------------------------
This SF.NET email is sponsored by: FREE  SSL Guide from Thawte
are you planning your Web Server Security? Click here to get a FREE
Thawte SSL guide and find the answers to all your  SSL security issues.
http://ads.sourceforge.net/cgi-bin/redirect.pl?thaw0026en
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Re: nfsd tuning - please help me! (Alan Powell)

[Alan Powell]

Yes indeed, the problem is probably that I have too
many files in each directory. Recommended # of
files/directory is a different topic, so I'll start a
new thread for that. Thanks!

(by the way, I did follow each and every suggestion in
the NFS Tuning How-To)


--- "Heflin, Roger A."
<Roger.A.Heflin@conocophillips.com> wrote:
> 
> 
> 
> >  Date: Fri, 14 Feb 2003 09:44:53 -0800 (PST)
> > From: Alan Powell <>
> > Subject: Re: [NFS] nfsd tuning - please help me!
> > To: Steve Dickson <SteveD@RedHat.com>,
> nfs@lists.sourceforge.net
> > 
> > Unfortunately, we've tried all that already. So
> given
> > that we are not hardware/network constrained, does
> all
> > this mean that the Linux kernel NFS runs into
> > performance issues beyond 100 file reads/sec?
> > 
> > 
> 	I have been able to get closer to 10-20MBytes per
> second with
> 	linux nfs.  The netapps will do around 4-5 times
> that though at
> 	a higher cost.  And you can get it out of linux, by
> putting more
> 	cheap smaller servers to obtain the same rate.
> 
> 	What are you underlying disks?  You could still be
> hardware
> 	constrained depending on what your underlying disks
> are,
> 	and what you underlying disk controller is.   Both
> can have
> 	issues.
> 
> 	I have machines that are servicing around 2500 8k
> reads
> 	per second and seem to work fine, though mine may
> break
> 	down to fewer larger reads.
> 
> 	Other things that will get you in trouble is having
> lots of files
> 	in a single directory (in the several thousand
> range will hurt
> 	pretty bad), also check to make sure you aren't
> accumulating lots
> 	of .nfs* files in the directies in question, I had
> a situation where
> 	there where lots of files being messed with (read
> and write) and
> 	lots of these files accumulated and pretty much
> brought
> 	performance to its knees.  The solution was to run
> a cron job
> 	to clean up the .nfs* files.  The .nfs files are
> created when you
> 	are reading a file that is being deleted by another
> process at
> 	the same time, the .nfs* stays around to service
> the reader,
> 	and does not always go away (this is on all NFS
> implementations
> 	I have seen).
> 
> 	Do a ls -ld dirname and see the size of the
> directories, and include it
> 	in the next message if one of the above don't pan
> out.
> 
> 							Roger
> 
> 					
> 
> 
>
-------------------------------------------------------
> This SF.NET email is sponsored by: FREE  SSL Guide
> from Thawte
> are you planning your Web Server Security? Click
> here to get a FREE
> Thawte SSL guide and find the answers to all your 
> SSL security issues.
>
http://ads.sourceforge.net/cgi-bin/redirect.pl?thaw0026en
> _______________________________________________
> NFS maillist  -  NFS@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs


__________________________________________________
Do you Yahoo!?
Yahoo! Shopping - Send Flowers for Valentine's Day
http://shopping.yahoo.com


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs