All of lore.kernel.org
 help / color / mirror / Atom feed
* 2.4.19 NFSALL performance oddity
@ 2002-10-04 21:45 Heflin, Roger A.
  2002-10-04 23:46 ` Trond Myklebust
  0 siblings, 1 reply; 14+ messages in thread
From: Heflin, Roger A. @ 2002-10-04 21:45 UTC (permalink / raw)
  To: nfs


I am running some tests using 2.4.19 with all posted nfs patchs, =
everything is
being run sync.   All runs are with NFS3 over UDP.

I have noticed that the IO rates (writes) start slowly (about =
1.5MB/second) and
slowly build up to a peak of 2.5MB/second over several minutes, with the
test averaging 2.32 MB/second over a 2GB file write.  This is=20
with a server with Gigabit copper and the client with 100BT ethernet.  =20
No packets are being lost from what I can tell, on either the server or=20
the client, both machines are dual cpu with 2GB of ram and otherwise=20
unloaded.

The 2.2.19 client (with almost no NFS patches) runs about 3.98MB/second =
over
a 2GB file write, against the same 2.4.19 NFS server, and hits that rate =

almost immediately.

Using an older 2.4.19 NFS server (that I don't believe has any extra NFS
patches) gets a rate of 6.50 average over a 2GB file from
a 2.2.19 client and gets 2.00  MB/second average from one of the
original 2.4 test clients.

Both rsize and wsize are 8192, I have tried it with 32768 on at least
the 2.4 client side, but it does not appear to affect the above results.
I am tried with 32 and 64 nfsd and this did not appear to change
any of the results.   The rmem/wmem parameters have been upped
to 256K and did not appear to make any difference on the server.

The disk is locally capable of 20MB/second writes over a 4GB file, using
the same test as was used above.

Is there a way to make the IO start fast and stay fast?   What code =
could
I look at to see if I can affect the way this works?   Any parameters =
that
I could try changing that might improve things?

It appears to partially be client issue since the 2.2.19 NFS client is =
noticeably
faster, but since the older 2.4.19 server and the 2.2.19 client is even =
faster it
appears to also be a server issue.

So in summary:

2.2.19 -> 2.4.19 no patches -> 6.5MB/second
2.4.19nfsall -> 2.4.19 no patches ->  2.00 MB/second
2.2.19 -> 2.4.19 nfsall patch -> 3.98MB/second
2.4.19nfsall -> 2.4.19 nfsall patch -> 2.32 MB/second


				Roger


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

^ permalink raw reply	[flat|nested] 14+ messages in thread
* RE: 2.4.19 NFSALL performance oddity
@ 2002-10-06  1:56 Heflin, Roger A.
  0 siblings, 0 replies; 14+ messages in thread
From: Heflin, Roger A. @ 2002-10-06  1:56 UTC (permalink / raw)
  To: Bryan O'Sullivan, Trond Myklebust; +Cc: nfs


The only extra patches to 2.4.19 are the NFS patches, so
I don't believe so unless it was included in 2.4.19.

We are using config_himem, because at least some of the
machines we have will have a bit too much memory.

I will try turning that off and see if it makes a difference, is
this on the client or the server or both?   I am pretty sure
we have it enabled on all of the nfsall patched 2.4.19 builts.

				Roger

> -----Original Message-----
> From:	Bryan O'Sullivan [SMTP:bos@serpentine.com]
> Sent:	Saturday, October 05, 2002 3:55 PM
> To:	Trond Myklebust
> Cc:	Heflin, Roger A.; nfs@lists.sourceforge.net
> Subject:	Re: [NFS] 2.4.19 NFSALL performance oddity
>=20
> On Sat, 2002-10-05 at 09:18, Trond Myklebust wrote:
>=20
> > I'm seeing major slowdowns, mainly on NFS reads, when I enable
> > CONFIG_HIGHMEM. I haven't yet had time to investigate this.
>=20
> I'll try to take a look on a local system in the next few days.  Are =
you
> or Roger using Jens Axboe's block-highmem patch or any of the drivers =
it
> touches?  Knowing this should help control for at least one variable.
>=20
> 	<b
>=20


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

^ permalink raw reply	[flat|nested] 14+ messages in thread
* RE: 2.4.19 NFSALL performance oddity
@ 2002-10-07 21:27 Heflin, Roger A.
  2002-10-07 22:20 ` Trond Myklebust
  0 siblings, 1 reply; 14+ messages in thread
From: Heflin, Roger A. @ 2002-10-07 21:27 UTC (permalink / raw)
  To: nfs; +Cc: Trond Myklebust


Update on the issue:

And given that I am using MS outlook, who knows
what this actual message will look like.

Original Test:
> 2.2.19 -> 2.4.19 no patches -> 6.5MB/second
> 2.4.19nfsall -> 2.4.19 no patches ->  2.00 MB/second
> 2.2.19 -> 2.4.19 nfsall patch -> 3.98MB/second
> 2.4.19nfsall -> 2.4.19 nfsall patch -> 2.32 MB/second
>=20
	All 2.4.19 kernels have at least 4GB himem enabled.
	The 2.4.19 kernels with nfsall have 64GB himem enabled.

	New tests:

	The write is a sequential write of a 2GB file,=20
	the read is a sequentail read of a 2GB file, the=20
	machines memory is just under 2GB, and it appears that
	the file is just large enough on the read test to make=20
	sure little or none of  the file is ever read straight=20
	from the cache.

	From the results, it looks like something in nfsall
	slows down both the client and the server.   The=20
	picture all of this data paints is rather unclear to me.
	If anyone sees any other test that would be possibly
	useful, suggest them.

	64GB/4GB/none does not appear to make a=20
	difference, for either the client or the server.   =20
	I did get some wide variations
	in some limited testing by adjusting the rmem/wmem
	and the number of nfsds, generally always slowing=20
	things down by a large amount, none of the numbers=20
	below are any of the especially slow cases.

	Tests with nohimem on NFSALL 2.4.19 server (32K blocks):
		nfsd=3D8, mem=3D65536, nohimem
			Client nohimem
				write: 2.312MB
				read: 8.938MB
			Client himem (64GB)
				write: 2.312MB
				read: 8.938MB=09

		nfsd=3D8, mem=3D262144, nohimem
			Client nohimem:=20
				write:  2.250MB
				read:   8.875MB
			Client himem (64GB)
				write: 2.312MB=20
				read:  8.812MB
	=09
	Tests with 4GB himem on nfsall 2.4.19 server (32k blocks)
			Client nohimem:
				write: 2.312MB
				read: 8.932MB
			Client 2.2.19:
				write: 4.50MB
				read:  8.812MB

	Tests with himem on NFSALL 2.4.19 server (8k blocks, different machine =
that 32k blocksize))
		nfsd=3D8, mem=3D65536, himem
			Client nohimem:
				write:	1.875MB
				read:	8.688MB
			Client himem (64GB)
				write:	1.875MB
				read:	8.625MB

	Tests with 4GB himem server, 2.4.19 server nohimem (8k blocks, =
different machine than other servers)
		nfsd=3D8, mem=3D65536
			Client nohimem, 2.4.19 nfsall
				write: 2.00 MB
				read:  2.00 MB
			Client 2.2.19:
				write:  6.5MB
				read:  10.0MB
			Client 2.4.19 no nfs patches, 64GB himem:
				write: not yet done
				read:

	Test with 2.2.19 server - these results seem to be unclear on their =
meaning:
			Client 2.4.19 64GB himem, nfsall:
				write: 3.875MB
				read:  4.875MB=09
			Client 2.2.19:
				write: 4.00 MB
				read: 8.188MB

> 				Roger


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

^ permalink raw reply	[flat|nested] 14+ messages in thread
* RE: 2.4.19 NFSALL performance oddity
@ 2002-10-08 20:53 Heflin, Roger A.
  0 siblings, 0 replies; 14+ messages in thread
From: Heflin, Roger A. @ 2002-10-08 20:53 UTC (permalink / raw)
  To: trond.myklebust; +Cc: nfs

Ok,

I applied the new patch, it does not appear to have made any difference=20
in my setup and tests, it also does not appear to break anything.
I am still analyzing the results, but not finding=20
anything obvious about what is going on.  2.2.19 appears to make alot
better client to a 2.4.19 server than 2.4.1[89][no patch/nfsall]
does.   And 2.4.19 with no patches appears to make a better
server than 2.4.19 (for the writes) with the NFS patches
(using 2.2.19 as a client).

Using 2.4.xx as a client, a 2.4.1[89][nfsall/nopatch] server is=20
about the same on writes, but 2.4.19 nfsall is much better
on the reads than the previous 2.4.1[89] without
the nfsall patch.

I guess the read numbers with NFSall look pretty good,=20
the write numbers probably need to be better, though the write=20
numbers do look ok  (2.5x larger) with 2.2.19 as
a client, they don't look good with 2.4.19 nfsall as a client.  =20
And from 2.4.19 to 2.4.19 nfsall the write numbers got a bit worse=20
for a 2.2.19 client, but did not appear to change a large amount
when using a 2.4.19  client.

I have a excel spreadsheet of the various results, that try to
summarize up all of the recent emails.  If anyone wants this=20
spreadsheet I will sent it out.


2.4.19 NFSALLnew Server:
	2.4.19 NFSALLnew Client
		write: 2.312MB
		read: 8.875MB
	2.4.18 Client:
		write: 2.125
		read: 8.125

2.4.19 NFSALLold Server:
	2.4.19 NFSALLnew Client:
		write: 2.312 MB
		read:  8.688 MB

2.4.18 no patches Server:
	2.4.19 NFSALL Client:
		write: 2.00 MB
		read:  2.50 MB
	2.4.19 nopatches client:
		write: 2.0MB
		read: 3.0MB
	2.2.19 stock client:
		write: 6.00 MB
		read: 10.0 MB



=20
			Roger
> -----Original Message-----
> From:	Trond Myklebust [SMTP:trond.myklebust@fys.uio.no]
> Sent:	Monday, October 07, 2002 5:21 PM
> To:	Heflin, Roger A.
> Cc:	nfs@lists.sourceforge.net
> Subject:	RE: 2.4.19 NFSALL performance oddity
>=20
>=20
> FYI: I updated 2.4.19-NFS_ALL on Saturday with a couple of
> bugfixes. Among them was one which fixes a queueing bottleneck when a
> timeout+resend occurs.
>=20
> Cheers,
>   Trond


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

^ permalink raw reply	[flat|nested] 14+ messages in thread
end of thread, other threads:[~2002-10-08 20:53 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2002-10-04 21:45 2.4.19 NFSALL performance oddity Heflin, Roger A.
2002-10-04 23:46 ` Trond Myklebust
2002-10-05  4:48   ` Bryan O'Sullivan
2002-10-05 16:18     ` Trond Myklebust
2002-10-05 20:55       ` Bryan O'Sullivan
2002-10-05 21:51         ` Trond Myklebust
2002-10-05 23:41           ` H. J. Lu
2002-10-06  4:01             ` Bryan O'Sullivan
2002-10-06  4:09               ` H. J. Lu
2002-10-06  4:16           ` Bryan O'Sullivan
  -- strict thread matches above, loose matches on Subject: below --
2002-10-06  1:56 Heflin, Roger A.
2002-10-07 21:27 Heflin, Roger A.
2002-10-07 22:20 ` Trond Myklebust
2002-10-08 20:53 Heflin, Roger A.

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.