All of lore.kernel.org
 help / color / mirror / Atom feed
* RE: 2.6.0 -- nfs client runs 15x faster if client app runs as root.
@ 2004-01-07 20:46 Lever, Charles
  2004-01-07 21:00 ` Eric Whiting
  0 siblings, 1 reply; 8+ messages in thread
From: Lever, Charles @ 2004-01-07 20:46 UTC (permalink / raw)
  To: Eric Whiting, nfs

are you writing to a set-uid file?  in the normal user case,
the SETATTR is required by Posix to unset the set-* bits on
the file.

> -----Original Message-----
> From: Eric Whiting [mailto:ewhiting@amis.com]
> Sent: Wednesday, January 07, 2004 3:14 PM
> To: nfs@lists.sourceforge.net
> Subject: [NFS] 2.6.0 -- nfs client runs 15x faster if client=20
> app runs as
> root.
>=20
>=20
> My 2.6.0 nfs clients to a netapps 940 NFS server run very=20
> differently depending
> on who the user is. Both tcp and udp mounts seem to show the=20
> same behavior. This
> is very apparent when doing the bonnie putc() tests. =20
>=20
> nfsstat and tcpdump show a lot of extra setattr calls when=20
> running as a normal
> user. I assume this is the reason for the slowdown.=20
>=20
> Is there a fix? Or is this the way it is supposed to be?
>=20
> This is not how this same hardware acts in 2.4.23.
>=20
> Thanks,
> eric
>=20
>=20
>=20
> Running as a 'user' doing bonnie putc()
> Writing with putc()...  done:    802 kB/s  10.6 %CPU
>=20
> 13:06:15.841823 172.16.17.176.3073181638 > 172.16.32.90.2049:=20
> 144 setattr [|nfs]
> (DF)
> 13:06:15.842024 172.16.32.90.2049 > 172.16.17.176.3073181638:=20
> reply ok 144
> setattr [|nfs]
> 13:06:15.842361 172.16.17.176.3089958854 > 172.16.32.90.2049:=20
> 648 write [|nfs]
> (DF)
> 13:06:15.842611 172.16.32.90.2049 > 172.16.17.176.3089958854:=20
> reply ok 160 write
> [|nfs]
> 13:06:15.842880 172.16.17.176.3106736070 > 172.16.32.90.2049:=20
> 144 setattr [|nfs]
> (DF)
> 13:06:15.843079 172.16.32.90.2049 > 172.16.17.176.3106736070:=20
> reply ok 144
> setattr [|nfs]
> 13:06:15.843506 172.16.17.176.3123513286 > 172.16.32.90.2049:=20
> 648 write [|nfs]
> (DF)
> 13:06:15.843783 172.16.32.90.2049 > 172.16.17.176.3123513286:=20
> reply ok 160 write
> [|nfs]
> 13:06:15.844069 172.16.17.176.3140290502 > 172.16.32.90.2049:=20
> 144 setattr [|nfs]
> (DF)
> 13:06:15.844253 172.16.32.90.2049 > 172.16.17.176.3140290502:=20
> reply ok 144
> setattr [|nfs]
> 13:06:15.844604 172.16.17.176.3157067718 > 172.16.32.90.2049:=20
> 648 write [|nfs]
> (DF)
>=20
> Running as root on the client box doing a bonnie putc()
> Writing with putc()...         done:  11625 kB/s  70.5 %CPU
>=20
> 13:04:34.530773 172.16.32.90.2049 > 172.16.17.176.2934638278:=20
> reply ok 160 write
> [|nfs]
> 13:04:34.533983 172.16.17.176.2951415494 > 172.16.32.90.2049:=20
> 1472 write [|nfs]
> (frag 58503:1480@0+)
> 13:04:34.533995 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@1480+)
> 13:04:34.533999 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@2960+)
> 13:04:34.534003 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@4440+)
> 13:04:34.534007 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@5920+)
> 13:04:34.534010 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@7400+)
> 13:04:34.534014 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@8880+)
> 13:04:34.534018 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@10360+)
> 13:04:34.534021 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@11840+)
> 13:04:34.534024 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@13320+)
> 13:04:34.534028 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@14800+)
> 13:04:34.534032 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@16280+)
> 13:04:34.534035 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@17760+)
> 13:04:34.534039 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@19240+)
> 13:04:34.534042 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@20720+)
> 13:04:34.534045 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@22200+)
> 13:04:34.534049 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@23680+)
> 13:04:34.534053 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@25160+)
> 13:04:34.534056 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@26640+)
> 13:04:34.534060 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@28120+)
> 13:04:34.534063 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@29600+)
> 13:04:34.534067 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:1480@31080+)
> 13:04:34.534070 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58503:368@32560)
> 13:04:34.537357 172.16.32.90.2049 > 172.16.17.176.2951415494:=20
> reply ok 160 write
> [|nfs]
> 13:04:34.540586 172.16.17.176.2968192710 > 172.16.32.90.2049:=20
> 1472 write [|nfs]
> (frag 58504:1480@0+)
> 13:04:34.540598 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58504:1480@1480+)
> 13:04:34.540603 172.16.17.176 > 172.16.32.90: udp (frag=20
> 58504:1480@2960+)
>=20
>=20
> Client nfs v3:
> null       getattr    setattr    lookup     access     readlink
> 0       0% 8724    0% 1113998 47% 5225    0% 3084    0% 98      0%
> read       write      create     mkdir      symlink    mknod
> 20776   0% 1213403 51% 237     0% 1       0% 3       0% 0       0%
> remove     rmdir      rename     link       readdir    readdirplus
> 206     0% 0       0% 95      0% 24      0% 48      0% 416     0%
> fsstat     fsinfo     pathconf   commit
> 118     0% 10      0% 0       0% 0       0%
>=20
>=20
> -------------------------------------------------------
> This SF.net email is sponsored by: Perforce Software.
> Perforce is the Fast Software Configuration Management System offering
> advanced branching capabilities and atomic changes on 50+ platforms.
> Free Eval! http://www.perforce.com/perforce/loadprog.html
> _______________________________________________
> NFS maillist  -  NFS@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs
>=20


-------------------------------------------------------
This SF.net email is sponsored by: Perforce Software.
Perforce is the Fast Software Configuration Management System offering
advanced branching capabilities and atomic changes on 50+ platforms.
Free Eval! http://www.perforce.com/perforce/loadprog.html
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

^ permalink raw reply	[flat|nested] 8+ messages in thread
* Re: 2.6.0 -- nfs client runs 15x faster if client app runs as root.
@ 2004-01-08 15:12 trond.myklebust
  2004-01-08 15:23 ` trond.myklebust
  0 siblings, 1 reply; 8+ messages in thread
From: trond.myklebust @ 2004-01-08 15:12 UTC (permalink / raw)
  To: gael.roualland; +Cc: ewhiting, nfs

[-- Attachment #1: Type: text/plain, Size: 622 bytes --]

På to , 08/01/2004 klokka 09:15, skreiv Gaël Roualland:
> Has the file exec permission set ?
> If yes this would look like an earlier problem that seems to still
> occur on 2.6.0.
> See http://marc.theaimsgroup.com/?l=linux-nfs&m=106966481311224 and
> http://marc.theaimsgroup.com/?l=linux-nfs&m=106981736010493.

Yeah. It turned out to be remove_suid() that was causing an empty
SETATTR to be sent after every write(). The solution is just to filter
out all requests that will result in empty SETATTRs. The server will
automatically do the honours w.r.t. the suid bit as soon as it receives
our write.

Cheers,
  Trond



[-- Attachment #2: untitled-1.2 --]
[-- Type: text/html, Size: 1712 bytes --]

[-- Attachment #3: linux-2.6.1-01-fix_setattr.dif --]
[-- Type: text/plain, Size: 1072 bytes --]

 inode.c |    9 +++++++++
 1 files changed, 9 insertions(+)

diff -u --recursive --new-file --show-c-function linux-2.6.1-rc2/fs/nfs/inode.c linux-2.6.1-01-fix_setattr/fs/nfs/inode.c
--- linux-2.6.1-rc2/fs/nfs/inode.c	2004-01-07 22:57:48.000000000 -0500
+++ linux-2.6.1-01-fix_setattr/fs/nfs/inode.c	2004-01-08 00:22:49.000000000 -0500
@@ -791,6 +791,8 @@ out_no_inode:
 	goto out;
 }
 
+#define NFS_VALID_ATTRS (ATTR_MODE|ATTR_UID|ATTR_GID|ATTR_SIZE|ATTR_ATIME_SET|ATTR_MTIME_SET)
+
 int
 nfs_setattr(struct dentry *dentry, struct iattr *attr)
 {
@@ -798,6 +800,11 @@ nfs_setattr(struct dentry *dentry, struc
 	struct nfs_fattr fattr;
 	int error;
 
+	/* Optimization: if the end result is no change, don't RPC */
+	attr->ia_valid &= NFS_VALID_ATTRS;
+	if (attr->ia_valid == 0)
+		return 0;
+
 	lock_kernel();
 
 	/*
@@ -813,6 +820,8 @@ printk("nfs_setattr: revalidate failed, 
 
 	if (!S_ISREG(inode->i_mode)) {
 		attr->ia_valid &= ~ATTR_SIZE;
+		if (attr->ia_valid == 0)
+			goto out;
 	} else {
 		filemap_fdatawrite(inode->i_mapping);
 		error = nfs_wb_all(inode);


^ permalink raw reply	[flat|nested] 8+ messages in thread
* 2.6.0 -- nfs client runs 15x faster if client app runs as root.
@ 2004-01-07 21:54 trond.myklebust
  2004-01-08 14:15 ` Gaël Roualland
  0 siblings, 1 reply; 8+ messages in thread
From: trond.myklebust @ 2004-01-07 21:54 UTC (permalink / raw)
  To: ewhiting; +Cc: nfs

> My 2.6.0 nfs clients to a netapps 940 NFS server run very differently
depending
> on who the user is. Both tcp and udp mounts seem to show the same behavior.
> This is very apparent when doing the bonnie putc() tests.
> nfsstat and tcpdump show a lot of extra setattr calls when running as a
normal
> user. I assume this is the reason for the slowdown. Is there a fix? Or
is this the
> way it is supposed to be?

Huh?
I'm certainly not seeing one setattr call per write on my 2.6 clients. Is
this stock
2.6.0 or is it patched? Does 2.6.1-rc2 change anything?

If not, can you capture a binary tcpdump of these setattr calls, so we can
have a
look at what they are trying to do? "tcpdump -w dump.out -s 9000" should
suffice.

Cheers,
  Trond




-------------------------------------------------------
This SF.net email is sponsored by: Perforce Software.
Perforce is the Fast Software Configuration Management System offering
advanced branching capabilities and atomic changes on 50+ platforms.
Free Eval! http://www.perforce.com/perforce/loadprog.html
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

^ permalink raw reply	[flat|nested] 8+ messages in thread
* 2.6.0 -- nfs client runs 15x faster if client app runs as root.
@ 2004-01-07 20:13 Eric Whiting
  0 siblings, 0 replies; 8+ messages in thread
From: Eric Whiting @ 2004-01-07 20:13 UTC (permalink / raw)
  To: nfs@lists.sourceforge.net

My 2.6.0 nfs clients to a netapps 940 NFS server run very differently depending
on who the user is. Both tcp and udp mounts seem to show the same behavior. This
is very apparent when doing the bonnie putc() tests.  

nfsstat and tcpdump show a lot of extra setattr calls when running as a normal
user. I assume this is the reason for the slowdown. 

Is there a fix? Or is this the way it is supposed to be?

This is not how this same hardware acts in 2.4.23.

Thanks,
eric



Running as a 'user' doing bonnie putc()
Writing with putc()...  done:    802 kB/s  10.6 %CPU

13:06:15.841823 172.16.17.176.3073181638 > 172.16.32.90.2049: 144 setattr [|nfs]
(DF)
13:06:15.842024 172.16.32.90.2049 > 172.16.17.176.3073181638: reply ok 144
setattr [|nfs]
13:06:15.842361 172.16.17.176.3089958854 > 172.16.32.90.2049: 648 write [|nfs]
(DF)
13:06:15.842611 172.16.32.90.2049 > 172.16.17.176.3089958854: reply ok 160 write
[|nfs]
13:06:15.842880 172.16.17.176.3106736070 > 172.16.32.90.2049: 144 setattr [|nfs]
(DF)
13:06:15.843079 172.16.32.90.2049 > 172.16.17.176.3106736070: reply ok 144
setattr [|nfs]
13:06:15.843506 172.16.17.176.3123513286 > 172.16.32.90.2049: 648 write [|nfs]
(DF)
13:06:15.843783 172.16.32.90.2049 > 172.16.17.176.3123513286: reply ok 160 write
[|nfs]
13:06:15.844069 172.16.17.176.3140290502 > 172.16.32.90.2049: 144 setattr [|nfs]
(DF)
13:06:15.844253 172.16.32.90.2049 > 172.16.17.176.3140290502: reply ok 144
setattr [|nfs]
13:06:15.844604 172.16.17.176.3157067718 > 172.16.32.90.2049: 648 write [|nfs]
(DF)

Running as root on the client box doing a bonnie putc()
Writing with putc()...         done:  11625 kB/s  70.5 %CPU

13:04:34.530773 172.16.32.90.2049 > 172.16.17.176.2934638278: reply ok 160 write
[|nfs]
13:04:34.533983 172.16.17.176.2951415494 > 172.16.32.90.2049: 1472 write [|nfs]
(frag 58503:1480@0+)
13:04:34.533995 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@1480+)
13:04:34.533999 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@2960+)
13:04:34.534003 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@4440+)
13:04:34.534007 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@5920+)
13:04:34.534010 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@7400+)
13:04:34.534014 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@8880+)
13:04:34.534018 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@10360+)
13:04:34.534021 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@11840+)
13:04:34.534024 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@13320+)
13:04:34.534028 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@14800+)
13:04:34.534032 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@16280+)
13:04:34.534035 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@17760+)
13:04:34.534039 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@19240+)
13:04:34.534042 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@20720+)
13:04:34.534045 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@22200+)
13:04:34.534049 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@23680+)
13:04:34.534053 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@25160+)
13:04:34.534056 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@26640+)
13:04:34.534060 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@28120+)
13:04:34.534063 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@29600+)
13:04:34.534067 172.16.17.176 > 172.16.32.90: udp (frag 58503:1480@31080+)
13:04:34.534070 172.16.17.176 > 172.16.32.90: udp (frag 58503:368@32560)
13:04:34.537357 172.16.32.90.2049 > 172.16.17.176.2951415494: reply ok 160 write
[|nfs]
13:04:34.540586 172.16.17.176.2968192710 > 172.16.32.90.2049: 1472 write [|nfs]
(frag 58504:1480@0+)
13:04:34.540598 172.16.17.176 > 172.16.32.90: udp (frag 58504:1480@1480+)
13:04:34.540603 172.16.17.176 > 172.16.32.90: udp (frag 58504:1480@2960+)


Client nfs v3:
null       getattr    setattr    lookup     access     readlink
0       0% 8724    0% 1113998 47% 5225    0% 3084    0% 98      0%
read       write      create     mkdir      symlink    mknod
20776   0% 1213403 51% 237     0% 1       0% 3       0% 0       0%
remove     rmdir      rename     link       readdir    readdirplus
206     0% 0       0% 95      0% 24      0% 48      0% 416     0%
fsstat     fsinfo     pathconf   commit
118     0% 10      0% 0       0% 0       0%


-------------------------------------------------------
This SF.net email is sponsored by: Perforce Software.
Perforce is the Fast Software Configuration Management System offering
advanced branching capabilities and atomic changes on 50+ platforms.
Free Eval! http://www.perforce.com/perforce/loadprog.html
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

^ permalink raw reply	[flat|nested] 8+ messages in thread
end of thread, other threads:[~2004-01-08 15:44 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2004-01-07 20:46 2.6.0 -- nfs client runs 15x faster if client app runs as root Lever, Charles
2004-01-07 21:00 ` Eric Whiting
  -- strict thread matches above, loose matches on Subject: below --
2004-01-08 15:12 trond.myklebust
2004-01-08 15:23 ` trond.myklebust
2004-01-07 21:54 trond.myklebust
2004-01-08 14:15 ` Gaël Roualland
2004-01-08 15:12   ` Eric Whiting
2004-01-07 20:13 Eric Whiting

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.