FYI: readdirplus responsiveness problem after 2.4 -> 2.6 client upgrade

FYI: readdirplus responsiveness problem after 2.4 -> 2.6 client upgrade

[Frank van Maarseveen]

Server is tru64 (OSF1 server V5.1 1885 alpha) using advfs.

client:
After replacing a 2.4.29 kernel by a 2.6 (.11) we started to notice
unresponsiveness in bash command- and name completion once in a while.
This still persists in 2.6.12.2.

A few quick investigations pointed to the readdir -> readdirplus
change. The following tcpdump snippets illustrates this (first column
is differential time in usec with previous packet):

example 1:
------
000093 ip 230: client.3377456114 > server.nfs: 188 readdirplus fh 4079,976512/13921 512 bytes @ 0x000000000 (DF)
645029 ip 1514: server.nfs > client.3377456114: reply ok 1472 readdirplus (frag 19592:1480@0+)
000058 ip 778: server > client: udp (frag 19592:744@1480)
------

example 2:
------
000094 ip 230: client.743433202 > server.nfs: 188 readdirplus fh 4079,976512/404 512 bytes @ 0x000000000 (DF)
102655 ip 230: client.743433202 > server.nfs: 188 readdirplus fh 4079,976512/404 512 bytes @ 0x000000000 (DF)
205974 ip 230: client.743433202 > server.nfs: 188 readdirplus fh 4079,976512/404 512 bytes @ 0x000000000 (DF)
248905 ip 1514: server.nfs > client.743433202: reply ok 1472 readdirplus (frag 58739:1480@0+)
000085 ip 1106: server > client: udp (frag 58739:1072@1480)

000128 ip 1514: server.nfs > client.743433202: reply ok 1472 readdirplus (frag 58740:1480@0+)
000057 ip 230: client.760210418 > server.nfs: 188 readdirplus fh 4079,976512/404 512 bytes @ 0x0000001e4 (DF)
000028 ip 1106: server > client: udp (frag 58740:1072@1480)

000125 ip 1514: server.nfs > client.743433202: reply ok 1472 readdirplus (frag 58741:1480@0+)
000088 ip 1106: server > client: udp (frag 58741:1072@1480)

022016 ip 506: server.nfs > client.760210418: reply ok 464 readdirplus
------

AFAICS this only shows a (sometimes) slow responding server. In the second
example the client appears to be retransmitting much sooner than in the
first example. Don't know if this can be true or that the differential
timestamps are fooling me.

In practice there are a significant number of cases without retransmission
when the server appears unresponsive to the user with readdirplus. Since
it is induced by bash name completion it is very visible to the user:
occasionally command-completion takes 3..10 seconds now to give an idea.
Name completion for arguments is hurt too sometimes.

A quick peek into the source suggested:

-	reduce NFS_LIMIT_READDIRPLUS?
-	"recognize" name-completion kind of use and revert
	NFS_INO_ADVISE_RDPLUS?

Other options (except replacing the server)?

-- 
Frank


-------------------------------------------------------
This SF.Net email is sponsored by the 'Do More With Dual!' webinar happening
July 14 at 8am PDT/11am EDT. We invite you to explore the latest in dual
core and dual graphics technology at this free one hour event hosted by HP, 
AMD, and NVIDIA.  To register visit http://www.hp.com/go/dualwebinar
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: FYI: readdirplus responsiveness problem after 2.4 -> 2.6 client upgrade

[Trond Myklebust]

m=C3=A5 den 11.07.2005 Klokka 18:51 (+0200) skreiv Frank van Maarseveen:
> In practice there are a significant number of cases without retransmissio=
n
> when the server appears unresponsive to the user with readdirplus. Since
> it is induced by bash name completion it is very visible to the user:
> occasionally command-completion takes 3..10 seconds now to give an idea.
> Name completion for arguments is hurt too sometimes.
>=20
> A quick peek into the source suggested:
>=20
> -	reduce NFS_LIMIT_READDIRPLUS?
> -	"recognize" name-completion kind of use and revert
> 	NFS_INO_ADVISE_RDPLUS?
>=20
> Other options (except replacing the server)?

I suppose you could mask out NFS_CAP_READDIRPLUS on the client, but it
sounds very strange that the OSF implementation should be so incredibly
slow. Is there nothing in the administrators manual about tuning this
sort of operation?

Cheers,
  Trond



-------------------------------------------------------
This SF.Net email is sponsored by the 'Do More With Dual!' webinar happening
July 14 at 8am PDT/11am EDT. We invite you to explore the latest in dual
core and dual graphics technology at this free one hour event hosted by HP, 
AMD, and NVIDIA.  To register visit http://www.hp.com/go/dualwebinar
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: FYI: readdirplus responsiveness problem after 2.4 -> 2.6 client upgrade

[Frank van Maarseveen]

On Mon, Jul 11, 2005 at 01:17:16PM -0400, Trond Myklebust wrote:
> m=E5 den 11.07.2005 Klokka 18:51 (+0200) skreiv Frank van Maarseveen:
> > In practice there are a significant number of cases without retransmiss=
ion
> > when the server appears unresponsive to the user with readdirplus. Since
> > it is induced by bash name completion it is very visible to the user:
> > occasionally command-completion takes 3..10 seconds now to give an idea.
> > Name completion for arguments is hurt too sometimes.
>=20
> I suppose you could mask out NFS_CAP_READDIRPLUS on the client, but it
> sounds very strange that the OSF implementation should be so incredibly
> slow. Is there nothing in the administrators manual about tuning this
> sort of operation?

Maybe. I'm not an tru64 expert. But I do know that the server in
question is underpowered, e.g. lookups, getattr are a bit slow at times
as well. Anyway, the "user experience" seems better with readdirplus
disabled on the client so I decided to do that.

--=20
Frank


-------------------------------------------------------
This SF.Net email is sponsored by the 'Do More With Dual!' webinar happening
July 14 at 8am PDT/11am EDT. We invite you to explore the latest in dual
core and dual graphics technology at this free one hour event hosted by HP, 
AMD, and NVIDIA.  To register visit http://www.hp.com/go/dualwebinar
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs