Re: Long-standing NFSv3 UDP client performance problem, probably due to RPC?

[Steve Dickson <SteveD@redhat.com>]

On 08/10/12 11:08, Quentin Barnes wrote:
> On Mon, Oct 8, 2012 at 6:10 AM, Steve Dickson <SteveD@redhat.com> wrote:
>> On 05/10/12 13:57, Quentin Barnes wrote:
> [...]
>>> Since for our work, for other reasons, we've switched over to
>>> using NFSv3 TCP mounts, so I can't justify spending a lot of time
>>> debugging this UDP/RPC problem.  However, for example if someone
>>> wants me to try something out and gather some new test results or
>>> a patch to test, I can squeeze that in.
> [...]
>>
>> I think there probably has been a steady decline in UPD performance
>> over the years.
> 
> In my data, after the initial big hit between RHEL4 and RHEL5,
> NFSv3/UDP performance went back up peaking with 2.6.31, then declined
> with 2.6.32 and RHEL6, and has then held steady ever since.
Wow... Impressive... Very rarely do we get a such a time line 
in WRT performance... I'm not sure what happen in the 2.6.32 kernel,
but maybe it has something to do with the RPC slot table???
That pure speculation... 

> 
> Now I have seen a significant dip my NFSv3/TCP performance data
> after 3.3 with 3.6 (I don't have data points for 3.4 & 3.5), but
> didn't want to get into that here and I hadn't looked into it hard
> enough yet to verify it.
Now this is not good... I do remember come claims that RHEL5 was
quicker than RHEL6, but there was never any numbers to back
it up... 
> 
>> The main reason is that nobody uses it since TCP is a
>> much better transport to use with NFS...
> 
> I disagree somewhat, at least for my particular configuration and
> networks.  With my testing and tuning with FreeBSD and 2.6.9 and
> earlier Linux kernels, NFSv3/UDP overall performance is generally
> 10%-15% better than NFSv3/TCP.
I did meant in production... We too still test v2 over UDP and TCP... 

> 
>> Why are you still using UDP as your transport?
> 
> We're not.  See my above quoted paragraph.  I still measure and
> monitor NFSv3/UDP's performance as part of my kernel development
> work improving the kernel's NFS performance for our needs, but since
> no one uses UDP mounts in house currently, I can't justify the time
> to find and fix the bug.
Agreed... Justifying working/fixing technology what we are moving
away from is tough... 

steved.