NFS/UDP/IP performance - 2.4.19 v/s 2.4.20, 2.4.20-pre3

[Arun Dharankar <ADharankar@ATTBI.Com>]

Greetings.

There seem to be a remarkable performance difference
between 2.4.19 and 2.4.20/2.4.21-pre3 in regards to
NFS writes/reads. I am not sure, but the problem may not
in NFS but somewhere lower (UDP/IP or core).



For example, in my kernel and network configuration a 
write to a new file over NFS on 2.4.19 for 5MB takes 2.5
seconds or so. With everything same (including kernel
configuration) 2.4.20 and 2.4.21-pre3 the same takes
11 or more seconds.

Also, when this file write is in progress, the system
time goes up to 15% on 2.4.19, whereas on 2.4.20/21-pre3,
it is about 4%. (I use sar/sysstat for this).


Memory accesses dont seem to be the issue either. Test
program to check this show same times and are ok (as I
expect on the board I use).


"netstat -s" or ifconfig or tcpdump traces dont seem to
point to dropped messages, collisions, retransmissions
etc.


The hardware configuration is PowerPC based, and there
are no changes in the board specific IO subsystem between 
2.4.19 and 2.4.20/21-pre3. The same compiler is used for 
building both the kernels, and have tried this even with 
GCC 3.2, with same results. 

So, I dont suspect this is either board or compiler
related issue.


Also, I see some differences in handling of the bottom
halves in net/core/dev.c between 2.4.19 and 2.4.20/21-pre3.
Although, I have not gone through these in details to
assert that this is indeed the problem area.



Questions:

  - Has anyone seen this? Perhaps on other platforms (x86 etc)?
    Is there some tunable that has been added (or is different)
    after 2.4.19, and which needs to be tuned?


  - I have tried to enable kernel profiling to find any
    potential problem code areas. But given the low cpu
    utilization during these copies, I am not sure if this
    can give any useful info.

    Could anyone offer any ideas to debug this?



I would appreciate if you copy me on any responses to this post, I
dont subscribe to this list.

Best regards,
-Arun.
ADharankar@ATTBI.Com