Re: latency doubled on tg3 device from 2.6.11 to 2.6.12

[Eric Dumazet <dada1@cosmosbay.com>]

Rick Warner a écrit :
> Hello,
>  We have been testing latency and bandwidth using our proprietary MPI link 
> checker tool (http://www.microway.com/mpilinkchecker.html) and have found 
> that the latency increased from ~25ms to ~45ms going from 2.6.11 to 2.6.12.  
> 2.6.13 has the same result.  We also tried the latest bcm5700 from broadcom 
> (8.2.18) and got the same ~45ms latencies.  This was tried on several 
> different opteron and em64t motherboards.
> 
>  We see 20-25ms latencies with the e1000 driver (with some module options) on 
> all 3 kernel versions.  For those interested, the e1000 options used are:
> 
>  InterruptThrottleRate=0 RxIntDelay=0 TxIntDelay=0 RxAbsIntDelay=0 
> TxAbsIntDelay=0
> 
>  Digging through source, it seems that a new locking mechanism for tg3 was put 
> in place in 2.6.12.  Is this the likely cause?  What can we do to restore our 
> lower latency?
> 
> 

Could you please define latency ?

tg3 driver was recently updated to use coalescing.

So when the nic receives one frame, it may delay up to XXXX us ( XXXX < 1024) 
the interrupt.

But 25 ms is far more than 1024 us, so I dont think this coalescing can 
explain your problem.

The HZ change from 1000 to 250 could be the root of the problem ?

Using a simple ping between 2 machines with tg3, I get less than 1ms time.

Eric