Re: Network latency regressions from 2.6.22 to 2.6.29

[Eric Dumazet <dada1@cosmosbay.com>]

Eric Dumazet a écrit :
> Christoph Lameter a écrit :
>> The following are results of lantency measurements using udpping
>> (available from http://gentwo.org/ll). It shows that significant latencies
>> were added since 2.6.27. I surely wish we could get back to times below 90
>> microseconds.
>>
>> The tests were done over 1G ethernet using
>> 09:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5754
>> Gigabit Ethernet PCI Express (rev 02)
>>
>> Results:
>>
>>            2.6.22 2.6.23 2.6.24 2.6.25 2.6.26 2.6.27 2.6.28 2.6.29
>> 40 Bytes    89.50  90.75  89.61  91.51  91.89  99.17  99.80  99.34
>> 400 Bytes   98.58 101.44  97.85  99.61 100.36 117.96 118.10 126.79
>> 1400 Bytes 152.76 153.75 153.85	156.22 156.66 163.92 165.54 166.04
>>
>> Compared to 2.6.22 2.6.23 2.6.24 2.6.25 2.6.26 2.6.27 2.6.28 2.6.29
>> 40 Bytes            -1.4%  -0.1%  -2.2%  -2.6%  -9.8%  -10.3% -9.9%
>> 400 Bytes           -2.8%   0.7%  -1.0%  -1.8% -16.4%  -16.5%-22.2%
>> 1400 Bytes          -0.6%  -0.7%  -2.2%  -2.5%  -6.8%  -7.7%  -8.0%
>>
>> I presented these numbers with some nice graphs at the Linux Collab Summit
>> last week.
>>
>> See
>> http://www.kernel.org/pub/linux/kernel/people/christoph/collab-spring-2009/Collab-summit-2009-sf.pdf
>>
> 
> Is it a bnx2 or tg3 driver ?
> 
> copybreak feature could explain a regression for small packets, since
> an extra copy is done.
> 
> It would be sane to always set on both machines :
> ethtool -C eth0 rx-usecs 0 rx-frames 1
> prior to your tests...
> 

Also you might check /proc/interrupts, because recent linux versions are
distributing interrupts on all cpus instead of one on previous version

Old kernel :
# grep eth1 /proc/interrupts
 58:   25754498          0          0          0          0          0          0          0         PCI-MSI  eth1


New kernel :
# grep eth1 /proc/interrupts
221:   16476622   16478581   16478007   16478793   16478237   16472191   16474083   16476374   PCI-MSI-edge      eth1

(and /proc/irq/221/smp_affinity is not taken into account on some NICS)