Re: big picture UDP/IP performance question re 2.6.18 -> 2.6.32

[Eric Dumazet <eric.dumazet@gmail.com>]

Le mercredi 05 octobre 2011 à 02:58 -0400, starlight@binnacle.cx a
écrit :
> Final note:
> 
> I had captured latency measurements for
> two of the three kernels.  Just ran
> 2.6.18(rhel5) and the results are
> stunning.  The older kernel is much,
> much better then the newer kernel.
> 
> Average latency is three times better
> and the standard deviation is six
> time better.  As in 300% and 600%.
> 
> Latency here is the time it takes
> a packet to travel from the kernel
> (where it is timestamped) till it
> reaches the final consumption point
> in the application.
> 
> Makes me think that the old kernel
> is better at keeping caches hot and
> scheduling woken threads on the same
> cores as the threads that triggered
> them.
> 

Note :

Your results are from a combination of a user application and kernel
default strategies.

On other combinations, results can be completely different.

A wakeup strategy is somewhat tricky : 

- Should we affine or not.
- Should we queue the wakeup on a remote CPU, to keep scheduler data hot
in a single cpu cache.
- Should we use RPS/RFS to queue the packet to another CPU before  even
handling it in our stack, to keep network data hot in a single cpu
cache. (check Documentation/networking/scaling.txt)

At least, with recent kernels, we have many available choices to tune a
workload.