Re: [RFC PATCH] net: add additional lock to qdisc to increase enqueue/dequeue fairness

Eric Dumazet
2010-03-23T20:54:31Z

Le mardi 23 mars 2010 à 13:25 -0700, Alexander Duyck a écrit :
> The qdisc layer shows a significant issue when you start transmitting from
> multiple CPUs. The issue is that the transmit rate drops significantly, and I
> believe it is due to the fact that the spinlock is shared between the 1
> dequeue, and n-1 enqueue cpu threads. In order to improve this situation I am
> adding one additional lock which will need to be obtained during the enqueue
> portion of the path. This essentially allows sch_direct_xmit to jump to
> near the head of the line when attempting to obtain the lock after
> completing a transmit.
>
> Running the script below I saw an increase from 200K packets per second to
> 1.07M packets per second as a result of this patch.
>
> for j in `seq 0 15`; do
> for i in `seq 0 7`; do
> netperf -H <ip> -t UDP_STREAM -l 600 -N -T $i -- -m 6 &
> done
> done
>
> Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
> ---
>

Hi Alexander

Thats a pretty good topic :)

So to speedup a pathological case (dozen of cpus all sending to same
queue), you suggest adding a spin_lock to fast path, slowing down normal
cases ?

Quite frankly, the real problem in this case is not the reduced
throughput, but fact that one cpu can stay a long time doing the xmits
to device, of skb queued by other cpus. This can hurt latencies a lot,
for real time threads for example...

I wonder if ticket spinlocks are not the problem. Maybe we want a
variant of spinlocks, so that cpu doing transmits can get the lock
before other cpus...