Re: [PATCH net-next-2.6] net: speedup udp receive path

[jamal <hadi@cyberus.ca>]

On Thu, 2010-04-29 at 14:45 +0200, Eric Dumazet wrote:

> 
> Changli, I wonder how you can cook "performance" patches without testing
> them at all for real... This cannot be true ?

Eric, I am with you, however you are in the minority of people who test
and produce numbers ;-> The system rewards people for sending patches
not much for anything else - so i cant blame Changli ;->

> When the cpu doing the device softirq is flooded, it handles 300 packets
> per net_rx_action() round (netdev_budget), so sends at most 6 ipis per
> 300 packets, with or without my patch, with or without your patch as
> well.
> 
> (At most because if remote cpus are flooded as well, they dont
> napi_complete so no IPI needed at all)
>
> (My patch had an effect only on normal load, ie one packet received in a
> while... up to 50.000 pps I would say). And it also has a nice effect on
> non RPS loads (mostly the more typical load for following years).
> If a second packet comes 3us after the first one, and before 2nd CPU
> handled it, we _can_ afford an extra IPI.
> 
> 750.000/50 = 15.000 IPI per second.

Could we have some stat in there that shows IPIs being produced? I think
it would help to at least observe any changes over variety of tests.
I did try to patch my system during the first few tests to record IPIs
but it seems to make more sense to have it as a perf stat.

> Even with 200.000 IPI per second, 'perf top -C CPU_IPI_sender' shows
> that sending IPI is very cheap (maybe ~1% of cpu cycles)
> 
> # Samples: 32033467127
> #

One thing i observed is our profiles seem different. Could you send me
your .config for a single nehalem and i will try to go as close as
possible to it? I have a sky2 instead of bnx - but i suspect everything
else will be very similar...
I apologize i dont have much time to look into details - but what i can
do is test at least.

cheers,
jamal