Re: rps: question

[Tom Herbert <therbert@google.com>]

On Sun, Feb 7, 2010 at 10:42 AM, jamal <hadi@cyberus.ca> wrote:
>
> Hi Tom,
>
> First off: Kudos on the numbers you are seeing; they are
> impressive. Do you have any numbers on a forwarding path test?
>
I don't have specific numbers, although we are using this on
application doing forwarding and numbers seem in line with what we see
for an end host.

> My first impression when i saw the numbers was one of suprise.
> Back in the days when we tried to split stack processing the way
> you did(it was one of the experiments on early NAPI), IPIs were
> _damn_ expensive. What changed in current architecture that makes
> this more palatable? IPIs are still synchronous AFAIK (and the more
> IPI receiver there are, the worse the ACK latency). Did you test this
> across other archs or say 3-4 year old machines?
>

No, the cost of the IPIs hasn't been an issue for us performance-wise.
 We are using them extensively-- up to one per core per device
interrupt.

We're calling __smp_call_function_single which is asynchronous in that
the caller provides the call structure and there is not waiting for
the IPI to complete.  A flag is used with each call structure that is
set when the IPI is in progress, this prevents simultaneous use of a
call structure.

I haven't seen any architectural specific issues with the IPIs, I
believe they are completing in < 2 usecs on platforms we're running
(some opteron systems that are over 3yrs old).

Tom

> cheers,
> jamal
>