Re: [PATCH v5] rfs: Receive Flow Steering

[Andi Kleen <andi@firstfloor.org>]

On Fri, Apr 16, 2010 at 10:05:15AM -0400, jamal wrote:
> On Fri, 2010-04-16 at 15:42 +0200, Andi Kleen wrote:
> > On Fri, Apr 16, 2010 at 09:32:06AM -0400, jamal wrote:
> > > How are you going to schedule the net softirq on an empty queue if you
> > > do this?
> > 
> > Sorry don't understand the question? 
> > 
> > You can always do the flow as if rps was not there.
> 
> Meaning you schedule the other side netrx softirq if queue is empty?

You handle the packet like if rps wasn't enabled. softirq on current
CPU and it queues it on the socket.

> > I meant an IPI to a sibling is not useful. You send it to the IPI
> > to get cache locality in the target, but if the target has the same
> > cache locality as you you can as well avoid the cost of the IPI
> > and process directly.
> > 
> 
> Isnt the purpose of the IPI to signal remote side that theres something
> for it to do? 

The current CPU can queue on that socket as well.

The whole point of the IPI is to do it with cache locality.
But if cache locality is already there on the current CPU you don't
need the IPI.

> Does it also sync the remote cache?

No, the caches are always coherent.

> 
> > For thread sibling I'm pretty sure it's useless. Not full sure about
> > socket sibling. Maybe.
> > 
> 
> Agreed, the SMT threads share L2. All the cores share L3. And it is
> inclusive, so if it is missing it is in L1 of one thread it must be
> present in L2 of shared cache as well as L3. Across the QPI i dont think
> that is true.
> But if you speacial case this - arent you being specific to Nehalem?

Other CPUs have SMT too (Niagara, POWER 6/7, mips, ...). It should
be the same there.

Assuming L3 affinity helps it might need to be a CPU specific tunable
yes. The scheduler has some information about this.

-Andi
-- 
ak@linux.intel.com -- Speaking for myself only.