Re: [PATCH v2] Receive Packet Steering

[David Miller <davem@davemloft.net>]

From: Tom Herbert <therbert@google.com>
Date: Mon, 13 Jul 2009 15:04:51 -0700

> On Mon, Jul 13, 2009 at 10:49 AM, David Miller <davem@davemloft.net> wrote:
>>
>> Why not take this to it's logical end point, which is to monitor
>> transmits using a tiny flow lookup table, and map receives of the same
>> flow to the same cpu?
> 
> Is it better do use transmits, or monitor where recvmsg was called?

First of all, using recvmsg would be on the more stateful side, and
we try to avoid that.  All of the logic should work if we had to
put it purely inside of dev_hard_xmit() and netif_receive_skb(),
way below the stack itself ever gets involved.

Second, Intel has committed the "monitor the TXs to device where to
steer RX" logic into silicon with their Flow Director stuff.  What
does that tell you about it's effectiveness? :-)

It seems the best heuristic to me, especially since it satisfies
many of the design constraints.

> Right.  In fact, just using the hash as the key is what you want when
> device provides the hash (i.e. Toeplitz).  The caveat is that we
> should to prevent OOO packets when threads migrate, I think I have a
> reasonable solution for that following your earlier suggestion.

But as you found, compared to jhash, Toeplitz is expensive to compute
and you would need to do this if implemented by monitoring transmits.

And furthermore, from the device perspective:

1) some (such as NIU) provide different hash values, not Toeplitz

2) it is expensive to extract this value (must enable different,
   larger, RX descriptor modes, thus more DMA traffic, etc.)

I really don't think Toeplitz is a net win, to be honest.

> I hope to have a new patch soon for steering packets to application
> CPU and using device hash, thanks for bugging me on it!

I had no choice, as I'm giving a presentation on this stuff tomorrow
night here in NYC :-)