Re: [PATCH v2] Receive Packet Steering

[Tom Herbert <therbert@google.com>]

On Mon, Jul 13, 2009 at 10:49 AM, David Miller <davem@davemloft.net> wrote:
>
> From: Tom Herbert <therbert@google.com>
> Date: Sun, 3 May 2009 21:03:01 -0700
>
> > @@ -758,6 +758,8 @@ struct net_device
> >       void                    *ax25_ptr;      /* AX.25 specific data */
> >       struct wireless_dev     *ieee80211_ptr; /* IEEE 802.11 specific data,
> >                                                  assign before registering */
> > +     u16                     *rps_map;
> > +     int                     rps_map_len;
> >
> >  /*
> >   * Cache line mostly used on receive path (including eth_type_trans())
>
> So essentially this table is a user defined (via sysctl) group of cpus
> among which to distribute incoming traffic for a device, right?

Yes.  It's a simple static table.

>
> Why not take this to it's logical end point, which is to monitor
> transmits using a tiny flow lookup table, and map receives of the same
> flow to the same cpu?

Is it better do use transmits, or monitor where recvmsg was called?

>
> You can even "cheat" and not store the whole flow key in the small
> lookup table, just use the resulting hash value as the key.  Also,
> if "best effort" is considered OK you can even do away with hash
> chaining as well, further decreasing the space cost of the table.

Right.  In fact, just using the hash as the key is what you want when
device provides the hash (i.e. Toeplitz).  The caveat is that we
should to prevent OOO packets when threads migrate, I think I have a
reasonable solution for that following your earlier suggestion.

I hope to have a new patch soon for steering packets to application
CPU and using device hash, thanks for bugging me on it!

>
> If your goal is to steer traffic to the cpu on which the receiving
> application is operating, that seems to me to be the only way to
> reliably and consistently hit that target.