Re: [PATCH v7] rps: Receive Packet Steering

[Eric Dumazet <eric.dumazet@gmail.com>]

Le mercredi 17 mars 2010 à 15:59 +0800, Changli Gao a écrit :
> On Wed, Mar 17, 2010 at 3:07 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> > Le mercredi 17 mars 2010 à 09:54 +0800, Changli Gao a écrit :
> >> On Wed, Mar 17, 2010 at 5:13 AM, David Miller <davem@davemloft.net> wrote:
> >> >
> >> > I'll integrate this as soon as I open up net-next-2.6
> >>
> >> It is really a good news. Now linux also can dispatch packets as
> >> FreeBSD does via netisr. Can we walk farer, and support weighted
> >> distribution?
> >>
> >
> > May I ask why ? What would be the goal ?
> >
> 
> For example, I have a firewall with dual core CPU, and use core 0 for
> IRQ and dispatching. If I use both core 0 and core 1 for the left
> processing, core 0 will be overloaded, and if I use core 1 for the
> left processing, core 0 will be light load. In order to take full of
> advantage of hardware, I need weighted the packet distribution.

I would not use RPS at all, weighted or not, unless your firewall must
perform heavy duty work (l7 or complex rules)

If the firewall setup is expensive, then IRQ processing has minor cost,
and RPS fits the bill (cpu handling IRQ will be a bit more loaded than
its buddy).

RPS is a win when _some_ TCP/UDP processing occurs, and we try to do
this processing on a cpu that will also run user space thread with data
in cpu cache (if process scheduler does a good job)
Not much applicable for routers...


Anyway, current sysfs RPS interface exposes
a /sys/class/net/eth0/queues/rx-0/rps_cpus bitmap,

I guess we could expose another file,
/sys/class/net/eth0/queues/rx-0/rps_map
to give different weight to cpus :

echo "0 1 0 1 0 1 1 1 1 1" >/sys/class/net/eth0/queues/rx-0/rps_map

cpu0 would get 30% of the postprocessing load, cpu1 70%

Using /sys/class/net/eth0/queues/rx-0/rps_cpus interface would give an
equal weight to each cpu :


# echo "0 1 0 1 0 1 1 1 1 1" >/sys/class/net/eth0/queues/rx-0/rps_map
# cat /sys/class/net/eth0/queues/rx-0/rps_cpus
3
# cat /sys/class/net/eth0/queues/rx-0/rps_map
0 1 0 1 0 1 1 1 1 1
# echo 3 >/sys/class/net/eth0/queues/rx-0/rps_cpus
# cat /sys/class/net/eth0/queues/rx-0/rps_map
0 1