Re: [PATCH v7] rps: Receive Packet Steering

[Tom Herbert <therbert@google.com>]

On Wed, Mar 17, 2010 at 7:09 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Le mercredi 17 mars 2010 à 15:59 +0800, Changli Gao a écrit :
>> On Wed, Mar 17, 2010 at 3:07 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>> > Le mercredi 17 mars 2010 à 09:54 +0800, Changli Gao a écrit :
>> >> On Wed, Mar 17, 2010 at 5:13 AM, David Miller <davem@davemloft.net> wrote:
>> >> >
>> >> > I'll integrate this as soon as I open up net-next-2.6
>> >>
>> >> It is really a good news. Now linux also can dispatch packets as
>> >> FreeBSD does via netisr. Can we walk farer, and support weighted
>> >> distribution?
>> >>
>> >
>> > May I ask why ? What would be the goal ?
>> >
>>
>> For example, I have a firewall with dual core CPU, and use core 0 for
>> IRQ and dispatching. If I use both core 0 and core 1 for the left
>> processing, core 0 will be overloaded, and if I use core 1 for the
>> left processing, core 0 will be light load. In order to take full of
>> advantage of hardware, I need weighted the packet distribution.
>
> I would not use RPS at all, weighted or not, unless your firewall must
> perform heavy duty work (l7 or complex rules)
>
> If the firewall setup is expensive, then IRQ processing has minor cost,
> and RPS fits the bill (cpu handling IRQ will be a bit more loaded than
> its buddy).
>
> RPS is a win when _some_ TCP/UDP processing occurs, and we try to do
> this processing on a cpu that will also run user space thread with data
> in cpu cache (if process scheduler does a good job)
> Not much applicable for routers...
>
>
> Anyway, current sysfs RPS interface exposes
> a /sys/class/net/eth0/queues/rx-0/rps_cpus bitmap,
>
> I guess we could expose another file,
> /sys/class/net/eth0/queues/rx-0/rps_map
> to give different weight to cpus :
>
> echo "0 1 0 1 0 1 1 1 1 1" >/sys/class/net/eth0/queues/rx-0/rps_map
>
> cpu0 would get 30% of the postprocessing load, cpu1 70%
>
> Using /sys/class/net/eth0/queues/rx-0/rps_cpus interface would give an
> equal weight to each cpu :
>
>
> # echo "0 1 0 1 0 1 1 1 1 1" >/sys/class/net/eth0/queues/rx-0/rps_map
> # cat /sys/class/net/eth0/queues/rx-0/rps_cpus
> 3
> # cat /sys/class/net/eth0/queues/rx-0/rps_map
> 0 1 0 1 0 1 1 1 1 1
> # echo 3 >/sys/class/net/eth0/queues/rx-0/rps_cpus
> # cat /sys/class/net/eth0/queues/rx-0/rps_map
> 0 1

Alternatively, the rps_map could be specified explicitly, which will
allow weighting.  For example "0 0 0 0 2 10 10 10"  would select CPUs
0, 2, 10 for the map with weights four, one, and three respectively.
This would go back to have sysfs files with multiple values in them,
so it might not be the right interface.

Tom