Re: RPS will assign different smp_processor_id for the same packet?

[Tom Herbert <therbert@google.com>]

On Sat, Apr 23, 2011 at 8:31 AM, zhou rui <zhourui.cn@gmail.com> wrote:
> one more question is:
>
> in the function "int netif_receive_skb(struct sk_buff *skb)"
>
> cpu = get_rps_cpu(skb->dev, skb, &rflow);
> if (cpu >= 0) {
>  ret = enqueue_to_backlog(skb, cpu, &rflow->last_qtail);
> ....
>
> probably the cpu is different from the current processor id?(smp_processor_id)
> let's say: get_rps_cpu->cpu 0, smp_processor_id->cpu1
> when this happen, does it mean that cpu1 is handling the softirq but
> have to divert the packet to cpu0?(via a new softirq?)
>
> so for one packet it involve 2 softirqs?
>
> possible to get_rps_cpu in interrupt,then let the target cpu do only
> one softirq to hanle the packet?
>
Yes, this is what a non-NAPI driver would do.

Tom


> thanks
>
> On Fri, Apr 22, 2011 at 12:29 AM, zhou rui <zhourui.cn@gmail.com> wrote:
>> On Friday, April 22, 2011, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>>> Le jeudi 21 avril 2011 à 18:08 +0200, Eric Dumazet a écrit :
>>>> Le jeudi 21 avril 2011 à 23:50 +0800, zhou rui a écrit :
>>>> > kernel 2.6.36.4
>>>> > CONFIG_RPS=y but not set the cpu mask
>>>> >
>>>> > /sys/class/net/eth1/queues/rx-0 # cat rps_cpus
>>>> > 00
>>>> >
>>>> > register a hook func:
>>>> >   prot_hook.func = packet_rcv;
>>>> >   prot_hook.type = htons(ETH_P_ALL);
>>>> >   dev_add_pack(&prot_hook);
>>>> >
>>>> >
>>>> > replay the same traffic in very slow speed, printk the
>>>> > smp_processor_id in packet_rcv():
>>>> > first time:
>>>> > cpu=4
>>>> > cpu=3
>>>> > cpu=6
>>>> > cpu=7
>>>> >
>>>> > second time:
>>>> > cpu=7
>>>> > cpu=1
>>>> > cpu=5
>>>> > cpu=2
>>>> >
>>>> > is it normal?
>>>>
>>>> Yes it is.
>>>>
>>>> What would you expect ?
>>>>
>>>
>>> If rps_cpus contains only '0' bits, it basically means RPS is not active
>>> for this input queue.
>>>
>>> CPU is therefore not changed : The cpu handling NAPI on your network
>>> device directly calls upper linux stack.
>>>
>>> Seeing your traces, it also means your device spreads its interrupts on
>>> many different cpus, this might be not optimal.
>>>
>>> Check /proc/irq/{irq_number}/smp_affinity, it probably contains "ff"
>>>
>>>
>>>
>>>
>> Thanks,just saw this email
>>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>