Re: [RFC][PATCH 2/2] bgmac: pass received packet to the netif instead of copying it

[Felix Fietkau <nbd@openwrt.org>]

On 2013-08-15 10:21 PM, Rafał Miłecki wrote:
> 2013/8/15 Felix Fietkau <nbd@openwrt.org>:
>> On 2013-08-15 1:36 PM, Rafał Miłecki wrote:
>>> 2013/8/11 Rafał Miłecki <zajec5@gmail.com>:
>>>> It makes more sense to allocate new (empty) skb and pass it to the
>>>> hardware. That way we avoid copying whole packet into new skb which
>>>> should result in better performance.
>>>
>>> I did some testing of this patch using "perf" tool and iperf -s
>>> running on the OpenWrt machine (with bgmac supported hardware).
>>>
>>> So you can see that __copy_user_common usage has really decreased with
>>> this patch!
>>>
>>> Unfortunately it didn't result in better performance... no idea why :(
>> Running iperf on the router is not useful as an indicator of routing
>> performance. Please focus on tests where you only push traffic through
>> the router, not directly to it.
> 
> OK, so I started "iperf -s" on notebook plugged into WAN port, and
> then played with "iperf -c" on notebook connected to LAN#2.
> 
> With some old 3.6.11 based OpenWrt build I got:
> [ 4] 0.0-60.0 sec 690 MBytes 96.4 Mbits/sec
> 
> With very recent 3.10.4 based OpenWrt build:
> [ 4] 0.0-60.0 sec 667 MBytes 93.2 Mbits/sec
> 
> After applying my patch on top of that 3.10.4:
> [ 5] 0.0-60.0 sec 759 MBytes 106 Mbits/sec
> 
> And some dumps from "perf top":
> 
> 3.10.4
> 6.75% [kernel] [k] __copy_user_common
> 6.73% [ip_tables] [k] ipt_do_table
> 4.33% [kernel] [k] arch_cpu_idle
> 3.96% [kernel] [k] arch_local_irq_restore
> 3.42% [bgmac] [k] 0x000007e0
> 3.35% [nf_conntrack] [k] nf_conntrack_proto_fini
> 2.72% [nf_conntrack] [k] nf_conntrack_in
> 2.50% [kernel] [k] __netif_receive_skb_core
> 2.42% [kernel] [k] r4k_dma_cache_inv
> 2.38% [kernel] [k] fib_table_lookup
> 2.20% [kernel] [k] dev_queue_xmit
> 2.11% [xt_conntrack] [k] 0x00000360
> 2.10% [kernel] [k] ip_route_input_noref
> 2.06% [nf_conntrack_ipv4] [k] need_ipv4_conntrack
> 
> 3.10.4 + 0002-bgmac-pass-received-packet-to-the-netif-instead-of-c.patch
> 6.09% [ip_tables] [k] ipt_do_table
> 4.71% [kernel] [k] arch_cpu_idle
> 4.48% [bgmac] [k] 0x00000d7c
> 3.50% [nf_conntrack] [k] nf_conntrack_in
> 3.22% [kernel] [k] arch_local_irq_restore
> 3.16% [nf_conntrack] [k] nf_conntrack_proto_fini
> 2.88% [kernel] [k] __netif_receive_skb_core
> 2.78% [xt_conntrack] [k] 0x0000011c
> 2.69% [kernel] [k] r4k_dma_cache_inv
> 2.67% [iptable_nat] [k] 0x000002a0
> 2.36% [kernel] [k] ip_route_input_noref
> 2.27% [kernel] [k] ip_rcv
> 2.25% [nf_conntrack_ipv4] [k] need_ipv4_conntrack
> 2.23% [kernel] [k] nf_iterate
> 
> I've compiled bgmac into the kernel and it seems that the magic 0xd7c
> was bgmac_poll.
> 
> I'm afraid this "perf top" output doesn't really tell us where to look
> for optimizations :| I'll still try Felix ideas tomorrow, but I'm not
> sure if they help, since there isn't __copy_user_common anymore in the
> "perf top" output...
What's the CPU load while passing traffic without running perf?
Have you tested bridging performance?

- Felix
_______________________________________________
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel