Re: [PATCH v2 net-next RFC] Generic XDP

[Daniel Borkmann <daniel@iogearbox.net>]

On 04/10/2017 08:39 PM, Andy Gospodarek wrote:
[...]
> I ran this on a desktop-class system I have (i7-6700 CPU @ 3.40GHz)
> and used pktgen_sample03_burst_single_flow.sh from another system to
> throw ~6.5Mpps as a single UDP stream towards the system running XDP.
>
> I just commented out the ndo_xdp op in bnxt.c to test this patch.  The
> differences were pretty dramatic.
>
> Using the ndo_xdp ndo in bnxt_en, my perf report output looked like
> this (not sure why X is still running on this system!):
>
>      38.08%  swapper          [sysimgblt]                   [k] 0x0000000000005bd0
>      11.80%  swapper          [kernel.vmlinux]              [k] intel_idle
>      10.49%  swapper          [bnxt_en]                     [k] bnxt_rx_pkt
>       6.31%  swapper          [bnxt_en]                     [k] bnxt_rx_xdp
>       5.64%  swapper          [bnxt_en]                     [k] bnxt_poll
>       4.22%  swapper          [kernel.vmlinux]              [k] poll_idle
>       3.46%  swapper          [kernel.vmlinux]              [k] irq_entries_start
>       2.95%  swapper          [kernel.vmlinux]              [k] napi_complete_done
>       1.79%  swapper          [kernel.vmlinux]              [k] cpuidle_enter_state
>       1.53%  swapper          [kernel.vmlinux]              [k] menu_select
>       1.19%  swapper          [bnxt_en]                     [k] bnxt_reuse_rx_data
>       1.00%  swapper          [sysimgblt]                   [k] 0x0000000000005c6f
>       0.92%  swapper          [kernel.vmlinux]              [k] __next_timer_interrupt
>       0.71%  swapper          [kernel.vmlinux]              [k] _raw_spin_lock_irqsave
>       0.71%  swapper          [kernel.vmlinux]              [k] bpf_map_lookup_elem
>
> mpstat reports that the CPU receiving and dropping the traffic is
> basically running idle.  Dropping this amount of traffic in the driver
> has very little impact on the system.
>
> With v2 of this patch I see the following from perf report:
>
>      19.69%  ksoftirqd/3      [kernel.vmlinux]              [k] memcpy_erms
>      16.30%  ksoftirqd/3      [kernel.vmlinux]              [k] __bpf_prog_run

Forgot echo 1 > /proc/sys/net/core/bpf_jit_enable? Was it disabled in both cases?

>      10.11%  ksoftirqd/3      [bnxt_en]                     [k] bnxt_rx_pkt
>       7.69%  ksoftirqd/3      [kernel.vmlinux]              [k] __build_skb
>       4.25%  ksoftirqd/3      [kernel.vmlinux]              [k] inet_gro_receive
>       3.74%  ksoftirqd/3      [kernel.vmlinux]              [k] kmem_cache_alloc
>       3.53%  ksoftirqd/3      [kernel.vmlinux]              [k] dev_gro_receive
>       3.43%  ksoftirqd/3      [kernel.vmlinux]              [k] page_frag_free
>       3.12%  ksoftirqd/3      [kernel.vmlinux]              [k] kmem_cache_free
>       2.56%  ksoftirqd/3      [bnxt_en]                     [k] bnxt_poll
>       2.46%  ksoftirqd/3      [kernel.vmlinux]              [k] netif_receive_skb_internal
>       2.13%  ksoftirqd/3      [kernel.vmlinux]              [k] __udp4_lib_lookup
>       1.63%  ksoftirqd/3      [kernel.vmlinux]              [k] __napi_alloc_skb
>       1.51%  ksoftirqd/3      [kernel.vmlinux]              [k] eth_type_trans
>       1.42%  ksoftirqd/3      [kernel.vmlinux]              [k] udp_gro_receive
>       1.29%  ksoftirqd/3      [kernel.vmlinux]              [k] napi_gro_receive
>       1.25%  ksoftirqd/3      [kernel.vmlinux]              [k] udp4_gro_receive
>       1.18%  ksoftirqd/3      [bnxt_en]                     [k] bnxt_rx_xdp
>       1.17%  ksoftirqd/3      [kernel.vmlinux]              [k] skb_release_data
>       1.11%  ksoftirqd/3      [bnxt_en]                     [k] bnxt_reuse_rx_data
>       1.07%  ksoftirqd/3      [kernel.vmlinux]              [k] net_rx_action