* Re: [PATCH resend] macvlan: use per-cpu queues for broadcast and multicast packets
From: David Miller @ 2018-11-01 18:53 UTC (permalink / raw)
To: khlebnikov; +Cc: netdev, linux-kernel, vfedorenko
In-Reply-To: <154108883150.919735.10620997593741055347.stgit@buzz>
From: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Date: Thu, 01 Nov 2018 19:13:51 +0300
> Currently macvlan has single per-port queue for broadcast and multicast.
> This disrupts order of packets when flows from different cpus are mixed.
>
> This patch replaces this queue with single set of per-cpu queues.
> Pointer to macvlan port is passed in skb control block.
>
> Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
> Reported-add-tested-by: Vadim Fedorenko <vfedorenko@yandex-team.ru>
Konstantin, all of your postings to vger.kernel.org are being rejected.
They are because of the:
"List-Unsubscribe: <https://ml.yandex-team.ru/lists/vfedorenko@yandex-team.ru/unsubscribe-click>"
and similar strings that appear in your email headers.
Please disable whatever is producing these email headers, otherwise
nobody (including me) will see and process your postings.
Thank you.
^ permalink raw reply
* Re: Kernel 4.19 network performance - forwarding/routing normal users traffic
From: Saeed Mahameed @ 2018-11-01 9:50 UTC (permalink / raw)
To: pstaszewski@itcare.pl, netdev@vger.kernel.org
In-Reply-To: <61697e49-e839-befc-8330-fc00187c48ee@itcare.pl>
On Wed, 2018-10-31 at 22:57 +0100, Paweł Staszewski wrote:
> Hi
>
> So maybee someone will be interested how linux kernel handles normal
> traffic (not pktgen :) )
>
>
> Server HW configuration:
>
> CPU : Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz
>
> NIC's: 2x 100G Mellanox ConnectX-4 (connected to x16 pcie 8GT)
>
>
> Server software:
>
> FRR - as routing daemon
>
> enp175s0f0 (100G) - 16 vlans from upstreams (28 RSS binded to local
> numa
> node)
>
> enp175s0f1 (100G) - 343 vlans to clients (28 RSS binded to local numa
> node)
>
>
> Maximum traffic that server can handle:
>
> Bandwidth
>
> bwm-ng v0.6.1 (probing every 1.000s), press 'h' for help
> input: /proc/net/dev type: rate
> \ iface Rx Tx Total
> =====================================================================
> =========
> enp175s0f1: 28.51 Gb/s 37.24
> Gb/s
> 65.74 Gb/s
> enp175s0f0: 38.07 Gb/s 28.44
> Gb/s
> 66.51 Gb/s
> -------------------------------------------------------------------
> -----------
> total: 66.58 Gb/s 65.67
> Gb/s
> 132.25 Gb/s
>
>
> Packets per second:
>
> bwm-ng v0.6.1 (probing every 1.000s), press 'h' for help
> input: /proc/net/dev type: rate
> - iface Rx Tx Total
> =====================================================================
> =========
> enp175s0f1: 5248589.00 P/s 3486617.75 P/s
> 8735207.00 P/s
> enp175s0f0: 3557944.25 P/s 5232516.00 P/s
> 8790460.00 P/s
> -------------------------------------------------------------------
> -----------
> total: 8806533.00 P/s 8719134.00 P/s
> 17525668.00 P/s
>
>
> After reaching that limits nics on the upstream side (more RX
> traffic)
> start to drop packets
>
>
> I just dont understand that server can't handle more bandwidth
> (~40Gbit/s is limit where all cpu's are 100% util) - where pps on RX
> side are increasing.
>
Where do you see 40 Gb/s ? you showed that both ports on the same NIC (
same pcie link) are doing 66.58 Gb/s (RX) + 65.67 Gb/s (TX) = 132.25
Gb/s which aligns with your pcie link limit, what am i missing ?
> Was thinking that maybee reached some pcie x16 limit - but x16 8GT
> is
> 126Gbit - and also when testing with pktgen i can reach more bw and
> pps
> (like 4x more comparing to normal internet traffic)
>
Are you forwarding when using pktgen as well or you just testing the RX
side pps ?
> And wondering if there is something that can be improved here.
>
>
>
> Some more informations / counters / stats and perf top below:
>
> Perf top flame graph:
>
> https://uploadfiles.io/7zo6u
>
>
>
> System configuration(long):
>
>
> cat /sys/devices/system/node/node1/cpulist
> 14-27,42-55
> cat /sys/class/net/enp175s0f0/device/numa_node
> 1
> cat /sys/class/net/enp175s0f1/device/numa_node
> 1
>
>
>
>
>
> ip -s -d link ls dev enp175s0f0
> 6: enp175s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq
> state
> UP mode DEFAULT group default qlen 8192
> link/ether 0c:c4:7a:d8:5d:1c brd ff:ff:ff:ff:ff:ff promiscuity
> 0
> addrgenmode eui64 numtxqueues 448 numrxqueues 56 gso_max_size 65536
> gso_max_segs 65535
> RX: bytes packets errors dropped overrun mcast
> 184142375840858 141347715974 2 2806325 0 85050528
> TX: bytes packets errors dropped carrier collsns
> 99270697277430 172227994003 0 0 0 0
>
> ip -s -d link ls dev enp175s0f1
> 7: enp175s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq
> state
> UP mode DEFAULT group default qlen 8192
> link/ether 0c:c4:7a:d8:5d:1d brd ff:ff:ff:ff:ff:ff promiscuity
> 0
> addrgenmode eui64 numtxqueues 448 numrxqueues 56 gso_max_size 65536
> gso_max_segs 65535
> RX: bytes packets errors dropped overrun mcast
> 99686284170801 173507590134 61 669685 0 100304421
> TX: bytes packets errors dropped carrier collsns
> 184435107970545 142383178304 0 0 0 0
>
>
> ./softnet.sh
> cpu total dropped squeezed collision rps flow_limit
> 0 3961392822 0 1221478 0 0 0
> 1 3701952251 0 1258234 0 0 0
> 2 3879522030 0 1584282 0 0 0
> 3 3731349789 0 1529029 0 0 0
> 4 1323956701 0 2176371 0 0 0
> 5 420528963 0 1880146 0 0 0
> 6 348720322 0 1830142 0 0 0
> 7 372736328 0 1820891 0 0 0
> 8 567888751 0 1414763 0 0 0
> 9 476075775 0 1868150 0 0 0
> 10 468946725 0 1841428 0 0 0
> 11 676591958 0 1900160 0 0 0
> 12 346803472 0 1834600 0 0 0
> 13 457960872 0 1874529 0 0 0
> 14 1990279665 0 4699000 0 0 0
> 15 1211873601 0 4541281 0 0 0
> 16 1123871928 0 4544712 0 0 0
> 17 1014957263 0 4152355 0 0 0
> 18 2603779724 0 4593869 0 0 0
> 19 2181924054 0 4930618 0 0 0
> 20 2273502182 0 4894627 0 0 0
> 21 2232030947 0 4860048 0 0 0
> 22 2203555394 0 4603830 0 0 0
> 23 2194756800 0 4921294 0 0 0
> 24 2347158294 0 4818354 0 0 0
> 25 2291097883 0 4744469 0 0 0
> 26 2206945011 0 4836483 0 0 0
> 27 2318530217 0 4917617 0 0 0
> 28 512797543 0 1895200 0 0 0
> 29 597279474 0 1532134 0 0 0
> 30 475317503 0 1451523 0 0 0
> 31 499172796 0 1901207 0 0 0
> 32 493874745 0 1915382 0 0 0
> 33 296056288 0 1865535 0 0 0
> 34 3905097041 0 1580822 0 0 0
> 35 3905112345 0 1536105 0 0 0
> 36 3900358950 0 1166319 0 0 0
> 37 3940978093 0 1600219 0 0 0
> 38 3878632215 0 1180389 0 0 0
> 39 3814804736 0 1584925 0 0 0
> 40 4152934337 0 1663660 0 0 0
> 41 3855273904 0 1552219 0 0 0
> 42 2319538182 0 4884480 0 0 0
> 43 2448606991 0 4387456 0 0 0
> 44 1436136753 0 4485073 0 0 0
> 45 1200500141 0 4537284 0 0 0
> 46 1307799923 0 4534156 0 0 0
> 47 1586575293 0 4272997 0 0 0
> 48 3852574 0 4162653 0 0 0
> 49 391449390 0 3935202 0 0 0
> 50 791388200 0 4290738 0 0 0
> 51 127107573 0 3907750 0 0 0
> 52 115622148 0 4012843 0 0 0
> 53 71098871 0 4200625 0 0 0
> 54 305121466 0 4365614 0 0 0
> 55 10914257 0 4369426 0 0 0
>
>
>
>
> PerfTop: 108490 irqs/sec kernel:99.6% exact: 0.0% [4000Hz
> cycles], (all, 56 CPUs)
> -------------------------------------------------------------------
> -------------------------------------------------------------------
> -------------------------------------------------------------------
> ------
>
> 26.78% [kernel] [k] queued_spin_lock_slowpath
> 9.09% [kernel] [k] mlx5e_skb_from_cqe_linear
> 4.94% [kernel] [k] mlx5e_sq_xmit
> 3.63% [kernel] [k] memcpy_erms
> 3.30% [kernel] [k] fib_table_lookup
> 3.26% [kernel] [k] build_skb
> 2.41% [kernel] [k] mlx5e_poll_tx_cq
> 2.11% [kernel] [k] get_page_from_freelist
> 1.51% [kernel] [k] vlan_do_receive
> 1.51% [kernel] [k] _raw_spin_lock
> 1.43% [kernel] [k] __dev_queue_xmit
> 1.41% [kernel] [k] dev_gro_receive
> 1.34% [kernel] [k] mlx5e_poll_rx_cq
> 1.26% [kernel] [k] tcp_gro_receive
> 1.21% [kernel] [k] free_one_page
> 1.13% [kernel] [k] swiotlb_map_page
> 1.13% [kernel] [k] mlx5e_post_rx_wqes
> 1.05% [kernel] [k] pfifo_fast_dequeue
> 1.05% [kernel] [k] mlx5e_handle_rx_cqe
> 1.03% [kernel] [k] ip_finish_output2
> 1.02% [kernel] [k] ipt_do_table
> 0.96% [kernel] [k] inet_gro_receive
> 0.91% [kernel] [k] mlx5_eq_int
> 0.88% [kernel] [k] __slab_free.isra.79
> 0.86% [kernel] [k] __build_skb
> 0.84% [kernel] [k] page_frag_free
> 0.76% [kernel] [k] skb_release_data
> 0.75% [kernel] [k] __netif_receive_skb_core
> 0.75% [kernel] [k] irq_entries_start
> 0.71% [kernel] [k] ip_route_input_rcu
> 0.65% [kernel] [k] vlan_dev_hard_start_xmit
> 0.56% [kernel] [k] ip_forward
> 0.56% [kernel] [k] __memcpy
> 0.52% [kernel] [k] kmem_cache_alloc
> 0.52% [kernel] [k] kmem_cache_free_bulk
> 0.49% [kernel] [k] mlx5e_page_release
> 0.47% [kernel] [k] netif_skb_features
> 0.47% [kernel] [k] mlx5e_build_rx_skb
> 0.47% [kernel] [k] dev_hard_start_xmit
> 0.43% [kernel] [k] __page_pool_put_page
> 0.43% [kernel] [k] __netif_schedule
> 0.43% [kernel] [k] mlx5e_xmit
> 0.41% [kernel] [k] __qdisc_run
> 0.41% [kernel] [k] validate_xmit_skb.isra.142
> 0.41% [kernel] [k] swiotlb_unmap_page
> 0.40% [kernel] [k] inet_lookup_ifaddr_rcu
> 0.34% [kernel] [k] ip_rcv_core.isra.20.constprop.25
> 0.34% [kernel] [k] tcp4_gro_receive
> 0.29% [kernel] [k] _raw_spin_lock_irqsave
> 0.29% [kernel] [k] napi_consume_skb
> 0.29% [kernel] [k] skb_gro_receive
> 0.29% [kernel] [k] ___slab_alloc.isra.80
> 0.27% [kernel] [k] eth_type_trans
> 0.26% [kernel] [k] __free_pages_ok
> 0.26% [kernel] [k] __get_xps_queue_idx
> 0.24% [kernel] [k] _raw_spin_trylock
> 0.23% [kernel] [k] __local_bh_enable_ip
> 0.22% [kernel] [k] pfifo_fast_enqueue
> 0.21% [kernel] [k] tasklet_action_common.isra.21
> 0.21% [kernel] [k] sch_direct_xmit
> 0.21% [kernel] [k] skb_network_protocol
> 0.21% [kernel] [k] kmem_cache_free
> 0.20% [kernel] [k] netdev_pick_tx
> 0.18% [kernel] [k] napi_gro_complete
> 0.18% [kernel] [k] __sched_text_start
> 0.18% [kernel] [k] mlx5e_xdp_handle
> 0.17% [kernel] [k] ip_finish_output
> 0.16% [kernel] [k] napi_gro_flush
> 0.16% [kernel] [k] vlan_passthru_hard_header
> 0.16% [kernel] [k] skb_segment
> 0.15% [kernel] [k] __alloc_pages_nodemask
> 0.15% [kernel] [k] mlx5e_features_check
> 0.15% [kernel] [k] mlx5e_napi_poll
> 0.15% [kernel] [k] napi_gro_receive
> 0.14% [kernel] [k] fib_validate_source
> 0.14% [kernel] [k] _raw_spin_lock_irq
> 0.14% [kernel] [k] inet_gro_complete
> 0.14% [kernel] [k] get_partial_node.isra.78
> 0.13% [kernel] [k] napi_complete_done
> 0.13% [kernel] [k] ip_rcv_finish_core.isra.17
> 0.13% [kernel] [k] cmd_exec
>
>
>
> ethtool -S enp175s0f1
> NIC statistics:
> rx_packets: 173730800927
> rx_bytes: 99827422751332
> tx_packets: 142532009512
> tx_bytes: 184633045911222
> tx_tso_packets: 25989113891
> tx_tso_bytes: 132933363384458
> tx_tso_inner_packets: 0
> tx_tso_inner_bytes: 0
> tx_added_vlan_packets: 74630239613
> tx_nop: 2029817748
> rx_lro_packets: 0
> rx_lro_bytes: 0
> rx_ecn_mark: 0
> rx_removed_vlan_packets: 173730800927
> rx_csum_unnecessary: 0
> rx_csum_none: 434357
> rx_csum_complete: 173730366570
> rx_csum_unnecessary_inner: 0
> rx_xdp_drop: 0
> rx_xdp_redirect: 0
> rx_xdp_tx_xmit: 0
> rx_xdp_tx_full: 0
> rx_xdp_tx_err: 0
> rx_xdp_tx_cqe: 0
> tx_csum_none: 38260960853
> tx_csum_partial: 36369278774
> tx_csum_partial_inner: 0
> tx_queue_stopped: 1
> tx_queue_dropped: 0
> tx_xmit_more: 748638099
> tx_recover: 0
> tx_cqes: 73881645031
> tx_queue_wake: 1
> tx_udp_seg_rem: 0
> tx_cqe_err: 0
> tx_xdp_xmit: 0
> tx_xdp_full: 0
> tx_xdp_err: 0
> tx_xdp_cqes: 0
> rx_wqe_err: 0
> rx_mpwqe_filler_cqes: 0
> rx_mpwqe_filler_strides: 0
> rx_buff_alloc_err: 0
> rx_cqe_compress_blks: 0
> rx_cqe_compress_pkts: 0
If this is a pcie bottleneck it might be useful to enable CQE
compression (to reduce PCIe completion descriptors transactions)
you should see the above rx_cqe_compress_pkts increasing when enabled.
$ ethtool --set-priv-flags enp175s0f1 rx_cqe_compress on
$ ethtool --show-priv-flags enp175s0f1
Private flags for p6p1:
rx_cqe_moder : on
cqe_moder : off
rx_cqe_compress : on
...
try this on both interfaces.
> rx_page_reuse: 0
> rx_cache_reuse: 14441066823
> rx_cache_full: 51126004413
> rx_cache_empty: 21297344082
> rx_cache_busy: 51127247487
> rx_cache_waive: 21298322293
> rx_congst_umr: 0
> rx_arfs_err: 0
> ch_events: 24603119858
> ch_poll: 25180949074
> ch_arm: 24480437587
> ch_aff_change: 75
> ch_eq_rearm: 0
> rx_out_of_buffer: 669685
comparing this to rx_vport_unicast_packets, it is a very small
percentage of dropped packets due to stalled rx cpu, so rx cpu is not a
bottleneck, at least for the driver rx rings.
> rx_if_down_packets: 61
> rx_vport_unicast_packets: 173731641945
> rx_vport_unicast_bytes: 100522745036693
> tx_vport_unicast_packets: 142531901313
> tx_vport_unicast_bytes: 185189071776429
> rx_vport_multicast_packets: 100360886
> rx_vport_multicast_bytes: 6639236688
> tx_vport_multicast_packets: 32837
> tx_vport_multicast_bytes: 2978810
> rx_vport_broadcast_packets: 44854
> rx_vport_broadcast_bytes: 6313510
> tx_vport_broadcast_packets: 72258
> tx_vport_broadcast_bytes: 4335480
> rx_vport_rdma_unicast_packets: 0
> rx_vport_rdma_unicast_bytes: 0
> tx_vport_rdma_unicast_packets: 0
> tx_vport_rdma_unicast_bytes: 0
> rx_vport_rdma_multicast_packets: 0
> rx_vport_rdma_multicast_bytes: 0
> tx_vport_rdma_multicast_packets: 0
> tx_vport_rdma_multicast_bytes: 0
> tx_packets_phy: 142532004669
> rx_packets_phy: 173980375752
> rx_crc_errors_phy: 0
> tx_bytes_phy: 185759204762903
> rx_bytes_phy: 101326109361379
> tx_multicast_phy: 32837
> tx_broadcast_phy: 72258
> rx_multicast_phy: 100360885
> rx_broadcast_phy: 44854
> rx_in_range_len_errors_phy: 2
> rx_out_of_range_len_phy: 0
> rx_oversize_pkts_phy: 59
> rx_symbol_err_phy: 0
> tx_mac_control_phy: 0
> rx_mac_control_phy: 0
> rx_unsupported_op_phy: 0
> rx_pause_ctrl_phy: 0
> tx_pause_ctrl_phy: 0
> rx_discards_phy: 148328738
> tx_discards_phy: 0
> tx_errors_phy: 0
> rx_undersize_pkts_phy: 0
> rx_fragments_phy: 0
> rx_jabbers_phy: 0
> rx_64_bytes_phy: 36551843112
> rx_65_to_127_bytes_phy: 65102131735
> rx_128_to_255_bytes_phy: 5755731137
> rx_256_to_511_bytes_phy: 2475619839
> rx_512_to_1023_bytes_phy: 2826971156
> rx_1024_to_1518_bytes_phy: 42474023107
> rx_1519_to_2047_bytes_phy: 18794051270
> rx_2048_to_4095_bytes_phy: 0
> rx_4096_to_8191_bytes_phy: 0
> rx_8192_to_10239_bytes_phy: 0
> link_down_events_phy: 0
> rx_pcs_symbol_err_phy: 0
> rx_corrected_bits_phy: 0
> rx_pci_signal_integrity: 0
> tx_pci_signal_integrity: 48
> rx_prio0_bytes: 101316322498995
> rx_prio0_packets: 173711151686
> tx_prio0_bytes: 185759176566814
> tx_prio0_packets: 142531983704
> rx_prio1_bytes: 47062768
> rx_prio1_packets: 228932
> tx_prio1_bytes: 0
> tx_prio1_packets: 0
> rx_prio2_bytes: 12434759
> rx_prio2_packets: 83773
> tx_prio2_bytes: 0
> tx_prio2_packets: 0
> rx_prio3_bytes: 288843134
> rx_prio3_packets: 982102
> tx_prio3_bytes: 0
> tx_prio3_packets: 0
> rx_prio4_bytes: 699797236
> rx_prio4_packets: 8109231
> tx_prio4_bytes: 0
> tx_prio4_packets: 0
> rx_prio5_bytes: 1385386738
> rx_prio5_packets: 9661187
> tx_prio5_bytes: 0
> tx_prio5_packets: 0
> rx_prio6_bytes: 317092102
> rx_prio6_packets: 1951538
> tx_prio6_bytes: 0
> tx_prio6_packets: 0
> rx_prio7_bytes: 7015734695
> rx_prio7_packets: 99847456
> tx_prio7_bytes: 0
> tx_prio7_packets: 0
> module_unplug: 0
> module_bus_stuck: 0
> module_high_temp: 0
> module_bad_shorted: 0
> ch0_events: 936264703
> ch0_poll: 963766474
> ch0_arm: 930246079
> ch0_aff_change: 0
> ch0_eq_rearm: 0
> ch1_events: 869408429
> ch1_poll: 896099392
> ch1_arm: 864336861
> ch1_aff_change: 0
> ch1_eq_rearm: 0
> ch2_events: 843345698
> ch2_poll: 869749522
> ch2_arm: 838186113
> ch2_aff_change: 2
> ch2_eq_rearm: 0
> ch3_events: 850261340
> ch3_poll: 876721111
> ch3_arm: 845295235
> ch3_aff_change: 3
> ch3_eq_rearm: 0
> ch4_events: 974985780
> ch4_poll: 997781915
> ch4_arm: 969618250
> ch4_aff_change: 3
> ch4_eq_rearm: 0
> ch5_events: 888559089
> ch5_poll: 912783615
> ch5_arm: 883826078
> ch5_aff_change: 2
> ch5_eq_rearm: 0
> ch6_events: 873730730
> ch6_poll: 899635752
> ch6_arm: 868677574
> ch6_aff_change: 4
> ch6_eq_rearm: 0
> ch7_events: 873478411
> ch7_poll: 899216716
> ch7_arm: 868693645
> ch7_aff_change: 3
> ch7_eq_rearm: 0
> ch8_events: 871900967
> ch8_poll: 898575518
> ch8_arm: 866763693
> ch8_aff_change: 3
> ch8_eq_rearm: 0
> ch9_events: 880325565
> ch9_poll: 904983269
> ch9_arm: 875643922
> ch9_aff_change: 2
> ch9_eq_rearm: 0
> ch10_events: 889919775
> ch10_poll: 915335809
> ch10_arm: 885110225
> ch10_aff_change: 4
> ch10_eq_rearm: 0
> ch11_events: 962709175
> ch11_poll: 983963451
> ch11_arm: 958117526
> ch11_aff_change: 2
> ch11_eq_rearm: 0
> ch12_events: 941333837
> ch12_poll: 964625523
> ch12_arm: 936409706
> ch12_aff_change: 2
> ch12_eq_rearm: 0
> ch13_events: 914996974
> ch13_poll: 937441049
> ch13_arm: 910478393
> ch13_aff_change: 4
> ch13_eq_rearm: 0
> ch14_events: 888050001
> ch14_poll: 911818008
> ch14_arm: 883465035
> ch14_aff_change: 4
> ch14_eq_rearm: 0
> ch15_events: 947547704
> ch15_poll: 969073194
> ch15_arm: 942686515
> ch15_aff_change: 4
> ch15_eq_rearm: 0
> ch16_events: 825804904
> ch16_poll: 840630747
> ch16_arm: 822227488
> ch16_aff_change: 2
> ch16_eq_rearm: 0
> ch17_events: 861673823
> ch17_poll: 874754041
> ch17_arm: 858520448
> ch17_aff_change: 2
> ch17_eq_rearm: 0
> ch18_events: 879413440
> ch18_poll: 893962529
> ch18_arm: 875983204
> ch18_aff_change: 4
> ch18_eq_rearm: 0
> ch19_events: 896073709
> ch19_poll: 909216857
> ch19_arm: 893022121
> ch19_aff_change: 4
> ch19_eq_rearm: 0
> ch20_events: 865188535
> ch20_poll: 880692345
> ch20_arm: 861440265
> ch20_aff_change: 3
> ch20_eq_rearm: 0
> ch21_events: 862709303
> ch21_poll: 878104242
> ch21_arm: 859041767
> ch21_aff_change: 2
> ch21_eq_rearm: 0
> ch22_events: 887720551
> ch22_poll: 904122074
> ch22_arm: 883983794
> ch22_aff_change: 2
> ch22_eq_rearm: 0
> ch23_events: 813355027
> ch23_poll: 828074467
> ch23_arm: 809912398
> ch23_aff_change: 4
> ch23_eq_rearm: 0
> ch24_events: 822366675
> ch24_poll: 839917937
> ch24_arm: 818422754
> ch24_aff_change: 2
> ch24_eq_rearm: 0
> ch25_events: 826642292
> ch25_poll: 842630121
> ch25_arm: 822642618
> ch25_aff_change: 2
> ch25_eq_rearm: 0
> ch26_events: 826392584
> ch26_poll: 843406973
> ch26_arm: 822455000
> ch26_aff_change: 3
> ch26_eq_rearm: 0
> ch27_events: 828960899
> ch27_poll: 843866518
> ch27_arm: 825230937
> ch27_aff_change: 3
> ch27_eq_rearm: 0
> ch28_events: 7
> ch28_poll: 7
> ch28_arm: 7
> ch28_aff_change: 0
> ch28_eq_rearm: 0
> ch29_events: 4
> ch29_poll: 4
> ch29_arm: 4
> ch29_aff_change: 0
> ch29_eq_rearm: 0
> ch30_events: 4
> ch30_poll: 4
> ch30_arm: 4
> ch30_aff_change: 0
> ch30_eq_rearm: 0
> ch31_events: 4
> ch31_poll: 4
> ch31_arm: 4
> ch31_aff_change: 0
> ch31_eq_rearm: 0
> ch32_events: 4
> ch32_poll: 4
> ch32_arm: 4
> ch32_aff_change: 0
> ch32_eq_rearm: 0
> ch33_events: 4
> ch33_poll: 4
> ch33_arm: 4
> ch33_aff_change: 0
> ch33_eq_rearm: 0
> ch34_events: 4
> ch34_poll: 4
> ch34_arm: 4
> ch34_aff_change: 0
> ch34_eq_rearm: 0
> ch35_events: 4
> ch35_poll: 4
> ch35_arm: 4
> ch35_aff_change: 0
> ch35_eq_rearm: 0
> ch36_events: 4
> ch36_poll: 4
> ch36_arm: 4
> ch36_aff_change: 0
> ch36_eq_rearm: 0
> ch37_events: 4
> ch37_poll: 4
> ch37_arm: 4
> ch37_aff_change: 0
> ch37_eq_rearm: 0
> ch38_events: 4
> ch38_poll: 4
> ch38_arm: 4
> ch38_aff_change: 0
> ch38_eq_rearm: 0
> ch39_events: 4
> ch39_poll: 4
> ch39_arm: 4
> ch39_aff_change: 0
> ch39_eq_rearm: 0
> ch40_events: 4
> ch40_poll: 4
> ch40_arm: 4
> ch40_aff_change: 0
> ch40_eq_rearm: 0
> ch41_events: 4
> ch41_poll: 4
> ch41_arm: 4
> ch41_aff_change: 0
> ch41_eq_rearm: 0
> ch42_events: 4
> ch42_poll: 4
> ch42_arm: 4
> ch42_aff_change: 0
> ch42_eq_rearm: 0
> ch43_events: 4
> ch43_poll: 4
> ch43_arm: 4
> ch43_aff_change: 0
> ch43_eq_rearm: 0
> ch44_events: 4
> ch44_poll: 4
> ch44_arm: 4
> ch44_aff_change: 0
> ch44_eq_rearm: 0
> ch45_events: 4
> ch45_poll: 4
> ch45_arm: 4
> ch45_aff_change: 0
> ch45_eq_rearm: 0
> ch46_events: 4
> ch46_poll: 4
> ch46_arm: 4
> ch46_aff_change: 0
> ch46_eq_rearm: 0
> ch47_events: 4
> ch47_poll: 4
> ch47_arm: 4
> ch47_aff_change: 0
> ch47_eq_rearm: 0
> ch48_events: 4
> ch48_poll: 4
> ch48_arm: 4
> ch48_aff_change: 0
> ch48_eq_rearm: 0
> ch49_events: 4
> ch49_poll: 4
> ch49_arm: 4
> ch49_aff_change: 0
> ch49_eq_rearm: 0
> ch50_events: 4
> ch50_poll: 4
> ch50_arm: 4
> ch50_aff_change: 0
> ch50_eq_rearm: 0
> ch51_events: 4
> ch51_poll: 4
> ch51_arm: 4
> ch51_aff_change: 0
> ch51_eq_rearm: 0
> ch52_events: 4
> ch52_poll: 4
> ch52_arm: 4
> ch52_aff_change: 0
> ch52_eq_rearm: 0
> ch53_events: 4
> ch53_poll: 4
> ch53_arm: 4
> ch53_aff_change: 0
> ch53_eq_rearm: 0
> ch54_events: 4
> ch54_poll: 4
> ch54_arm: 4
> ch54_aff_change: 0
> ch54_eq_rearm: 0
> ch55_events: 4
> ch55_poll: 4
> ch55_arm: 4
> ch55_aff_change: 0
> ch55_eq_rearm: 0
> rx0_packets: 7284057433
> rx0_bytes: 4330611281319
> rx0_csum_complete: 7283623076
> rx0_csum_unnecessary: 0
> rx0_csum_unnecessary_inner: 0
> rx0_csum_none: 434357
> rx0_xdp_drop: 0
> rx0_xdp_redirect: 0
> rx0_lro_packets: 0
> rx0_lro_bytes: 0
> rx0_ecn_mark: 0
> rx0_removed_vlan_packets: 7284057433
> rx0_wqe_err: 0
> rx0_mpwqe_filler_cqes: 0
> rx0_mpwqe_filler_strides: 0
> rx0_buff_alloc_err: 0
> rx0_cqe_compress_blks: 0
> rx0_cqe_compress_pkts: 0
> rx0_page_reuse: 0
> rx0_cache_reuse: 1989731589
> rx0_cache_full: 28213297
> rx0_cache_empty: 1624089822
> rx0_cache_busy: 28213961
> rx0_cache_waive: 1624083610
> rx0_congst_umr: 0
> rx0_arfs_err: 0
> rx0_xdp_tx_xmit: 0
> rx0_xdp_tx_full: 0
> rx0_xdp_tx_err: 0
> rx0_xdp_tx_cqes: 0
> rx1_packets: 6691319211
> rx1_bytes: 3799580210608
> rx1_csum_complete: 6691319211
> rx1_csum_unnecessary: 0
> rx1_csum_unnecessary_inner: 0
> rx1_csum_none: 0
> rx1_xdp_drop: 0
> rx1_xdp_redirect: 0
> rx1_lro_packets: 0
> rx1_lro_bytes: 0
> rx1_ecn_mark: 0
> rx1_removed_vlan_packets: 6691319211
> rx1_wqe_err: 0
> rx1_mpwqe_filler_cqes: 0
> rx1_mpwqe_filler_strides: 0
> rx1_buff_alloc_err: 0
> rx1_cqe_compress_blks: 0
> rx1_cqe_compress_pkts: 0
> rx1_page_reuse: 0
> rx1_cache_reuse: 2270019
> rx1_cache_full: 3343389331
> rx1_cache_empty: 6656
> rx1_cache_busy: 3343389585
> rx1_cache_waive: 0
> rx1_congst_umr: 0
> rx1_arfs_err: 0
> rx1_xdp_tx_xmit: 0
> rx1_xdp_tx_full: 0
> rx1_xdp_tx_err: 0
> rx1_xdp_tx_cqes: 0
> rx2_packets: 6618370416
> rx2_bytes: 3762508364015
> rx2_csum_complete: 6618370416
> rx2_csum_unnecessary: 0
> rx2_csum_unnecessary_inner: 0
> rx2_csum_none: 0
> rx2_xdp_drop: 0
> rx2_xdp_redirect: 0
> rx2_lro_packets: 0
> rx2_lro_bytes: 0
> rx2_ecn_mark: 0
> rx2_removed_vlan_packets: 6618370416
> rx2_wqe_err: 0
> rx2_mpwqe_filler_cqes: 0
> rx2_mpwqe_filler_strides: 0
> rx2_buff_alloc_err: 0
> rx2_cqe_compress_blks: 0
> rx2_cqe_compress_pkts: 0
> rx2_page_reuse: 0
> rx2_cache_reuse: 111419328
> rx2_cache_full: 1807563903
> rx2_cache_empty: 1390208158
> rx2_cache_busy: 1807564378
> rx2_cache_waive: 1390201722
> rx2_congst_umr: 0
> rx2_arfs_err: 0
> rx2_xdp_tx_xmit: 0
> rx2_xdp_tx_full: 0
> rx2_xdp_tx_err: 0
> rx2_xdp_tx_cqes: 0
> rx3_packets: 6665308976
> rx3_bytes: 3828546206006
> rx3_csum_complete: 6665308976
> rx3_csum_unnecessary: 0
> rx3_csum_unnecessary_inner: 0
> rx3_csum_none: 0
> rx3_xdp_drop: 0
> rx3_xdp_redirect: 0
> rx3_lro_packets: 0
> rx3_lro_bytes: 0
> rx3_ecn_mark: 0
> rx3_removed_vlan_packets: 6665308976
> rx3_wqe_err: 0
> rx3_mpwqe_filler_cqes: 0
> rx3_mpwqe_filler_strides: 0
> rx3_buff_alloc_err: 0
> rx3_cqe_compress_blks: 0
> rx3_cqe_compress_pkts: 0
> rx3_page_reuse: 0
> rx3_cache_reuse: 215779091
> rx3_cache_full: 1720040649
> rx3_cache_empty: 1396840926
> rx3_cache_busy: 1720041127
> rx3_cache_waive: 1396834493
> rx3_congst_umr: 0
> rx3_arfs_err: 0
> rx3_xdp_tx_xmit: 0
> rx3_xdp_tx_full: 0
> rx3_xdp_tx_err: 0
> rx3_xdp_tx_cqes: 0
> rx4_packets: 6764448165
> rx4_bytes: 3883101339142
> rx4_csum_complete: 6764448165
> rx4_csum_unnecessary: 0
> rx4_csum_unnecessary_inner: 0
> rx4_csum_none: 0
> rx4_xdp_drop: 0
> rx4_xdp_redirect: 0
> rx4_lro_packets: 0
> rx4_lro_bytes: 0
> rx4_ecn_mark: 0
> rx4_removed_vlan_packets: 6764448165
> rx4_wqe_err: 0
> rx4_mpwqe_filler_cqes: 0
> rx4_mpwqe_filler_strides: 0
> rx4_buff_alloc_err: 0
> rx4_cqe_compress_blks: 0
> rx4_cqe_compress_pkts: 0
> rx4_page_reuse: 0
> rx4_cache_reuse: 1930710653
> rx4_cache_full: 6490815
> rx4_cache_empty: 1445028605
> rx4_cache_busy: 6491478
> rx4_cache_waive: 1445022392
> rx4_congst_umr: 0
> rx4_arfs_err: 0
> rx4_xdp_tx_xmit: 0
> rx4_xdp_tx_full: 0
> rx4_xdp_tx_err: 0
> rx4_xdp_tx_cqes: 0
> rx5_packets: 6736853264
> rx5_bytes: 3925186068552
> rx5_csum_complete: 6736853264
> rx5_csum_unnecessary: 0
> rx5_csum_unnecessary_inner: 0
> rx5_csum_none: 0
> rx5_xdp_drop: 0
> rx5_xdp_redirect: 0
> rx5_lro_packets: 0
> rx5_lro_bytes: 0
> rx5_ecn_mark: 0
> rx5_removed_vlan_packets: 6736853264
> rx5_wqe_err: 0
> rx5_mpwqe_filler_cqes: 0
> rx5_mpwqe_filler_strides: 0
> rx5_buff_alloc_err: 0
> rx5_cqe_compress_blks: 0
> rx5_cqe_compress_pkts: 0
> rx5_page_reuse: 0
> rx5_cache_reuse: 7283914
> rx5_cache_full: 3361142463
> rx5_cache_empty: 6656
> rx5_cache_busy: 3361142718
> rx5_cache_waive: 0
> rx5_congst_umr: 0
> rx5_arfs_err: 0
> rx5_xdp_tx_xmit: 0
> rx5_xdp_tx_full: 0
> rx5_xdp_tx_err: 0
> rx5_xdp_tx_cqes: 0
> rx6_packets: 6751588828
> rx6_bytes: 3860537598885
> rx6_csum_complete: 6751588828
> rx6_csum_unnecessary: 0
> rx6_csum_unnecessary_inner: 0
> rx6_csum_none: 0
> rx6_xdp_drop: 0
> rx6_xdp_redirect: 0
> rx6_lro_packets: 0
> rx6_lro_bytes: 0
> rx6_ecn_mark: 0
> rx6_removed_vlan_packets: 6751588828
> rx6_wqe_err: 0
> rx6_mpwqe_filler_cqes: 0
> rx6_mpwqe_filler_strides: 0
> rx6_buff_alloc_err: 0
> rx6_cqe_compress_blks: 0
> rx6_cqe_compress_pkts: 0
> rx6_page_reuse: 0
> rx6_cache_reuse: 96032126
> rx6_cache_full: 1857890923
> rx6_cache_empty: 1421877543
> rx6_cache_busy: 1857891399
> rx6_cache_waive: 1421871110
> rx6_congst_umr: 0
> rx6_arfs_err: 0
> rx6_xdp_tx_xmit: 0
> rx6_xdp_tx_full: 0
> rx6_xdp_tx_err: 0
> rx6_xdp_tx_cqes: 0
> rx7_packets: 6935300074
> rx7_bytes: 4004713524388
> rx7_csum_complete: 6935300074
> rx7_csum_unnecessary: 0
> rx7_csum_unnecessary_inner: 0
> rx7_csum_none: 0
> rx7_xdp_drop: 0
> rx7_xdp_redirect: 0
> rx7_lro_packets: 0
> rx7_lro_bytes: 0
> rx7_ecn_mark: 0
> rx7_removed_vlan_packets: 6935300074
> rx7_wqe_err: 0
> rx7_mpwqe_filler_cqes: 0
> rx7_mpwqe_filler_strides: 0
> rx7_buff_alloc_err: 0
> rx7_cqe_compress_blks: 0
> rx7_cqe_compress_pkts: 0
> rx7_page_reuse: 0
> rx7_cache_reuse: 17555187
> rx7_cache_full: 3450094595
> rx7_cache_empty: 6656
> rx7_cache_busy: 3450094849
> rx7_cache_waive: 0
> rx7_congst_umr: 0
> rx7_arfs_err: 0
> rx7_xdp_tx_xmit: 0
> rx7_xdp_tx_full: 0
> rx7_xdp_tx_err: 0
> rx7_xdp_tx_cqes: 0
> rx8_packets: 6678640094
> rx8_bytes: 3783722686028
> rx8_csum_complete: 6678640094
> rx8_csum_unnecessary: 0
> rx8_csum_unnecessary_inner: 0
> rx8_csum_none: 0
> rx8_xdp_drop: 0
> rx8_xdp_redirect: 0
> rx8_lro_packets: 0
> rx8_lro_bytes: 0
> rx8_ecn_mark: 0
> rx8_removed_vlan_packets: 6678640094
> rx8_wqe_err: 0
> rx8_mpwqe_filler_cqes: 0
> rx8_mpwqe_filler_strides: 0
> rx8_buff_alloc_err: 0
> rx8_cqe_compress_blks: 0
> rx8_cqe_compress_pkts: 0
> rx8_page_reuse: 0
> rx8_cache_reuse: 71006578
> rx8_cache_full: 1879380649
> rx8_cache_empty: 1388938999
> rx8_cache_busy: 1879381123
> rx8_cache_waive: 1388932565
> rx8_congst_umr: 0
> rx8_arfs_err: 0
> rx8_xdp_tx_xmit: 0
> rx8_xdp_tx_full: 0
> rx8_xdp_tx_err: 0
> rx8_xdp_tx_cqes: 0
> rx9_packets: 6709855557
> rx9_bytes: 3849522227880
> rx9_csum_complete: 6709855557
> rx9_csum_unnecessary: 0
> rx9_csum_unnecessary_inner: 0
> rx9_csum_none: 0
> rx9_xdp_drop: 0
> rx9_xdp_redirect: 0
> rx9_lro_packets: 0
> rx9_lro_bytes: 0
> rx9_ecn_mark: 0
> rx9_removed_vlan_packets: 6709855557
> rx9_wqe_err: 0
> rx9_mpwqe_filler_cqes: 0
> rx9_mpwqe_filler_strides: 0
> rx9_buff_alloc_err: 0
> rx9_cqe_compress_blks: 0
> rx9_cqe_compress_pkts: 0
> rx9_page_reuse: 0
> rx9_cache_reuse: 108980215
> rx9_cache_full: 1822730121
> rx9_cache_empty: 1423223623
> rx9_cache_busy: 1822730594
> rx9_cache_waive: 1423217187
> rx9_congst_umr: 0
> rx9_arfs_err: 0
> rx9_xdp_tx_xmit: 0
> rx9_xdp_tx_full: 0
> rx9_xdp_tx_err: 0
> rx9_xdp_tx_cqes: 0
> rx10_packets: 6761861066
> rx10_bytes: 3816266733385
> rx10_csum_complete: 6761861066
> rx10_csum_unnecessary: 0
> rx10_csum_unnecessary_inner: 0
> rx10_csum_none: 0
> rx10_xdp_drop: 0
> rx10_xdp_redirect: 0
> rx10_lro_packets: 0
> rx10_lro_bytes: 0
> rx10_ecn_mark: 0
> rx10_removed_vlan_packets: 6761861066
> rx10_wqe_err: 0
> rx10_mpwqe_filler_cqes: 0
> rx10_mpwqe_filler_strides: 0
> rx10_buff_alloc_err: 0
> rx10_cqe_compress_blks: 0
> rx10_cqe_compress_pkts: 0
> rx10_page_reuse: 0
> rx10_cache_reuse: 3489300
> rx10_cache_full: 3377440977
> rx10_cache_empty: 6656
> rx10_cache_busy: 3377441216
> rx10_cache_waive: 0
> rx10_congst_umr: 0
> rx10_arfs_err: 0
> rx10_xdp_tx_xmit: 0
> rx10_xdp_tx_full: 0
> rx10_xdp_tx_err: 0
> rx10_xdp_tx_cqes: 0
> rx11_packets: 6868113938
> rx11_bytes: 4048196300710
> rx11_csum_complete: 6868113938
> rx11_csum_unnecessary: 0
> rx11_csum_unnecessary_inner: 0
> rx11_csum_none: 0
> rx11_xdp_drop: 0
> rx11_xdp_redirect: 0
> rx11_lro_packets: 0
> rx11_lro_bytes: 0
> rx11_ecn_mark: 0
> rx11_removed_vlan_packets: 6868113938
> rx11_wqe_err: 0
> rx11_mpwqe_filler_cqes: 0
> rx11_mpwqe_filler_strides: 0
> rx11_buff_alloc_err: 0
> rx11_cqe_compress_blks: 0
> rx11_cqe_compress_pkts: 0
> rx11_page_reuse: 0
> rx11_cache_reuse: 1948516819
> rx11_cache_full: 17132157
> rx11_cache_empty: 1468413985
> rx11_cache_busy: 17132820
> rx11_cache_waive: 1468407772
> rx11_congst_umr: 0
> rx11_arfs_err: 0
> rx11_xdp_tx_xmit: 0
> rx11_xdp_tx_full: 0
> rx11_xdp_tx_err: 0
> rx11_xdp_tx_cqes: 0
> rx12_packets: 6742955386
> rx12_bytes: 3865747629271
> rx12_csum_complete: 6742955386
> rx12_csum_unnecessary: 0
> rx12_csum_unnecessary_inner: 0
> rx12_csum_none: 0
> rx12_xdp_drop: 0
> rx12_xdp_redirect: 0
> rx12_lro_packets: 0
> rx12_lro_bytes: 0
> rx12_ecn_mark: 0
> rx12_removed_vlan_packets: 6742955386
> rx12_wqe_err: 0
> rx12_mpwqe_filler_cqes: 0
> rx12_mpwqe_filler_strides: 0
> rx12_buff_alloc_err: 0
> rx12_cqe_compress_blks: 0
> rx12_cqe_compress_pkts: 0
> rx12_page_reuse: 0
> rx12_cache_reuse: 30809331
> rx12_cache_full: 3340668106
> rx12_cache_empty: 6656
> rx12_cache_busy: 3340668333
> rx12_cache_waive: 0
> rx12_congst_umr: 0
> rx12_arfs_err: 0
> rx12_xdp_tx_xmit: 0
> rx12_xdp_tx_full: 0
> rx12_xdp_tx_err: 0
> rx12_xdp_tx_cqes: 0
> rx13_packets: 6707028036
> rx13_bytes: 3813462190623
> rx13_csum_complete: 6707028036
> rx13_csum_unnecessary: 0
> rx13_csum_unnecessary_inner: 0
> rx13_csum_none: 0
> rx13_xdp_drop: 0
> rx13_xdp_redirect: 0
> rx13_lro_packets: 0
> rx13_lro_bytes: 0
> rx13_ecn_mark: 0
> rx13_removed_vlan_packets: 6707028036
> rx13_wqe_err: 0
> rx13_mpwqe_filler_cqes: 0
> rx13_mpwqe_filler_strides: 0
> rx13_buff_alloc_err: 0
> rx13_cqe_compress_blks: 0
> rx13_cqe_compress_pkts: 0
> rx13_page_reuse: 0
> rx13_cache_reuse: 14951053
> rx13_cache_full: 3338562710
> rx13_cache_empty: 6656
> rx13_cache_busy: 3338562963
> rx13_cache_waive: 0
> rx13_congst_umr: 0
> rx13_arfs_err: 0
> rx13_xdp_tx_xmit: 0
> rx13_xdp_tx_full: 0
> rx13_xdp_tx_err: 0
> rx13_xdp_tx_cqes: 0
> rx14_packets: 6737074410
> rx14_bytes: 3868905276119
> rx14_csum_complete: 6737074410
> rx14_csum_unnecessary: 0
> rx14_csum_unnecessary_inner: 0
> rx14_csum_none: 0
> rx14_xdp_drop: 0
> rx14_xdp_redirect: 0
> rx14_lro_packets: 0
> rx14_lro_bytes: 0
> rx14_ecn_mark: 0
> rx14_removed_vlan_packets: 6737074410
> rx14_wqe_err: 0
> rx14_mpwqe_filler_cqes: 0
> rx14_mpwqe_filler_strides: 0
> rx14_buff_alloc_err: 0
> rx14_cqe_compress_blks: 0
> rx14_cqe_compress_pkts: 0
> rx14_page_reuse: 0
> rx14_cache_reuse: 967799432
> rx14_cache_full: 982704312
> rx14_cache_empty: 1418039639
> rx14_cache_busy: 982704789
> rx14_cache_waive: 1418033206
> rx14_congst_umr: 0
> rx14_arfs_err: 0
> rx14_xdp_tx_xmit: 0
> rx14_xdp_tx_full: 0
> rx14_xdp_tx_err: 0
> rx14_xdp_tx_cqes: 0
> rx15_packets: 6641887441
> rx15_bytes: 3742874400402
> rx15_csum_complete: 6641887441
> rx15_csum_unnecessary: 0
> rx15_csum_unnecessary_inner: 0
> rx15_csum_none: 0
> rx15_xdp_drop: 0
> rx15_xdp_redirect: 0
> rx15_lro_packets: 0
> rx15_lro_bytes: 0
> rx15_ecn_mark: 0
> rx15_removed_vlan_packets: 6641887441
> rx15_wqe_err: 0
> rx15_mpwqe_filler_cqes: 0
> rx15_mpwqe_filler_strides: 0
> rx15_buff_alloc_err: 0
> rx15_cqe_compress_blks: 0
> rx15_cqe_compress_pkts: 0
> rx15_page_reuse: 0
> rx15_cache_reuse: 1920227538
> rx15_cache_full: 19386129
> rx15_cache_empty: 1381335137
> rx15_cache_busy: 19387693
> rx15_cache_waive: 1381329825
> rx15_congst_umr: 0
> rx15_arfs_err: 0
> rx15_xdp_tx_xmit: 0
> rx15_xdp_tx_full: 0
> rx15_xdp_tx_err: 0
> rx15_xdp_tx_cqes: 0
> rx16_packets: 5420472874
> rx16_bytes: 3079293332581
> rx16_csum_complete: 5420472874
> rx16_csum_unnecessary: 0
> rx16_csum_unnecessary_inner: 0
> rx16_csum_none: 0
> rx16_xdp_drop: 0
> rx16_xdp_redirect: 0
> rx16_lro_packets: 0
> rx16_lro_bytes: 0
> rx16_ecn_mark: 0
> rx16_removed_vlan_packets: 5420472874
> rx16_wqe_err: 0
> rx16_mpwqe_filler_cqes: 0
> rx16_mpwqe_filler_strides: 0
> rx16_buff_alloc_err: 0
> rx16_cqe_compress_blks: 0
> rx16_cqe_compress_pkts: 0
> rx16_page_reuse: 0
> rx16_cache_reuse: 2361079
> rx16_cache_full: 2707875103
> rx16_cache_empty: 6656
> rx16_cache_busy: 2707875349
> rx16_cache_waive: 0
> rx16_congst_umr: 0
> rx16_arfs_err: 0
> rx16_xdp_tx_xmit: 0
> rx16_xdp_tx_full: 0
> rx16_xdp_tx_err: 0
> rx16_xdp_tx_cqes: 0
> rx17_packets: 5428380986
> rx17_bytes: 3080981893118
> rx17_csum_complete: 5428380986
> rx17_csum_unnecessary: 0
> rx17_csum_unnecessary_inner: 0
> rx17_csum_none: 0
> rx17_xdp_drop: 0
> rx17_xdp_redirect: 0
> rx17_lro_packets: 0
> rx17_lro_bytes: 0
> rx17_ecn_mark: 0
> rx17_removed_vlan_packets: 5428380986
> rx17_wqe_err: 0
> rx17_mpwqe_filler_cqes: 0
> rx17_mpwqe_filler_strides: 0
> rx17_buff_alloc_err: 0
> rx17_cqe_compress_blks: 0
> rx17_cqe_compress_pkts: 0
> rx17_page_reuse: 0
> rx17_cache_reuse: 1552266402
> rx17_cache_full: 5947505
> rx17_cache_empty: 1155981856
> rx17_cache_busy: 5948870
> rx17_cache_waive: 1155976345
> rx17_congst_umr: 0
> rx17_arfs_err: 0
> rx17_xdp_tx_xmit: 0
> rx17_xdp_tx_full: 0
> rx17_xdp_tx_err: 0
> rx17_xdp_tx_cqes: 0
> rx18_packets: 5529118410
> rx18_bytes: 3254749573833
> rx18_csum_complete: 5529118410
> rx18_csum_unnecessary: 0
> rx18_csum_unnecessary_inner: 0
> rx18_csum_none: 0
> rx18_xdp_drop: 0
> rx18_xdp_redirect: 0
> rx18_lro_packets: 0
> rx18_lro_bytes: 0
> rx18_ecn_mark: 0
> rx18_removed_vlan_packets: 5529118410
> rx18_wqe_err: 0
> rx18_mpwqe_filler_cqes: 0
> rx18_mpwqe_filler_strides: 0
> rx18_buff_alloc_err: 0
> rx18_cqe_compress_blks: 0
> rx18_cqe_compress_pkts: 0
> rx18_page_reuse: 0
> rx18_cache_reuse: 67438840
> rx18_cache_full: 1536718472
> rx18_cache_empty: 1160408072
> rx18_cache_busy: 1536718932
> rx18_cache_waive: 1160401638
> rx18_congst_umr: 0
> rx18_arfs_err: 0
> rx18_xdp_tx_xmit: 0
> rx18_xdp_tx_full: 0
> rx18_xdp_tx_err: 0
> rx18_xdp_tx_cqes: 0
> rx19_packets: 5449932653
> rx19_bytes: 3148726579411
> rx19_csum_complete: 5449932653
> rx19_csum_unnecessary: 0
> rx19_csum_unnecessary_inner: 0
> rx19_csum_none: 0
> rx19_xdp_drop: 0
> rx19_xdp_redirect: 0
> rx19_lro_packets: 0
> rx19_lro_bytes: 0
> rx19_ecn_mark: 0
> rx19_removed_vlan_packets: 5449932653
> rx19_wqe_err: 0
> rx19_mpwqe_filler_cqes: 0
> rx19_mpwqe_filler_strides: 0
> rx19_buff_alloc_err: 0
> rx19_cqe_compress_blks: 0
> rx19_cqe_compress_pkts: 0
> rx19_page_reuse: 0
> rx19_cache_reuse: 1537841743
> rx19_cache_full: 9920960
> rx19_cache_empty: 1177208938
> rx19_cache_busy: 9922299
> rx19_cache_waive: 1177203401
> rx19_congst_umr: 0
> rx19_arfs_err: 0
> rx19_xdp_tx_xmit: 0
> rx19_xdp_tx_full: 0
> rx19_xdp_tx_err: 0
> rx19_xdp_tx_cqes: 0
> rx20_packets: 5407910071
> rx20_bytes: 3123560861922
> rx20_csum_complete: 5407910071
> rx20_csum_unnecessary: 0
> rx20_csum_unnecessary_inner: 0
> rx20_csum_none: 0
> rx20_xdp_drop: 0
> rx20_xdp_redirect: 0
> rx20_lro_packets: 0
> rx20_lro_bytes: 0
> rx20_ecn_mark: 0
> rx20_removed_vlan_packets: 5407910071
> rx20_wqe_err: 0
> rx20_mpwqe_filler_cqes: 0
> rx20_mpwqe_filler_strides: 0
> rx20_buff_alloc_err: 0
> rx20_cqe_compress_blks: 0
> rx20_cqe_compress_pkts: 0
> rx20_page_reuse: 0
> rx20_cache_reuse: 10255209
> rx20_cache_full: 2693699571
> rx20_cache_empty: 6656
> rx20_cache_busy: 2693699823
> rx20_cache_waive: 0
> rx20_congst_umr: 0
> rx20_arfs_err: 0
> rx20_xdp_tx_xmit: 0
> rx20_xdp_tx_full: 0
> rx20_xdp_tx_err: 0
> rx20_xdp_tx_cqes: 0
> rx21_packets: 5417498508
> rx21_bytes: 3131335892379
> rx21_csum_complete: 5417498508
> rx21_csum_unnecessary: 0
> rx21_csum_unnecessary_inner: 0
> rx21_csum_none: 0
> rx21_xdp_drop: 0
> rx21_xdp_redirect: 0
> rx21_lro_packets: 0
> rx21_lro_bytes: 0
> rx21_ecn_mark: 0
> rx21_removed_vlan_packets: 5417498508
> rx21_wqe_err: 0
> rx21_mpwqe_filler_cqes: 0
> rx21_mpwqe_filler_strides: 0
> rx21_buff_alloc_err: 0
> rx21_cqe_compress_blks: 0
> rx21_cqe_compress_pkts: 0
> rx21_page_reuse: 0
> rx21_cache_reuse: 192662917
> rx21_cache_full: 1374120417
> rx21_cache_empty: 1141972100
> rx21_cache_busy: 1374120891
> rx21_cache_waive: 1141965665
> rx21_congst_umr: 0
> rx21_arfs_err: 0
> rx21_xdp_tx_xmit: 0
> rx21_xdp_tx_full: 0
> rx21_xdp_tx_err: 0
> rx21_xdp_tx_cqes: 0
> rx22_packets: 5613634706
> rx22_bytes: 3240055099058
> rx22_csum_complete: 5613634706
> rx22_csum_unnecessary: 0
> rx22_csum_unnecessary_inner: 0
> rx22_csum_none: 0
> rx22_xdp_drop: 0
> rx22_xdp_redirect: 0
> rx22_lro_packets: 0
> rx22_lro_bytes: 0
> rx22_ecn_mark: 0
> rx22_removed_vlan_packets: 5613634706
> rx22_wqe_err: 0
> rx22_mpwqe_filler_cqes: 0
> rx22_mpwqe_filler_strides: 0
> rx22_buff_alloc_err: 0
> rx22_cqe_compress_blks: 0
> rx22_cqe_compress_pkts: 0
> rx22_page_reuse: 0
> rx22_cache_reuse: 12161531
> rx22_cache_full: 2794655567
> rx22_cache_empty: 6656
> rx22_cache_busy: 2794655821
> rx22_cache_waive: 0
> rx22_congst_umr: 0
> rx22_arfs_err: 0
> rx22_xdp_tx_xmit: 0
> rx22_xdp_tx_full: 0
> rx22_xdp_tx_err: 0
> rx22_xdp_tx_cqes: 0
> rx23_packets: 5389977167
> rx23_bytes: 3054270771559
> rx23_csum_complete: 5389977167
> rx23_csum_unnecessary: 0
> rx23_csum_unnecessary_inner: 0
> rx23_csum_none: 0
> rx23_xdp_drop: 0
> rx23_xdp_redirect: 0
> rx23_lro_packets: 0
> rx23_lro_bytes: 0
> rx23_ecn_mark: 0
> rx23_removed_vlan_packets: 5389977167
> rx23_wqe_err: 0
> rx23_mpwqe_filler_cqes: 0
> rx23_mpwqe_filler_strides: 0
> rx23_buff_alloc_err: 0
> rx23_cqe_compress_blks: 0
> rx23_cqe_compress_pkts: 0
> rx23_page_reuse: 0
> rx23_cache_reuse: 709328
> rx23_cache_full: 2694279000
> rx23_cache_empty: 6656
> rx23_cache_busy: 2694279252
> rx23_cache_waive: 0
> rx23_congst_umr: 0
> rx23_arfs_err: 0
> rx23_xdp_tx_xmit: 0
> rx23_xdp_tx_full: 0
> rx23_xdp_tx_err: 0
> rx23_xdp_tx_cqes: 0
> rx24_packets: 5547561932
> rx24_bytes: 3166602453443
> rx24_csum_complete: 5547561932
> rx24_csum_unnecessary: 0
> rx24_csum_unnecessary_inner: 0
> rx24_csum_none: 0
> rx24_xdp_drop: 0
> rx24_xdp_redirect: 0
> rx24_lro_packets: 0
> rx24_lro_bytes: 0
> rx24_ecn_mark: 0
> rx24_removed_vlan_packets: 5547561932
> rx24_wqe_err: 0
> rx24_mpwqe_filler_cqes: 0
> rx24_mpwqe_filler_strides: 0
> rx24_buff_alloc_err: 0
> rx24_cqe_compress_blks: 0
> rx24_cqe_compress_pkts: 0
> rx24_page_reuse: 0
> rx24_cache_reuse: 57885119
> rx24_cache_full: 1529450077
> rx24_cache_empty: 1186451948
> rx24_cache_busy: 1529450553
> rx24_cache_waive: 1186445515
> rx24_congst_umr: 0
> rx24_arfs_err: 0
> rx24_xdp_tx_xmit: 0
> rx24_xdp_tx_full: 0
> rx24_xdp_tx_err: 0
> rx24_xdp_tx_cqes: 0
> rx25_packets: 5414569326
> rx25_bytes: 3184757708091
> rx25_csum_complete: 5414569326
> rx25_csum_unnecessary: 0
> rx25_csum_unnecessary_inner: 0
> rx25_csum_none: 0
> rx25_xdp_drop: 0
> rx25_xdp_redirect: 0
> rx25_lro_packets: 0
> rx25_lro_bytes: 0
> rx25_ecn_mark: 0
> rx25_removed_vlan_packets: 5414569326
> rx25_wqe_err: 0
> rx25_mpwqe_filler_cqes: 0
> rx25_mpwqe_filler_strides: 0
> rx25_buff_alloc_err: 0
> rx25_cqe_compress_blks: 0
> rx25_cqe_compress_pkts: 0
> rx25_page_reuse: 0
> rx25_cache_reuse: 5080853
> rx25_cache_full: 2702203555
> rx25_cache_empty: 6656
> rx25_cache_busy: 2702203807
> rx25_cache_waive: 0
> rx25_congst_umr: 0
> rx25_arfs_err: 0
> rx25_xdp_tx_xmit: 0
> rx25_xdp_tx_full: 0
> rx25_xdp_tx_err: 0
> rx25_xdp_tx_cqes: 0
> rx26_packets: 5479972151
> rx26_bytes: 3110642276239
> rx26_csum_complete: 5479972151
> rx26_csum_unnecessary: 0
> rx26_csum_unnecessary_inner: 0
> rx26_csum_none: 0
> rx26_xdp_drop: 0
> rx26_xdp_redirect: 0
> rx26_lro_packets: 0
> rx26_lro_bytes: 0
> rx26_ecn_mark: 0
> rx26_removed_vlan_packets: 5479972151
> rx26_wqe_err: 0
> rx26_mpwqe_filler_cqes: 0
> rx26_mpwqe_filler_strides: 0
> rx26_buff_alloc_err: 0
> rx26_cqe_compress_blks: 0
> rx26_cqe_compress_pkts: 0
> rx26_page_reuse: 0
> rx26_cache_reuse: 26543335
> rx26_cache_full: 2713442485
> rx26_cache_empty: 6656
> rx26_cache_busy: 2713442737
> rx26_cache_waive: 0
> rx26_congst_umr: 0
> rx26_arfs_err: 0
> rx26_xdp_tx_xmit: 0
> rx26_xdp_tx_full: 0
> rx26_xdp_tx_err: 0
> rx26_xdp_tx_cqes: 0
> rx27_packets: 5337113900
> rx27_bytes: 3068966906075
> rx27_csum_complete: 5337113900
> rx27_csum_unnecessary: 0
> rx27_csum_unnecessary_inner: 0
> rx27_csum_none: 0
> rx27_xdp_drop: 0
> rx27_xdp_redirect: 0
> rx27_lro_packets: 0
> rx27_lro_bytes: 0
> rx27_ecn_mark: 0
> rx27_removed_vlan_packets: 5337113900
> rx27_wqe_err: 0
> rx27_mpwqe_filler_cqes: 0
> rx27_mpwqe_filler_strides: 0
> rx27_buff_alloc_err: 0
> rx27_cqe_compress_blks: 0
> rx27_cqe_compress_pkts: 0
> rx27_page_reuse: 0
> rx27_cache_reuse: 1539298962
> rx27_cache_full: 10861919
> rx27_cache_empty: 1117173179
> rx27_cache_busy: 12091463
> rx27_cache_waive: 1118395847
> rx27_congst_umr: 0
> rx27_arfs_err: 0
> rx27_xdp_tx_xmit: 0
> rx27_xdp_tx_full: 0
> rx27_xdp_tx_err: 0
> rx27_xdp_tx_cqes: 0
> rx28_packets: 0
> rx28_bytes: 0
> rx28_csum_complete: 0
> rx28_csum_unnecessary: 0
> rx28_csum_unnecessary_inner: 0
> rx28_csum_none: 0
> rx28_xdp_drop: 0
> rx28_xdp_redirect: 0
> rx28_lro_packets: 0
> rx28_lro_bytes: 0
> rx28_ecn_mark: 0
> rx28_removed_vlan_packets: 0
> rx28_wqe_err: 0
> rx28_mpwqe_filler_cqes: 0
> rx28_mpwqe_filler_strides: 0
> rx28_buff_alloc_err: 0
> rx28_cqe_compress_blks: 0
> rx28_cqe_compress_pkts: 0
> rx28_page_reuse: 0
> rx28_cache_reuse: 0
> rx28_cache_full: 0
> rx28_cache_empty: 2560
> rx28_cache_busy: 0
> rx28_cache_waive: 0
> rx28_congst_umr: 0
> rx28_arfs_err: 0
> rx28_xdp_tx_xmit: 0
> rx28_xdp_tx_full: 0
> rx28_xdp_tx_err: 0
> rx28_xdp_tx_cqes: 0
> rx29_packets: 0
> rx29_bytes: 0
> rx29_csum_complete: 0
> rx29_csum_unnecessary: 0
> rx29_csum_unnecessary_inner: 0
> rx29_csum_none: 0
> rx29_xdp_drop: 0
> rx29_xdp_redirect: 0
> rx29_lro_packets: 0
> rx29_lro_bytes: 0
> rx29_ecn_mark: 0
> rx29_removed_vlan_packets: 0
> rx29_wqe_err: 0
> rx29_mpwqe_filler_cqes: 0
> rx29_mpwqe_filler_strides: 0
> rx29_buff_alloc_err: 0
> rx29_cqe_compress_blks: 0
> rx29_cqe_compress_pkts: 0
> rx29_page_reuse: 0
> rx29_cache_reuse: 0
> rx29_cache_full: 0
> rx29_cache_empty: 2560
> rx29_cache_busy: 0
> rx29_cache_waive: 0
> rx29_congst_umr: 0
> rx29_arfs_err: 0
> rx29_xdp_tx_xmit: 0
> rx29_xdp_tx_full: 0
> rx29_xdp_tx_err: 0
> rx29_xdp_tx_cqes: 0
> rx30_packets: 0
> rx30_bytes: 0
> rx30_csum_complete: 0
> rx30_csum_unnecessary: 0
> rx30_csum_unnecessary_inner: 0
> rx30_csum_none: 0
> rx30_xdp_drop: 0
> rx30_xdp_redirect: 0
> rx30_lro_packets: 0
> rx30_lro_bytes: 0
> rx30_ecn_mark: 0
> rx30_removed_vlan_packets: 0
> rx30_wqe_err: 0
> rx30_mpwqe_filler_cqes: 0
> rx30_mpwqe_filler_strides: 0
> rx30_buff_alloc_err: 0
> rx30_cqe_compress_blks: 0
> rx30_cqe_compress_pkts: 0
> rx30_page_reuse: 0
> rx30_cache_reuse: 0
> rx30_cache_full: 0
> rx30_cache_empty: 2560
> rx30_cache_busy: 0
> rx30_cache_waive: 0
> rx30_congst_umr: 0
> rx30_arfs_err: 0
> rx30_xdp_tx_xmit: 0
> rx30_xdp_tx_full: 0
> rx30_xdp_tx_err: 0
> rx30_xdp_tx_cqes: 0
> rx31_packets: 0
> rx31_bytes: 0
> rx31_csum_complete: 0
> rx31_csum_unnecessary: 0
> rx31_csum_unnecessary_inner: 0
> rx31_csum_none: 0
> rx31_xdp_drop: 0
> rx31_xdp_redirect: 0
> rx31_lro_packets: 0
> rx31_lro_bytes: 0
> rx31_ecn_mark: 0
> rx31_removed_vlan_packets: 0
> rx31_wqe_err: 0
> rx31_mpwqe_filler_cqes: 0
> rx31_mpwqe_filler_strides: 0
> rx31_buff_alloc_err: 0
> rx31_cqe_compress_blks: 0
> rx31_cqe_compress_pkts: 0
> rx31_page_reuse: 0
> rx31_cache_reuse: 0
> rx31_cache_full: 0
> rx31_cache_empty: 2560
> rx31_cache_busy: 0
> rx31_cache_waive: 0
> rx31_congst_umr: 0
> rx31_arfs_err: 0
> rx31_xdp_tx_xmit: 0
> rx31_xdp_tx_full: 0
> rx31_xdp_tx_err: 0
> rx31_xdp_tx_cqes: 0
> rx32_packets: 0
> rx32_bytes: 0
> rx32_csum_complete: 0
> rx32_csum_unnecessary: 0
> rx32_csum_unnecessary_inner: 0
> rx32_csum_none: 0
> rx32_xdp_drop: 0
> rx32_xdp_redirect: 0
> rx32_lro_packets: 0
> rx32_lro_bytes: 0
> rx32_ecn_mark: 0
> rx32_removed_vlan_packets: 0
> rx32_wqe_err: 0
> rx32_mpwqe_filler_cqes: 0
> rx32_mpwqe_filler_strides: 0
> rx32_buff_alloc_err: 0
> rx32_cqe_compress_blks: 0
> rx32_cqe_compress_pkts: 0
> rx32_page_reuse: 0
> rx32_cache_reuse: 0
> rx32_cache_full: 0
> rx32_cache_empty: 2560
> rx32_cache_busy: 0
> rx32_cache_waive: 0
> rx32_congst_umr: 0
> rx32_arfs_err: 0
> rx32_xdp_tx_xmit: 0
> rx32_xdp_tx_full: 0
> rx32_xdp_tx_err: 0
> rx32_xdp_tx_cqes: 0
> rx33_packets: 0
> rx33_bytes: 0
> rx33_csum_complete: 0
> rx33_csum_unnecessary: 0
> rx33_csum_unnecessary_inner: 0
> rx33_csum_none: 0
> rx33_xdp_drop: 0
> rx33_xdp_redirect: 0
> rx33_lro_packets: 0
> rx33_lro_bytes: 0
> rx33_ecn_mark: 0
> rx33_removed_vlan_packets: 0
> rx33_wqe_err: 0
> rx33_mpwqe_filler_cqes: 0
> rx33_mpwqe_filler_strides: 0
> rx33_buff_alloc_err: 0
> rx33_cqe_compress_blks: 0
> rx33_cqe_compress_pkts: 0
> rx33_page_reuse: 0
> rx33_cache_reuse: 0
> rx33_cache_full: 0
> rx33_cache_empty: 2560
> rx33_cache_busy: 0
> rx33_cache_waive: 0
> rx33_congst_umr: 0
> rx33_arfs_err: 0
> rx33_xdp_tx_xmit: 0
> rx33_xdp_tx_full: 0
> rx33_xdp_tx_err: 0
> rx33_xdp_tx_cqes: 0
> rx34_packets: 0
> rx34_bytes: 0
> rx34_csum_complete: 0
> rx34_csum_unnecessary: 0
> rx34_csum_unnecessary_inner: 0
> rx34_csum_none: 0
> rx34_xdp_drop: 0
> rx34_xdp_redirect: 0
> rx34_lro_packets: 0
> rx34_lro_bytes: 0
> rx34_ecn_mark: 0
> rx34_removed_vlan_packets: 0
> rx34_wqe_err: 0
> rx34_mpwqe_filler_cqes: 0
> rx34_mpwqe_filler_strides: 0
> rx34_buff_alloc_err: 0
> rx34_cqe_compress_blks: 0
> rx34_cqe_compress_pkts: 0
> rx34_page_reuse: 0
> rx34_cache_reuse: 0
> rx34_cache_full: 0
> rx34_cache_empty: 2560
> rx34_cache_busy: 0
> rx34_cache_waive: 0
> rx34_congst_umr: 0
> rx34_arfs_err: 0
> rx34_xdp_tx_xmit: 0
> rx34_xdp_tx_full: 0
> rx34_xdp_tx_err: 0
> rx34_xdp_tx_cqes: 0
> rx35_packets: 0
> rx35_bytes: 0
> rx35_csum_complete: 0
> rx35_csum_unnecessary: 0
> rx35_csum_unnecessary_inner: 0
> rx35_csum_none: 0
> rx35_xdp_drop: 0
> rx35_xdp_redirect: 0
> rx35_lro_packets: 0
> rx35_lro_bytes: 0
> rx35_ecn_mark: 0
> rx35_removed_vlan_packets: 0
> rx35_wqe_err: 0
> rx35_mpwqe_filler_cqes: 0
> rx35_mpwqe_filler_strides: 0
> rx35_buff_alloc_err: 0
> rx35_cqe_compress_blks: 0
> rx35_cqe_compress_pkts: 0
> rx35_page_reuse: 0
> rx35_cache_reuse: 0
> rx35_cache_full: 0
> rx35_cache_empty: 2560
> rx35_cache_busy: 0
> rx35_cache_waive: 0
> rx35_congst_umr: 0
> rx35_arfs_err: 0
> rx35_xdp_tx_xmit: 0
> rx35_xdp_tx_full: 0
> rx35_xdp_tx_err: 0
> rx35_xdp_tx_cqes: 0
> rx36_packets: 0
> rx36_bytes: 0
> rx36_csum_complete: 0
> rx36_csum_unnecessary: 0
> rx36_csum_unnecessary_inner: 0
> rx36_csum_none: 0
> rx36_xdp_drop: 0
> rx36_xdp_redirect: 0
> rx36_lro_packets: 0
> rx36_lro_bytes: 0
> rx36_ecn_mark: 0
> rx36_removed_vlan_packets: 0
> rx36_wqe_err: 0
> rx36_mpwqe_filler_cqes: 0
> rx36_mpwqe_filler_strides: 0
> rx36_buff_alloc_err: 0
> rx36_cqe_compress_blks: 0
> rx36_cqe_compress_pkts: 0
> rx36_page_reuse: 0
> rx36_cache_reuse: 0
> rx36_cache_full: 0
> rx36_cache_empty: 2560
> rx36_cache_busy: 0
> rx36_cache_waive: 0
> rx36_congst_umr: 0
> rx36_arfs_err: 0
> rx36_xdp_tx_xmit: 0
> rx36_xdp_tx_full: 0
> rx36_xdp_tx_err: 0
> rx36_xdp_tx_cqes: 0
> rx37_packets: 0
> rx37_bytes: 0
> rx37_csum_complete: 0
> rx37_csum_unnecessary: 0
> rx37_csum_unnecessary_inner: 0
> rx37_csum_none: 0
> rx37_xdp_drop: 0
> rx37_xdp_redirect: 0
> rx37_lro_packets: 0
> rx37_lro_bytes: 0
> rx37_ecn_mark: 0
> rx37_removed_vlan_packets: 0
> rx37_wqe_err: 0
> rx37_mpwqe_filler_cqes: 0
> rx37_mpwqe_filler_strides: 0
> rx37_buff_alloc_err: 0
> rx37_cqe_compress_blks: 0
> rx37_cqe_compress_pkts: 0
> rx37_page_reuse: 0
> rx37_cache_reuse: 0
> rx37_cache_full: 0
> rx37_cache_empty: 2560
> rx37_cache_busy: 0
> rx37_cache_waive: 0
> rx37_congst_umr: 0
> rx37_arfs_err: 0
> rx37_xdp_tx_xmit: 0
> rx37_xdp_tx_full: 0
> rx37_xdp_tx_err: 0
> rx37_xdp_tx_cqes: 0
> rx38_packets: 0
> rx38_bytes: 0
> rx38_csum_complete: 0
> rx38_csum_unnecessary: 0
> rx38_csum_unnecessary_inner: 0
> rx38_csum_none: 0
> rx38_xdp_drop: 0
> rx38_xdp_redirect: 0
> rx38_lro_packets: 0
> rx38_lro_bytes: 0
> rx38_ecn_mark: 0
> rx38_removed_vlan_packets: 0
> rx38_wqe_err: 0
> rx38_mpwqe_filler_cqes: 0
> rx38_mpwqe_filler_strides: 0
> rx38_buff_alloc_err: 0
> rx38_cqe_compress_blks: 0
> rx38_cqe_compress_pkts: 0
> rx38_page_reuse: 0
> rx38_cache_reuse: 0
> rx38_cache_full: 0
> rx38_cache_empty: 2560
> rx38_cache_busy: 0
> rx38_cache_waive: 0
> rx38_congst_umr: 0
> rx38_arfs_err: 0
> rx38_xdp_tx_xmit: 0
> rx38_xdp_tx_full: 0
> rx38_xdp_tx_err: 0
> rx38_xdp_tx_cqes: 0
> rx39_packets: 0
> rx39_bytes: 0
> rx39_csum_complete: 0
> rx39_csum_unnecessary: 0
> rx39_csum_unnecessary_inner: 0
> rx39_csum_none: 0
> rx39_xdp_drop: 0
> rx39_xdp_redirect: 0
> rx39_lro_packets: 0
> rx39_lro_bytes: 0
> rx39_ecn_mark: 0
> rx39_removed_vlan_packets: 0
> rx39_wqe_err: 0
> rx39_mpwqe_filler_cqes: 0
> rx39_mpwqe_filler_strides: 0
> rx39_buff_alloc_err: 0
> rx39_cqe_compress_blks: 0
> rx39_cqe_compress_pkts: 0
> rx39_page_reuse: 0
> rx39_cache_reuse: 0
> rx39_cache_full: 0
> rx39_cache_empty: 2560
> rx39_cache_busy: 0
> rx39_cache_waive: 0
> rx39_congst_umr: 0
> rx39_arfs_err: 0
> rx39_xdp_tx_xmit: 0
> rx39_xdp_tx_full: 0
> rx39_xdp_tx_err: 0
> rx39_xdp_tx_cqes: 0
> rx40_packets: 0
> rx40_bytes: 0
> rx40_csum_complete: 0
> rx40_csum_unnecessary: 0
> rx40_csum_unnecessary_inner: 0
> rx40_csum_none: 0
> rx40_xdp_drop: 0
> rx40_xdp_redirect: 0
> rx40_lro_packets: 0
> rx40_lro_bytes: 0
> rx40_ecn_mark: 0
> rx40_removed_vlan_packets: 0
> rx40_wqe_err: 0
> rx40_mpwqe_filler_cqes: 0
> rx40_mpwqe_filler_strides: 0
> rx40_buff_alloc_err: 0
> rx40_cqe_compress_blks: 0
> rx40_cqe_compress_pkts: 0
> rx40_page_reuse: 0
> rx40_cache_reuse: 0
> rx40_cache_full: 0
> rx40_cache_empty: 2560
> rx40_cache_busy: 0
> rx40_cache_waive: 0
> rx40_congst_umr: 0
> rx40_arfs_err: 0
> rx40_xdp_tx_xmit: 0
> rx40_xdp_tx_full: 0
> rx40_xdp_tx_err: 0
> rx40_xdp_tx_cqes: 0
> rx41_packets: 0
> rx41_bytes: 0
> rx41_csum_complete: 0
> rx41_csum_unnecessary: 0
> rx41_csum_unnecessary_inner: 0
> rx41_csum_none: 0
> rx41_xdp_drop: 0
> rx41_xdp_redirect: 0
> rx41_lro_packets: 0
> rx41_lro_bytes: 0
> rx41_ecn_mark: 0
> rx41_removed_vlan_packets: 0
> rx41_wqe_err: 0
> rx41_mpwqe_filler_cqes: 0
> rx41_mpwqe_filler_strides: 0
> rx41_buff_alloc_err: 0
> rx41_cqe_compress_blks: 0
> rx41_cqe_compress_pkts: 0
> rx41_page_reuse: 0
> rx41_cache_reuse: 0
> rx41_cache_full: 0
> rx41_cache_empty: 2560
> rx41_cache_busy: 0
> rx41_cache_waive: 0
> rx41_congst_umr: 0
> rx41_arfs_err: 0
> rx41_xdp_tx_xmit: 0
> rx41_xdp_tx_full: 0
> rx41_xdp_tx_err: 0
> rx41_xdp_tx_cqes: 0
> rx42_packets: 0
> rx42_bytes: 0
> rx42_csum_complete: 0
> rx42_csum_unnecessary: 0
> rx42_csum_unnecessary_inner: 0
> rx42_csum_none: 0
> rx42_xdp_drop: 0
> rx42_xdp_redirect: 0
> rx42_lro_packets: 0
> rx42_lro_bytes: 0
> rx42_ecn_mark: 0
> rx42_removed_vlan_packets: 0
> rx42_wqe_err: 0
> rx42_mpwqe_filler_cqes: 0
> rx42_mpwqe_filler_strides: 0
> rx42_buff_alloc_err: 0
> rx42_cqe_compress_blks: 0
> rx42_cqe_compress_pkts: 0
> rx42_page_reuse: 0
> rx42_cache_reuse: 0
> rx42_cache_full: 0
> rx42_cache_empty: 2560
> rx42_cache_busy: 0
> rx42_cache_waive: 0
> rx42_congst_umr: 0
> rx42_arfs_err: 0
> rx42_xdp_tx_xmit: 0
> rx42_xdp_tx_full: 0
> rx42_xdp_tx_err: 0
> rx42_xdp_tx_cqes: 0
> rx43_packets: 0
> rx43_bytes: 0
> rx43_csum_complete: 0
> rx43_csum_unnecessary: 0
> rx43_csum_unnecessary_inner: 0
> rx43_csum_none: 0
> rx43_xdp_drop: 0
> rx43_xdp_redirect: 0
> rx43_lro_packets: 0
> rx43_lro_bytes: 0
> rx43_ecn_mark: 0
> rx43_removed_vlan_packets: 0
> rx43_wqe_err: 0
> rx43_mpwqe_filler_cqes: 0
> rx43_mpwqe_filler_strides: 0
> rx43_buff_alloc_err: 0
> rx43_cqe_compress_blks: 0
> rx43_cqe_compress_pkts: 0
> rx43_page_reuse: 0
> rx43_cache_reuse: 0
> rx43_cache_full: 0
> rx43_cache_empty: 2560
> rx43_cache_busy: 0
> rx43_cache_waive: 0
> rx43_congst_umr: 0
> rx43_arfs_err: 0
> rx43_xdp_tx_xmit: 0
> rx43_xdp_tx_full: 0
> rx43_xdp_tx_err: 0
> rx43_xdp_tx_cqes: 0
> rx44_packets: 0
> rx44_bytes: 0
> rx44_csum_complete: 0
> rx44_csum_unnecessary: 0
> rx44_csum_unnecessary_inner: 0
> rx44_csum_none: 0
> rx44_xdp_drop: 0
> rx44_xdp_redirect: 0
> rx44_lro_packets: 0
> rx44_lro_bytes: 0
> rx44_ecn_mark: 0
> rx44_removed_vlan_packets: 0
> rx44_wqe_err: 0
> rx44_mpwqe_filler_cqes: 0
> rx44_mpwqe_filler_strides: 0
> rx44_buff_alloc_err: 0
> rx44_cqe_compress_blks: 0
> rx44_cqe_compress_pkts: 0
> rx44_page_reuse: 0
> rx44_cache_reuse: 0
> rx44_cache_full: 0
> rx44_cache_empty: 2560
> rx44_cache_busy: 0
> rx44_cache_waive: 0
> rx44_congst_umr: 0
> rx44_arfs_err: 0
> rx44_xdp_tx_xmit: 0
> rx44_xdp_tx_full: 0
> rx44_xdp_tx_err: 0
> rx44_xdp_tx_cqes: 0
> rx45_packets: 0
> rx45_bytes: 0
> rx45_csum_complete: 0
> rx45_csum_unnecessary: 0
> rx45_csum_unnecessary_inner: 0
> rx45_csum_none: 0
> rx45_xdp_drop: 0
> rx45_xdp_redirect: 0
> rx45_lro_packets: 0
> rx45_lro_bytes: 0
> rx45_ecn_mark: 0
> rx45_removed_vlan_packets: 0
> rx45_wqe_err: 0
> rx45_mpwqe_filler_cqes: 0
> rx45_mpwqe_filler_strides: 0
> rx45_buff_alloc_err: 0
> rx45_cqe_compress_blks: 0
> rx45_cqe_compress_pkts: 0
> rx45_page_reuse: 0
> rx45_cache_reuse: 0
> rx45_cache_full: 0
> rx45_cache_empty: 2560
> rx45_cache_busy: 0
> rx45_cache_waive: 0
> rx45_congst_umr: 0
> rx45_arfs_err: 0
> rx45_xdp_tx_xmit: 0
> rx45_xdp_tx_full: 0
> rx45_xdp_tx_err: 0
> rx45_xdp_tx_cqes: 0
> rx46_packets: 0
> rx46_bytes: 0
> rx46_csum_complete: 0
> rx46_csum_unnecessary: 0
> rx46_csum_unnecessary_inner: 0
> rx46_csum_none: 0
> rx46_xdp_drop: 0
> rx46_xdp_redirect: 0
> rx46_lro_packets: 0
> rx46_lro_bytes: 0
> rx46_ecn_mark: 0
> rx46_removed_vlan_packets: 0
> rx46_wqe_err: 0
> rx46_mpwqe_filler_cqes: 0
> rx46_mpwqe_filler_strides: 0
> rx46_buff_alloc_err: 0
> rx46_cqe_compress_blks: 0
> rx46_cqe_compress_pkts: 0
> rx46_page_reuse: 0
> rx46_cache_reuse: 0
> rx46_cache_full: 0
> rx46_cache_empty: 2560
> rx46_cache_busy: 0
> rx46_cache_waive: 0
> rx46_congst_umr: 0
> rx46_arfs_err: 0
> rx46_xdp_tx_xmit: 0
> rx46_xdp_tx_full: 0
> rx46_xdp_tx_err: 0
> rx46_xdp_tx_cqes: 0
> rx47_packets: 0
> rx47_bytes: 0
> rx47_csum_complete: 0
> rx47_csum_unnecessary: 0
> rx47_csum_unnecessary_inner: 0
> rx47_csum_none: 0
> rx47_xdp_drop: 0
> rx47_xdp_redirect: 0
> rx47_lro_packets: 0
> rx47_lro_bytes: 0
> rx47_ecn_mark: 0
> rx47_removed_vlan_packets: 0
> rx47_wqe_err: 0
> rx47_mpwqe_filler_cqes: 0
> rx47_mpwqe_filler_strides: 0
> rx47_buff_alloc_err: 0
> rx47_cqe_compress_blks: 0
> rx47_cqe_compress_pkts: 0
> rx47_page_reuse: 0
> rx47_cache_reuse: 0
> rx47_cache_full: 0
> rx47_cache_empty: 2560
> rx47_cache_busy: 0
> rx47_cache_waive: 0
> rx47_congst_umr: 0
> rx47_arfs_err: 0
> rx47_xdp_tx_xmit: 0
> rx47_xdp_tx_full: 0
> rx47_xdp_tx_err: 0
> rx47_xdp_tx_cqes: 0
> rx48_packets: 0
> rx48_bytes: 0
> rx48_csum_complete: 0
> rx48_csum_unnecessary: 0
> rx48_csum_unnecessary_inner: 0
> rx48_csum_none: 0
> rx48_xdp_drop: 0
> rx48_xdp_redirect: 0
> rx48_lro_packets: 0
> rx48_lro_bytes: 0
> rx48_ecn_mark: 0
> rx48_removed_vlan_packets: 0
> rx48_wqe_err: 0
> rx48_mpwqe_filler_cqes: 0
> rx48_mpwqe_filler_strides: 0
> rx48_buff_alloc_err: 0
> rx48_cqe_compress_blks: 0
> rx48_cqe_compress_pkts: 0
> rx48_page_reuse: 0
> rx48_cache_reuse: 0
> rx48_cache_full: 0
> rx48_cache_empty: 2560
> rx48_cache_busy: 0
> rx48_cache_waive: 0
> rx48_congst_umr: 0
> rx48_arfs_err: 0
> rx48_xdp_tx_xmit: 0
> rx48_xdp_tx_full: 0
> rx48_xdp_tx_err: 0
> rx48_xdp_tx_cqes: 0
> rx49_packets: 0
> rx49_bytes: 0
> rx49_csum_complete: 0
> rx49_csum_unnecessary: 0
> rx49_csum_unnecessary_inner: 0
> rx49_csum_none: 0
> rx49_xdp_drop: 0
> rx49_xdp_redirect: 0
> rx49_lro_packets: 0
> rx49_lro_bytes: 0
> rx49_ecn_mark: 0
> rx49_removed_vlan_packets: 0
> rx49_wqe_err: 0
> rx49_mpwqe_filler_cqes: 0
> rx49_mpwqe_filler_strides: 0
> rx49_buff_alloc_err: 0
> rx49_cqe_compress_blks: 0
> rx49_cqe_compress_pkts: 0
> rx49_page_reuse: 0
> rx49_cache_reuse: 0
> rx49_cache_full: 0
> rx49_cache_empty: 2560
> rx49_cache_busy: 0
> rx49_cache_waive: 0
> rx49_congst_umr: 0
> rx49_arfs_err: 0
> rx49_xdp_tx_xmit: 0
> rx49_xdp_tx_full: 0
> rx49_xdp_tx_err: 0
> rx49_xdp_tx_cqes: 0
> rx50_packets: 0
> rx50_bytes: 0
> rx50_csum_complete: 0
> rx50_csum_unnecessary: 0
> rx50_csum_unnecessary_inner: 0
> rx50_csum_none: 0
> rx50_xdp_drop: 0
> rx50_xdp_redirect: 0
> rx50_lro_packets: 0
> rx50_lro_bytes: 0
> rx50_ecn_mark: 0
> rx50_removed_vlan_packets: 0
> rx50_wqe_err: 0
> rx50_mpwqe_filler_cqes: 0
> rx50_mpwqe_filler_strides: 0
> rx50_buff_alloc_err: 0
> rx50_cqe_compress_blks: 0
> rx50_cqe_compress_pkts: 0
> rx50_page_reuse: 0
> rx50_cache_reuse: 0
> rx50_cache_full: 0
> rx50_cache_empty: 2560
> rx50_cache_busy: 0
> rx50_cache_waive: 0
> rx50_congst_umr: 0
> rx50_arfs_err: 0
> rx50_xdp_tx_xmit: 0
> rx50_xdp_tx_full: 0
> rx50_xdp_tx_err: 0
> rx50_xdp_tx_cqes: 0
> rx51_packets: 0
> rx51_bytes: 0
> rx51_csum_complete: 0
> rx51_csum_unnecessary: 0
> rx51_csum_unnecessary_inner: 0
> rx51_csum_none: 0
> rx51_xdp_drop: 0
> rx51_xdp_redirect: 0
> rx51_lro_packets: 0
> rx51_lro_bytes: 0
> rx51_ecn_mark: 0
> rx51_removed_vlan_packets: 0
> rx51_wqe_err: 0
> rx51_mpwqe_filler_cqes: 0
> rx51_mpwqe_filler_strides: 0
> rx51_buff_alloc_err: 0
> rx51_cqe_compress_blks: 0
> rx51_cqe_compress_pkts: 0
> rx51_page_reuse: 0
> rx51_cache_reuse: 0
> rx51_cache_full: 0
> rx51_cache_empty: 2560
> rx51_cache_busy: 0
> rx51_cache_waive: 0
> rx51_congst_umr: 0
> rx51_arfs_err: 0
> rx51_xdp_tx_xmit: 0
> rx51_xdp_tx_full: 0
> rx51_xdp_tx_err: 0
> rx51_xdp_tx_cqes: 0
> rx52_packets: 0
> rx52_bytes: 0
> rx52_csum_complete: 0
> rx52_csum_unnecessary: 0
> rx52_csum_unnecessary_inner: 0
> rx52_csum_none: 0
> rx52_xdp_drop: 0
> rx52_xdp_redirect: 0
> rx52_lro_packets: 0
> rx52_lro_bytes: 0
> rx52_ecn_mark: 0
> rx52_removed_vlan_packets: 0
> rx52_wqe_err: 0
> rx52_mpwqe_filler_cqes: 0
> rx52_mpwqe_filler_strides: 0
> rx52_buff_alloc_err: 0
> rx52_cqe_compress_blks: 0
> rx52_cqe_compress_pkts: 0
> rx52_page_reuse: 0
> rx52_cache_reuse: 0
> rx52_cache_full: 0
> rx52_cache_empty: 2560
> rx52_cache_busy: 0
> rx52_cache_waive: 0
> rx52_congst_umr: 0
> rx52_arfs_err: 0
> rx52_xdp_tx_xmit: 0
> rx52_xdp_tx_full: 0
> rx52_xdp_tx_err: 0
> rx52_xdp_tx_cqes: 0
> rx53_packets: 0
> rx53_bytes: 0
> rx53_csum_complete: 0
> rx53_csum_unnecessary: 0
> rx53_csum_unnecessary_inner: 0
> rx53_csum_none: 0
> rx53_xdp_drop: 0
> rx53_xdp_redirect: 0
> rx53_lro_packets: 0
> rx53_lro_bytes: 0
> rx53_ecn_mark: 0
> rx53_removed_vlan_packets: 0
> rx53_wqe_err: 0
> rx53_mpwqe_filler_cqes: 0
> rx53_mpwqe_filler_strides: 0
> rx53_buff_alloc_err: 0
> rx53_cqe_compress_blks: 0
> rx53_cqe_compress_pkts: 0
> rx53_page_reuse: 0
> rx53_cache_reuse: 0
> rx53_cache_full: 0
> rx53_cache_empty: 2560
> rx53_cache_busy: 0
> rx53_cache_waive: 0
> rx53_congst_umr: 0
> rx53_arfs_err: 0
> rx53_xdp_tx_xmit: 0
> rx53_xdp_tx_full: 0
> rx53_xdp_tx_err: 0
> rx53_xdp_tx_cqes: 0
> rx54_packets: 0
> rx54_bytes: 0
> rx54_csum_complete: 0
> rx54_csum_unnecessary: 0
> rx54_csum_unnecessary_inner: 0
> rx54_csum_none: 0
> rx54_xdp_drop: 0
> rx54_xdp_redirect: 0
> rx54_lro_packets: 0
> rx54_lro_bytes: 0
> rx54_ecn_mark: 0
> rx54_removed_vlan_packets: 0
> rx54_wqe_err: 0
> rx54_mpwqe_filler_cqes: 0
> rx54_mpwqe_filler_strides: 0
> rx54_buff_alloc_err: 0
> rx54_cqe_compress_blks: 0
> rx54_cqe_compress_pkts: 0
> rx54_page_reuse: 0
> rx54_cache_reuse: 0
> rx54_cache_full: 0
> rx54_cache_empty: 2560
> rx54_cache_busy: 0
> rx54_cache_waive: 0
> rx54_congst_umr: 0
> rx54_arfs_err: 0
> rx54_xdp_tx_xmit: 0
> rx54_xdp_tx_full: 0
> rx54_xdp_tx_err: 0
> rx54_xdp_tx_cqes: 0
> rx55_packets: 0
> rx55_bytes: 0
> rx55_csum_complete: 0
> rx55_csum_unnecessary: 0
> rx55_csum_unnecessary_inner: 0
> rx55_csum_none: 0
> rx55_xdp_drop: 0
> rx55_xdp_redirect: 0
> rx55_lro_packets: 0
> rx55_lro_bytes: 0
> rx55_ecn_mark: 0
> rx55_removed_vlan_packets: 0
> rx55_wqe_err: 0
> rx55_mpwqe_filler_cqes: 0
> rx55_mpwqe_filler_strides: 0
> rx55_buff_alloc_err: 0
> rx55_cqe_compress_blks: 0
> rx55_cqe_compress_pkts: 0
> rx55_page_reuse: 0
> rx55_cache_reuse: 0
> rx55_cache_full: 0
> rx55_cache_empty: 2560
> rx55_cache_busy: 0
> rx55_cache_waive: 0
> rx55_congst_umr: 0
> rx55_arfs_err: 0
> rx55_xdp_tx_xmit: 0
> rx55_xdp_tx_full: 0
> rx55_xdp_tx_err: 0
> rx55_xdp_tx_cqes: 0
> tx0_packets: 5868971166
> tx0_bytes: 7384241881537
> tx0_tso_packets: 1005089669
> tx0_tso_bytes: 5138882499687
> tx0_tso_inner_packets: 0
> tx0_tso_inner_bytes: 0
> tx0_csum_partial: 1405330470
> tx0_csum_partial_inner: 0
> tx0_added_vlan_packets: 3247061022
> tx0_nop: 83925216
> tx0_csum_none: 1841730552
> tx0_stopped: 0
> tx0_dropped: 0
> tx0_xmit_more: 29664303
> tx0_recover: 0
> tx0_cqes: 3217398842
> tx0_wake: 0
> tx0_cqe_err: 0
> tx1_packets: 5599378674
> tx1_bytes: 7272236466962
> tx1_tso_packets: 1024612268
> tx1_tso_bytes: 5244192050917
> tx1_tso_inner_packets: 0
> tx1_tso_inner_bytes: 0
> tx1_csum_partial: 1438007932
> tx1_csum_partial_inner: 0
> tx1_added_vlan_packets: 2919765857
> tx1_nop: 79661231
> tx1_csum_none: 1481757925
> tx1_stopped: 0
> tx1_dropped: 0
> tx1_xmit_more: 29485355
> tx1_recover: 0
> tx1_cqes: 2890282176
> tx1_wake: 0
> tx1_cqe_err: 0
> tx2_packets: 5413821094
> tx2_bytes: 7033951631334
> tx2_tso_packets: 1002868589
> tx2_tso_bytes: 5089549008985
> tx2_tso_inner_packets: 0
> tx2_tso_inner_bytes: 0
> tx2_csum_partial: 1404186175
> tx2_csum_partial_inner: 0
> tx2_added_vlan_packets: 2822670460
> tx2_nop: 77115408
> tx2_csum_none: 1418484285
> tx2_stopped: 0
> tx2_dropped: 0
> tx2_xmit_more: 29321129
> tx2_recover: 0
> tx2_cqes: 2793351019
> tx2_wake: 0
> tx2_cqe_err: 0
> tx3_packets: 5479609727
> tx3_bytes: 7116904107659
> tx3_tso_packets: 1002992639
> tx3_tso_bytes: 5154225081979
> tx3_tso_inner_packets: 0
> tx3_tso_inner_bytes: 0
> tx3_csum_partial: 1415739849
> tx3_csum_partial_inner: 0
> tx3_added_vlan_packets: 2842823811
> tx3_nop: 78060813
> tx3_csum_none: 1427083971
> tx3_stopped: 0
> tx3_dropped: 0
> tx3_xmit_more: 28575040
> tx3_recover: 0
> tx3_cqes: 2814250785
> tx3_wake: 0
> tx3_cqe_err: 0
> tx4_packets: 5508297397
> tx4_bytes: 7127659369902
> tx4_tso_packets: 1007356432
> tx4_tso_bytes: 5145975736034
> tx4_tso_inner_packets: 0
> tx4_tso_inner_bytes: 0
> tx4_csum_partial: 1411271000
> tx4_csum_partial_inner: 0
> tx4_added_vlan_packets: 2882086825
> tx4_nop: 78433610
> tx4_csum_none: 1470815825
> tx4_stopped: 0
> tx4_dropped: 0
> tx4_xmit_more: 28632444
> tx4_recover: 0
> tx4_cqes: 2853456464
> tx4_wake: 0
> tx4_cqe_err: 0
> tx5_packets: 5513864156
> tx5_bytes: 7165864145517
> tx5_tso_packets: 1014046485
> tx5_tso_bytes: 5192635614477
> tx5_tso_inner_packets: 0
> tx5_tso_inner_bytes: 0
> tx5_csum_partial: 1420810473
> tx5_csum_partial_inner: 0
> tx5_added_vlan_packets: 2861370556
> tx5_nop: 78481355
> tx5_csum_none: 1440560083
> tx5_stopped: 0
> tx5_dropped: 0
> tx5_xmit_more: 28222467
> tx5_recover: 0
> tx5_cqes: 2833149758
> tx5_wake: 0
> tx5_cqe_err: 0
> tx6_packets: 5560724761
> tx6_bytes: 7210309972086
> tx6_tso_packets: 994050514
> tx6_tso_bytes: 5171393741595
> tx6_tso_inner_packets: 0
> tx6_tso_inner_bytes: 0
> tx6_csum_partial: 1414303265
> tx6_csum_partial_inner: 0
> tx6_added_vlan_packets: 2905794177
> tx6_nop: 79353318
> tx6_csum_none: 1491490912
> tx6_stopped: 0
> tx6_dropped: 0
> tx6_xmit_more: 31246664
> tx6_recover: 0
> tx6_cqes: 2874549217
> tx6_wake: 0
> tx6_cqe_err: 0
> tx7_packets: 5557594170
> tx7_bytes: 7223138778685
> tx7_tso_packets: 1013475396
> tx7_tso_bytes: 5241530065484
> tx7_tso_inner_packets: 0
> tx7_tso_inner_bytes: 0
> tx7_csum_partial: 1438604314
> tx7_csum_partial_inner: 0
> tx7_added_vlan_packets: 2873917552
> tx7_nop: 79057059
> tx7_csum_none: 1435313239
> tx7_stopped: 0
> tx7_dropped: 0
> tx7_xmit_more: 29258761
> tx7_recover: 0
> tx7_cqes: 2844660578
> tx7_wake: 0
> tx7_cqe_err: 0
> tx8_packets: 5521254733
> tx8_bytes: 7208043146297
> tx8_tso_packets: 1014670801
> tx8_tso_bytes: 5185842447246
> tx8_tso_inner_packets: 0
> tx8_tso_inner_bytes: 0
> tx8_csum_partial: 1431631562
> tx8_csum_partial_inner: 0
> tx8_added_vlan_packets: 2872641129
> tx8_nop: 78545776
> tx8_csum_none: 1441009567
> tx8_stopped: 0
> tx8_dropped: 0
> tx8_xmit_more: 29106291
> tx8_recover: 0
> tx8_cqes: 2843536748
> tx8_wake: 0
> tx8_cqe_err: 0
> tx9_packets: 5528889957
> tx9_bytes: 7191793816058
> tx9_tso_packets: 1015955476
> tx9_tso_bytes: 5207232047828
> tx9_tso_inner_packets: 0
> tx9_tso_inner_bytes: 0
> tx9_csum_partial: 1421266796
> tx9_csum_partial_inner: 0
> tx9_added_vlan_packets: 2869523921
> tx9_nop: 78586218
> tx9_csum_none: 1448257125
> tx9_stopped: 0
> tx9_dropped: 0
> tx9_xmit_more: 29483347
> tx9_recover: 0
> tx9_cqes: 2840042245
> tx9_wake: 0
> tx9_cqe_err: 0
> tx10_packets: 5556351222
> tx10_bytes: 7254798330757
> tx10_tso_packets: 1028554460
> tx10_tso_bytes: 5246179615774
> tx10_tso_inner_packets: 0
> tx10_tso_inner_bytes: 0
> tx10_csum_partial: 1430459021
> tx10_csum_partial_inner: 0
> tx10_added_vlan_packets: 2881683382
> tx10_nop: 79139584
> tx10_csum_none: 1451224361
> tx10_stopped: 0
> tx10_dropped: 0
> tx10_xmit_more: 29217190
> tx10_recover: 0
> tx10_cqes: 2852467898
> tx10_wake: 0
> tx10_cqe_err: 0
> tx11_packets: 5455631854
> tx11_bytes: 7061121713772
> tx11_tso_packets: 992133383
> tx11_tso_bytes: 5089419722682
> tx11_tso_inner_packets: 0
> tx11_tso_inner_bytes: 0
> tx11_csum_partial: 1395542033
> tx11_csum_partial_inner: 0
> tx11_added_vlan_packets: 2852589093
> tx11_nop: 77799857
> tx11_csum_none: 1457047060
> tx11_stopped: 0
> tx11_dropped: 0
> tx11_xmit_more: 29559927
> tx11_recover: 0
> tx11_cqes: 2823031110
> tx11_wake: 0
> tx11_cqe_err: 0
> tx12_packets: 5488286808
> tx12_bytes: 7137087569303
> tx12_tso_packets: 1006435537
> tx12_tso_bytes: 5163371416750
> tx12_tso_inner_packets: 0
> tx12_tso_inner_bytes: 0
> tx12_csum_partial: 1414799411
> tx12_csum_partial_inner: 0
> tx12_added_vlan_packets: 2841679543
> tx12_nop: 78387039
> tx12_csum_none: 1426880132
> tx12_stopped: 0
> tx12_dropped: 0
> tx12_xmit_more: 28607526
> tx12_recover: 0
> tx12_cqes: 2813073557
> tx12_wake: 0
> tx12_cqe_err: 0
> tx13_packets: 5594132290
> tx13_bytes: 7251106284829
> tx13_tso_packets: 1035172061
> tx13_tso_bytes: 5251200286298
> tx13_tso_inner_packets: 0
> tx13_tso_inner_bytes: 0
> tx13_csum_partial: 1443665981
> tx13_csum_partial_inner: 0
> tx13_added_vlan_packets: 2916604799
> tx13_nop: 79670465
> tx13_csum_none: 1472938818
> tx13_stopped: 0
> tx13_dropped: 0
> tx13_xmit_more: 27797067
> tx13_recover: 0
> tx13_cqes: 2888809352
> tx13_wake: 0
> tx13_cqe_err: 0
> tx14_packets: 5548790952
> tx14_bytes: 7194211868411
> tx14_tso_packets: 1021015561
> tx14_tso_bytes: 5231483708869
> tx14_tso_inner_packets: 0
> tx14_tso_inner_bytes: 0
> tx14_csum_partial: 1427711576
> tx14_csum_partial_inner: 0
> tx14_added_vlan_packets: 2875288572
> tx14_nop: 78900224
> tx14_csum_none: 1447576996
> tx14_stopped: 0
> tx14_dropped: 0
> tx14_xmit_more: 30003496
> tx14_recover: 0
> tx14_cqes: 2845286732
> tx14_wake: 0
> tx14_cqe_err: 0
> tx15_packets: 5609310963
> tx15_bytes: 7271380831798
> tx15_tso_packets: 1027830118
> tx15_tso_bytes: 5229697431506
> tx15_tso_inner_packets: 0
> tx15_tso_inner_bytes: 0
> tx15_csum_partial: 1429209941
> tx15_csum_partial_inner: 0
> tx15_added_vlan_packets: 2940315402
> tx15_nop: 79950883
> tx15_csum_none: 1511105462
> tx15_stopped: 0
> tx15_dropped: 0
> tx15_xmit_more: 28820740
> tx15_recover: 0
> tx15_cqes: 2911496633
> tx15_wake: 0
> tx15_cqe_err: 0
> tx16_packets: 4465363036
> tx16_bytes: 5769771803704
> tx16_tso_packets: 817101913
> tx16_tso_bytes: 4180172833814
> tx16_tso_inner_packets: 0
> tx16_tso_inner_bytes: 0
> tx16_csum_partial: 1136731404
> tx16_csum_partial_inner: 0
> tx16_added_vlan_packets: 2332178232
> tx16_nop: 63458573
> tx16_csum_none: 1195446828
> tx16_stopped: 0
> tx16_dropped: 0
> tx16_xmit_more: 23756254
> tx16_recover: 0
> tx16_cqes: 2308423025
> tx16_wake: 0
> tx16_cqe_err: 0
> tx17_packets: 4380386348
> tx17_bytes: 5708702994526
> tx17_tso_packets: 813638023
> tx17_tso_bytes: 4130806014947
> tx17_tso_inner_packets: 0
> tx17_tso_inner_bytes: 0
> tx17_csum_partial: 1133007164
> tx17_csum_partial_inner: 0
> tx17_added_vlan_packets: 2277314787
> tx17_nop: 62377372
> tx17_csum_none: 1144307623
> tx17_stopped: 0
> tx17_dropped: 0
> tx17_xmit_more: 23731361
> tx17_recover: 0
> tx17_cqes: 2253584638
> tx17_wake: 0
> tx17_cqe_err: 0
> tx18_packets: 4450359743
> tx18_bytes: 5758968674820
> tx18_tso_packets: 815791601
> tx18_tso_bytes: 4179942688909
> tx18_tso_inner_packets: 0
> tx18_tso_inner_bytes: 0
> tx18_csum_partial: 1137649257
> tx18_csum_partial_inner: 0
> tx18_added_vlan_packets: 2314556550
> tx18_nop: 63271085
> tx18_csum_none: 1176907293
> tx18_stopped: 0
> tx18_dropped: 0
> tx18_xmit_more: 23055770
> tx18_recover: 0
> tx18_cqes: 2291501928
> tx18_wake: 0
> tx18_cqe_err: 0
> tx19_packets: 4596064378
> tx19_bytes: 5916675706535
> tx19_tso_packets: 825788649
> tx19_tso_bytes: 4208046929921
> tx19_tso_inner_packets: 0
> tx19_tso_inner_bytes: 0
> tx19_csum_partial: 1150666569
> tx19_csum_partial_inner: 0
> tx19_added_vlan_packets: 2450567026
> tx19_nop: 65468504
> tx19_csum_none: 1299900457
> tx19_stopped: 0
> tx19_dropped: 0
> tx19_xmit_more: 23846250
> tx19_recover: 0
> tx19_cqes: 2426722127
> tx19_wake: 0
> tx19_cqe_err: 0
> tx20_packets: 4424935388
> tx20_bytes: 5757631205901
> tx20_tso_packets: 804875006
> tx20_tso_bytes: 4156262736109
> tx20_tso_inner_packets: 0
> tx20_tso_inner_bytes: 0
> tx20_csum_partial: 1134144916
> tx20_csum_partial_inner: 0
> tx20_added_vlan_packets: 2294839665
> tx20_nop: 63023986
> tx20_csum_none: 1160694749
> tx20_stopped: 0
> tx20_dropped: 0
> tx20_xmit_more: 23393201
> tx20_recover: 0
> tx20_cqes: 2271447623
> tx20_wake: 0
> tx20_cqe_err: 0
> tx21_packets: 4595062285
> tx21_bytes: 5958671993467
> tx21_tso_packets: 821936215
> tx21_tso_bytes: 4187977870684
> tx21_tso_inner_packets: 0
> tx21_tso_inner_bytes: 0
> tx21_csum_partial: 1143339787
> tx21_csum_partial_inner: 0
> tx21_added_vlan_packets: 2457167412
> tx21_nop: 65697763
> tx21_csum_none: 1313827625
> tx21_stopped: 0
> tx21_dropped: 0
> tx21_xmit_more: 23858345
> tx21_recover: 0
> tx21_cqes: 2433310348
> tx21_wake: 0
> tx21_cqe_err: 0
> tx22_packets: 4664446513
> tx22_bytes: 5931429292082
> tx22_tso_packets: 814457881
> tx22_tso_bytes: 4148607956533
> tx22_tso_inner_packets: 0
> tx22_tso_inner_bytes: 0
> tx22_csum_partial: 1127284783
> tx22_csum_partial_inner: 0
> tx22_added_vlan_packets: 2548650146
> tx22_nop: 66299909
> tx22_csum_none: 1421365363
> tx22_stopped: 0
> tx22_dropped: 0
> tx22_xmit_more: 23800911
> tx22_recover: 0
> tx22_cqes: 2524850415
> tx22_wake: 0
> tx22_cqe_err: 0
> tx23_packets: 4416221747
> tx23_bytes: 5721472587985
> tx23_tso_packets: 823538520
> tx23_tso_bytes: 4163520218617
> tx23_tso_inner_packets: 0
> tx23_tso_inner_bytes: 0
> tx23_csum_partial: 1135996006
> tx23_csum_partial_inner: 0
> tx23_added_vlan_packets: 2292404120
> tx23_nop: 62709432
> tx23_csum_none: 1156408114
> tx23_stopped: 0
> tx23_dropped: 0
> tx23_xmit_more: 22299889
> tx23_recover: 0
> tx23_cqes: 2270105487
> tx23_wake: 0
> tx23_cqe_err: 0
> tx24_packets: 4420014824
> tx24_bytes: 5740767318521
> tx24_tso_packets: 820838072
> tx24_tso_bytes: 4183722948422
> tx24_tso_inner_packets: 0
> tx24_tso_inner_bytes: 0
> tx24_csum_partial: 1138070059
> tx24_csum_partial_inner: 0
> tx24_added_vlan_packets: 2289043946
> tx24_nop: 62797341
> tx24_csum_none: 1150973887
> tx24_stopped: 0
> tx24_dropped: 0
> tx24_xmit_more: 22744690
> tx24_recover: 0
> tx24_cqes: 2266300568
> tx24_wake: 0
> tx24_cqe_err: 0
> tx25_packets: 4413225545
> tx25_bytes: 5716162617155
> tx25_tso_packets: 808274341
> tx25_tso_bytes: 4138408857714
> tx25_tso_inner_packets: 0
> tx25_tso_inner_bytes: 0
> tx25_csum_partial: 1134587898
> tx25_csum_partial_inner: 0
> tx25_added_vlan_packets: 2297149310
> tx25_nop: 62958238
> tx25_csum_none: 1162561412
> tx25_stopped: 0
> tx25_dropped: 0
> tx25_xmit_more: 24463552
> tx25_recover: 0
> tx25_cqes: 2272686971
> tx25_wake: 0
> tx25_cqe_err: 0
> tx26_packets: 4524907591
> tx26_bytes: 5865394280699
> tx26_tso_packets: 807270022
> tx26_tso_bytes: 4148754705317
> tx26_tso_inner_packets: 0
> tx26_tso_inner_bytes: 0
> tx26_csum_partial: 1130306933
> tx26_csum_partial_inner: 0
> tx26_added_vlan_packets: 2402682460
> tx26_nop: 64474322
> tx26_csum_none: 1272375527
> tx26_stopped: 1
> tx26_dropped: 0
> tx26_xmit_more: 23316186
> tx26_recover: 0
> tx26_cqes: 2379367502
> tx26_wake: 1
> tx26_cqe_err: 0
> tx27_packets: 4376114969
> tx27_bytes: 5683551238304
> tx27_tso_packets: 809344829
> tx27_tso_bytes: 4124331859270
> tx27_tso_inner_packets: 0
> tx27_tso_inner_bytes: 0
> tx27_csum_partial: 1124954937
> tx27_csum_partial_inner: 0
> tx27_added_vlan_packets: 2267871300
> tx27_nop: 62213214
> tx27_csum_none: 1142916363
> tx27_stopped: 0
> tx27_dropped: 0
> tx27_xmit_more: 23369974
> tx27_recover: 0
> tx27_cqes: 2244502686
> tx27_wake: 0
> tx27_cqe_err: 0
> tx28_packets: 3
> tx28_bytes: 266
> tx28_tso_packets: 0
> tx28_tso_bytes: 0
> tx28_tso_inner_packets: 0
> tx28_tso_inner_bytes: 0
> tx28_csum_partial: 0
> tx28_csum_partial_inner: 0
> tx28_added_vlan_packets: 0
> tx28_nop: 0
> tx28_csum_none: 3
> tx28_stopped: 0
> tx28_dropped: 0
> tx28_xmit_more: 0
> tx28_recover: 0
> tx28_cqes: 3
> tx28_wake: 0
> tx28_cqe_err: 0
> tx29_packets: 0
> tx29_bytes: 0
> tx29_tso_packets: 0
> tx29_tso_bytes: 0
> tx29_tso_inner_packets: 0
> tx29_tso_inner_bytes: 0
> tx29_csum_partial: 0
> tx29_csum_partial_inner: 0
> tx29_added_vlan_packets: 0
> tx29_nop: 0
> tx29_csum_none: 0
> tx29_stopped: 0
> tx29_dropped: 0
> tx29_xmit_more: 0
> tx29_recover: 0
> tx29_cqes: 0
> tx29_wake: 0
> tx29_cqe_err: 0
> tx30_packets: 0
> tx30_bytes: 0
> tx30_tso_packets: 0
> tx30_tso_bytes: 0
> tx30_tso_inner_packets: 0
> tx30_tso_inner_bytes: 0
> tx30_csum_partial: 0
> tx30_csum_partial_inner: 0
> tx30_added_vlan_packets: 0
> tx30_nop: 0
> tx30_csum_none: 0
> tx30_stopped: 0
> tx30_dropped: 0
> tx30_xmit_more: 0
> tx30_recover: 0
> tx30_cqes: 0
> tx30_wake: 0
> tx30_cqe_err: 0
> tx31_packets: 0
> tx31_bytes: 0
> tx31_tso_packets: 0
> tx31_tso_bytes: 0
> tx31_tso_inner_packets: 0
> tx31_tso_inner_bytes: 0
> tx31_csum_partial: 0
> tx31_csum_partial_inner: 0
> tx31_added_vlan_packets: 0
> tx31_nop: 0
> tx31_csum_none: 0
> tx31_stopped: 0
> tx31_dropped: 0
> tx31_xmit_more: 0
> tx31_recover: 0
> tx31_cqes: 0
> tx31_wake: 0
> tx31_cqe_err: 0
> tx32_packets: 0
> tx32_bytes: 0
> tx32_tso_packets: 0
> tx32_tso_bytes: 0
> tx32_tso_inner_packets: 0
> tx32_tso_inner_bytes: 0
> tx32_csum_partial: 0
> tx32_csum_partial_inner: 0
> tx32_added_vlan_packets: 0
> tx32_nop: 0
> tx32_csum_none: 0
> tx32_stopped: 0
> tx32_dropped: 0
> tx32_xmit_more: 0
> tx32_recover: 0
> tx32_cqes: 0
> tx32_wake: 0
> tx32_cqe_err: 0
> tx33_packets: 0
> tx33_bytes: 0
> tx33_tso_packets: 0
> tx33_tso_bytes: 0
> tx33_tso_inner_packets: 0
> tx33_tso_inner_bytes: 0
> tx33_csum_partial: 0
> tx33_csum_partial_inner: 0
> tx33_added_vlan_packets: 0
> tx33_nop: 0
> tx33_csum_none: 0
> tx33_stopped: 0
> tx33_dropped: 0
> tx33_xmit_more: 0
> tx33_recover: 0
> tx33_cqes: 0
> tx33_wake: 0
> tx33_cqe_err: 0
> tx34_packets: 0
> tx34_bytes: 0
> tx34_tso_packets: 0
> tx34_tso_bytes: 0
> tx34_tso_inner_packets: 0
> tx34_tso_inner_bytes: 0
> tx34_csum_partial: 0
> tx34_csum_partial_inner: 0
> tx34_added_vlan_packets: 0
> tx34_nop: 0
> tx34_csum_none: 0
> tx34_stopped: 0
> tx34_dropped: 0
> tx34_xmit_more: 0
> tx34_recover: 0
> tx34_cqes: 0
> tx34_wake: 0
> tx34_cqe_err: 0
> tx35_packets: 0
> tx35_bytes: 0
> tx35_tso_packets: 0
> tx35_tso_bytes: 0
> tx35_tso_inner_packets: 0
> tx35_tso_inner_bytes: 0
> tx35_csum_partial: 0
> tx35_csum_partial_inner: 0
> tx35_added_vlan_packets: 0
> tx35_nop: 0
> tx35_csum_none: 0
> tx35_stopped: 0
> tx35_dropped: 0
> tx35_xmit_more: 0
> tx35_recover: 0
> tx35_cqes: 0
> tx35_wake: 0
> tx35_cqe_err: 0
> tx36_packets: 0
> tx36_bytes: 0
> tx36_tso_packets: 0
> tx36_tso_bytes: 0
> tx36_tso_inner_packets: 0
> tx36_tso_inner_bytes: 0
> tx36_csum_partial: 0
> tx36_csum_partial_inner: 0
> tx36_added_vlan_packets: 0
> tx36_nop: 0
> tx36_csum_none: 0
> tx36_stopped: 0
> tx36_dropped: 0
> tx36_xmit_more: 0
> tx36_recover: 0
> tx36_cqes: 0
> tx36_wake: 0
> tx36_cqe_err: 0
> tx37_packets: 0
> tx37_bytes: 0
> tx37_tso_packets: 0
> tx37_tso_bytes: 0
> tx37_tso_inner_packets: 0
> tx37_tso_inner_bytes: 0
> tx37_csum_partial: 0
> tx37_csum_partial_inner: 0
> tx37_added_vlan_packets: 0
> tx37_nop: 0
> tx37_csum_none: 0
> tx37_stopped: 0
> tx37_dropped: 0
> tx37_xmit_more: 0
> tx37_recover: 0
> tx37_cqes: 0
> tx37_wake: 0
> tx37_cqe_err: 0
> tx38_packets: 0
> tx38_bytes: 0
> tx38_tso_packets: 0
> tx38_tso_bytes: 0
> tx38_tso_inner_packets: 0
> tx38_tso_inner_bytes: 0
> tx38_csum_partial: 0
> tx38_csum_partial_inner: 0
> tx38_added_vlan_packets: 0
> tx38_nop: 0
> tx38_csum_none: 0
> tx38_stopped: 0
> tx38_dropped: 0
> tx38_xmit_more: 0
> tx38_recover: 0
> tx38_cqes: 0
> tx38_wake: 0
> tx38_cqe_err: 0
> tx39_packets: 0
> tx39_bytes: 0
> tx39_tso_packets: 0
> tx39_tso_bytes: 0
> tx39_tso_inner_packets: 0
> tx39_tso_inner_bytes: 0
> tx39_csum_partial: 0
> tx39_csum_partial_inner: 0
> tx39_added_vlan_packets: 0
> tx39_nop: 0
> tx39_csum_none: 0
> tx39_stopped: 0
> tx39_dropped: 0
> tx39_xmit_more: 0
> tx39_recover: 0
> tx39_cqes: 0
> tx39_wake: 0
> tx39_cqe_err: 0
> tx40_packets: 0
> tx40_bytes: 0
> tx40_tso_packets: 0
> tx40_tso_bytes: 0
> tx40_tso_inner_packets: 0
> tx40_tso_inner_bytes: 0
> tx40_csum_partial: 0
> tx40_csum_partial_inner: 0
> tx40_added_vlan_packets: 0
> tx40_nop: 0
> tx40_csum_none: 0
> tx40_stopped: 0
> tx40_dropped: 0
> tx40_xmit_more: 0
> tx40_recover: 0
> tx40_cqes: 0
> tx40_wake: 0
> tx40_cqe_err: 0
> tx41_packets: 0
> tx41_bytes: 0
> tx41_tso_packets: 0
> tx41_tso_bytes: 0
> tx41_tso_inner_packets: 0
> tx41_tso_inner_bytes: 0
> tx41_csum_partial: 0
> tx41_csum_partial_inner: 0
> tx41_added_vlan_packets: 0
> tx41_nop: 0
> tx41_csum_none: 0
> tx41_stopped: 0
> tx41_dropped: 0
> tx41_xmit_more: 0
> tx41_recover: 0
> tx41_cqes: 0
> tx41_wake: 0
> tx41_cqe_err: 0
> tx42_packets: 0
> tx42_bytes: 0
> tx42_tso_packets: 0
> tx42_tso_bytes: 0
> tx42_tso_inner_packets: 0
> tx42_tso_inner_bytes: 0
> tx42_csum_partial: 0
> tx42_csum_partial_inner: 0
> tx42_added_vlan_packets: 0
> tx42_nop: 0
> tx42_csum_none: 0
> tx42_stopped: 0
> tx42_dropped: 0
> tx42_xmit_more: 0
> tx42_recover: 0
> tx42_cqes: 0
> tx42_wake: 0
> tx42_cqe_err: 0
> tx43_packets: 0
> tx43_bytes: 0
> tx43_tso_packets: 0
> tx43_tso_bytes: 0
> tx43_tso_inner_packets: 0
> tx43_tso_inner_bytes: 0
> tx43_csum_partial: 0
> tx43_csum_partial_inner: 0
> tx43_added_vlan_packets: 0
> tx43_nop: 0
> tx43_csum_none: 0
> tx43_stopped: 0
> tx43_dropped: 0
> tx43_xmit_more: 0
> tx43_recover: 0
> tx43_cqes: 0
> tx43_wake: 0
> tx43_cqe_err: 0
> tx44_packets: 0
> tx44_bytes: 0
> tx44_tso_packets: 0
> tx44_tso_bytes: 0
> tx44_tso_inner_packets: 0
> tx44_tso_inner_bytes: 0
> tx44_csum_partial: 0
> tx44_csum_partial_inner: 0
> tx44_added_vlan_packets: 0
> tx44_nop: 0
> tx44_csum_none: 0
> tx44_stopped: 0
> tx44_dropped: 0
> tx44_xmit_more: 0
> tx44_recover: 0
> tx44_cqes: 0
> tx44_wake: 0
> tx44_cqe_err: 0
> tx45_packets: 0
> tx45_bytes: 0
> tx45_tso_packets: 0
> tx45_tso_bytes: 0
> tx45_tso_inner_packets: 0
> tx45_tso_inner_bytes: 0
> tx45_csum_partial: 0
> tx45_csum_partial_inner: 0
> tx45_added_vlan_packets: 0
> tx45_nop: 0
> tx45_csum_none: 0
> tx45_stopped: 0
> tx45_dropped: 0
> tx45_xmit_more: 0
> tx45_recover: 0
> tx45_cqes: 0
> tx45_wake: 0
> tx45_cqe_err: 0
> tx46_packets: 0
> tx46_bytes: 0
> tx46_tso_packets: 0
> tx46_tso_bytes: 0
> tx46_tso_inner_packets: 0
> tx46_tso_inner_bytes: 0
> tx46_csum_partial: 0
> tx46_csum_partial_inner: 0
> tx46_added_vlan_packets: 0
> tx46_nop: 0
> tx46_csum_none: 0
> tx46_stopped: 0
> tx46_dropped: 0
> tx46_xmit_more: 0
> tx46_recover: 0
> tx46_cqes: 0
> tx46_wake: 0
> tx46_cqe_err: 0
> tx47_packets: 0
> tx47_bytes: 0
> tx47_tso_packets: 0
> tx47_tso_bytes: 0
> tx47_tso_inner_packets: 0
> tx47_tso_inner_bytes: 0
> tx47_csum_partial: 0
> tx47_csum_partial_inner: 0
> tx47_added_vlan_packets: 0
> tx47_nop: 0
> tx47_csum_none: 0
> tx47_stopped: 0
> tx47_dropped: 0
> tx47_xmit_more: 0
> tx47_recover: 0
> tx47_cqes: 0
> tx47_wake: 0
> tx47_cqe_err: 0
> tx48_packets: 0
> tx48_bytes: 0
> tx48_tso_packets: 0
> tx48_tso_bytes: 0
> tx48_tso_inner_packets: 0
> tx48_tso_inner_bytes: 0
> tx48_csum_partial: 0
> tx48_csum_partial_inner: 0
> tx48_added_vlan_packets: 0
> tx48_nop: 0
> tx48_csum_none: 0
> tx48_stopped: 0
> tx48_dropped: 0
> tx48_xmit_more: 0
> tx48_recover: 0
> tx48_cqes: 0
> tx48_wake: 0
> tx48_cqe_err: 0
> tx49_packets: 0
> tx49_bytes: 0
> tx49_tso_packets: 0
> tx49_tso_bytes: 0
> tx49_tso_inner_packets: 0
> tx49_tso_inner_bytes: 0
> tx49_csum_partial: 0
> tx49_csum_partial_inner: 0
> tx49_added_vlan_packets: 0
> tx49_nop: 0
> tx49_csum_none: 0
> tx49_stopped: 0
> tx49_dropped: 0
> tx49_xmit_more: 0
> tx49_recover: 0
> tx49_cqes: 0
> tx49_wake: 0
> tx49_cqe_err: 0
> tx50_packets: 0
> tx50_bytes: 0
> tx50_tso_packets: 0
> tx50_tso_bytes: 0
> tx50_tso_inner_packets: 0
> tx50_tso_inner_bytes: 0
> tx50_csum_partial: 0
> tx50_csum_partial_inner: 0
> tx50_added_vlan_packets: 0
> tx50_nop: 0
> tx50_csum_none: 0
> tx50_stopped: 0
> tx50_dropped: 0
> tx50_xmit_more: 0
> tx50_recover: 0
> tx50_cqes: 0
> tx50_wake: 0
> tx50_cqe_err: 0
> tx51_packets: 0
> tx51_bytes: 0
> tx51_tso_packets: 0
> tx51_tso_bytes: 0
> tx51_tso_inner_packets: 0
> tx51_tso_inner_bytes: 0
> tx51_csum_partial: 0
> tx51_csum_partial_inner: 0
> tx51_added_vlan_packets: 0
> tx51_nop: 0
> tx51_csum_none: 0
> tx51_stopped: 0
> tx51_dropped: 0
> tx51_xmit_more: 0
> tx51_recover: 0
> tx51_cqes: 0
> tx51_wake: 0
> tx51_cqe_err: 0
> tx52_packets: 0
> tx52_bytes: 0
> tx52_tso_packets: 0
> tx52_tso_bytes: 0
> tx52_tso_inner_packets: 0
> tx52_tso_inner_bytes: 0
> tx52_csum_partial: 0
> tx52_csum_partial_inner: 0
> tx52_added_vlan_packets: 0
> tx52_nop: 0
> tx52_csum_none: 0
> tx52_stopped: 0
> tx52_dropped: 0
> tx52_xmit_more: 0
> tx52_recover: 0
> tx52_cqes: 0
> tx52_wake: 0
> tx52_cqe_err: 0
> tx53_packets: 0
> tx53_bytes: 0
> tx53_tso_packets: 0
> tx53_tso_bytes: 0
> tx53_tso_inner_packets: 0
> tx53_tso_inner_bytes: 0
> tx53_csum_partial: 0
> tx53_csum_partial_inner: 0
> tx53_added_vlan_packets: 0
> tx53_nop: 0
> tx53_csum_none: 0
> tx53_stopped: 0
> tx53_dropped: 0
> tx53_xmit_more: 0
> tx53_recover: 0
> tx53_cqes: 0
> tx53_wake: 0
> tx53_cqe_err: 0
> tx54_packets: 0
> tx54_bytes: 0
> tx54_tso_packets: 0
> tx54_tso_bytes: 0
> tx54_tso_inner_packets: 0
> tx54_tso_inner_bytes: 0
> tx54_csum_partial: 0
> tx54_csum_partial_inner: 0
> tx54_added_vlan_packets: 0
> tx54_nop: 0
> tx54_csum_none: 0
> tx54_stopped: 0
> tx54_dropped: 0
> tx54_xmit_more: 0
> tx54_recover: 0
> tx54_cqes: 0
> tx54_wake: 0
> tx54_cqe_err: 0
> tx55_packets: 0
> tx55_bytes: 0
> tx55_tso_packets: 0
> tx55_tso_bytes: 0
> tx55_tso_inner_packets: 0
> tx55_tso_inner_bytes: 0
> tx55_csum_partial: 0
> tx55_csum_partial_inner: 0
> tx55_added_vlan_packets: 0
> tx55_nop: 0
> tx55_csum_none: 0
> tx55_stopped: 0
> tx55_dropped: 0
> tx55_xmit_more: 0
> tx55_recover: 0
> tx55_cqes: 0
> tx55_wake: 0
> tx55_cqe_err: 0
> tx0_xdp_xmit: 0
> tx0_xdp_full: 0
> tx0_xdp_err: 0
> tx0_xdp_cqes: 0
> tx1_xdp_xmit: 0
> tx1_xdp_full: 0
> tx1_xdp_err: 0
> tx1_xdp_cqes: 0
> tx2_xdp_xmit: 0
> tx2_xdp_full: 0
> tx2_xdp_err: 0
> tx2_xdp_cqes: 0
> tx3_xdp_xmit: 0
> tx3_xdp_full: 0
> tx3_xdp_err: 0
> tx3_xdp_cqes: 0
> tx4_xdp_xmit: 0
> tx4_xdp_full: 0
> tx4_xdp_err: 0
> tx4_xdp_cqes: 0
> tx5_xdp_xmit: 0
> tx5_xdp_full: 0
> tx5_xdp_err: 0
> tx5_xdp_cqes: 0
> tx6_xdp_xmit: 0
> tx6_xdp_full: 0
> tx6_xdp_err: 0
> tx6_xdp_cqes: 0
> tx7_xdp_xmit: 0
> tx7_xdp_full: 0
> tx7_xdp_err: 0
> tx7_xdp_cqes: 0
> tx8_xdp_xmit: 0
> tx8_xdp_full: 0
> tx8_xdp_err: 0
> tx8_xdp_cqes: 0
> tx9_xdp_xmit: 0
> tx9_xdp_full: 0
> tx9_xdp_err: 0
> tx9_xdp_cqes: 0
> tx10_xdp_xmit: 0
> tx10_xdp_full: 0
> tx10_xdp_err: 0
> tx10_xdp_cqes: 0
> tx11_xdp_xmit: 0
> tx11_xdp_full: 0
> tx11_xdp_err: 0
> tx11_xdp_cqes: 0
> tx12_xdp_xmit: 0
> tx12_xdp_full: 0
> tx12_xdp_err: 0
> tx12_xdp_cqes: 0
> tx13_xdp_xmit: 0
> tx13_xdp_full: 0
> tx13_xdp_err: 0
> tx13_xdp_cqes: 0
> tx14_xdp_xmit: 0
> tx14_xdp_full: 0
> tx14_xdp_err: 0
> tx14_xdp_cqes: 0
> tx15_xdp_xmit: 0
> tx15_xdp_full: 0
> tx15_xdp_err: 0
> tx15_xdp_cqes: 0
> tx16_xdp_xmit: 0
> tx16_xdp_full: 0
> tx16_xdp_err: 0
> tx16_xdp_cqes: 0
> tx17_xdp_xmit: 0
> tx17_xdp_full: 0
> tx17_xdp_err: 0
> tx17_xdp_cqes: 0
> tx18_xdp_xmit: 0
> tx18_xdp_full: 0
> tx18_xdp_err: 0
> tx18_xdp_cqes: 0
> tx19_xdp_xmit: 0
> tx19_xdp_full: 0
> tx19_xdp_err: 0
> tx19_xdp_cqes: 0
> tx20_xdp_xmit: 0
> tx20_xdp_full: 0
> tx20_xdp_err: 0
> tx20_xdp_cqes: 0
> tx21_xdp_xmit: 0
> tx21_xdp_full: 0
> tx21_xdp_err: 0
> tx21_xdp_cqes: 0
> tx22_xdp_xmit: 0
> tx22_xdp_full: 0
> tx22_xdp_err: 0
> tx22_xdp_cqes: 0
> tx23_xdp_xmit: 0
> tx23_xdp_full: 0
> tx23_xdp_err: 0
> tx23_xdp_cqes: 0
> tx24_xdp_xmit: 0
> tx24_xdp_full: 0
> tx24_xdp_err: 0
> tx24_xdp_cqes: 0
> tx25_xdp_xmit: 0
> tx25_xdp_full: 0
> tx25_xdp_err: 0
> tx25_xdp_cqes: 0
> tx26_xdp_xmit: 0
> tx26_xdp_full: 0
> tx26_xdp_err: 0
> tx26_xdp_cqes: 0
> tx27_xdp_xmit: 0
> tx27_xdp_full: 0
> tx27_xdp_err: 0
> tx27_xdp_cqes: 0
> tx28_xdp_xmit: 0
> tx28_xdp_full: 0
> tx28_xdp_err: 0
> tx28_xdp_cqes: 0
> tx29_xdp_xmit: 0
> tx29_xdp_full: 0
> tx29_xdp_err: 0
> tx29_xdp_cqes: 0
> tx30_xdp_xmit: 0
> tx30_xdp_full: 0
> tx30_xdp_err: 0
> tx30_xdp_cqes: 0
> tx31_xdp_xmit: 0
> tx31_xdp_full: 0
> tx31_xdp_err: 0
> tx31_xdp_cqes: 0
> tx32_xdp_xmit: 0
> tx32_xdp_full: 0
> tx32_xdp_err: 0
> tx32_xdp_cqes: 0
> tx33_xdp_xmit: 0
> tx33_xdp_full: 0
> tx33_xdp_err: 0
> tx33_xdp_cqes: 0
> tx34_xdp_xmit: 0
> tx34_xdp_full: 0
> tx34_xdp_err: 0
> tx34_xdp_cqes: 0
> tx35_xdp_xmit: 0
> tx35_xdp_full: 0
> tx35_xdp_err: 0
> tx35_xdp_cqes: 0
> tx36_xdp_xmit: 0
> tx36_xdp_full: 0
> tx36_xdp_err: 0
> tx36_xdp_cqes: 0
> tx37_xdp_xmit: 0
> tx37_xdp_full: 0
> tx37_xdp_err: 0
> tx37_xdp_cqes: 0
> tx38_xdp_xmit: 0
> tx38_xdp_full: 0
> tx38_xdp_err: 0
> tx38_xdp_cqes: 0
> tx39_xdp_xmit: 0
> tx39_xdp_full: 0
> tx39_xdp_err: 0
> tx39_xdp_cqes: 0
> tx40_xdp_xmit: 0
> tx40_xdp_full: 0
> tx40_xdp_err: 0
> tx40_xdp_cqes: 0
> tx41_xdp_xmit: 0
> tx41_xdp_full: 0
> tx41_xdp_err: 0
> tx41_xdp_cqes: 0
> tx42_xdp_xmit: 0
> tx42_xdp_full: 0
> tx42_xdp_err: 0
> tx42_xdp_cqes: 0
> tx43_xdp_xmit: 0
> tx43_xdp_full: 0
> tx43_xdp_err: 0
> tx43_xdp_cqes: 0
> tx44_xdp_xmit: 0
> tx44_xdp_full: 0
> tx44_xdp_err: 0
> tx44_xdp_cqes: 0
> tx45_xdp_xmit: 0
> tx45_xdp_full: 0
> tx45_xdp_err: 0
> tx45_xdp_cqes: 0
> tx46_xdp_xmit: 0
> tx46_xdp_full: 0
> tx46_xdp_err: 0
> tx46_xdp_cqes: 0
> tx47_xdp_xmit: 0
> tx47_xdp_full: 0
> tx47_xdp_err: 0
> tx47_xdp_cqes: 0
> tx48_xdp_xmit: 0
> tx48_xdp_full: 0
> tx48_xdp_err: 0
> tx48_xdp_cqes: 0
> tx49_xdp_xmit: 0
> tx49_xdp_full: 0
> tx49_xdp_err: 0
> tx49_xdp_cqes: 0
> tx50_xdp_xmit: 0
> tx50_xdp_full: 0
> tx50_xdp_err: 0
> tx50_xdp_cqes: 0
> tx51_xdp_xmit: 0
> tx51_xdp_full: 0
> tx51_xdp_err: 0
> tx51_xdp_cqes: 0
> tx52_xdp_xmit: 0
> tx52_xdp_full: 0
> tx52_xdp_err: 0
> tx52_xdp_cqes: 0
> tx53_xdp_xmit: 0
> tx53_xdp_full: 0
> tx53_xdp_err: 0
> tx53_xdp_cqes: 0
> tx54_xdp_xmit: 0
> tx54_xdp_full: 0
> tx54_xdp_err: 0
> tx54_xdp_cqes: 0
> tx55_xdp_xmit: 0
> tx55_xdp_full: 0
> tx55_xdp_err: 0
> tx55_xdp_cqes: 0
>
> ethtool -S enp175s0f0
> NIC statistics:
> rx_packets: 141574897253
> rx_bytes: 184445040406258
> tx_packets: 172569543894
> tx_bytes: 99486882076365
> tx_tso_packets: 9367664195
> tx_tso_bytes: 56435233992948
> tx_tso_inner_packets: 0
> tx_tso_inner_bytes: 0
> tx_added_vlan_packets: 141297671626
> tx_nop: 2102916272
> rx_lro_packets: 0
> rx_lro_bytes: 0
> rx_ecn_mark: 0
> rx_removed_vlan_packets: 141574897252
> rx_csum_unnecessary: 0
> rx_csum_none: 23135854
> rx_csum_complete: 141551761398
> rx_csum_unnecessary_inner: 0
> rx_xdp_drop: 0
> rx_xdp_redirect: 0
> rx_xdp_tx_xmit: 0
> rx_xdp_tx_full: 0
> rx_xdp_tx_err: 0
> rx_xdp_tx_cqe: 0
> tx_csum_none: 127934791664
It is a good idea to look into this, tx is not requesting hw tx
csumming for a lot of packets, maybe you are wasting a lot of cpu on
calculating csum, or maybe this is just the rx csum complete..
> tx_csum_partial: 13362879974
> tx_csum_partial_inner: 0
> tx_queue_stopped: 232561
TX queues are stalling, could be an indentation for the pcie
bottelneck.
> tx_queue_dropped: 0
> tx_xmit_more: 1266021946
> tx_recover: 0
> tx_cqes: 140031716469
> tx_queue_wake: 232561
> tx_udp_seg_rem: 0
> tx_cqe_err: 0
> tx_xdp_xmit: 0
> tx_xdp_full: 0
> tx_xdp_err: 0
> tx_xdp_cqes: 0
> rx_wqe_err: 0
> rx_mpwqe_filler_cqes: 0
> rx_mpwqe_filler_strides: 0
> rx_buff_alloc_err: 0
> rx_cqe_compress_blks: 0
> rx_cqe_compress_pkts: 0
> rx_page_reuse: 0
> rx_cache_reuse: 16625975793
> rx_cache_full: 54161465914
> rx_cache_empty: 258048
> rx_cache_busy: 54161472735
> rx_cache_waive: 0
> rx_congst_umr: 0
> rx_arfs_err: 0
> ch_events: 40572621887
> ch_poll: 40885650979
> ch_arm: 40429276692
> ch_aff_change: 0
> ch_eq_rearm: 0
> rx_out_of_buffer: 2791690
> rx_if_down_packets: 74
> rx_vport_unicast_packets: 141843476308
> rx_vport_unicast_bytes: 185421265403318
> tx_vport_unicast_packets: 172569484005
> tx_vport_unicast_bytes: 100019940094298
> rx_vport_multicast_packets: 85122935
> rx_vport_multicast_bytes: 5761316431
> tx_vport_multicast_packets: 6452
> tx_vport_multicast_bytes: 643540
> rx_vport_broadcast_packets: 22423624
> rx_vport_broadcast_bytes: 1390127090
> tx_vport_broadcast_packets: 22024
> tx_vport_broadcast_bytes: 1321440
> rx_vport_rdma_unicast_packets: 0
> rx_vport_rdma_unicast_bytes: 0
> tx_vport_rdma_unicast_packets: 0
> tx_vport_rdma_unicast_bytes: 0
> rx_vport_rdma_multicast_packets: 0
> rx_vport_rdma_multicast_bytes: 0
> tx_vport_rdma_multicast_packets: 0
> tx_vport_rdma_multicast_bytes: 0
> tx_packets_phy: 172569501577
> rx_packets_phy: 142871314588
> rx_crc_errors_phy: 0
> tx_bytes_phy: 100710212814151
> rx_bytes_phy: 187209224289564
> tx_multicast_phy: 6452
> tx_broadcast_phy: 22024
> rx_multicast_phy: 85122933
> rx_broadcast_phy: 22423623
> rx_in_range_len_errors_phy: 2
> rx_out_of_range_len_phy: 0
> rx_oversize_pkts_phy: 0
> rx_symbol_err_phy: 0
> tx_mac_control_phy: 0
> rx_mac_control_phy: 0
> rx_unsupported_op_phy: 0
> rx_pause_ctrl_phy: 0
> tx_pause_ctrl_phy: 0
> rx_discards_phy: 920161423
Ok, this port seem to be suffering more, RX is congested, maybe due to
the pcie bottleneck.
> tx_discards_phy: 0
> tx_errors_phy: 0
> rx_undersize_pkts_phy: 0
> rx_fragments_phy: 0
> rx_jabbers_phy: 0
> rx_64_bytes_phy: 412006326
> rx_65_to_127_bytes_phy: 11934371453
> rx_128_to_255_bytes_phy: 3415281165
> rx_256_to_511_bytes_phy: 2072955511
> rx_512_to_1023_bytes_phy: 2415393005
> rx_1024_to_1518_bytes_phy: 72182391608
> rx_1519_to_2047_bytes_phy: 50438902587
> rx_2048_to_4095_bytes_phy: 0
> rx_4096_to_8191_bytes_phy: 0
> rx_8192_to_10239_bytes_phy: 0
> link_down_events_phy: 0
> rx_pcs_symbol_err_phy: 0
> rx_corrected_bits_phy: 0
> rx_pci_signal_integrity: 0
> tx_pci_signal_integrity: 48
> rx_prio0_bytes: 186709842592642
> rx_prio0_packets: 141481966007
> tx_prio0_bytes: 100710171118138
> tx_prio0_packets: 172569437949
> rx_prio1_bytes: 492288152326
> rx_prio1_packets: 385996045
> tx_prio1_bytes: 0
> tx_prio1_packets: 0
> rx_prio2_bytes: 22119952
> rx_prio2_packets: 70788
> tx_prio2_bytes: 0
> tx_prio2_packets: 0
> rx_prio3_bytes: 546141102
> rx_prio3_packets: 681608
> tx_prio3_bytes: 0
> tx_prio3_packets: 0
> rx_prio4_bytes: 14665067
> rx_prio4_packets: 29486
> tx_prio4_bytes: 0
> tx_prio4_packets: 0
> rx_prio5_bytes: 158862504
> rx_prio5_packets: 965307
> tx_prio5_bytes: 0
> tx_prio5_packets: 0
> rx_prio6_bytes: 669337783
> rx_prio6_packets: 1475775
> tx_prio6_bytes: 0
> tx_prio6_packets: 0
> rx_prio7_bytes: 5623481349
> rx_prio7_packets: 79926412
> tx_prio7_bytes: 0
> tx_prio7_packets: 0
> module_unplug: 0
> module_bus_stuck: 0
> module_high_temp: 0
> module_bad_shorted: 0
> ch0_events: 1446162630
> ch0_poll: 1463312972
> ch0_arm: 1440728278
> ch0_aff_change: 0
> ch0_eq_rearm: 0
> ch1_events: 1384301405
> ch1_poll: 1399210915
> ch1_arm: 1378636486
> ch1_aff_change: 0
> ch1_eq_rearm: 0
> ch2_events: 1382788887
> ch2_poll: 1397231470
> ch2_arm: 1377058116
> ch2_aff_change: 0
> ch2_eq_rearm: 0
> ch3_events: 1461956995
> ch3_poll: 1475553146
> ch3_arm: 1456571625
> ch3_aff_change: 0
> ch3_eq_rearm: 0
> ch4_events: 1497359109
> ch4_poll: 1511021037
> ch4_arm: 1491733757
> ch4_aff_change: 0
> ch4_eq_rearm: 0
> ch5_events: 1387736262
> ch5_poll: 1400964615
> ch5_arm: 1382382834
> ch5_aff_change: 0
> ch5_eq_rearm: 0
> ch6_events: 1376772405
> ch6_poll: 1390851449
> ch6_arm: 1371551764
> ch6_aff_change: 0
> ch6_eq_rearm: 0
> ch7_events: 1431271514
> ch7_poll: 1445049729
> ch7_arm: 1425753718
> ch7_aff_change: 0
> ch7_eq_rearm: 0
> ch8_events: 1426976374
> ch8_poll: 1439938692
> ch8_arm: 1421392984
> ch8_aff_change: 0
> ch8_eq_rearm: 0
> ch9_events: 1456160031
> ch9_poll: 1468922870
> ch9_arm: 1450930446
> ch9_aff_change: 0
> ch9_eq_rearm: 0
> ch10_events: 1443640165
> ch10_poll: 1456812203
> ch10_arm: 1438425101
> ch10_aff_change: 0
> ch10_eq_rearm: 0
> ch11_events: 1381104776
> ch11_poll: 1393811057
> ch11_arm: 1376059326
> ch11_aff_change: 0
> ch11_eq_rearm: 0
> ch12_events: 1365223276
> ch12_poll: 1378406059
> ch12_arm: 1359950494
> ch12_aff_change: 0
> ch12_eq_rearm: 0
> ch13_events: 1421622259
> ch13_poll: 1434670996
> ch13_arm: 1416241801
> ch13_aff_change: 0
> ch13_eq_rearm: 0
> ch14_events: 1379084590
> ch14_poll: 1392425015
> ch14_arm: 1373675179
> ch14_aff_change: 0
> ch14_eq_rearm: 0
> ch15_events: 1531217338
> ch15_poll: 1543353833
> ch15_arm: 1526350453
> ch15_aff_change: 0
> ch15_eq_rearm: 0
> ch16_events: 1460469776
> ch16_poll: 1467995928
> ch16_arm: 1456010194
> ch16_aff_change: 0
> ch16_eq_rearm: 0
> ch17_events: 1494067670
> ch17_poll: 1500856680
> ch17_arm: 1489232674
> ch17_aff_change: 0
> ch17_eq_rearm: 0
> ch18_events: 1530126866
> ch18_poll: 1537293620
> ch18_arm: 1525476123
> ch18_aff_change: 0
> ch18_eq_rearm: 0
> ch19_events: 1499526149
> ch19_poll: 1506789309
> ch19_arm: 1495161602
> ch19_aff_change: 0
> ch19_eq_rearm: 0
> ch20_events: 1451479763
> ch20_poll: 1459767921
> ch20_arm: 1446360801
> ch20_aff_change: 0
> ch20_eq_rearm: 0
> ch21_events: 1521413613
> ch21_poll: 1529345146
> ch21_arm: 1517229314
> ch21_aff_change: 0
> ch21_eq_rearm: 0
> ch22_events: 1471950045
> ch22_poll: 1479746764
> ch22_arm: 1467681629
> ch22_aff_change: 0
> ch22_eq_rearm: 0
> ch23_events: 1502968393
> ch23_poll: 1510419909
> ch23_arm: 1498168438
> ch23_aff_change: 0
> ch23_eq_rearm: 0
> ch24_events: 1473451639
> ch24_poll: 1482606899
> ch24_arm: 1468212489
> ch24_aff_change: 0
> ch24_eq_rearm: 0
> ch25_events: 1440399182
> ch25_poll: 1448897475
> ch25_arm: 1435044786
> ch25_aff_change: 0
> ch25_eq_rearm: 0
> ch26_events: 1436831565
> ch26_poll: 1445485731
> ch26_arm: 1431827527
> ch26_aff_change: 0
> ch26_eq_rearm: 0
> ch27_events: 1516560621
> ch27_poll: 1524911010
> ch27_arm: 1511430164
> ch27_aff_change: 0
> ch27_eq_rearm: 0
> ch28_events: 4
> ch28_poll: 4
> ch28_arm: 4
> ch28_aff_change: 0
> ch28_eq_rearm: 0
> ch29_events: 6
> ch29_poll: 6
> ch29_arm: 6
> ch29_aff_change: 0
> ch29_eq_rearm: 0
> ch30_events: 4
> ch30_poll: 4
> ch30_arm: 4
> ch30_aff_change: 0
> ch30_eq_rearm: 0
> ch31_events: 4
> ch31_poll: 4
> ch31_arm: 4
> ch31_aff_change: 0
> ch31_eq_rearm: 0
> ch32_events: 4
> ch32_poll: 4
> ch32_arm: 4
> ch32_aff_change: 0
> ch32_eq_rearm: 0
> ch33_events: 4
> ch33_poll: 4
> ch33_arm: 4
> ch33_aff_change: 0
> ch33_eq_rearm: 0
> ch34_events: 4
> ch34_poll: 4
> ch34_arm: 4
> ch34_aff_change: 0
> ch34_eq_rearm: 0
> ch35_events: 4
> ch35_poll: 4
> ch35_arm: 4
> ch35_aff_change: 0
> ch35_eq_rearm: 0
> ch36_events: 4
> ch36_poll: 4
> ch36_arm: 4
> ch36_aff_change: 0
> ch36_eq_rearm: 0
> ch37_events: 4
> ch37_poll: 4
> ch37_arm: 4
> ch37_aff_change: 0
> ch37_eq_rearm: 0
> ch38_events: 4
> ch38_poll: 4
> ch38_arm: 4
> ch38_aff_change: 0
> ch38_eq_rearm: 0
> ch39_events: 4
> ch39_poll: 4
> ch39_arm: 4
> ch39_aff_change: 0
> ch39_eq_rearm: 0
> ch40_events: 4
> ch40_poll: 4
> ch40_arm: 4
> ch40_aff_change: 0
> ch40_eq_rearm: 0
> ch41_events: 4
> ch41_poll: 4
> ch41_arm: 4
> ch41_aff_change: 0
> ch41_eq_rearm: 0
> ch42_events: 4
> ch42_poll: 4
> ch42_arm: 4
> ch42_aff_change: 0
> ch42_eq_rearm: 0
> ch43_events: 4
> ch43_poll: 4
> ch43_arm: 4
> ch43_aff_change: 0
> ch43_eq_rearm: 0
> ch44_events: 4
> ch44_poll: 4
> ch44_arm: 4
> ch44_aff_change: 0
> ch44_eq_rearm: 0
> ch45_events: 4
> ch45_poll: 4
> ch45_arm: 4
> ch45_aff_change: 0
> ch45_eq_rearm: 0
> ch46_events: 4
> ch46_poll: 4
> ch46_arm: 4
> ch46_aff_change: 0
> ch46_eq_rearm: 0
> ch47_events: 4
> ch47_poll: 4
> ch47_arm: 4
> ch47_aff_change: 0
> ch47_eq_rearm: 0
> ch48_events: 4
> ch48_poll: 4
> ch48_arm: 4
> ch48_aff_change: 0
> ch48_eq_rearm: 0
> ch49_events: 4
> ch49_poll: 4
> ch49_arm: 4
> ch49_aff_change: 0
> ch49_eq_rearm: 0
> ch50_events: 4
> ch50_poll: 4
> ch50_arm: 4
> ch50_aff_change: 0
> ch50_eq_rearm: 0
> ch51_events: 4
> ch51_poll: 4
> ch51_arm: 4
> ch51_aff_change: 0
> ch51_eq_rearm: 0
> ch52_events: 4
> ch52_poll: 4
> ch52_arm: 4
> ch52_aff_change: 0
> ch52_eq_rearm: 0
> ch53_events: 4
> ch53_poll: 4
> ch53_arm: 4
> ch53_aff_change: 0
> ch53_eq_rearm: 0
> ch54_events: 4
> ch54_poll: 4
> ch54_arm: 4
> ch54_aff_change: 0
> ch54_eq_rearm: 0
> ch55_events: 4
> ch55_poll: 4
> ch55_arm: 4
> ch55_aff_change: 0
> ch55_eq_rearm: 0
> rx0_packets: 5861448653
> rx0_bytes: 7389128595728
> rx0_csum_complete: 5838312798
> rx0_csum_unnecessary: 0
> rx0_csum_unnecessary_inner: 0
> rx0_csum_none: 23135855
> rx0_xdp_drop: 0
> rx0_xdp_redirect: 0
> rx0_lro_packets: 0
> rx0_lro_bytes: 0
> rx0_ecn_mark: 0
> rx0_removed_vlan_packets: 5861448653
> rx0_wqe_err: 0
> rx0_mpwqe_filler_cqes: 0
> rx0_mpwqe_filler_strides: 0
> rx0_buff_alloc_err: 0
> rx0_cqe_compress_blks: 0
> rx0_cqe_compress_pkts: 0
> rx0_page_reuse: 0
> rx0_cache_reuse: 2559
> rx0_cache_full: 2930721512
> rx0_cache_empty: 6656
> rx0_cache_busy: 2930721765
> rx0_cache_waive: 0
> rx0_congst_umr: 0
> rx0_arfs_err: 0
> rx0_xdp_tx_xmit: 0
> rx0_xdp_tx_full: 0
> rx0_xdp_tx_err: 0
> rx0_xdp_tx_cqes: 0
> rx1_packets: 5550585106
> rx1_bytes: 7255635262803
> rx1_csum_complete: 5550585106
> rx1_csum_unnecessary: 0
> rx1_csum_unnecessary_inner: 0
> rx1_csum_none: 0
> rx1_xdp_drop: 0
> rx1_xdp_redirect: 0
> rx1_lro_packets: 0
> rx1_lro_bytes: 0
> rx1_ecn_mark: 0
> rx1_removed_vlan_packets: 5550585106
> rx1_wqe_err: 0
> rx1_mpwqe_filler_cqes: 0
> rx1_mpwqe_filler_strides: 0
> rx1_buff_alloc_err: 0
> rx1_cqe_compress_blks: 0
> rx1_cqe_compress_pkts: 0
> rx1_page_reuse: 0
> rx1_cache_reuse: 2918845
> rx1_cache_full: 2772373453
> rx1_cache_empty: 6656
> rx1_cache_busy: 2772373707
> rx1_cache_waive: 0
> rx1_congst_umr: 0
> rx1_arfs_err: 0
> rx1_xdp_tx_xmit: 0
> rx1_xdp_tx_full: 0
> rx1_xdp_tx_err: 0
> rx1_xdp_tx_cqes: 0
> rx2_packets: 5383874739
> rx2_bytes: 7031545423967
> rx2_csum_complete: 5383874739
> rx2_csum_unnecessary: 0
> rx2_csum_unnecessary_inner: 0
> rx2_csum_none: 0
> rx2_xdp_drop: 0
> rx2_xdp_redirect: 0
> rx2_lro_packets: 0
> rx2_lro_bytes: 0
> rx2_ecn_mark: 0
> rx2_removed_vlan_packets: 5383874739
> rx2_wqe_err: 0
> rx2_mpwqe_filler_cqes: 0
> rx2_mpwqe_filler_strides: 0
> rx2_buff_alloc_err: 0
> rx2_cqe_compress_blks: 0
> rx2_cqe_compress_pkts: 0
> rx2_page_reuse: 0
> rx2_cache_reuse: 2173370
> rx2_cache_full: 2689763744
> rx2_cache_empty: 6656
> rx2_cache_busy: 2689763998
> rx2_cache_waive: 0
> rx2_congst_umr: 0
> rx2_arfs_err: 0
> rx2_xdp_tx_xmit: 0
> rx2_xdp_tx_full: 0
> rx2_xdp_tx_err: 0
> rx2_xdp_tx_cqes: 0
> rx3_packets: 5456494012
> rx3_bytes: 7120241119485
> rx3_csum_complete: 5456494012
> rx3_csum_unnecessary: 0
> rx3_csum_unnecessary_inner: 0
> rx3_csum_none: 0
> rx3_xdp_drop: 0
> rx3_xdp_redirect: 0
> rx3_lro_packets: 0
> rx3_lro_bytes: 0
> rx3_ecn_mark: 0
> rx3_removed_vlan_packets: 5456494012
> rx3_wqe_err: 0
> rx3_mpwqe_filler_cqes: 0
> rx3_mpwqe_filler_strides: 0
> rx3_buff_alloc_err: 0
> rx3_cqe_compress_blks: 0
> rx3_cqe_compress_pkts: 0
> rx3_page_reuse: 0
> rx3_cache_reuse: 2120123
> rx3_cache_full: 2726126628
> rx3_cache_empty: 6656
> rx3_cache_busy: 2726126881
> rx3_cache_waive: 0
> rx3_congst_umr: 0
> rx3_arfs_err: 0
> rx3_xdp_tx_xmit: 0
> rx3_xdp_tx_full: 0
> rx3_xdp_tx_err: 0
> rx3_xdp_tx_cqes: 0
> rx4_packets: 5475216251
> rx4_bytes: 7123129170196
> rx4_csum_complete: 5475216251
> rx4_csum_unnecessary: 0
> rx4_csum_unnecessary_inner: 0
> rx4_csum_none: 0
> rx4_xdp_drop: 0
> rx4_xdp_redirect: 0
> rx4_lro_packets: 0
> rx4_lro_bytes: 0
> rx4_ecn_mark: 0
> rx4_removed_vlan_packets: 5475216251
> rx4_wqe_err: 0
> rx4_mpwqe_filler_cqes: 0
> rx4_mpwqe_filler_strides: 0
> rx4_buff_alloc_err: 0
> rx4_cqe_compress_blks: 0
> rx4_cqe_compress_pkts: 0
> rx4_page_reuse: 0
> rx4_cache_reuse: 2668296355
> rx4_cache_full: 69311549
> rx4_cache_empty: 6656
> rx4_cache_busy: 69311769
> rx4_cache_waive: 0
> rx4_congst_umr: 0
> rx4_arfs_err: 0
> rx4_xdp_tx_xmit: 0
> rx4_xdp_tx_full: 0
> rx4_xdp_tx_err: 0
> rx4_xdp_tx_cqes: 0
> rx5_packets: 5474372232
> rx5_bytes: 7159146801926
> rx5_csum_complete: 5474372232
> rx5_csum_unnecessary: 0
> rx5_csum_unnecessary_inner: 0
> rx5_csum_none: 0
> rx5_xdp_drop: 0
> rx5_xdp_redirect: 0
> rx5_lro_packets: 0
> rx5_lro_bytes: 0
> rx5_ecn_mark: 0
> rx5_removed_vlan_packets: 5474372232
> rx5_wqe_err: 0
> rx5_mpwqe_filler_cqes: 0
> rx5_mpwqe_filler_strides: 0
> rx5_buff_alloc_err: 0
> rx5_cqe_compress_blks: 0
> rx5_cqe_compress_pkts: 0
> rx5_page_reuse: 0
> rx5_cache_reuse: 626187
> rx5_cache_full: 2736559674
> rx5_cache_empty: 6656
> rx5_cache_busy: 2736559929
> rx5_cache_waive: 0
> rx5_congst_umr: 0
> rx5_arfs_err: 0
> rx5_xdp_tx_xmit: 0
> rx5_xdp_tx_full: 0
> rx5_xdp_tx_err: 0
> rx5_xdp_tx_cqes: 0
> rx6_packets: 5533622456
> rx6_bytes: 7207308809081
> rx6_csum_complete: 5533622456
> rx6_csum_unnecessary: 0
> rx6_csum_unnecessary_inner: 0
> rx6_csum_none: 0
> rx6_xdp_drop: 0
> rx6_xdp_redirect: 0
> rx6_lro_packets: 0
> rx6_lro_bytes: 0
> rx6_ecn_mark: 0
> rx6_removed_vlan_packets: 5533622456
> rx6_wqe_err: 0
> rx6_mpwqe_filler_cqes: 0
> rx6_mpwqe_filler_strides: 0
> rx6_buff_alloc_err: 0
> rx6_cqe_compress_blks: 0
> rx6_cqe_compress_pkts: 0
> rx6_page_reuse: 0
> rx6_cache_reuse: 2325217
> rx6_cache_full: 2764485756
> rx6_cache_empty: 6656
> rx6_cache_busy: 2764486011
> rx6_cache_waive: 0
> rx6_congst_umr: 0
> rx6_arfs_err: 0
> rx6_xdp_tx_xmit: 0
> rx6_xdp_tx_full: 0
> rx6_xdp_tx_err: 0
> rx6_xdp_tx_cqes: 0
> rx7_packets: 5533901822
> rx7_bytes: 7227441240536
> rx7_csum_complete: 5533901822
> rx7_csum_unnecessary: 0
> rx7_csum_unnecessary_inner: 0
> rx7_csum_none: 0
> rx7_xdp_drop: 0
> rx7_xdp_redirect: 0
> rx7_lro_packets: 0
> rx7_lro_bytes: 0
> rx7_ecn_mark: 0
> rx7_removed_vlan_packets: 5533901822
> rx7_wqe_err: 0
> rx7_mpwqe_filler_cqes: 0
> rx7_mpwqe_filler_strides: 0
> rx7_buff_alloc_err: 0
> rx7_cqe_compress_blks: 0
> rx7_cqe_compress_pkts: 0
> rx7_page_reuse: 0
> rx7_cache_reuse: 2372505
> rx7_cache_full: 2764578151
> rx7_cache_empty: 6656
> rx7_cache_busy: 2764578403
> rx7_cache_waive: 0
> rx7_congst_umr: 0
> rx7_arfs_err: 0
> rx7_xdp_tx_xmit: 0
> rx7_xdp_tx_full: 0
> rx7_xdp_tx_err: 0
> rx7_xdp_tx_cqes: 0
> rx8_packets: 5485670137
> rx8_bytes: 7203339989013
> rx8_csum_complete: 5485670137
> rx8_csum_unnecessary: 0
> rx8_csum_unnecessary_inner: 0
> rx8_csum_none: 0
> rx8_xdp_drop: 0
> rx8_xdp_redirect: 0
> rx8_lro_packets: 0
> rx8_lro_bytes: 0
> rx8_ecn_mark: 0
> rx8_removed_vlan_packets: 5485670137
> rx8_wqe_err: 0
> rx8_mpwqe_filler_cqes: 0
> rx8_mpwqe_filler_strides: 0
> rx8_buff_alloc_err: 0
> rx8_cqe_compress_blks: 0
> rx8_cqe_compress_pkts: 0
> rx8_page_reuse: 0
> rx8_cache_reuse: 7522232
> rx8_cache_full: 2735312581
> rx8_cache_empty: 6656
> rx8_cache_busy: 2735312836
> rx8_cache_waive: 0
> rx8_congst_umr: 0
> rx8_arfs_err: 0
> rx8_xdp_tx_xmit: 0
> rx8_xdp_tx_full: 0
> rx8_xdp_tx_err: 0
> rx8_xdp_tx_cqes: 0
> rx9_packets: 5482212354
> rx9_bytes: 7169663341718
> rx9_csum_complete: 5482212354
> rx9_csum_unnecessary: 0
> rx9_csum_unnecessary_inner: 0
> rx9_csum_none: 0
> rx9_xdp_drop: 0
> rx9_xdp_redirect: 0
> rx9_lro_packets: 0
> rx9_lro_bytes: 0
> rx9_ecn_mark: 0
> rx9_removed_vlan_packets: 5482212354
> rx9_wqe_err: 0
> rx9_mpwqe_filler_cqes: 0
> rx9_mpwqe_filler_strides: 0
> rx9_buff_alloc_err: 0
> rx9_cqe_compress_blks: 0
> rx9_cqe_compress_pkts: 0
> rx9_page_reuse: 0
> rx9_cache_reuse: 37279961
> rx9_cache_full: 2703825961
> rx9_cache_empty: 6656
> rx9_cache_busy: 2703826215
> rx9_cache_waive: 0
> rx9_congst_umr: 0
> rx9_arfs_err: 0
> rx9_xdp_tx_xmit: 0
> rx9_xdp_tx_full: 0
> rx9_xdp_tx_err: 0
> rx9_xdp_tx_cqes: 0
> rx10_packets: 5524679952
> rx10_bytes: 7248301275181
> rx10_csum_complete: 5524679952
> rx10_csum_unnecessary: 0
> rx10_csum_unnecessary_inner: 0
> rx10_csum_none: 0
> rx10_xdp_drop: 0
> rx10_xdp_redirect: 0
> rx10_lro_packets: 0
> rx10_lro_bytes: 0
> rx10_ecn_mark: 0
> rx10_removed_vlan_packets: 5524679952
> rx10_wqe_err: 0
> rx10_mpwqe_filler_cqes: 0
> rx10_mpwqe_filler_strides: 0
> rx10_buff_alloc_err: 0
> rx10_cqe_compress_blks: 0
> rx10_cqe_compress_pkts: 0
> rx10_page_reuse: 0
> rx10_cache_reuse: 2049666
> rx10_cache_full: 2760290055
> rx10_cache_empty: 6656
> rx10_cache_busy: 2760290310
> rx10_cache_waive: 0
> rx10_congst_umr: 0
> rx10_arfs_err: 0
> rx10_xdp_tx_xmit: 0
> rx10_xdp_tx_full: 0
> rx10_xdp_tx_err: 0
> rx10_xdp_tx_cqes: 0
> rx11_packets: 5394633545
> rx11_bytes: 7033509636092
> rx11_csum_complete: 5394633545
> rx11_csum_unnecessary: 0
> rx11_csum_unnecessary_inner: 0
> rx11_csum_none: 0
> rx11_xdp_drop: 0
> rx11_xdp_redirect: 0
> rx11_lro_packets: 0
> rx11_lro_bytes: 0
> rx11_ecn_mark: 0
> rx11_removed_vlan_packets: 5394633545
> rx11_wqe_err: 0
> rx11_mpwqe_filler_cqes: 0
> rx11_mpwqe_filler_strides: 0
> rx11_buff_alloc_err: 0
> rx11_cqe_compress_blks: 0
> rx11_cqe_compress_pkts: 0
> rx11_page_reuse: 0
> rx11_cache_reuse: 2617466268
> rx11_cache_full: 79850284
> rx11_cache_empty: 6656
> rx11_cache_busy: 79850504
> rx11_cache_waive: 0
> rx11_congst_umr: 0
> rx11_arfs_err: 0
> rx11_xdp_tx_xmit: 0
> rx11_xdp_tx_full: 0
> rx11_xdp_tx_err: 0
> rx11_xdp_tx_cqes: 0
> rx12_packets: 5458907385
> rx12_bytes: 7134867867515
> rx12_csum_complete: 5458907385
> rx12_csum_unnecessary: 0
> rx12_csum_unnecessary_inner: 0
> rx12_csum_none: 0
> rx12_xdp_drop: 0
> rx12_xdp_redirect: 0
> rx12_lro_packets: 0
> rx12_lro_bytes: 0
> rx12_ecn_mark: 0
> rx12_removed_vlan_packets: 5458907385
> rx12_wqe_err: 0
> rx12_mpwqe_filler_cqes: 0
> rx12_mpwqe_filler_strides: 0
> rx12_buff_alloc_err: 0
> rx12_cqe_compress_blks: 0
> rx12_cqe_compress_pkts: 0
> rx12_page_reuse: 0
> rx12_cache_reuse: 2650214169
> rx12_cache_full: 79239303
> rx12_cache_empty: 6656
> rx12_cache_busy: 79239523
> rx12_cache_waive: 0
> rx12_congst_umr: 0
> rx12_arfs_err: 0
> rx12_xdp_tx_xmit: 0
> rx12_xdp_tx_full: 0
> rx12_xdp_tx_err: 0
> rx12_xdp_tx_cqes: 0
> rx13_packets: 5549932912
> rx13_bytes: 7232548705586
> rx13_csum_complete: 5549932912
> rx13_csum_unnecessary: 0
> rx13_csum_unnecessary_inner: 0
> rx13_csum_none: 0
> rx13_xdp_drop: 0
> rx13_xdp_redirect: 0
> rx13_lro_packets: 0
> rx13_lro_bytes: 0
> rx13_ecn_mark: 0
> rx13_removed_vlan_packets: 5549932912
> rx13_wqe_err: 0
> rx13_mpwqe_filler_cqes: 0
> rx13_mpwqe_filler_strides: 0
> rx13_buff_alloc_err: 0
> rx13_cqe_compress_blks: 0
> rx13_cqe_compress_pkts: 0
> rx13_page_reuse: 0
> rx13_cache_reuse: 2417696
> rx13_cache_full: 2772548505
> rx13_cache_empty: 6656
> rx13_cache_busy: 2772548760
> rx13_cache_waive: 0
> rx13_congst_umr: 0
> rx13_arfs_err: 0
> rx13_xdp_tx_xmit: 0
> rx13_xdp_tx_full: 0
> rx13_xdp_tx_err: 0
> rx13_xdp_tx_cqes: 0
> rx14_packets: 5517712329
> rx14_bytes: 7192111965227
> rx14_csum_complete: 5517712329
> rx14_csum_unnecessary: 0
> rx14_csum_unnecessary_inner: 0
> rx14_csum_none: 0
> rx14_xdp_drop: 0
> rx14_xdp_redirect: 0
> rx14_lro_packets: 0
> rx14_lro_bytes: 0
> rx14_ecn_mark: 0
> rx14_removed_vlan_packets: 5517712329
> rx14_wqe_err: 0
> rx14_mpwqe_filler_cqes: 0
> rx14_mpwqe_filler_strides: 0
> rx14_buff_alloc_err: 0
> rx14_cqe_compress_blks: 0
> rx14_cqe_compress_pkts: 0
> rx14_page_reuse: 0
> rx14_cache_reuse: 1830206
> rx14_cache_full: 2757025703
> rx14_cache_empty: 6656
> rx14_cache_busy: 2757025958
> rx14_cache_waive: 0
> rx14_congst_umr: 0
> rx14_arfs_err: 0
> rx14_xdp_tx_xmit: 0
> rx14_xdp_tx_full: 0
> rx14_xdp_tx_err: 0
> rx14_xdp_tx_cqes: 0
> rx15_packets: 5578343373
> rx15_bytes: 7268484501219
> rx15_csum_complete: 5578343373
> rx15_csum_unnecessary: 0
> rx15_csum_unnecessary_inner: 0
> rx15_csum_none: 0
> rx15_xdp_drop: 0
> rx15_xdp_redirect: 0
> rx15_lro_packets: 0
> rx15_lro_bytes: 0
> rx15_ecn_mark: 0
> rx15_removed_vlan_packets: 5578343373
> rx15_wqe_err: 0
> rx15_mpwqe_filler_cqes: 0
> rx15_mpwqe_filler_strides: 0
> rx15_buff_alloc_err: 0
> rx15_cqe_compress_blks: 0
> rx15_cqe_compress_pkts: 0
> rx15_page_reuse: 0
> rx15_cache_reuse: 2317165
> rx15_cache_full: 2786854266
> rx15_cache_empty: 6656
> rx15_cache_busy: 2786854519
> rx15_cache_waive: 0
> rx15_congst_umr: 0
> rx15_arfs_err: 0
> rx15_xdp_tx_xmit: 0
> rx15_xdp_tx_full: 0
> rx15_xdp_tx_err: 0
> rx15_xdp_tx_cqes: 0
> rx16_packets: 4435773951
> rx16_bytes: 5766665272007
> rx16_csum_complete: 4435773951
> rx16_csum_unnecessary: 0
> rx16_csum_unnecessary_inner: 0
> rx16_csum_none: 0
> rx16_xdp_drop: 0
> rx16_xdp_redirect: 0
> rx16_lro_packets: 0
> rx16_lro_bytes: 0
> rx16_ecn_mark: 0
> rx16_removed_vlan_packets: 4435773951
> rx16_wqe_err: 0
> rx16_mpwqe_filler_cqes: 0
> rx16_mpwqe_filler_strides: 0
> rx16_buff_alloc_err: 0
> rx16_cqe_compress_blks: 0
> rx16_cqe_compress_pkts: 0
> rx16_page_reuse: 0
> rx16_cache_reuse: 2033793
> rx16_cache_full: 2215852927
> rx16_cache_empty: 6656
> rx16_cache_busy: 2215853179
> rx16_cache_waive: 0
> rx16_congst_umr: 0
> rx16_arfs_err: 0
> rx16_xdp_tx_xmit: 0
> rx16_xdp_tx_full: 0
> rx16_xdp_tx_err: 0
> rx16_xdp_tx_cqes: 0
> rx17_packets: 4344087587
> rx17_bytes: 5695006496323
> rx17_csum_complete: 4344087587
> rx17_csum_unnecessary: 0
> rx17_csum_unnecessary_inner: 0
> rx17_csum_none: 0
> rx17_xdp_drop: 0
> rx17_xdp_redirect: 0
> rx17_lro_packets: 0
> rx17_lro_bytes: 0
> rx17_ecn_mark: 0
> rx17_removed_vlan_packets: 4344087587
> rx17_wqe_err: 0
> rx17_mpwqe_filler_cqes: 0
> rx17_mpwqe_filler_strides: 0
> rx17_buff_alloc_err: 0
> rx17_cqe_compress_blks: 0
> rx17_cqe_compress_pkts: 0
> rx17_page_reuse: 0
> rx17_cache_reuse: 2652127
> rx17_cache_full: 2169391411
> rx17_cache_empty: 6656
> rx17_cache_busy: 2169391665
> rx17_cache_waive: 0
> rx17_congst_umr: 0
> rx17_arfs_err: 0
> rx17_xdp_tx_xmit: 0
> rx17_xdp_tx_full: 0
> rx17_xdp_tx_err: 0
> rx17_xdp_tx_cqes: 0
> rx18_packets: 4407422804
> rx18_bytes: 5741134634177
> rx18_csum_complete: 4407422804
> rx18_csum_unnecessary: 0
> rx18_csum_unnecessary_inner: 0
> rx18_csum_none: 0
> rx18_xdp_drop: 0
> rx18_xdp_redirect: 0
> rx18_lro_packets: 0
> rx18_lro_bytes: 0
> rx18_ecn_mark: 0
> rx18_removed_vlan_packets: 4407422804
> rx18_wqe_err: 0
> rx18_mpwqe_filler_cqes: 0
> rx18_mpwqe_filler_strides: 0
> rx18_buff_alloc_err: 0
> rx18_cqe_compress_blks: 0
> rx18_cqe_compress_pkts: 0
> rx18_page_reuse: 0
> rx18_cache_reuse: 2156080239
> rx18_cache_full: 47630941
> rx18_cache_empty: 6656
> rx18_cache_busy: 47631161
> rx18_cache_waive: 0
> rx18_congst_umr: 0
> rx18_arfs_err: 0
> rx18_xdp_tx_xmit: 0
> rx18_xdp_tx_full: 0
> rx18_xdp_tx_err: 0
> rx18_xdp_tx_cqes: 0
> rx19_packets: 4545554180
> rx19_bytes: 5905277503466
> rx19_csum_complete: 4545554180
> rx19_csum_unnecessary: 0
> rx19_csum_unnecessary_inner: 0
> rx19_csum_none: 0
> rx19_xdp_drop: 0
> rx19_xdp_redirect: 0
> rx19_lro_packets: 0
> rx19_lro_bytes: 0
> rx19_ecn_mark: 0
> rx19_removed_vlan_packets: 4545554180
> rx19_wqe_err: 0
> rx19_mpwqe_filler_cqes: 0
> rx19_mpwqe_filler_strides: 0
> rx19_buff_alloc_err: 0
> rx19_cqe_compress_blks: 0
> rx19_cqe_compress_pkts: 0
> rx19_page_reuse: 0
> rx19_cache_reuse: 11112455
> rx19_cache_full: 2261664379
> rx19_cache_empty: 6656
> rx19_cache_busy: 2261664601
> rx19_cache_waive: 0
> rx19_congst_umr: 0
> rx19_arfs_err: 0
> rx19_xdp_tx_xmit: 0
> rx19_xdp_tx_full: 0
> rx19_xdp_tx_err: 0
> rx19_xdp_tx_cqes: 0
> rx20_packets: 4397428553
> rx20_bytes: 5757329184301
> rx20_csum_complete: 4397428553
> rx20_csum_unnecessary: 0
> rx20_csum_unnecessary_inner: 0
> rx20_csum_none: 0
> rx20_xdp_drop: 0
> rx20_xdp_redirect: 0
> rx20_lro_packets: 0
> rx20_lro_bytes: 0
> rx20_ecn_mark: 0
> rx20_removed_vlan_packets: 4397428553
> rx20_wqe_err: 0
> rx20_mpwqe_filler_cqes: 0
> rx20_mpwqe_filler_strides: 0
> rx20_buff_alloc_err: 0
> rx20_cqe_compress_blks: 0
> rx20_cqe_compress_pkts: 0
> rx20_page_reuse: 0
> rx20_cache_reuse: 2168116995
> rx20_cache_full: 30597061
> rx20_cache_empty: 6656
> rx20_cache_busy: 30597281
> rx20_cache_waive: 0
> rx20_congst_umr: 0
> rx20_arfs_err: 0
> rx20_xdp_tx_xmit: 0
> rx20_xdp_tx_full: 0
> rx20_xdp_tx_err: 0
> rx20_xdp_tx_cqes: 0
> rx21_packets: 4552564821
> rx21_bytes: 5944840329249
> rx21_csum_complete: 4552564821
> rx21_csum_unnecessary: 0
> rx21_csum_unnecessary_inner: 0
> rx21_csum_none: 0
> rx21_xdp_drop: 0
> rx21_xdp_redirect: 0
> rx21_lro_packets: 0
> rx21_lro_bytes: 0
> rx21_ecn_mark: 0
> rx21_removed_vlan_packets: 4552564821
> rx21_wqe_err: 0
> rx21_mpwqe_filler_cqes: 0
> rx21_mpwqe_filler_strides: 0
> rx21_buff_alloc_err: 0
> rx21_cqe_compress_blks: 0
> rx21_cqe_compress_pkts: 0
> rx21_page_reuse: 0
> rx21_cache_reuse: 2295681
> rx21_cache_full: 2273986474
> rx21_cache_empty: 6656
> rx21_cache_busy: 2273986727
> rx21_cache_waive: 0
> rx21_congst_umr: 0
> rx21_arfs_err: 0
> rx21_xdp_tx_xmit: 0
> rx21_xdp_tx_full: 0
> rx21_xdp_tx_err: 0
> rx21_xdp_tx_cqes: 0
> rx22_packets: 4629499740
> rx22_bytes: 5924206566499
> rx22_csum_complete: 4629499740
> rx22_csum_unnecessary: 0
> rx22_csum_unnecessary_inner: 0
> rx22_csum_none: 0
> rx22_xdp_drop: 0
> rx22_xdp_redirect: 0
> rx22_lro_packets: 0
> rx22_lro_bytes: 0
> rx22_ecn_mark: 0
> rx22_removed_vlan_packets: 4629499740
> rx22_wqe_err: 0
> rx22_mpwqe_filler_cqes: 0
> rx22_mpwqe_filler_strides: 0
> rx22_buff_alloc_err: 0
> rx22_cqe_compress_blks: 0
> rx22_cqe_compress_pkts: 0
> rx22_page_reuse: 0
> rx22_cache_reuse: 1407527
> rx22_cache_full: 2313342088
> rx22_cache_empty: 6656
> rx22_cache_busy: 2313342341
> rx22_cache_waive: 0
> rx22_congst_umr: 0
> rx22_arfs_err: 0
> rx22_xdp_tx_xmit: 0
> rx22_xdp_tx_full: 0
> rx22_xdp_tx_err: 0
> rx22_xdp_tx_cqes: 0
> rx23_packets: 4387124505
> rx23_bytes: 5718118678470
> rx23_csum_complete: 4387124505
> rx23_csum_unnecessary: 0
> rx23_csum_unnecessary_inner: 0
> rx23_csum_none: 0
> rx23_xdp_drop: 0
> rx23_xdp_redirect: 0
> rx23_lro_packets: 0
> rx23_lro_bytes: 0
> rx23_ecn_mark: 0
> rx23_removed_vlan_packets: 4387124505
> rx23_wqe_err: 0
> rx23_mpwqe_filler_cqes: 0
> rx23_mpwqe_filler_strides: 0
> rx23_buff_alloc_err: 0
> rx23_cqe_compress_blks: 0
> rx23_cqe_compress_pkts: 0
> rx23_page_reuse: 0
> rx23_cache_reuse: 2013280
> rx23_cache_full: 2191548717
> rx23_cache_empty: 6656
> rx23_cache_busy: 2191548972
> rx23_cache_waive: 0
> rx23_congst_umr: 0
> rx23_arfs_err: 0
> rx23_xdp_tx_xmit: 0
> rx23_xdp_tx_full: 0
> rx23_xdp_tx_err: 0
> rx23_xdp_tx_cqes: 0
> rx24_packets: 4398791634
> rx24_bytes: 5744875564632
> rx24_csum_complete: 4398791634
> rx24_csum_unnecessary: 0
> rx24_csum_unnecessary_inner: 0
> rx24_csum_none: 0
> rx24_xdp_drop: 0
> rx24_xdp_redirect: 0
> rx24_lro_packets: 0
> rx24_lro_bytes: 0
> rx24_ecn_mark: 0
> rx24_removed_vlan_packets: 4398791634
> rx24_wqe_err: 0
> rx24_mpwqe_filler_cqes: 0
> rx24_mpwqe_filler_strides: 0
> rx24_buff_alloc_err: 0
> rx24_cqe_compress_blks: 0
> rx24_cqe_compress_pkts: 0
> rx24_page_reuse: 0
> rx24_cache_reuse: 2143926100
> rx24_cache_full: 55469496
> rx24_cache_empty: 6656
> rx24_cache_busy: 55469716
> rx24_cache_waive: 0
> rx24_congst_umr: 0
> rx24_arfs_err: 0
> rx24_xdp_tx_xmit: 0
> rx24_xdp_tx_full: 0
> rx24_xdp_tx_err: 0
> rx24_xdp_tx_cqes: 0
> rx25_packets: 4377204935
> rx25_bytes: 5710369124105
> rx25_csum_complete: 4377204935
> rx25_csum_unnecessary: 0
> rx25_csum_unnecessary_inner: 0
> rx25_csum_none: 0
> rx25_xdp_drop: 0
> rx25_xdp_redirect: 0
> rx25_lro_packets: 0
> rx25_lro_bytes: 0
> rx25_ecn_mark: 0
> rx25_removed_vlan_packets: 4377204935
> rx25_wqe_err: 0
> rx25_mpwqe_filler_cqes: 0
> rx25_mpwqe_filler_strides: 0
> rx25_buff_alloc_err: 0
> rx25_cqe_compress_blks: 0
> rx25_cqe_compress_pkts: 0
> rx25_page_reuse: 0
> rx25_cache_reuse: 2132658660
> rx25_cache_full: 55943584
> rx25_cache_empty: 6656
> rx25_cache_busy: 55943804
> rx25_cache_waive: 0
> rx25_congst_umr: 0
> rx25_arfs_err: 0
> rx25_xdp_tx_xmit: 0
> rx25_xdp_tx_full: 0
> rx25_xdp_tx_err: 0
> rx25_xdp_tx_cqes: 0
> rx26_packets: 4496003688
> rx26_bytes: 5862180715503
> rx26_csum_complete: 4496003688
> rx26_csum_unnecessary: 0
> rx26_csum_unnecessary_inner: 0
> rx26_csum_none: 0
> rx26_xdp_drop: 0
> rx26_xdp_redirect: 0
> rx26_lro_packets: 0
> rx26_lro_bytes: 0
> rx26_ecn_mark: 0
> rx26_removed_vlan_packets: 4496003688
> rx26_wqe_err: 0
> rx26_mpwqe_filler_cqes: 0
> rx26_mpwqe_filler_strides: 0
> rx26_buff_alloc_err: 0
> rx26_cqe_compress_blks: 0
> rx26_cqe_compress_pkts: 0
> rx26_page_reuse: 0
> rx26_cache_reuse: 8
> rx26_cache_full: 2248001581
> rx26_cache_empty: 6656
> rx26_cache_busy: 2248001836
> rx26_cache_waive: 0
> rx26_congst_umr: 0
> rx26_arfs_err: 0
> rx26_xdp_tx_xmit: 0
> rx26_xdp_tx_full: 0
> rx26_xdp_tx_err: 0
> rx26_xdp_tx_cqes: 0
> rx27_packets: 4341849333
> rx27_bytes: 5678653545018
> rx27_csum_complete: 4341849333
> rx27_csum_unnecessary: 0
> rx27_csum_unnecessary_inner: 0
> rx27_csum_none: 0
> rx27_xdp_drop: 0
> rx27_xdp_redirect: 0
> rx27_lro_packets: 0
> rx27_lro_bytes: 0
> rx27_ecn_mark: 0
> rx27_removed_vlan_packets: 4341849333
> rx27_wqe_err: 0
> rx27_mpwqe_filler_cqes: 0
> rx27_mpwqe_filler_strides: 0
> rx27_buff_alloc_err: 0
> rx27_cqe_compress_blks: 0
> rx27_cqe_compress_pkts: 0
> rx27_page_reuse: 0
> rx27_cache_reuse: 1748188
> rx27_cache_full: 2169176223
> rx27_cache_empty: 6656
> rx27_cache_busy: 2169176476
> rx27_cache_waive: 0
> rx27_congst_umr: 0
> rx27_arfs_err: 0
> rx27_xdp_tx_xmit: 0
> rx27_xdp_tx_full: 0
> rx27_xdp_tx_err: 0
> rx27_xdp_tx_cqes: 0
> rx28_packets: 0
> rx28_bytes: 0
> rx28_csum_complete: 0
> rx28_csum_unnecessary: 0
> rx28_csum_unnecessary_inner: 0
> rx28_csum_none: 0
> rx28_xdp_drop: 0
> rx28_xdp_redirect: 0
> rx28_lro_packets: 0
> rx28_lro_bytes: 0
> rx28_ecn_mark: 0
> rx28_removed_vlan_packets: 0
> rx28_wqe_err: 0
> rx28_mpwqe_filler_cqes: 0
> rx28_mpwqe_filler_strides: 0
> rx28_buff_alloc_err: 0
> rx28_cqe_compress_blks: 0
> rx28_cqe_compress_pkts: 0
> rx28_page_reuse: 0
> rx28_cache_reuse: 0
> rx28_cache_full: 0
> rx28_cache_empty: 2560
> rx28_cache_busy: 0
> rx28_cache_waive: 0
> rx28_congst_umr: 0
> rx28_arfs_err: 0
> rx28_xdp_tx_xmit: 0
> rx28_xdp_tx_full: 0
> rx28_xdp_tx_err: 0
> rx28_xdp_tx_cqes: 0
> rx29_packets: 0
> rx29_bytes: 0
> rx29_csum_complete: 0
> rx29_csum_unnecessary: 0
> rx29_csum_unnecessary_inner: 0
> rx29_csum_none: 0
> rx29_xdp_drop: 0
> rx29_xdp_redirect: 0
> rx29_lro_packets: 0
> rx29_lro_bytes: 0
> rx29_ecn_mark: 0
> rx29_removed_vlan_packets: 0
> rx29_wqe_err: 0
> rx29_mpwqe_filler_cqes: 0
> rx29_mpwqe_filler_strides: 0
> rx29_buff_alloc_err: 0
> rx29_cqe_compress_blks: 0
> rx29_cqe_compress_pkts: 0
> rx29_page_reuse: 0
> rx29_cache_reuse: 0
> rx29_cache_full: 0
> rx29_cache_empty: 2560
> rx29_cache_busy: 0
> rx29_cache_waive: 0
> rx29_congst_umr: 0
> rx29_arfs_err: 0
> rx29_xdp_tx_xmit: 0
> rx29_xdp_tx_full: 0
> rx29_xdp_tx_err: 0
> rx29_xdp_tx_cqes: 0
> rx30_packets: 0
> rx30_bytes: 0
> rx30_csum_complete: 0
> rx30_csum_unnecessary: 0
> rx30_csum_unnecessary_inner: 0
> rx30_csum_none: 0
> rx30_xdp_drop: 0
> rx30_xdp_redirect: 0
> rx30_lro_packets: 0
> rx30_lro_bytes: 0
> rx30_ecn_mark: 0
> rx30_removed_vlan_packets: 0
> rx30_wqe_err: 0
> rx30_mpwqe_filler_cqes: 0
> rx30_mpwqe_filler_strides: 0
> rx30_buff_alloc_err: 0
> rx30_cqe_compress_blks: 0
> rx30_cqe_compress_pkts: 0
> rx30_page_reuse: 0
> rx30_cache_reuse: 0
> rx30_cache_full: 0
> rx30_cache_empty: 2560
> rx30_cache_busy: 0
> rx30_cache_waive: 0
> rx30_congst_umr: 0
> rx30_arfs_err: 0
> rx30_xdp_tx_xmit: 0
> rx30_xdp_tx_full: 0
> rx30_xdp_tx_err: 0
> rx30_xdp_tx_cqes: 0
> rx31_packets: 0
> rx31_bytes: 0
> rx31_csum_complete: 0
> rx31_csum_unnecessary: 0
> rx31_csum_unnecessary_inner: 0
> rx31_csum_none: 0
> rx31_xdp_drop: 0
> rx31_xdp_redirect: 0
> rx31_lro_packets: 0
> rx31_lro_bytes: 0
> rx31_ecn_mark: 0
> rx31_removed_vlan_packets: 0
> rx31_wqe_err: 0
> rx31_mpwqe_filler_cqes: 0
> rx31_mpwqe_filler_strides: 0
> rx31_buff_alloc_err: 0
> rx31_cqe_compress_blks: 0
> rx31_cqe_compress_pkts: 0
> rx31_page_reuse: 0
> rx31_cache_reuse: 0
> rx31_cache_full: 0
> rx31_cache_empty: 2560
> rx31_cache_busy: 0
> rx31_cache_waive: 0
> rx31_congst_umr: 0
> rx31_arfs_err: 0
> rx31_xdp_tx_xmit: 0
> rx31_xdp_tx_full: 0
> rx31_xdp_tx_err: 0
> rx31_xdp_tx_cqes: 0
> rx32_packets: 0
> rx32_bytes: 0
> rx32_csum_complete: 0
> rx32_csum_unnecessary: 0
> rx32_csum_unnecessary_inner: 0
> rx32_csum_none: 0
> rx32_xdp_drop: 0
> rx32_xdp_redirect: 0
> rx32_lro_packets: 0
> rx32_lro_bytes: 0
> rx32_ecn_mark: 0
> rx32_removed_vlan_packets: 0
> rx32_wqe_err: 0
> rx32_mpwqe_filler_cqes: 0
> rx32_mpwqe_filler_strides: 0
> rx32_buff_alloc_err: 0
> rx32_cqe_compress_blks: 0
> rx32_cqe_compress_pkts: 0
> rx32_page_reuse: 0
> rx32_cache_reuse: 0
> rx32_cache_full: 0
> rx32_cache_empty: 2560
> rx32_cache_busy: 0
> rx32_cache_waive: 0
> rx32_congst_umr: 0
> rx32_arfs_err: 0
> rx32_xdp_tx_xmit: 0
> rx32_xdp_tx_full: 0
> rx32_xdp_tx_err: 0
> rx32_xdp_tx_cqes: 0
> rx33_packets: 0
> rx33_bytes: 0
> rx33_csum_complete: 0
> rx33_csum_unnecessary: 0
> rx33_csum_unnecessary_inner: 0
> rx33_csum_none: 0
> rx33_xdp_drop: 0
> rx33_xdp_redirect: 0
> rx33_lro_packets: 0
> rx33_lro_bytes: 0
> rx33_ecn_mark: 0
> rx33_removed_vlan_packets: 0
> rx33_wqe_err: 0
> rx33_mpwqe_filler_cqes: 0
> rx33_mpwqe_filler_strides: 0
> rx33_buff_alloc_err: 0
> rx33_cqe_compress_blks: 0
> rx33_cqe_compress_pkts: 0
> rx33_page_reuse: 0
> rx33_cache_reuse: 0
> rx33_cache_full: 0
> rx33_cache_empty: 2560
> rx33_cache_busy: 0
> rx33_cache_waive: 0
> rx33_congst_umr: 0
> rx33_arfs_err: 0
> rx33_xdp_tx_xmit: 0
> rx33_xdp_tx_full: 0
> rx33_xdp_tx_err: 0
> rx33_xdp_tx_cqes: 0
> rx34_packets: 0
> rx34_bytes: 0
> rx34_csum_complete: 0
> rx34_csum_unnecessary: 0
> rx34_csum_unnecessary_inner: 0
> rx34_csum_none: 0
> rx34_xdp_drop: 0
> rx34_xdp_redirect: 0
> rx34_lro_packets: 0
> rx34_lro_bytes: 0
> rx34_ecn_mark: 0
> rx34_removed_vlan_packets: 0
> rx34_wqe_err: 0
> rx34_mpwqe_filler_cqes: 0
> rx34_mpwqe_filler_strides: 0
> rx34_buff_alloc_err: 0
> rx34_cqe_compress_blks: 0
> rx34_cqe_compress_pkts: 0
> rx34_page_reuse: 0
> rx34_cache_reuse: 0
> rx34_cache_full: 0
> rx34_cache_empty: 2560
> rx34_cache_busy: 0
> rx34_cache_waive: 0
> rx34_congst_umr: 0
> rx34_arfs_err: 0
> rx34_xdp_tx_xmit: 0
> rx34_xdp_tx_full: 0
> rx34_xdp_tx_err: 0
> rx34_xdp_tx_cqes: 0
> rx35_packets: 0
> rx35_bytes: 0
> rx35_csum_complete: 0
> rx35_csum_unnecessary: 0
> rx35_csum_unnecessary_inner: 0
> rx35_csum_none: 0
> rx35_xdp_drop: 0
> rx35_xdp_redirect: 0
> rx35_lro_packets: 0
> rx35_lro_bytes: 0
> rx35_ecn_mark: 0
> rx35_removed_vlan_packets: 0
> rx35_wqe_err: 0
> rx35_mpwqe_filler_cqes: 0
> rx35_mpwqe_filler_strides: 0
> rx35_buff_alloc_err: 0
> rx35_cqe_compress_blks: 0
> rx35_cqe_compress_pkts: 0
> rx35_page_reuse: 0
> rx35_cache_reuse: 0
> rx35_cache_full: 0
> rx35_cache_empty: 2560
> rx35_cache_busy: 0
> rx35_cache_waive: 0
> rx35_congst_umr: 0
> rx35_arfs_err: 0
> rx35_xdp_tx_xmit: 0
> rx35_xdp_tx_full: 0
> rx35_xdp_tx_err: 0
> rx35_xdp_tx_cqes: 0
> rx36_packets: 0
> rx36_bytes: 0
> rx36_csum_complete: 0
> rx36_csum_unnecessary: 0
> rx36_csum_unnecessary_inner: 0
> rx36_csum_none: 0
> rx36_xdp_drop: 0
> rx36_xdp_redirect: 0
> rx36_lro_packets: 0
> rx36_lro_bytes: 0
> rx36_ecn_mark: 0
> rx36_removed_vlan_packets: 0
> rx36_wqe_err: 0
> rx36_mpwqe_filler_cqes: 0
> rx36_mpwqe_filler_strides: 0
> rx36_buff_alloc_err: 0
> rx36_cqe_compress_blks: 0
> rx36_cqe_compress_pkts: 0
> rx36_page_reuse: 0
> rx36_cache_reuse: 0
> rx36_cache_full: 0
> rx36_cache_empty: 2560
> rx36_cache_busy: 0
> rx36_cache_waive: 0
> rx36_congst_umr: 0
> rx36_arfs_err: 0
> rx36_xdp_tx_xmit: 0
> rx36_xdp_tx_full: 0
> rx36_xdp_tx_err: 0
> rx36_xdp_tx_cqes: 0
> rx37_packets: 0
> rx37_bytes: 0
> rx37_csum_complete: 0
> rx37_csum_unnecessary: 0
> rx37_csum_unnecessary_inner: 0
> rx37_csum_none: 0
> rx37_xdp_drop: 0
> rx37_xdp_redirect: 0
> rx37_lro_packets: 0
> rx37_lro_bytes: 0
> rx37_ecn_mark: 0
> rx37_removed_vlan_packets: 0
> rx37_wqe_err: 0
> rx37_mpwqe_filler_cqes: 0
> rx37_mpwqe_filler_strides: 0
> rx37_buff_alloc_err: 0
> rx37_cqe_compress_blks: 0
> rx37_cqe_compress_pkts: 0
> rx37_page_reuse: 0
> rx37_cache_reuse: 0
> rx37_cache_full: 0
> rx37_cache_empty: 2560
> rx37_cache_busy: 0
> rx37_cache_waive: 0
> rx37_congst_umr: 0
> rx37_arfs_err: 0
> rx37_xdp_tx_xmit: 0
> rx37_xdp_tx_full: 0
> rx37_xdp_tx_err: 0
> rx37_xdp_tx_cqes: 0
> rx38_packets: 0
> rx38_bytes: 0
> rx38_csum_complete: 0
> rx38_csum_unnecessary: 0
> rx38_csum_unnecessary_inner: 0
> rx38_csum_none: 0
> rx38_xdp_drop: 0
> rx38_xdp_redirect: 0
> rx38_lro_packets: 0
> rx38_lro_bytes: 0
> rx38_ecn_mark: 0
> rx38_removed_vlan_packets: 0
> rx38_wqe_err: 0
> rx38_mpwqe_filler_cqes: 0
> rx38_mpwqe_filler_strides: 0
> rx38_buff_alloc_err: 0
> rx38_cqe_compress_blks: 0
> rx38_cqe_compress_pkts: 0
> rx38_page_reuse: 0
> rx38_cache_reuse: 0
> rx38_cache_full: 0
> rx38_cache_empty: 2560
> rx38_cache_busy: 0
> rx38_cache_waive: 0
> rx38_congst_umr: 0
> rx38_arfs_err: 0
> rx38_xdp_tx_xmit: 0
> rx38_xdp_tx_full: 0
> rx38_xdp_tx_err: 0
> rx38_xdp_tx_cqes: 0
> rx39_packets: 0
> rx39_bytes: 0
> rx39_csum_complete: 0
> rx39_csum_unnecessary: 0
> rx39_csum_unnecessary_inner: 0
> rx39_csum_none: 0
> rx39_xdp_drop: 0
> rx39_xdp_redirect: 0
> rx39_lro_packets: 0
> rx39_lro_bytes: 0
> rx39_ecn_mark: 0
> rx39_removed_vlan_packets: 0
> rx39_wqe_err: 0
> rx39_mpwqe_filler_cqes: 0
> rx39_mpwqe_filler_strides: 0
> rx39_buff_alloc_err: 0
> rx39_cqe_compress_blks: 0
> rx39_cqe_compress_pkts: 0
> rx39_page_reuse: 0
> rx39_cache_reuse: 0
> rx39_cache_full: 0
> rx39_cache_empty: 2560
> rx39_cache_busy: 0
> rx39_cache_waive: 0
> rx39_congst_umr: 0
> rx39_arfs_err: 0
> rx39_xdp_tx_xmit: 0
> rx39_xdp_tx_full: 0
> rx39_xdp_tx_err: 0
> rx39_xdp_tx_cqes: 0
> rx40_packets: 0
> rx40_bytes: 0
> rx40_csum_complete: 0
> rx40_csum_unnecessary: 0
> rx40_csum_unnecessary_inner: 0
> rx40_csum_none: 0
> rx40_xdp_drop: 0
> rx40_xdp_redirect: 0
> rx40_lro_packets: 0
> rx40_lro_bytes: 0
> rx40_ecn_mark: 0
> rx40_removed_vlan_packets: 0
> rx40_wqe_err: 0
> rx40_mpwqe_filler_cqes: 0
> rx40_mpwqe_filler_strides: 0
> rx40_buff_alloc_err: 0
> rx40_cqe_compress_blks: 0
> rx40_cqe_compress_pkts: 0
> rx40_page_reuse: 0
> rx40_cache_reuse: 0
> rx40_cache_full: 0
> rx40_cache_empty: 2560
> rx40_cache_busy: 0
> rx40_cache_waive: 0
> rx40_congst_umr: 0
> rx40_arfs_err: 0
> rx40_xdp_tx_xmit: 0
> rx40_xdp_tx_full: 0
> rx40_xdp_tx_err: 0
> rx40_xdp_tx_cqes: 0
> rx41_packets: 0
> rx41_bytes: 0
> rx41_csum_complete: 0
> rx41_csum_unnecessary: 0
> rx41_csum_unnecessary_inner: 0
> rx41_csum_none: 0
> rx41_xdp_drop: 0
> rx41_xdp_redirect: 0
> rx41_lro_packets: 0
> rx41_lro_bytes: 0
> rx41_ecn_mark: 0
> rx41_removed_vlan_packets: 0
> rx41_wqe_err: 0
> rx41_mpwqe_filler_cqes: 0
> rx41_mpwqe_filler_strides: 0
> rx41_buff_alloc_err: 0
> rx41_cqe_compress_blks: 0
> rx41_cqe_compress_pkts: 0
> rx41_page_reuse: 0
> rx41_cache_reuse: 0
> rx41_cache_full: 0
> rx41_cache_empty: 2560
> rx41_cache_busy: 0
> rx41_cache_waive: 0
> rx41_congst_umr: 0
> rx41_arfs_err: 0
> rx41_xdp_tx_xmit: 0
> rx41_xdp_tx_full: 0
> rx41_xdp_tx_err: 0
> rx41_xdp_tx_cqes: 0
> rx42_packets: 0
> rx42_bytes: 0
> rx42_csum_complete: 0
> rx42_csum_unnecessary: 0
> rx42_csum_unnecessary_inner: 0
> rx42_csum_none: 0
> rx42_xdp_drop: 0
> rx42_xdp_redirect: 0
> rx42_lro_packets: 0
> rx42_lro_bytes: 0
> rx42_ecn_mark: 0
> rx42_removed_vlan_packets: 0
> rx42_wqe_err: 0
> rx42_mpwqe_filler_cqes: 0
> rx42_mpwqe_filler_strides: 0
> rx42_buff_alloc_err: 0
> rx42_cqe_compress_blks: 0
> rx42_cqe_compress_pkts: 0
> rx42_page_reuse: 0
> rx42_cache_reuse: 0
> rx42_cache_full: 0
> rx42_cache_empty: 2560
> rx42_cache_busy: 0
> rx42_cache_waive: 0
> rx42_congst_umr: 0
> rx42_arfs_err: 0
> rx42_xdp_tx_xmit: 0
> rx42_xdp_tx_full: 0
> rx42_xdp_tx_err: 0
> rx42_xdp_tx_cqes: 0
> rx43_packets: 0
> rx43_bytes: 0
> rx43_csum_complete: 0
> rx43_csum_unnecessary: 0
> rx43_csum_unnecessary_inner: 0
> rx43_csum_none: 0
> rx43_xdp_drop: 0
> rx43_xdp_redirect: 0
> rx43_lro_packets: 0
> rx43_lro_bytes: 0
> rx43_ecn_mark: 0
> rx43_removed_vlan_packets: 0
> rx43_wqe_err: 0
> rx43_mpwqe_filler_cqes: 0
> rx43_mpwqe_filler_strides: 0
> rx43_buff_alloc_err: 0
> rx43_cqe_compress_blks: 0
> rx43_cqe_compress_pkts: 0
> rx43_page_reuse: 0
> rx43_cache_reuse: 0
> rx43_cache_full: 0
> rx43_cache_empty: 2560
> rx43_cache_busy: 0
> rx43_cache_waive: 0
> rx43_congst_umr: 0
> rx43_arfs_err: 0
> rx43_xdp_tx_xmit: 0
> rx43_xdp_tx_full: 0
> rx43_xdp_tx_err: 0
> rx43_xdp_tx_cqes: 0
> rx44_packets: 0
> rx44_bytes: 0
> rx44_csum_complete: 0
> rx44_csum_unnecessary: 0
> rx44_csum_unnecessary_inner: 0
> rx44_csum_none: 0
> rx44_xdp_drop: 0
> rx44_xdp_redirect: 0
> rx44_lro_packets: 0
> rx44_lro_bytes: 0
> rx44_ecn_mark: 0
> rx44_removed_vlan_packets: 0
> rx44_wqe_err: 0
> rx44_mpwqe_filler_cqes: 0
> rx44_mpwqe_filler_strides: 0
> rx44_buff_alloc_err: 0
> rx44_cqe_compress_blks: 0
> rx44_cqe_compress_pkts: 0
> rx44_page_reuse: 0
> rx44_cache_reuse: 0
> rx44_cache_full: 0
> rx44_cache_empty: 2560
> rx44_cache_busy: 0
> rx44_cache_waive: 0
> rx44_congst_umr: 0
> rx44_arfs_err: 0
> rx44_xdp_tx_xmit: 0
> rx44_xdp_tx_full: 0
> rx44_xdp_tx_err: 0
> rx44_xdp_tx_cqes: 0
> rx45_packets: 0
> rx45_bytes: 0
> rx45_csum_complete: 0
> rx45_csum_unnecessary: 0
> rx45_csum_unnecessary_inner: 0
> rx45_csum_none: 0
> rx45_xdp_drop: 0
> rx45_xdp_redirect: 0
> rx45_lro_packets: 0
> rx45_lro_bytes: 0
> rx45_ecn_mark: 0
> rx45_removed_vlan_packets: 0
> rx45_wqe_err: 0
> rx45_mpwqe_filler_cqes: 0
> rx45_mpwqe_filler_strides: 0
> rx45_buff_alloc_err: 0
> rx45_cqe_compress_blks: 0
> rx45_cqe_compress_pkts: 0
> rx45_page_reuse: 0
> rx45_cache_reuse: 0
> rx45_cache_full: 0
> rx45_cache_empty: 2560
> rx45_cache_busy: 0
> rx45_cache_waive: 0
> rx45_congst_umr: 0
> rx45_arfs_err: 0
> rx45_xdp_tx_xmit: 0
> rx45_xdp_tx_full: 0
> rx45_xdp_tx_err: 0
> rx45_xdp_tx_cqes: 0
> rx46_packets: 0
> rx46_bytes: 0
> rx46_csum_complete: 0
> rx46_csum_unnecessary: 0
> rx46_csum_unnecessary_inner: 0
> rx46_csum_none: 0
> rx46_xdp_drop: 0
> rx46_xdp_redirect: 0
> rx46_lro_packets: 0
> rx46_lro_bytes: 0
> rx46_ecn_mark: 0
> rx46_removed_vlan_packets: 0
> rx46_wqe_err: 0
> rx46_mpwqe_filler_cqes: 0
> rx46_mpwqe_filler_strides: 0
> rx46_buff_alloc_err: 0
> rx46_cqe_compress_blks: 0
> rx46_cqe_compress_pkts: 0
> rx46_page_reuse: 0
> rx46_cache_reuse: 0
> rx46_cache_full: 0
> rx46_cache_empty: 2560
> rx46_cache_busy: 0
> rx46_cache_waive: 0
> rx46_congst_umr: 0
> rx46_arfs_err: 0
> rx46_xdp_tx_xmit: 0
> rx46_xdp_tx_full: 0
> rx46_xdp_tx_err: 0
> rx46_xdp_tx_cqes: 0
> rx47_packets: 0
> rx47_bytes: 0
> rx47_csum_complete: 0
> rx47_csum_unnecessary: 0
> rx47_csum_unnecessary_inner: 0
> rx47_csum_none: 0
> rx47_xdp_drop: 0
> rx47_xdp_redirect: 0
> rx47_lro_packets: 0
> rx47_lro_bytes: 0
> rx47_ecn_mark: 0
> rx47_removed_vlan_packets: 0
> rx47_wqe_err: 0
> rx47_mpwqe_filler_cqes: 0
> rx47_mpwqe_filler_strides: 0
> rx47_buff_alloc_err: 0
> rx47_cqe_compress_blks: 0
> rx47_cqe_compress_pkts: 0
> rx47_page_reuse: 0
> rx47_cache_reuse: 0
> rx47_cache_full: 0
> rx47_cache_empty: 2560
> rx47_cache_busy: 0
> rx47_cache_waive: 0
> rx47_congst_umr: 0
> rx47_arfs_err: 0
> rx47_xdp_tx_xmit: 0
> rx47_xdp_tx_full: 0
> rx47_xdp_tx_err: 0
> rx47_xdp_tx_cqes: 0
> rx48_packets: 0
> rx48_bytes: 0
> rx48_csum_complete: 0
> rx48_csum_unnecessary: 0
> rx48_csum_unnecessary_inner: 0
> rx48_csum_none: 0
> rx48_xdp_drop: 0
> rx48_xdp_redirect: 0
> rx48_lro_packets: 0
> rx48_lro_bytes: 0
> rx48_ecn_mark: 0
> rx48_removed_vlan_packets: 0
> rx48_wqe_err: 0
> rx48_mpwqe_filler_cqes: 0
> rx48_mpwqe_filler_strides: 0
> rx48_buff_alloc_err: 0
> rx48_cqe_compress_blks: 0
> rx48_cqe_compress_pkts: 0
> rx48_page_reuse: 0
> rx48_cache_reuse: 0
> rx48_cache_full: 0
> rx48_cache_empty: 2560
> rx48_cache_busy: 0
> rx48_cache_waive: 0
> rx48_congst_umr: 0
> rx48_arfs_err: 0
> rx48_xdp_tx_xmit: 0
> rx48_xdp_tx_full: 0
> rx48_xdp_tx_err: 0
> rx48_xdp_tx_cqes: 0
> rx49_packets: 0
> rx49_bytes: 0
> rx49_csum_complete: 0
> rx49_csum_unnecessary: 0
> rx49_csum_unnecessary_inner: 0
> rx49_csum_none: 0
> rx49_xdp_drop: 0
> rx49_xdp_redirect: 0
> rx49_lro_packets: 0
> rx49_lro_bytes: 0
> rx49_ecn_mark: 0
> rx49_removed_vlan_packets: 0
> rx49_wqe_err: 0
> rx49_mpwqe_filler_cqes: 0
> rx49_mpwqe_filler_strides: 0
> rx49_buff_alloc_err: 0
> rx49_cqe_compress_blks: 0
> rx49_cqe_compress_pkts: 0
> rx49_page_reuse: 0
> rx49_cache_reuse: 0
> rx49_cache_full: 0
> rx49_cache_empty: 2560
> rx49_cache_busy: 0
> rx49_cache_waive: 0
> rx49_congst_umr: 0
> rx49_arfs_err: 0
> rx49_xdp_tx_xmit: 0
> rx49_xdp_tx_full: 0
> rx49_xdp_tx_err: 0
> rx49_xdp_tx_cqes: 0
> rx50_packets: 0
> rx50_bytes: 0
> rx50_csum_complete: 0
> rx50_csum_unnecessary: 0
> rx50_csum_unnecessary_inner: 0
> rx50_csum_none: 0
> rx50_xdp_drop: 0
> rx50_xdp_redirect: 0
> rx50_lro_packets: 0
> rx50_lro_bytes: 0
> rx50_ecn_mark: 0
> rx50_removed_vlan_packets: 0
> rx50_wqe_err: 0
> rx50_mpwqe_filler_cqes: 0
> rx50_mpwqe_filler_strides: 0
> rx50_buff_alloc_err: 0
> rx50_cqe_compress_blks: 0
> rx50_cqe_compress_pkts: 0
> rx50_page_reuse: 0
> rx50_cache_reuse: 0
> rx50_cache_full: 0
> rx50_cache_empty: 2560
> rx50_cache_busy: 0
> rx50_cache_waive: 0
> rx50_congst_umr: 0
> rx50_arfs_err: 0
> rx50_xdp_tx_xmit: 0
> rx50_xdp_tx_full: 0
> rx50_xdp_tx_err: 0
> rx50_xdp_tx_cqes: 0
> rx51_packets: 0
> rx51_bytes: 0
> rx51_csum_complete: 0
> rx51_csum_unnecessary: 0
> rx51_csum_unnecessary_inner: 0
> rx51_csum_none: 0
> rx51_xdp_drop: 0
> rx51_xdp_redirect: 0
> rx51_lro_packets: 0
> rx51_lro_bytes: 0
> rx51_ecn_mark: 0
> rx51_removed_vlan_packets: 0
> rx51_wqe_err: 0
> rx51_mpwqe_filler_cqes: 0
> rx51_mpwqe_filler_strides: 0
> rx51_buff_alloc_err: 0
> rx51_cqe_compress_blks: 0
> rx51_cqe_compress_pkts: 0
> rx51_page_reuse: 0
> rx51_cache_reuse: 0
> rx51_cache_full: 0
> rx51_cache_empty: 2560
> rx51_cache_busy: 0
> rx51_cache_waive: 0
> rx51_congst_umr: 0
> rx51_arfs_err: 0
> rx51_xdp_tx_xmit: 0
> rx51_xdp_tx_full: 0
> rx51_xdp_tx_err: 0
> rx51_xdp_tx_cqes: 0
> rx52_packets: 0
> rx52_bytes: 0
> rx52_csum_complete: 0
> rx52_csum_unnecessary: 0
> rx52_csum_unnecessary_inner: 0
> rx52_csum_none: 0
> rx52_xdp_drop: 0
> rx52_xdp_redirect: 0
> rx52_lro_packets: 0
> rx52_lro_bytes: 0
> rx52_ecn_mark: 0
> rx52_removed_vlan_packets: 0
> rx52_wqe_err: 0
> rx52_mpwqe_filler_cqes: 0
> rx52_mpwqe_filler_strides: 0
> rx52_buff_alloc_err: 0
> rx52_cqe_compress_blks: 0
> rx52_cqe_compress_pkts: 0
> rx52_page_reuse: 0
> rx52_cache_reuse: 0
> rx52_cache_full: 0
> rx52_cache_empty: 2560
> rx52_cache_busy: 0
> rx52_cache_waive: 0
> rx52_congst_umr: 0
> rx52_arfs_err: 0
> rx52_xdp_tx_xmit: 0
> rx52_xdp_tx_full: 0
> rx52_xdp_tx_err: 0
> rx52_xdp_tx_cqes: 0
> rx53_packets: 0
> rx53_bytes: 0
> rx53_csum_complete: 0
> rx53_csum_unnecessary: 0
> rx53_csum_unnecessary_inner: 0
> rx53_csum_none: 0
> rx53_xdp_drop: 0
> rx53_xdp_redirect: 0
> rx53_lro_packets: 0
> rx53_lro_bytes: 0
> rx53_ecn_mark: 0
> rx53_removed_vlan_packets: 0
> rx53_wqe_err: 0
> rx53_mpwqe_filler_cqes: 0
> rx53_mpwqe_filler_strides: 0
> rx53_buff_alloc_err: 0
> rx53_cqe_compress_blks: 0
> rx53_cqe_compress_pkts: 0
> rx53_page_reuse: 0
> rx53_cache_reuse: 0
> rx53_cache_full: 0
> rx53_cache_empty: 2560
> rx53_cache_busy: 0
> rx53_cache_waive: 0
> rx53_congst_umr: 0
> rx53_arfs_err: 0
> rx53_xdp_tx_xmit: 0
> rx53_xdp_tx_full: 0
> rx53_xdp_tx_err: 0
> rx53_xdp_tx_cqes: 0
> rx54_packets: 0
> rx54_bytes: 0
> rx54_csum_complete: 0
> rx54_csum_unnecessary: 0
> rx54_csum_unnecessary_inner: 0
> rx54_csum_none: 0
> rx54_xdp_drop: 0
> rx54_xdp_redirect: 0
> rx54_lro_packets: 0
> rx54_lro_bytes: 0
> rx54_ecn_mark: 0
> rx54_removed_vlan_packets: 0
> rx54_wqe_err: 0
> rx54_mpwqe_filler_cqes: 0
> rx54_mpwqe_filler_strides: 0
> rx54_buff_alloc_err: 0
> rx54_cqe_compress_blks: 0
> rx54_cqe_compress_pkts: 0
> rx54_page_reuse: 0
> rx54_cache_reuse: 0
> rx54_cache_full: 0
> rx54_cache_empty: 2560
> rx54_cache_busy: 0
> rx54_cache_waive: 0
> rx54_congst_umr: 0
> rx54_arfs_err: 0
> rx54_xdp_tx_xmit: 0
> rx54_xdp_tx_full: 0
> rx54_xdp_tx_err: 0
> rx54_xdp_tx_cqes: 0
> rx55_packets: 0
> rx55_bytes: 0
> rx55_csum_complete: 0
> rx55_csum_unnecessary: 0
> rx55_csum_unnecessary_inner: 0
> rx55_csum_none: 0
> rx55_xdp_drop: 0
> rx55_xdp_redirect: 0
> rx55_lro_packets: 0
> rx55_lro_bytes: 0
> rx55_ecn_mark: 0
> rx55_removed_vlan_packets: 0
> rx55_wqe_err: 0
> rx55_mpwqe_filler_cqes: 0
> rx55_mpwqe_filler_strides: 0
> rx55_buff_alloc_err: 0
> rx55_cqe_compress_blks: 0
> rx55_cqe_compress_pkts: 0
> rx55_page_reuse: 0
> rx55_cache_reuse: 0
> rx55_cache_full: 0
> rx55_cache_empty: 2560
> rx55_cache_busy: 0
> rx55_cache_waive: 0
> rx55_congst_umr: 0
> rx55_arfs_err: 0
> rx55_xdp_tx_xmit: 0
> rx55_xdp_tx_full: 0
> rx55_xdp_tx_err: 0
> rx55_xdp_tx_cqes: 0
> tx0_packets: 6019477917
> tx0_bytes: 3445238940825
> tx0_tso_packets: 311304622
> tx0_tso_bytes: 1897094773213
> tx0_tso_inner_packets: 0
> tx0_tso_inner_bytes: 0
> tx0_csum_partial: 457981794
> tx0_csum_partial_inner: 0
> tx0_added_vlan_packets: 4965567654
> tx0_nop: 72290329
> tx0_csum_none: 4507585860
> tx0_stopped: 9118
> tx0_dropped: 0
> tx0_xmit_more: 51651593
> tx0_recover: 0
> tx0_cqes: 4913918402
> tx0_wake: 9118
> tx0_cqe_err: 0
> tx1_packets: 5700413414
> tx1_bytes: 3340870662350
> tx1_tso_packets: 318201557
> tx1_tso_bytes: 1915233462303
> tx1_tso_inner_packets: 0
> tx1_tso_inner_bytes: 0
> tx1_csum_partial: 461736722
> tx1_csum_partial_inner: 0
> tx1_added_vlan_packets: 4638708749
> tx1_nop: 70061796
> tx1_csum_none: 4176972027
> tx1_stopped: 9248
> tx1_dropped: 0
> tx1_xmit_more: 39531959
> tx1_recover: 0
> tx1_cqes: 4599179178
> tx1_wake: 9248
> tx1_cqe_err: 0
> tx2_packets: 5795960848
> tx2_bytes: 3394876820271
> tx2_tso_packets: 322935065
> tx2_tso_bytes: 1910825901109
> tx2_tso_inner_packets: 0
> tx2_tso_inner_bytes: 0
> tx2_csum_partial: 460747092
> tx2_csum_partial_inner: 0
> tx2_added_vlan_packets: 4743705654
> tx2_nop: 72722430
> tx2_csum_none: 4282958562
> tx2_stopped: 8938
> tx2_dropped: 0
> tx2_xmit_more: 44084718
> tx2_recover: 0
> tx2_cqes: 4699623410
> tx2_wake: 8938
> tx2_cqe_err: 0
> tx3_packets: 5580215878
> tx3_bytes: 3191677257787
> tx3_tso_packets: 305771141
> tx3_tso_bytes: 1823265793476
> tx3_tso_inner_packets: 0
> tx3_tso_inner_bytes: 0
> tx3_csum_partial: 434976070
> tx3_csum_partial_inner: 0
> tx3_added_vlan_packets: 4569899956
> tx3_nop: 68184348
> tx3_csum_none: 4134923886
> tx3_stopped: 8383
> tx3_dropped: 0
> tx3_xmit_more: 41940375
> tx3_recover: 0
> tx3_cqes: 4527961924
> tx3_wake: 8383
> tx3_cqe_err: 0
> tx4_packets: 6795007068
> tx4_bytes: 3963890025270
> tx4_tso_packets: 358437617
> tx4_tso_bytes: 2154747995355
> tx4_tso_inner_packets: 0
> tx4_tso_inner_bytes: 0
> tx4_csum_partial: 504764524
> tx4_csum_partial_inner: 0
> tx4_added_vlan_packets: 5602510191
> tx4_nop: 81345604
> tx4_csum_none: 5097745667
> tx4_stopped: 10248
> tx4_dropped: 0
> tx4_xmit_more: 49068571
> tx4_recover: 0
> tx4_cqes: 5553444276
> tx4_wake: 10248
> tx4_cqe_err: 0
> tx5_packets: 6408089261
> tx5_bytes: 3676275848279
> tx5_tso_packets: 345129329
> tx5_tso_bytes: 2108447877473
> tx5_tso_inner_packets: 0
> tx5_tso_inner_bytes: 0
> tx5_csum_partial: 494705894
> tx5_csum_partial_inner: 0
> tx5_added_vlan_packets: 5235998343
> tx5_nop: 77694627
> tx5_csum_none: 4741292449
> tx5_stopped: 46
> tx5_dropped: 0
> tx5_xmit_more: 46675831
> tx5_recover: 0
> tx5_cqes: 5189323550
> tx5_wake: 46
> tx5_cqe_err: 0
> tx6_packets: 6382289663
> tx6_bytes: 3670991418150
> tx6_tso_packets: 342927826
> tx6_tso_bytes: 2075049679904
> tx6_tso_inner_packets: 0
> tx6_tso_inner_bytes: 0
> tx6_csum_partial: 490369221
> tx6_csum_partial_inner: 0
> tx6_added_vlan_packets: 5232144528
> tx6_nop: 77391246
> tx6_csum_none: 4741775307
> tx6_stopped: 10823
> tx6_dropped: 0
> tx6_xmit_more: 44487607
> tx6_recover: 0
> tx6_cqes: 5187659877
> tx6_wake: 10823
> tx6_cqe_err: 0
> tx7_packets: 6456378284
> tx7_bytes: 3758013320518
> tx7_tso_packets: 350958294
> tx7_tso_bytes: 2126833408524
> tx7_tso_inner_packets: 0
> tx7_tso_inner_bytes: 0
> tx7_csum_partial: 501804109
> tx7_csum_partial_inner: 0
> tx7_added_vlan_packets: 5275635204
> tx7_nop: 79010883
> tx7_csum_none: 4773831096
> tx7_stopped: 14684
> tx7_dropped: 0
> tx7_xmit_more: 44447469
> tx7_recover: 0
> tx7_cqes: 5231191770
> tx7_wake: 14684
> tx7_cqe_err: 0
> tx8_packets: 6401799768
> tx8_bytes: 3681210808766
> tx8_tso_packets: 342878228
> tx8_tso_bytes: 2089688012191
> tx8_tso_inner_packets: 0
> tx8_tso_inner_bytes: 0
> tx8_csum_partial: 494865145
> tx8_csum_partial_inner: 0
> tx8_added_vlan_packets: 5242288908
> tx8_nop: 77250910
> tx8_csum_none: 4747423763
> tx8_stopped: 2
> tx8_dropped: 0
> tx8_xmit_more: 44191737
> tx8_recover: 0
> tx8_cqes: 5198098454
> tx8_wake: 2
> tx8_cqe_err: 0
> tx9_packets: 6632882888
> tx9_bytes: 3820110338309
> tx9_tso_packets: 354189056
> tx9_tso_bytes: 2187883597128
> tx9_tso_inner_packets: 0
> tx9_tso_inner_bytes: 0
> tx9_csum_partial: 511108218
> tx9_csum_partial_inner: 0
> tx9_added_vlan_packets: 5413836353
> tx9_nop: 80560668
> tx9_csum_none: 4902728135
> tx9_stopped: 9091
> tx9_dropped: 0
> tx9_xmit_more: 54501293
> tx9_recover: 0
> tx9_cqes: 5359337638
> tx9_wake: 9091
> tx9_cqe_err: 0
> tx10_packets: 6421786406
> tx10_bytes: 3692798413429
> tx10_tso_packets: 346878943
> tx10_tso_bytes: 2111921062110
> tx10_tso_inner_packets: 0
> tx10_tso_inner_bytes: 0
> tx10_csum_partial: 494356645
> tx10_csum_partial_inner: 0
> tx10_added_vlan_packets: 5248274374
> tx10_nop: 77922624
> tx10_csum_none: 4753917730
> tx10_stopped: 9617
> tx10_dropped: 0
> tx10_xmit_more: 44473939
> tx10_recover: 0
> tx10_cqes: 5203802547
> tx10_wake: 9617
> tx10_cqe_err: 0
> tx11_packets: 6406750938
> tx11_bytes: 3660343565126
> tx11_tso_packets: 355917271
> tx11_tso_bytes: 2130812246956
> tx11_tso_inner_packets: 0
> tx11_tso_inner_bytes: 0
> tx11_csum_partial: 500336369
> tx11_csum_partial_inner: 0
> tx11_added_vlan_packets: 5228267547
> tx11_nop: 78906315
> tx11_csum_none: 4727931178
> tx11_stopped: 9607
> tx11_dropped: 0
> tx11_xmit_more: 40041492
> tx11_recover: 0
> tx11_cqes: 5188228290
> tx11_wake: 9607
> tx11_cqe_err: 0
> tx12_packets: 6422347846
> tx12_bytes: 3718772753227
> tx12_tso_packets: 355397223
> tx12_tso_bytes: 2162614059758
> tx12_tso_inner_packets: 0
> tx12_tso_inner_bytes: 0
> tx12_csum_partial: 511437844
> tx12_csum_partial_inner: 0
> tx12_added_vlan_packets: 5221373746
> tx12_nop: 78866779
> tx12_csum_none: 4709935902
> tx12_stopped: 10280
> tx12_dropped: 0
> tx12_xmit_more: 42189399
> tx12_recover: 0
> tx12_cqes: 5179187154
> tx12_wake: 10280
> tx12_cqe_err: 0
> tx13_packets: 6429383816
> tx13_bytes: 3725679445046
> tx13_tso_packets: 360934759
> tx13_tso_bytes: 2148016411436
> tx13_tso_inner_packets: 0
> tx13_tso_inner_bytes: 0
> tx13_csum_partial: 505245849
> tx13_csum_partial_inner: 0
> tx13_added_vlan_packets: 5240267441
> tx13_nop: 80295637
> tx13_csum_none: 4735021592
> tx13_stopped: 84
> tx13_dropped: 0
> tx13_xmit_more: 43118045
> tx13_recover: 0
> tx13_cqes: 5197150348
> tx13_wake: 84
> tx13_cqe_err: 0
> tx14_packets: 6375279148
> tx14_bytes: 3624267203336
> tx14_tso_packets: 344388148
> tx14_tso_bytes: 2094966273548
> tx14_tso_inner_packets: 0
> tx14_tso_inner_bytes: 0
> tx14_csum_partial: 494129407
> tx14_csum_partial_inner: 0
> tx14_added_vlan_packets: 5210749337
> tx14_nop: 77280615
> tx14_csum_none: 4716619930
> tx14_stopped: 13057
> tx14_dropped: 0
> tx14_xmit_more: 40849682
> tx14_recover: 0
> tx14_cqes: 5169902694
> tx14_wake: 13057
> tx14_cqe_err: 0
> tx15_packets: 6489306520
> tx15_bytes: 3775716194795
> tx15_tso_packets: 368716406
> tx15_tso_bytes: 2165876423354
> tx15_tso_inner_packets: 0
> tx15_tso_inner_bytes: 0
> tx15_csum_partial: 509887864
> tx15_csum_partial_inner: 0
> tx15_added_vlan_packets: 5296767390
> tx15_nop: 80803468
> tx15_csum_none: 4786879529
> tx15_stopped: 1
> tx15_dropped: 0
> tx15_xmit_more: 46979676
> tx15_recover: 0
> tx15_cqes: 5249789328
> tx15_wake: 1
> tx15_cqe_err: 0
> tx16_packets: 6559857761
> tx16_bytes: 3724080573905
> tx16_tso_packets: 350864176
> tx16_tso_bytes: 2099634006033
> tx16_tso_inner_packets: 0
> tx16_tso_inner_bytes: 0
> tx16_csum_partial: 489397232
> tx16_csum_partial_inner: 0
> tx16_added_vlan_packets: 5398869334
> tx16_nop: 79046075
> tx16_csum_none: 4909472106
> tx16_stopped: 4480
> tx16_dropped: 0
> tx16_xmit_more: 47273286
> tx16_recover: 0
> tx16_cqes: 5351598315
> tx16_wake: 4480
> tx16_cqe_err: 0
> tx17_packets: 6358711533
> tx17_bytes: 3650180865573
> tx17_tso_packets: 350723136
> tx17_tso_bytes: 2109426587128
> tx17_tso_inner_packets: 0
> tx17_tso_inner_bytes: 0
> tx17_csum_partial: 494719487
> tx17_csum_partial_inner: 0
> tx17_added_vlan_packets: 5190068796
> tx17_nop: 77285612
> tx17_csum_none: 4695349309
> tx17_stopped: 10443
> tx17_dropped: 0
> tx17_xmit_more: 45582108
> tx17_recover: 0
> tx17_cqes: 5144489363
> tx17_wake: 10443
> tx17_cqe_err: 0
> tx18_packets: 6655328437
> tx18_bytes: 3801768461807
> tx18_tso_packets: 356516373
> tx18_tso_bytes: 2164829247550
> tx18_tso_inner_packets: 0
> tx18_tso_inner_bytes: 0
> tx18_csum_partial: 500508446
> tx18_csum_partial_inner: 0
> tx18_added_vlan_packets: 5454166840
> tx18_nop: 80423007
> tx18_csum_none: 4953658394
> tx18_stopped: 14760
> tx18_dropped: 0
> tx18_xmit_more: 50837465
> tx18_recover: 0
> tx18_cqes: 5403332553
> tx18_wake: 14760
> tx18_cqe_err: 0
> tx19_packets: 6408680611
> tx19_bytes: 3644119934372
> tx19_tso_packets: 350727530
> tx19_tso_bytes: 2089896715365
> tx19_tso_inner_packets: 0
> tx19_tso_inner_bytes: 0
> tx19_csum_partial: 486536490
> tx19_csum_partial_inner: 0
> tx19_added_vlan_packets: 5255839020
> tx19_nop: 78525198
> tx19_csum_none: 4769302530
> tx19_stopped: 8614
> tx19_dropped: 0
> tx19_xmit_more: 43605232
> tx19_recover: 0
> tx19_cqes: 5212236833
> tx19_wake: 8614
> tx19_cqe_err: 0
> tx20_packets: 5609275141
> tx20_bytes: 3187279031581
> tx20_tso_packets: 298609303
> tx20_tso_bytes: 1794382229379
> tx20_tso_inner_packets: 0
> tx20_tso_inner_bytes: 0
> tx20_csum_partial: 430691178
> tx20_csum_partial_inner: 0
> tx20_added_vlan_packets: 4616844286
> tx20_nop: 67450040
> tx20_csum_none: 4186153108
> tx20_stopped: 9099
> tx20_dropped: 0
> tx20_xmit_more: 42040991
> tx20_recover: 0
> tx20_cqes: 4574805846
> tx20_wake: 9099
> tx20_cqe_err: 0
> tx21_packets: 5641621183
> tx21_bytes: 3279282331124
> tx21_tso_packets: 311297057
> tx21_tso_bytes: 1875735401012
> tx21_tso_inner_packets: 0
> tx21_tso_inner_bytes: 0
> tx21_csum_partial: 444333894
> tx21_csum_partial_inner: 0
> tx21_added_vlan_packets: 4603527701
> tx21_nop: 68857983
> tx21_csum_none: 4159193807
> tx21_stopped: 10082
> tx21_dropped: 0
> tx21_xmit_more: 43988081
> tx21_recover: 0
> tx21_cqes: 4559542410
> tx21_wake: 10082
> tx21_cqe_err: 0
> tx22_packets: 5822168288
> tx22_bytes: 3452026726862
> tx22_tso_packets: 308230791
> tx22_tso_bytes: 1859686450671
> tx22_tso_inner_packets: 0
> tx22_tso_inner_bytes: 0
> tx22_csum_partial: 442751518
> tx22_csum_partial_inner: 0
> tx22_added_vlan_packets: 4792100335
> tx22_nop: 70631706
> tx22_csum_none: 4349348817
> tx22_stopped: 9355
> tx22_dropped: 0
> tx22_xmit_more: 45165994
> tx22_recover: 0
> tx22_cqes: 4746936601
> tx22_wake: 9355
> tx22_cqe_err: 0
> tx23_packets: 5664896066
> tx23_bytes: 3207724186946
> tx23_tso_packets: 300418757
> tx23_tso_bytes: 1794180478679
> tx23_tso_inner_packets: 0
> tx23_tso_inner_bytes: 0
> tx23_csum_partial: 429898848
> tx23_csum_partial_inner: 0
> tx23_added_vlan_packets: 4674317320
> tx23_nop: 67899896
> tx23_csum_none: 4244418472
> tx23_stopped: 11684
> tx23_dropped: 0
> tx23_xmit_more: 43351132
> tx23_recover: 0
> tx23_cqes: 4630969028
> tx23_wake: 11684
> tx23_cqe_err: 0
> tx24_packets: 5663326601
> tx24_bytes: 3250127095110
> tx24_tso_packets: 301327422
> tx24_tso_bytes: 1831260534157
> tx24_tso_inner_packets: 0
> tx24_tso_inner_bytes: 0
> tx24_csum_partial: 438757312
> tx24_csum_partial_inner: 0
> tx24_added_vlan_packets: 4646014986
> tx24_nop: 68431153
> tx24_csum_none: 4207257674
> tx24_stopped: 9240
> tx24_dropped: 0
> tx24_xmit_more: 47699542
> tx24_recover: 0
> tx24_cqes: 4598317913
> tx24_wake: 9240
> tx24_cqe_err: 0
> tx25_packets: 5703883962
> tx25_bytes: 3291856915695
> tx25_tso_packets: 308900318
> tx25_tso_bytes: 1855516128386
> tx25_tso_inner_packets: 0
> tx25_tso_inner_bytes: 0
> tx25_csum_partial: 444753744
> tx25_csum_partial_inner: 0
> tx25_added_vlan_packets: 4676528924
> tx25_nop: 69230967
> tx25_csum_none: 4231775180
> tx25_stopped: 1140
> tx25_dropped: 0
> tx25_xmit_more: 40819195
> tx25_recover: 0
> tx25_cqes: 4635710966
> tx25_wake: 1140
> tx25_cqe_err: 0
> tx26_packets: 5803495984
> tx26_bytes: 3413564272139
> tx26_tso_packets: 319986230
> tx26_tso_bytes: 1929042839677
> tx26_tso_inner_packets: 0
> tx26_tso_inner_bytes: 0
> tx26_csum_partial: 464771163
> tx26_csum_partial_inner: 0
> tx26_added_vlan_packets: 4734767280
> tx26_nop: 71345080
> tx26_csum_none: 4269996117
> tx26_stopped: 10972
> tx26_dropped: 0
> tx26_xmit_more: 43793424
> tx26_recover: 0
> tx26_cqes: 4690976400
> tx26_wake: 10972
> tx26_cqe_err: 0
> tx27_packets: 5960955343
> tx27_bytes: 3444156164526
> tx27_tso_packets: 325099639
> tx27_tso_bytes: 1928378678784
> tx27_tso_inner_packets: 0
> tx27_tso_inner_bytes: 0
> tx27_csum_partial: 467310289
> tx27_csum_partial_inner: 0
> tx27_added_vlan_packets: 4888651368
> tx27_nop: 73201664
> tx27_csum_none: 4421341079
> tx27_stopped: 9465
> tx27_dropped: 0
> tx27_xmit_more: 53632121
> tx27_recover: 0
> tx27_cqes: 4835021398
> tx27_wake: 9465
> tx27_cqe_err: 0
> tx28_packets: 0
> tx28_bytes: 0
> tx28_tso_packets: 0
> tx28_tso_bytes: 0
> tx28_tso_inner_packets: 0
> tx28_tso_inner_bytes: 0
> tx28_csum_partial: 0
> tx28_csum_partial_inner: 0
> tx28_added_vlan_packets: 0
> tx28_nop: 0
> tx28_csum_none: 0
> tx28_stopped: 0
> tx28_dropped: 0
> tx28_xmit_more: 0
> tx28_recover: 0
> tx28_cqes: 0
> tx28_wake: 0
> tx28_cqe_err: 0
> tx29_packets: 3
> tx29_bytes: 266
> tx29_tso_packets: 0
> tx29_tso_bytes: 0
> tx29_tso_inner_packets: 0
> tx29_tso_inner_bytes: 0
> tx29_csum_partial: 0
> tx29_csum_partial_inner: 0
> tx29_added_vlan_packets: 0
> tx29_nop: 0
> tx29_csum_none: 3
> tx29_stopped: 0
> tx29_dropped: 0
> tx29_xmit_more: 1
> tx29_recover: 0
> tx29_cqes: 2
> tx29_wake: 0
> tx29_cqe_err: 0
> tx30_packets: 0
> tx30_bytes: 0
> tx30_tso_packets: 0
> tx30_tso_bytes: 0
> tx30_tso_inner_packets: 0
> tx30_tso_inner_bytes: 0
> tx30_csum_partial: 0
> tx30_csum_partial_inner: 0
> tx30_added_vlan_packets: 0
> tx30_nop: 0
> tx30_csum_none: 0
> tx30_stopped: 0
> tx30_dropped: 0
> tx30_xmit_more: 0
> tx30_recover: 0
> tx30_cqes: 0
> tx30_wake: 0
> tx30_cqe_err: 0
> tx31_packets: 0
> tx31_bytes: 0
> tx31_tso_packets: 0
> tx31_tso_bytes: 0
> tx31_tso_inner_packets: 0
> tx31_tso_inner_bytes: 0
> tx31_csum_partial: 0
> tx31_csum_partial_inner: 0
> tx31_added_vlan_packets: 0
> tx31_nop: 0
> tx31_csum_none: 0
> tx31_stopped: 0
> tx31_dropped: 0
> tx31_xmit_more: 0
> tx31_recover: 0
> tx31_cqes: 0
> tx31_wake: 0
> tx31_cqe_err: 0
> tx32_packets: 0
> tx32_bytes: 0
> tx32_tso_packets: 0
> tx32_tso_bytes: 0
> tx32_tso_inner_packets: 0
> tx32_tso_inner_bytes: 0
> tx32_csum_partial: 0
> tx32_csum_partial_inner: 0
> tx32_added_vlan_packets: 0
> tx32_nop: 0
> tx32_csum_none: 0
> tx32_stopped: 0
> tx32_dropped: 0
> tx32_xmit_more: 0
> tx32_recover: 0
> tx32_cqes: 0
> tx32_wake: 0
> tx32_cqe_err: 0
> tx33_packets: 0
> tx33_bytes: 0
> tx33_tso_packets: 0
> tx33_tso_bytes: 0
> tx33_tso_inner_packets: 0
> tx33_tso_inner_bytes: 0
> tx33_csum_partial: 0
> tx33_csum_partial_inner: 0
> tx33_added_vlan_packets: 0
> tx33_nop: 0
> tx33_csum_none: 0
> tx33_stopped: 0
> tx33_dropped: 0
> tx33_xmit_more: 0
> tx33_recover: 0
> tx33_cqes: 0
> tx33_wake: 0
> tx33_cqe_err: 0
> tx34_packets: 0
> tx34_bytes: 0
> tx34_tso_packets: 0
> tx34_tso_bytes: 0
> tx34_tso_inner_packets: 0
> tx34_tso_inner_bytes: 0
> tx34_csum_partial: 0
> tx34_csum_partial_inner: 0
> tx34_added_vlan_packets: 0
> tx34_nop: 0
> tx34_csum_none: 0
> tx34_stopped: 0
> tx34_dropped: 0
> tx34_xmit_more: 0
> tx34_recover: 0
> tx34_cqes: 0
> tx34_wake: 0
> tx34_cqe_err: 0
> tx35_packets: 0
> tx35_bytes: 0
> tx35_tso_packets: 0
> tx35_tso_bytes: 0
> tx35_tso_inner_packets: 0
> tx35_tso_inner_bytes: 0
> tx35_csum_partial: 0
> tx35_csum_partial_inner: 0
> tx35_added_vlan_packets: 0
> tx35_nop: 0
> tx35_csum_none: 0
> tx35_stopped: 0
> tx35_dropped: 0
> tx35_xmit_more: 0
> tx35_recover: 0
> tx35_cqes: 0
> tx35_wake: 0
> tx35_cqe_err: 0
> tx36_packets: 0
> tx36_bytes: 0
> tx36_tso_packets: 0
> tx36_tso_bytes: 0
> tx36_tso_inner_packets: 0
> tx36_tso_inner_bytes: 0
> tx36_csum_partial: 0
> tx36_csum_partial_inner: 0
> tx36_added_vlan_packets: 0
> tx36_nop: 0
> tx36_csum_none: 0
> tx36_stopped: 0
> tx36_dropped: 0
> tx36_xmit_more: 0
> tx36_recover: 0
> tx36_cqes: 0
> tx36_wake: 0
> tx36_cqe_err: 0
> tx37_packets: 0
> tx37_bytes: 0
> tx37_tso_packets: 0
> tx37_tso_bytes: 0
> tx37_tso_inner_packets: 0
> tx37_tso_inner_bytes: 0
> tx37_csum_partial: 0
> tx37_csum_partial_inner: 0
> tx37_added_vlan_packets: 0
> tx37_nop: 0
> tx37_csum_none: 0
> tx37_stopped: 0
> tx37_dropped: 0
> tx37_xmit_more: 0
> tx37_recover: 0
> tx37_cqes: 0
> tx37_wake: 0
> tx37_cqe_err: 0
> tx38_packets: 0
> tx38_bytes: 0
> tx38_tso_packets: 0
> tx38_tso_bytes: 0
> tx38_tso_inner_packets: 0
> tx38_tso_inner_bytes: 0
> tx38_csum_partial: 0
> tx38_csum_partial_inner: 0
> tx38_added_vlan_packets: 0
> tx38_nop: 0
> tx38_csum_none: 0
> tx38_stopped: 0
> tx38_dropped: 0
> tx38_xmit_more: 0
> tx38_recover: 0
> tx38_cqes: 0
> tx38_wake: 0
> tx38_cqe_err: 0
> tx39_packets: 0
> tx39_bytes: 0
> tx39_tso_packets: 0
> tx39_tso_bytes: 0
> tx39_tso_inner_packets: 0
> tx39_tso_inner_bytes: 0
> tx39_csum_partial: 0
> tx39_csum_partial_inner: 0
> tx39_added_vlan_packets: 0
> tx39_nop: 0
> tx39_csum_none: 0
> tx39_stopped: 0
> tx39_dropped: 0
> tx39_xmit_more: 0
> tx39_recover: 0
> tx39_cqes: 0
> tx39_wake: 0
> tx39_cqe_err: 0
> tx40_packets: 0
> tx40_bytes: 0
> tx40_tso_packets: 0
> tx40_tso_bytes: 0
> tx40_tso_inner_packets: 0
> tx40_tso_inner_bytes: 0
> tx40_csum_partial: 0
> tx40_csum_partial_inner: 0
> tx40_added_vlan_packets: 0
> tx40_nop: 0
> tx40_csum_none: 0
> tx40_stopped: 0
> tx40_dropped: 0
> tx40_xmit_more: 0
> tx40_recover: 0
> tx40_cqes: 0
> tx40_wake: 0
> tx40_cqe_err: 0
> tx41_packets: 0
> tx41_bytes: 0
> tx41_tso_packets: 0
> tx41_tso_bytes: 0
> tx41_tso_inner_packets: 0
> tx41_tso_inner_bytes: 0
> tx41_csum_partial: 0
> tx41_csum_partial_inner: 0
> tx41_added_vlan_packets: 0
> tx41_nop: 0
> tx41_csum_none: 0
> tx41_stopped: 0
> tx41_dropped: 0
> tx41_xmit_more: 0
> tx41_recover: 0
> tx41_cqes: 0
> tx41_wake: 0
> tx41_cqe_err: 0
> tx42_packets: 0
> tx42_bytes: 0
> tx42_tso_packets: 0
> tx42_tso_bytes: 0
> tx42_tso_inner_packets: 0
> tx42_tso_inner_bytes: 0
> tx42_csum_partial: 0
> tx42_csum_partial_inner: 0
> tx42_added_vlan_packets: 0
> tx42_nop: 0
> tx42_csum_none: 0
> tx42_stopped: 0
> tx42_dropped: 0
> tx42_xmit_more: 0
> tx42_recover: 0
> tx42_cqes: 0
> tx42_wake: 0
> tx42_cqe_err: 0
> tx43_packets: 0
> tx43_bytes: 0
> tx43_tso_packets: 0
> tx43_tso_bytes: 0
> tx43_tso_inner_packets: 0
> tx43_tso_inner_bytes: 0
> tx43_csum_partial: 0
> tx43_csum_partial_inner: 0
> tx43_added_vlan_packets: 0
> tx43_nop: 0
> tx43_csum_none: 0
> tx43_stopped: 0
> tx43_dropped: 0
> tx43_xmit_more: 0
> tx43_recover: 0
> tx43_cqes: 0
> tx43_wake: 0
> tx43_cqe_err: 0
> tx44_packets: 0
> tx44_bytes: 0
> tx44_tso_packets: 0
> tx44_tso_bytes: 0
> tx44_tso_inner_packets: 0
> tx44_tso_inner_bytes: 0
> tx44_csum_partial: 0
> tx44_csum_partial_inner: 0
> tx44_added_vlan_packets: 0
> tx44_nop: 0
> tx44_csum_none: 0
> tx44_stopped: 0
> tx44_dropped: 0
> tx44_xmit_more: 0
> tx44_recover: 0
> tx44_cqes: 0
> tx44_wake: 0
> tx44_cqe_err: 0
> tx45_packets: 0
> tx45_bytes: 0
> tx45_tso_packets: 0
> tx45_tso_bytes: 0
> tx45_tso_inner_packets: 0
> tx45_tso_inner_bytes: 0
> tx45_csum_partial: 0
> tx45_csum_partial_inner: 0
> tx45_added_vlan_packets: 0
> tx45_nop: 0
> tx45_csum_none: 0
> tx45_stopped: 0
> tx45_dropped: 0
> tx45_xmit_more: 0
> tx45_recover: 0
> tx45_cqes: 0
> tx45_wake: 0
> tx45_cqe_err: 0
> tx46_packets: 0
> tx46_bytes: 0
> tx46_tso_packets: 0
> tx46_tso_bytes: 0
> tx46_tso_inner_packets: 0
> tx46_tso_inner_bytes: 0
> tx46_csum_partial: 0
> tx46_csum_partial_inner: 0
> tx46_added_vlan_packets: 0
> tx46_nop: 0
> tx46_csum_none: 0
> tx46_stopped: 0
> tx46_dropped: 0
> tx46_xmit_more: 0
> tx46_recover: 0
> tx46_cqes: 0
> tx46_wake: 0
> tx46_cqe_err: 0
> tx47_packets: 0
> tx47_bytes: 0
> tx47_tso_packets: 0
> tx47_tso_bytes: 0
> tx47_tso_inner_packets: 0
> tx47_tso_inner_bytes: 0
> tx47_csum_partial: 0
> tx47_csum_partial_inner: 0
> tx47_added_vlan_packets: 0
> tx47_nop: 0
> tx47_csum_none: 0
> tx47_stopped: 0
> tx47_dropped: 0
> tx47_xmit_more: 0
> tx47_recover: 0
> tx47_cqes: 0
> tx47_wake: 0
> tx47_cqe_err: 0
> tx48_packets: 0
> tx48_bytes: 0
> tx48_tso_packets: 0
> tx48_tso_bytes: 0
> tx48_tso_inner_packets: 0
> tx48_tso_inner_bytes: 0
> tx48_csum_partial: 0
> tx48_csum_partial_inner: 0
> tx48_added_vlan_packets: 0
> tx48_nop: 0
> tx48_csum_none: 0
> tx48_stopped: 0
> tx48_dropped: 0
> tx48_xmit_more: 0
> tx48_recover: 0
> tx48_cqes: 0
> tx48_wake: 0
> tx48_cqe_err: 0
> tx49_packets: 0
> tx49_bytes: 0
> tx49_tso_packets: 0
> tx49_tso_bytes: 0
> tx49_tso_inner_packets: 0
> tx49_tso_inner_bytes: 0
> tx49_csum_partial: 0
> tx49_csum_partial_inner: 0
> tx49_added_vlan_packets: 0
> tx49_nop: 0
> tx49_csum_none: 0
> tx49_stopped: 0
> tx49_dropped: 0
> tx49_xmit_more: 0
> tx49_recover: 0
> tx49_cqes: 0
> tx49_wake: 0
> tx49_cqe_err: 0
> tx50_packets: 0
> tx50_bytes: 0
> tx50_tso_packets: 0
> tx50_tso_bytes: 0
> tx50_tso_inner_packets: 0
> tx50_tso_inner_bytes: 0
> tx50_csum_partial: 0
> tx50_csum_partial_inner: 0
> tx50_added_vlan_packets: 0
> tx50_nop: 0
> tx50_csum_none: 0
> tx50_stopped: 0
> tx50_dropped: 0
> tx50_xmit_more: 0
> tx50_recover: 0
> tx50_cqes: 0
> tx50_wake: 0
> tx50_cqe_err: 0
> tx51_packets: 0
> tx51_bytes: 0
> tx51_tso_packets: 0
> tx51_tso_bytes: 0
> tx51_tso_inner_packets: 0
> tx51_tso_inner_bytes: 0
> tx51_csum_partial: 0
> tx51_csum_partial_inner: 0
> tx51_added_vlan_packets: 0
> tx51_nop: 0
> tx51_csum_none: 0
> tx51_stopped: 0
> tx51_dropped: 0
> tx51_xmit_more: 0
> tx51_recover: 0
> tx51_cqes: 0
> tx51_wake: 0
> tx51_cqe_err: 0
> tx52_packets: 0
> tx52_bytes: 0
> tx52_tso_packets: 0
> tx52_tso_bytes: 0
> tx52_tso_inner_packets: 0
> tx52_tso_inner_bytes: 0
> tx52_csum_partial: 0
> tx52_csum_partial_inner: 0
> tx52_added_vlan_packets: 0
> tx52_nop: 0
> tx52_csum_none: 0
> tx52_stopped: 0
> tx52_dropped: 0
> tx52_xmit_more: 0
> tx52_recover: 0
> tx52_cqes: 0
> tx52_wake: 0
> tx52_cqe_err: 0
> tx53_packets: 0
> tx53_bytes: 0
> tx53_tso_packets: 0
> tx53_tso_bytes: 0
> tx53_tso_inner_packets: 0
> tx53_tso_inner_bytes: 0
> tx53_csum_partial: 0
> tx53_csum_partial_inner: 0
> tx53_added_vlan_packets: 0
> tx53_nop: 0
> tx53_csum_none: 0
> tx53_stopped: 0
> tx53_dropped: 0
> tx53_xmit_more: 0
> tx53_recover: 0
> tx53_cqes: 0
> tx53_wake: 0
> tx53_cqe_err: 0
> tx54_packets: 0
> tx54_bytes: 0
> tx54_tso_packets: 0
> tx54_tso_bytes: 0
> tx54_tso_inner_packets: 0
> tx54_tso_inner_bytes: 0
> tx54_csum_partial: 0
> tx54_csum_partial_inner: 0
> tx54_added_vlan_packets: 0
> tx54_nop: 0
> tx54_csum_none: 0
> tx54_stopped: 0
> tx54_dropped: 0
> tx54_xmit_more: 0
> tx54_recover: 0
> tx54_cqes: 0
> tx54_wake: 0
> tx54_cqe_err: 0
> tx55_packets: 0
> tx55_bytes: 0
> tx55_tso_packets: 0
> tx55_tso_bytes: 0
> tx55_tso_inner_packets: 0
> tx55_tso_inner_bytes: 0
> tx55_csum_partial: 0
> tx55_csum_partial_inner: 0
> tx55_added_vlan_packets: 0
> tx55_nop: 0
> tx55_csum_none: 0
> tx55_stopped: 0
> tx55_dropped: 0
> tx55_xmit_more: 0
> tx55_recover: 0
> tx55_cqes: 0
> tx55_wake: 0
> tx55_cqe_err: 0
> tx0_xdp_xmit: 0
> tx0_xdp_full: 0
> tx0_xdp_err: 0
> tx0_xdp_cqes: 0
> tx1_xdp_xmit: 0
> tx1_xdp_full: 0
> tx1_xdp_err: 0
> tx1_xdp_cqes: 0
> tx2_xdp_xmit: 0
> tx2_xdp_full: 0
> tx2_xdp_err: 0
> tx2_xdp_cqes: 0
> tx3_xdp_xmit: 0
> tx3_xdp_full: 0
> tx3_xdp_err: 0
> tx3_xdp_cqes: 0
> tx4_xdp_xmit: 0
> tx4_xdp_full: 0
> tx4_xdp_err: 0
> tx4_xdp_cqes: 0
> tx5_xdp_xmit: 0
> tx5_xdp_full: 0
> tx5_xdp_err: 0
> tx5_xdp_cqes: 0
> tx6_xdp_xmit: 0
> tx6_xdp_full: 0
> tx6_xdp_err: 0
> tx6_xdp_cqes: 0
> tx7_xdp_xmit: 0
> tx7_xdp_full: 0
> tx7_xdp_err: 0
> tx7_xdp_cqes: 0
> tx8_xdp_xmit: 0
> tx8_xdp_full: 0
> tx8_xdp_err: 0
> tx8_xdp_cqes: 0
> tx9_xdp_xmit: 0
> tx9_xdp_full: 0
> tx9_xdp_err: 0
> tx9_xdp_cqes: 0
> tx10_xdp_xmit: 0
> tx10_xdp_full: 0
> tx10_xdp_err: 0
> tx10_xdp_cqes: 0
> tx11_xdp_xmit: 0
> tx11_xdp_full: 0
> tx11_xdp_err: 0
> tx11_xdp_cqes: 0
> tx12_xdp_xmit: 0
> tx12_xdp_full: 0
> tx12_xdp_err: 0
> tx12_xdp_cqes: 0
> tx13_xdp_xmit: 0
> tx13_xdp_full: 0
> tx13_xdp_err: 0
> tx13_xdp_cqes: 0
> tx14_xdp_xmit: 0
> tx14_xdp_full: 0
> tx14_xdp_err: 0
> tx14_xdp_cqes: 0
> tx15_xdp_xmit: 0
> tx15_xdp_full: 0
> tx15_xdp_err: 0
> tx15_xdp_cqes: 0
> tx16_xdp_xmit: 0
> tx16_xdp_full: 0
> tx16_xdp_err: 0
> tx16_xdp_cqes: 0
> tx17_xdp_xmit: 0
> tx17_xdp_full: 0
> tx17_xdp_err: 0
> tx17_xdp_cqes: 0
> tx18_xdp_xmit: 0
> tx18_xdp_full: 0
> tx18_xdp_err: 0
> tx18_xdp_cqes: 0
> tx19_xdp_xmit: 0
> tx19_xdp_full: 0
> tx19_xdp_err: 0
> tx19_xdp_cqes: 0
> tx20_xdp_xmit: 0
> tx20_xdp_full: 0
> tx20_xdp_err: 0
> tx20_xdp_cqes: 0
> tx21_xdp_xmit: 0
> tx21_xdp_full: 0
> tx21_xdp_err: 0
> tx21_xdp_cqes: 0
> tx22_xdp_xmit: 0
> tx22_xdp_full: 0
> tx22_xdp_err: 0
> tx22_xdp_cqes: 0
> tx23_xdp_xmit: 0
> tx23_xdp_full: 0
> tx23_xdp_err: 0
> tx23_xdp_cqes: 0
> tx24_xdp_xmit: 0
> tx24_xdp_full: 0
> tx24_xdp_err: 0
> tx24_xdp_cqes: 0
> tx25_xdp_xmit: 0
> tx25_xdp_full: 0
> tx25_xdp_err: 0
> tx25_xdp_cqes: 0
> tx26_xdp_xmit: 0
> tx26_xdp_full: 0
> tx26_xdp_err: 0
> tx26_xdp_cqes: 0
> tx27_xdp_xmit: 0
> tx27_xdp_full: 0
> tx27_xdp_err: 0
> tx27_xdp_cqes: 0
> tx28_xdp_xmit: 0
> tx28_xdp_full: 0
> tx28_xdp_err: 0
> tx28_xdp_cqes: 0
> tx29_xdp_xmit: 0
> tx29_xdp_full: 0
> tx29_xdp_err: 0
> tx29_xdp_cqes: 0
> tx30_xdp_xmit: 0
> tx30_xdp_full: 0
> tx30_xdp_err: 0
> tx30_xdp_cqes: 0
> tx31_xdp_xmit: 0
> tx31_xdp_full: 0
> tx31_xdp_err: 0
> tx31_xdp_cqes: 0
> tx32_xdp_xmit: 0
> tx32_xdp_full: 0
> tx32_xdp_err: 0
> tx32_xdp_cqes: 0
> tx33_xdp_xmit: 0
> tx33_xdp_full: 0
> tx33_xdp_err: 0
> tx33_xdp_cqes: 0
> tx34_xdp_xmit: 0
> tx34_xdp_full: 0
> tx34_xdp_err: 0
> tx34_xdp_cqes: 0
> tx35_xdp_xmit: 0
> tx35_xdp_full: 0
> tx35_xdp_err: 0
> tx35_xdp_cqes: 0
> tx36_xdp_xmit: 0
> tx36_xdp_full: 0
> tx36_xdp_err: 0
> tx36_xdp_cqes: 0
> tx37_xdp_xmit: 0
> tx37_xdp_full: 0
> tx37_xdp_err: 0
> tx37_xdp_cqes: 0
> tx38_xdp_xmit: 0
> tx38_xdp_full: 0
> tx38_xdp_err: 0
> tx38_xdp_cqes: 0
> tx39_xdp_xmit: 0
> tx39_xdp_full: 0
> tx39_xdp_err: 0
> tx39_xdp_cqes: 0
> tx40_xdp_xmit: 0
> tx40_xdp_full: 0
> tx40_xdp_err: 0
> tx40_xdp_cqes: 0
> tx41_xdp_xmit: 0
> tx41_xdp_full: 0
> tx41_xdp_err: 0
> tx41_xdp_cqes: 0
> tx42_xdp_xmit: 0
> tx42_xdp_full: 0
> tx42_xdp_err: 0
> tx42_xdp_cqes: 0
> tx43_xdp_xmit: 0
> tx43_xdp_full: 0
> tx43_xdp_err: 0
> tx43_xdp_cqes: 0
> tx44_xdp_xmit: 0
> tx44_xdp_full: 0
> tx44_xdp_err: 0
> tx44_xdp_cqes: 0
> tx45_xdp_xmit: 0
> tx45_xdp_full: 0
> tx45_xdp_err: 0
> tx45_xdp_cqes: 0
> tx46_xdp_xmit: 0
> tx46_xdp_full: 0
> tx46_xdp_err: 0
> tx46_xdp_cqes: 0
> tx47_xdp_xmit: 0
> tx47_xdp_full: 0
> tx47_xdp_err: 0
> tx47_xdp_cqes: 0
> tx48_xdp_xmit: 0
> tx48_xdp_full: 0
> tx48_xdp_err: 0
> tx48_xdp_cqes: 0
> tx49_xdp_xmit: 0
> tx49_xdp_full: 0
> tx49_xdp_err: 0
> tx49_xdp_cqes: 0
> tx50_xdp_xmit: 0
> tx50_xdp_full: 0
> tx50_xdp_err: 0
> tx50_xdp_cqes: 0
> tx51_xdp_xmit: 0
> tx51_xdp_full: 0
> tx51_xdp_err: 0
> tx51_xdp_cqes: 0
> tx52_xdp_xmit: 0
> tx52_xdp_full: 0
> tx52_xdp_err: 0
> tx52_xdp_cqes: 0
> tx53_xdp_xmit: 0
> tx53_xdp_full: 0
> tx53_xdp_err: 0
> tx53_xdp_cqes: 0
> tx54_xdp_xmit: 0
> tx54_xdp_full: 0
> tx54_xdp_err: 0
> tx54_xdp_cqes: 0
> tx55_xdp_xmit: 0
> tx55_xdp_full: 0
> tx55_xdp_err: 0
> tx55_xdp_cqes: 0
>
>
> mpstat -P ALL 1 10
> Average: CPU %usr %nice %sys %iowait %irq %soft
> %steal
> %guest %gnice %idle
> Average: all 0.04 0.00 6.94 0.02 0.00 32.00
> 0.00 0.00 0.00 61.00
> Average: 0 0.00 0.00 1.20 0.00 0.00 0.00
> 0.00
> 0.00 0.00 98.80
> Average: 1 0.00 0.00 2.30 0.00 0.00 0.00
> 0.00
> 0.00 0.00 97.70
> Average: 2 0.10 0.00 0.00 0.00 0.00 0.00
> 0.00
> 0.00 0.00 99.90
> Average: 3 0.10 0.00 1.50 0.00 0.00 0.00
> 0.00
> 0.00 0.00 98.40
> Average: 4 0.50 0.00 2.50 0.00 0.00 0.00
> 0.00
> 0.00 0.00 97.00
> Average: 5 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00
> 0.00 0.00 100.00
> Average: 6 0.90 0.00 10.20 0.00 0.00 0.00
> 0.00
> 0.00 0.00 88.90
> Average: 7 0.00 0.00 0.00 1.40 0.00 0.00
> 0.00
> 0.00 0.00 98.60
> Average: 8 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00
> 0.00 0.00 100.00
> Average: 9 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00
> 0.00 0.00 100.00
> Average: 10 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00
> 0.00 0.00 100.00
> Average: 11 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00
> 0.00 0.00 100.00
> Average: 12 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00
> 0.00 0.00 100.00
> Average: 13 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00
> 0.00 0.00 100.00
> Average: 14 0.00 0.00 12.99 0.00 0.00 62.64
> 0.00 0.00 0.00 24.38
> Average: 15 0.00 0.00 12.70 0.00 0.00 63.40
> 0.00 0.00 0.00 23.90
> Average: 16 0.00 0.00 11.20 0.00 0.00 66.40
> 0.00 0.00 0.00 22.40
> Average: 17 0.00 0.00 16.60 0.00 0.00 52.10
> 0.00 0.00 0.00 31.30
> Average: 18 0.00 0.00 13.90 0.00 0.00 61.20
> 0.00 0.00 0.00 24.90
> Average: 19 0.00 0.00 9.99 0.00 0.00 70.33
> 0.00 0.00 0.00 19.68
> Average: 20 0.00 0.00 9.00 0.00 0.00 73.00
> 0.00 0.00 0.00 18.00
> Average: 21 0.00 0.00 8.70 0.00 0.00 73.90
> 0.00 0.00 0.00 17.40
> Average: 22 0.00 0.00 15.42 0.00 0.00 58.56
> 0.00 0.00 0.00 26.03
> Average: 23 0.00 0.00 10.81 0.00 0.00 71.67
> 0.00 0.00 0.00 17.52
> Average: 24 0.00 0.00 10.00 0.00 0.00 71.80
> 0.00 0.00 0.00 18.20
> Average: 25 0.00 0.00 11.19 0.00 0.00 71.13
> 0.00 0.00 0.00 17.68
> Average: 26 0.00 0.00 11.00 0.00 0.00 70.80
> 0.00 0.00 0.00 18.20
> Average: 27 0.00 0.00 10.01 0.00 0.00 69.57
> 0.00 0.00 0.00 20.42
The numa cores are not at 100% util, you have around 20% of idle on
each one.
> Average: 28 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00
> 0.00 0.00 100.00
> Average: 29 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00
> 0.00 0.00 100.00
> Average: 30 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00
> 0.00 0.00 100.00
> Average: 31 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00
> 0.00 0.00 100.00
> Average: 32 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00
> 0.00 0.00 100.00
> Average: 33 0.00 0.00 3.90 0.00 0.00 0.00
> 0.00
> 0.00 0.00 96.10
> Average: 34 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00
> 0.00 0.00 100.00
> Average: 35 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00
> 0.00 0.00 100.00
> Average: 36 0.10 0.00 0.20 0.00 0.00 0.00
> 0.00
> 0.00 0.00 99.70
> Average: 37 0.20 0.00 0.30 0.00 0.00 0.00
> 0.00
> 0.00 0.00 99.50
> Average: 38 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00
> 0.00 0.00 100.00
> Average: 39 0.00 0.00 2.60 0.00 0.00 0.00
> 0.00
> 0.00 0.00 97.40
> Average: 40 0.00 0.00 0.90 0.00 0.00 0.00
> 0.00
> 0.00 0.00 99.10
> Average: 41 0.10 0.00 0.50 0.00 0.00 0.00
> 0.00
> 0.00 0.00 99.40
> Average: 42 0.00 0.00 9.91 0.00 0.00 70.67
> 0.00 0.00 0.00 19.42
> Average: 43 0.00 0.00 15.90 0.00 0.00 57.50
> 0.00 0.00 0.00 26.60
> Average: 44 0.00 0.00 12.20 0.00 0.00 66.20
> 0.00 0.00 0.00 21.60
> Average: 45 0.00 0.00 12.00 0.00 0.00 67.50
> 0.00 0.00 0.00 20.50
> Average: 46 0.00 0.00 12.90 0.00 0.00 65.50
> 0.00 0.00 0.00 21.60
> Average: 47 0.00 0.00 14.59 0.00 0.00 60.84
> 0.00 0.00 0.00 24.58
> Average: 48 0.00 0.00 13.59 0.00 0.00 61.74
> 0.00 0.00 0.00 24.68
> Average: 49 0.00 0.00 18.36 0.00 0.00 53.29
> 0.00 0.00 0.00 28.34
> Average: 50 0.00 0.00 15.32 0.00 0.00 58.86
> 0.00 0.00 0.00 25.83
> Average: 51 0.00 0.00 17.60 0.00 0.00 55.20
> 0.00 0.00 0.00 27.20
> Average: 52 0.00 0.00 15.92 0.00 0.00 56.06
> 0.00 0.00 0.00 28.03
> Average: 53 0.00 0.00 13.00 0.00 0.00 62.30
> 0.00 0.00 0.00 24.70
> Average: 54 0.00 0.00 13.20 0.00 0.00 61.50
> 0.00 0.00 0.00 25.30
> Average: 55 0.00 0.00 14.59 0.00 0.00 58.64
> 0.00 0.00 0.00 26.77
>
>
> ethtool -k enp175s0f0
> Features for enp175s0f0:
> rx-checksumming: on
> tx-checksumming: on
> tx-checksum-ipv4: on
> tx-checksum-ip-generic: off [fixed]
> tx-checksum-ipv6: on
> tx-checksum-fcoe-crc: off [fixed]
> tx-checksum-sctp: off [fixed]
> scatter-gather: on
> tx-scatter-gather: on
> tx-scatter-gather-fraglist: off [fixed]
> tcp-segmentation-offload: on
> tx-tcp-segmentation: on
> tx-tcp-ecn-segmentation: off [fixed]
> tx-tcp-mangleid-segmentation: off
> tx-tcp6-segmentation: on
> udp-fragmentation-offload: off
> generic-segmentation-offload: on
> generic-receive-offload: on
> large-receive-offload: off [fixed]
> rx-vlan-offload: on
> tx-vlan-offload: on
> ntuple-filters: off
> receive-hashing: on
> highdma: on [fixed]
> rx-vlan-filter: on
> vlan-challenged: off [fixed]
> tx-lockless: off [fixed]
> netns-local: off [fixed]
> tx-gso-robust: off [fixed]
> tx-fcoe-segmentation: off [fixed]
> tx-gre-segmentation: on
> tx-gre-csum-segmentation: on
> tx-ipxip4-segmentation: off [fixed]
> tx-ipxip6-segmentation: off [fixed]
> tx-udp_tnl-segmentation: on
> tx-udp_tnl-csum-segmentation: on
> tx-gso-partial: on
> tx-sctp-segmentation: off [fixed]
> tx-esp-segmentation: off [fixed]
> tx-udp-segmentation: on
> fcoe-mtu: off [fixed]
> tx-nocache-copy: off
> loopback: off [fixed]
> rx-fcs: off
> rx-all: off
> tx-vlan-stag-hw-insert: on
> rx-vlan-stag-hw-parse: off [fixed]
> rx-vlan-stag-filter: on [fixed]
> l2-fwd-offload: off [fixed]
> hw-tc-offload: off
> esp-hw-offload: off [fixed]
> esp-tx-csum-hw-offload: off [fixed]
> rx-udp_tunnel-port-offload: on
> tls-hw-tx-offload: off [fixed]
> tls-hw-rx-offload: off [fixed]
> rx-gro-hw: off [fixed]
> tls-hw-record: off [fixed]
>
> ethtool -c enp175s0f0
> Coalesce parameters for enp175s0f0:
> Adaptive RX: off TX: on
> stats-block-usecs: 0
> sample-interval: 0
> pkt-rate-low: 0
> pkt-rate-high: 0
> dmac: 32703
>
> rx-usecs: 256
> rx-frames: 128
> rx-usecs-irq: 0
> rx-frames-irq: 0
>
> tx-usecs: 8
> tx-frames: 128
> tx-usecs-irq: 0
> tx-frames-irq: 0
>
> rx-usecs-low: 0
> rx-frame-low: 0
> tx-usecs-low: 0
> tx-frame-low: 0
>
> rx-usecs-high: 0
> rx-frame-high: 0
> tx-usecs-high: 0
> tx-frame-high: 0
>
> ethtool -g enp175s0f0
> Ring parameters for enp175s0f0:
> Pre-set maximums:
> RX: 8192
> RX Mini: 0
> RX Jumbo: 0
> TX: 8192
> Current hardware settings:
> RX: 4096
> RX Mini: 0
> RX Jumbo: 0
> TX: 4096
>
>
>
>
>
>
^ permalink raw reply
* Re: Kernel 4.19 network performance - forwarding/routing normal users traffic
From: Paweł Staszewski @ 2018-11-01 10:34 UTC (permalink / raw)
To: Jesper Dangaard Brouer
Cc: Eric Dumazet, netdev, Tariq Toukan, Ilias Apalodimas,
Yoel Caspersen, Mel Gorman, Aaron Lu
In-Reply-To: <20181101102213.2fa2643d@redhat.com>
W dniu 01.11.2018 o 10:22, Jesper Dangaard Brouer pisze:
> On Wed, 31 Oct 2018 23:20:01 +0100
> Paweł Staszewski <pstaszewski@itcare.pl> wrote:
>
>> W dniu 31.10.2018 o 23:09, Eric Dumazet pisze:
>>> On 10/31/2018 02:57 PM, Paweł Staszewski wrote:
>>>> Hi
>>>>
>>>> So maybee someone will be interested how linux kernel handles
>>>> normal traffic (not pktgen :) )
> Pawel is this live production traffic?
Yes moved server from testlab to production to check (risking a little -
but this is traffic switched to backup router : ) )
>
> I know Yoel (Cc) is very interested to know the real-life limitation of
> Linux as a router, especially with VLANs like you use.
So yes this is real-life traffic , real users - normal mixed internet
traffic forwarded (including ddos-es :) )
>
>
>>>> Server HW configuration:
>>>>
>>>> CPU : Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz
>>>>
>>>> NIC's: 2x 100G Mellanox ConnectX-4 (connected to x16 pcie 8GT)
>>>>
>>>>
>>>> Server software:
>>>>
>>>> FRR - as routing daemon
>>>>
>>>> enp175s0f0 (100G) - 16 vlans from upstreams (28 RSS binded to local numa node)
>>>>
>>>> enp175s0f1 (100G) - 343 vlans to clients (28 RSS binded to local numa node)
>>>>
>>>>
>>>> Maximum traffic that server can handle:
>>>>
>>>> Bandwidth
>>>>
>>>> bwm-ng v0.6.1 (probing every 1.000s), press 'h' for help
>>>> input: /proc/net/dev type: rate
>>>> \ iface Rx Tx Total
>>>> ==============================================================================
>>>> enp175s0f1: 28.51 Gb/s 37.24 Gb/s 65.74 Gb/s
>>>> enp175s0f0: 38.07 Gb/s 28.44 Gb/s 66.51 Gb/s
>>>> ------------------------------------------------------------------------------
>>>> total: 66.58 Gb/s 65.67 Gb/s 132.25 Gb/s
>>>>
> Actually rather impressive number for a Linux router.
>
>>>> Packets per second:
>>>>
>>>> bwm-ng v0.6.1 (probing every 1.000s), press 'h' for help
>>>> input: /proc/net/dev type: rate
>>>> - iface Rx Tx Total
>>>> ==============================================================================
>>>> enp175s0f1: 5248589.00 P/s 3486617.75 P/s 8735207.00 P/s
>>>> enp175s0f0: 3557944.25 P/s 5232516.00 P/s 8790460.00 P/s
>>>> ------------------------------------------------------------------------------
>>>> total: 8806533.00 P/s 8719134.00 P/s 17525668.00 P/s
>>>>
> Average packet size:
> (28.51*10^9/8)/5248589 = 678.99 bytes
> (38.07*10^9/8)/3557944 = 1337.49 bytes
>
>
>>>> After reaching that limits nics on the upstream side (more RX
>>>> traffic) start to drop packets
>>>>
>>>>
>>>> I just dont understand that server can't handle more bandwidth
>>>> (~40Gbit/s is limit where all cpu's are 100% util) - where pps on
>>>> RX side are increasing.
>>>>
>>>> Was thinking that maybee reached some pcie x16 limit - but x16 8GT
>>>> is 126Gbit - and also when testing with pktgen i can reach more bw
>>>> and pps (like 4x more comparing to normal internet traffic)
>>>>
>>>> And wondering if there is something that can be improved here.
>>>>
>>>>
>>>>
>>>> Some more informations / counters / stats and perf top below:
>>>>
>>>> Perf top flame graph:
>>>>
>>>> https://uploadfiles.io/7zo6u
> Thanks a lot for the flame graph!
>
>>>> System configuration(long):
>>>>
>>>>
>>>> cat /sys/devices/system/node/node1/cpulist
>>>> 14-27,42-55
>>>> cat /sys/class/net/enp175s0f0/device/numa_node
>>>> 1
>>>> cat /sys/class/net/enp175s0f1/device/numa_node
>>>> 1
>>>>
> Hint grep can give you nicer output that cat:
>
> $ grep -H . /sys/class/net/*/device/numa_node
Sure:
grep -H . /sys/class/net/*/device/numa_node
/sys/class/net/enp175s0f0/device/numa_node:1
/sys/class/net/enp175s0f1/device/numa_node:1
>
>>>>
>>>>
>>>>
>>>> ip -s -d link ls dev enp175s0f0
>>>> 6: enp175s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 8192
>>>> link/ether 0c:c4:7a:d8:5d:1c brd ff:ff:ff:ff:ff:ff promiscuity 0 addrgenmode eui64 numtxqueues 448 numrxqueues 56 gso_max_size 65536 gso_max_segs 65535
>>>> RX: bytes packets errors dropped overrun mcast
>>>> 184142375840858 141347715974 2 2806325 0 85050528
>>>> TX: bytes packets errors dropped carrier collsns
>>>> 99270697277430 172227994003 0 0 0 0
>>>>
>>>> ip -s -d link ls dev enp175s0f1
>>>> 7: enp175s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 8192
>>>> link/ether 0c:c4:7a:d8:5d:1d brd ff:ff:ff:ff:ff:ff promiscuity 0 addrgenmode eui64 numtxqueues 448 numrxqueues 56 gso_max_size 65536 gso_max_segs 65535
>>>> RX: bytes packets errors dropped overrun mcast
>>>> 99686284170801 173507590134 61 669685 0 100304421
>>>> TX: bytes packets errors dropped carrier collsns
>>>> 184435107970545 142383178304 0 0 0 0
>>>>
> You have increased the default (1000) qlen to 8192, why?
Was checking if higher txq will change anything
But no change for settings 1000,4096,8192
But yes i do not use there any traffic shaping like hfsc/hdb etc
- just default qdisc mq 0:
root pfifp_fast
tc qdisc show dev enp175s0f1
qdisc mq 0: root
qdisc pfifo_fast 0: parent :38 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1
1 1 1 1
qdisc pfifo_fast 0: parent :37 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1
1 1 1 1
qdisc pfifo_fast 0: parent :36 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1
1 1 1 1
...
...
And vlans are noqueue
tc -s -d qdisc show dev vlan1521
qdisc noqueue 0: root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
Weird is that no counters increasing but there is traffic in/out on that
vlans
ip -s -d link ls dev vlan1521
87: vlan1521@enp175s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 0c:c4:7a:d8:5d:1d brd ff:ff:ff:ff:ff:ff promiscuity 0
vlan protocol 802.1Q id 1521 <REORDER_HDR> addrgenmode eui64
numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
RX: bytes packets errors dropped overrun mcast
562964218394 1639370761 0 0 0 0
TX: bytes packets errors dropped carrier collsns
1417648713052 618271312 0 0 0 0
>
> What default qdisc do you run?... looking through your very detail main
> email report (I do love the details you give!). You run
> pfifo_fast_dequeue, thus this 8192 qlen is actually having effect.
>
> I would like to know if and how much qdisc_dequeue bulking is happening
> in this setup? Can you run:
>
> perf-stat-hist -m 8192 -P2 qdisc:qdisc_dequeue packets
>
> The perf-stat-hist is from Brendan Gregg's git-tree:
> https://github.com/brendangregg/perf-tools
> https://github.com/brendangregg/perf-tools/blob/master/misc/perf-stat-hist
>
./perf-stat-hist -m 8192 -P2 qdisc:qdisc_dequeue packets
Tracing qdisc:qdisc_dequeue, power-of-2, max 8192, until Ctrl-C...
^C
Range : Count Distribution
-> -1 : 0 | |
0 -> 0 : 43768349
|######################################|
1 -> 1 : 43895249
|######################################|
2 -> 3 : 352 |# |
4 -> 7 : 228 |# |
8 -> 15 : 135 |# |
16 -> 31 : 73 |# |
32 -> 63 : 7 |# |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 0 | |
512 -> 1023 : 0 | |
1024 -> 2047 : 0 | |
2048 -> 4095 : 0 | |
4096 -> 8191 : 0 | |
8192 -> : 0 | |
>>>> ./softnet.sh
>>>> cpu total dropped squeezed collision rps flow_limit
>>>>
>>>>
>>>>
>>>>
>>>> PerfTop: 108490 irqs/sec kernel:99.6% exact: 0.0% [4000Hz cycles], (all, 56 CPUs)
>>>> ------------------------------------------------------------------------------------------
>>>>
>>>> 26.78% [kernel] [k] queued_spin_lock_slowpath
>>> This is highly suspect.
>>>
> I agree! -- 26.78% spend in queued_spin_lock_slowpath. Hint if you see
> _raw_spin_lock then it is likely not a contended lock, but if you see
> queued_spin_lock_slowpath in a perf-report your workload is likely in
> trouble.
>
>
>>> A call graph (perf record -a -g sleep 1; perf report --stdio)
>>> would tell what is going on.
>> perf report:
>> https://ufile.io/rqp0h
>>
> Thanks for the output (my 30" screen is just large enough to see the
> full output). Together with the flame-graph, it is clear that this
> lock happens in the page allocator code.
>
> Section copied out:
>
> mlx5e_poll_tx_cq
> |
> --16.34%--napi_consume_skb
> |
> |--12.65%--__free_pages_ok
> | |
> | --11.86%--free_one_page
> | |
> | |--10.10%--queued_spin_lock_slowpath
> | |
> | --0.65%--_raw_spin_lock
> |
> |--1.55%--page_frag_free
> |
> --1.44%--skb_release_data
>
>
> Let me explain what (I think) happens. The mlx5 driver RX-page recycle
> mechanism is not effective in this workload, and pages have to go
> through the page allocator. The lock contention happens during mlx5
> DMA TX completion cycle. And the page allocator cannot keep up at
> these speeds.
>
> One solution is extend page allocator with a bulk free API. (This have
> been on my TODO list for a long time, but I don't have a
> micro-benchmark that trick the driver page-recycle to fail). It should
> fit nicely, as I can see that kmem_cache_free_bulk() does get
> activated (bulk freeing SKBs), which means that DMA TX completion do
> have a bulk of packets.
>
> We can (and should) also improve the page recycle scheme in the driver.
> After LPC, I have a project with Tariq and Ilias (Cc'ed) to improve the
> page_pool, and we will (attempt) to generalize this, for both high-end
> mlx5 and more low-end ARM64-boards (macchiatobin and espressobin).
>
> The MM-people is in parallel working to improve the performance of
> order-0 page returns. Thus, the explicit page bulk free API might
> actually become less important. I actually think (Cc.) Aaron have a
> patchset he would like you to test, which removes the (zone->)lock
> you hit in free_one_page().
>
Ok - Thank You Jesper
^ permalink raw reply
* Re: Kernel 4.19 network performance - forwarding/routing normal users traffic
From: Jesper Dangaard Brouer @ 2018-11-01 10:55 UTC (permalink / raw)
To: David Ahern; +Cc: brouer, Paweł Staszewski, netdev, Yoel Caspersen
In-Reply-To: <3a88bb53-9d17-3e85-638e-a605f5bfe0fb@gmail.com>
On Wed, 31 Oct 2018 21:37:16 -0600 David Ahern <dsahern@gmail.com> wrote:
> This is mainly a forwarding use case? Seems so based on the perf report.
> I suspect forwarding with XDP would show pretty good improvement.
Yes, significant performance improvements.
Notice Davids talk: "Leveraging Kernel Tables with XDP"
http://vger.kernel.org/lpc-networking2018.html#session-1
It looks like that you are doing "pure" IP-routing, without any
iptables conntrack stuff (from your perf report data). That will
actually be a really good use-case for accelerating this with XDP.
I want you to understand the philosophy behind how David and I want
people to leverage XDP. Think of XDP as a software offload layer for
the kernel network stack. Setup and use Linux kernel network stack, but
accelerate parts of it with XDP, e.g. the route FIB lookup.
Sample code avail here:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/samples/bpf/xdp_fwd_kern.c
(I do warn, what we just found a bug/crash in setup+tairdown for the
mlx5 driver you are using, that we/mlx _will_ fix soon)
> You need the vlan changes I have queued up though.
I know Yoel will be very interested in those changes too! I've
convinced Yoel to write an XDP program for his Border Network Gateway
(BNG) production system[1], and his is a heavy VLAN user. And the plan
is to Open Source this when he have-something-working.
[1] https://www.version2.dk/blog/software-router-del-5-linux-bng-1086060
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
LinkedIn: http://www.linkedin.com/in/brouer
^ permalink raw reply
* Re: Kernel 4.19 network performance - forwarding/routing normal users traffic
From: Paweł Staszewski @ 2018-11-01 11:09 UTC (permalink / raw)
To: Saeed Mahameed, netdev@vger.kernel.org
In-Reply-To: <659fbf4b481c815f45a58b2351481cc9f761445b.camel@mellanox.com>
W dniu 01.11.2018 o 10:50, Saeed Mahameed pisze:
> On Wed, 2018-10-31 at 22:57 +0100, Paweł Staszewski wrote:
>> Hi
>>
>> So maybee someone will be interested how linux kernel handles normal
>> traffic (not pktgen :) )
>>
>>
>> Server HW configuration:
>>
>> CPU : Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz
>>
>> NIC's: 2x 100G Mellanox ConnectX-4 (connected to x16 pcie 8GT)
>>
>>
>> Server software:
>>
>> FRR - as routing daemon
>>
>> enp175s0f0 (100G) - 16 vlans from upstreams (28 RSS binded to local
>> numa
>> node)
>>
>> enp175s0f1 (100G) - 343 vlans to clients (28 RSS binded to local numa
>> node)
>>
>>
>> Maximum traffic that server can handle:
>>
>> Bandwidth
>>
>> bwm-ng v0.6.1 (probing every 1.000s), press 'h' for help
>> input: /proc/net/dev type: rate
>> \ iface Rx Tx Total
>> =====================================================================
>> =========
>> enp175s0f1: 28.51 Gb/s 37.24
>> Gb/s
>> 65.74 Gb/s
>> enp175s0f0: 38.07 Gb/s 28.44
>> Gb/s
>> 66.51 Gb/s
>> -------------------------------------------------------------------
>> -----------
>> total: 66.58 Gb/s 65.67
>> Gb/s
>> 132.25 Gb/s
>>
>>
>> Packets per second:
>>
>> bwm-ng v0.6.1 (probing every 1.000s), press 'h' for help
>> input: /proc/net/dev type: rate
>> - iface Rx Tx Total
>> =====================================================================
>> =========
>> enp175s0f1: 5248589.00 P/s 3486617.75 P/s
>> 8735207.00 P/s
>> enp175s0f0: 3557944.25 P/s 5232516.00 P/s
>> 8790460.00 P/s
>> -------------------------------------------------------------------
>> -----------
>> total: 8806533.00 P/s 8719134.00 P/s
>> 17525668.00 P/s
>>
>>
>> After reaching that limits nics on the upstream side (more RX
>> traffic)
>> start to drop packets
>>
>>
>> I just dont understand that server can't handle more bandwidth
>> (~40Gbit/s is limit where all cpu's are 100% util) - where pps on RX
>> side are increasing.
>>
> Where do you see 40 Gb/s ? you showed that both ports on the same NIC (
> same pcie link) are doing 66.58 Gb/s (RX) + 65.67 Gb/s (TX) = 132.25
> Gb/s which aligns with your pcie link limit, what am i missing ?
hmm yes that was my concern also - cause cant find anywhere informations
about that bandwidth is uni or bidirectional - so if 126Gbit for x16 8GT
is unidir - then bidir will be 126/2 ~68Gbit - which will fit total bw
on both ports
This can explain maybee also why cpuload is rising rapidly from
120Gbit/s in total to 132Gbit (counters of bwmng are from /proc/net - so
there can be some error in reading them when offloading (gro/gso/tso) on
nic's is enabled that is why
>
>> Was thinking that maybee reached some pcie x16 limit - but x16 8GT
>> is
>> 126Gbit - and also when testing with pktgen i can reach more bw and
>> pps
>> (like 4x more comparing to normal internet traffic)
>>
> Are you forwarding when using pktgen as well or you just testing the RX
> side pps ?
Yes pktgen was tested on single port RX
Can check also forwarding to eliminate pciex limits
>
>> And wondering if there is something that can be improved here.
>>
>>
>>
>> Some more informations / counters / stats and perf top below:
>>
>> Perf top flame graph:
>>
>> https://uploadfiles.io/7zo6u
>>
>>
>>
>> System configuration(long):
>>
>>
>> cat /sys/devices/system/node/node1/cpulist
>> 14-27,42-55
>> cat /sys/class/net/enp175s0f0/device/numa_node
>> 1
>> cat /sys/class/net/enp175s0f1/device/numa_node
>> 1
>>
>>
>>
>>
>>
>> ip -s -d link ls dev enp175s0f0
>> 6: enp175s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq
>> state
>> UP mode DEFAULT group default qlen 8192
>> link/ether 0c:c4:7a:d8:5d:1c brd ff:ff:ff:ff:ff:ff promiscuity
>> 0
>> addrgenmode eui64 numtxqueues 448 numrxqueues 56 gso_max_size 65536
>> gso_max_segs 65535
>> RX: bytes packets errors dropped overrun mcast
>> 184142375840858 141347715974 2 2806325 0 85050528
>> TX: bytes packets errors dropped carrier collsns
>> 99270697277430 172227994003 0 0 0 0
>>
>> ip -s -d link ls dev enp175s0f1
>> 7: enp175s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq
>> state
>> UP mode DEFAULT group default qlen 8192
>> link/ether 0c:c4:7a:d8:5d:1d brd ff:ff:ff:ff:ff:ff promiscuity
>> 0
>> addrgenmode eui64 numtxqueues 448 numrxqueues 56 gso_max_size 65536
>> gso_max_segs 65535
>> RX: bytes packets errors dropped overrun mcast
>> 99686284170801 173507590134 61 669685 0 100304421
>> TX: bytes packets errors dropped carrier collsns
>> 184435107970545 142383178304 0 0 0 0
>>
>>
>> ./softnet.sh
>> cpu total dropped squeezed collision rps flow_limit
>> 0 3961392822 0 1221478 0 0 0
>> 1 3701952251 0 1258234 0 0 0
>> 2 3879522030 0 1584282 0 0 0
>> 3 3731349789 0 1529029 0 0 0
>> 4 1323956701 0 2176371 0 0 0
>> 5 420528963 0 1880146 0 0 0
>> 6 348720322 0 1830142 0 0 0
>> 7 372736328 0 1820891 0 0 0
>> 8 567888751 0 1414763 0 0 0
>> 9 476075775 0 1868150 0 0 0
>> 10 468946725 0 1841428 0 0 0
>> 11 676591958 0 1900160 0 0 0
>> 12 346803472 0 1834600 0 0 0
>> 13 457960872 0 1874529 0 0 0
>> 14 1990279665 0 4699000 0 0 0
>> 15 1211873601 0 4541281 0 0 0
>> 16 1123871928 0 4544712 0 0 0
>> 17 1014957263 0 4152355 0 0 0
>> 18 2603779724 0 4593869 0 0 0
>> 19 2181924054 0 4930618 0 0 0
>> 20 2273502182 0 4894627 0 0 0
>> 21 2232030947 0 4860048 0 0 0
>> 22 2203555394 0 4603830 0 0 0
>> 23 2194756800 0 4921294 0 0 0
>> 24 2347158294 0 4818354 0 0 0
>> 25 2291097883 0 4744469 0 0 0
>> 26 2206945011 0 4836483 0 0 0
>> 27 2318530217 0 4917617 0 0 0
>> 28 512797543 0 1895200 0 0 0
>> 29 597279474 0 1532134 0 0 0
>> 30 475317503 0 1451523 0 0 0
>> 31 499172796 0 1901207 0 0 0
>> 32 493874745 0 1915382 0 0 0
>> 33 296056288 0 1865535 0 0 0
>> 34 3905097041 0 1580822 0 0 0
>> 35 3905112345 0 1536105 0 0 0
>> 36 3900358950 0 1166319 0 0 0
>> 37 3940978093 0 1600219 0 0 0
>> 38 3878632215 0 1180389 0 0 0
>> 39 3814804736 0 1584925 0 0 0
>> 40 4152934337 0 1663660 0 0 0
>> 41 3855273904 0 1552219 0 0 0
>> 42 2319538182 0 4884480 0 0 0
>> 43 2448606991 0 4387456 0 0 0
>> 44 1436136753 0 4485073 0 0 0
>> 45 1200500141 0 4537284 0 0 0
>> 46 1307799923 0 4534156 0 0 0
>> 47 1586575293 0 4272997 0 0 0
>> 48 3852574 0 4162653 0 0 0
>> 49 391449390 0 3935202 0 0 0
>> 50 791388200 0 4290738 0 0 0
>> 51 127107573 0 3907750 0 0 0
>> 52 115622148 0 4012843 0 0 0
>> 53 71098871 0 4200625 0 0 0
>> 54 305121466 0 4365614 0 0 0
>> 55 10914257 0 4369426 0 0 0
>>
>>
>>
>>
>> PerfTop: 108490 irqs/sec kernel:99.6% exact: 0.0% [4000Hz
>> cycles], (all, 56 CPUs)
>> -------------------------------------------------------------------
>> -------------------------------------------------------------------
>> -------------------------------------------------------------------
>> ------
>>
>> 26.78% [kernel] [k] queued_spin_lock_slowpath
>> 9.09% [kernel] [k] mlx5e_skb_from_cqe_linear
>> 4.94% [kernel] [k] mlx5e_sq_xmit
>> 3.63% [kernel] [k] memcpy_erms
>> 3.30% [kernel] [k] fib_table_lookup
>> 3.26% [kernel] [k] build_skb
>> 2.41% [kernel] [k] mlx5e_poll_tx_cq
>> 2.11% [kernel] [k] get_page_from_freelist
>> 1.51% [kernel] [k] vlan_do_receive
>> 1.51% [kernel] [k] _raw_spin_lock
>> 1.43% [kernel] [k] __dev_queue_xmit
>> 1.41% [kernel] [k] dev_gro_receive
>> 1.34% [kernel] [k] mlx5e_poll_rx_cq
>> 1.26% [kernel] [k] tcp_gro_receive
>> 1.21% [kernel] [k] free_one_page
>> 1.13% [kernel] [k] swiotlb_map_page
>> 1.13% [kernel] [k] mlx5e_post_rx_wqes
>> 1.05% [kernel] [k] pfifo_fast_dequeue
>> 1.05% [kernel] [k] mlx5e_handle_rx_cqe
>> 1.03% [kernel] [k] ip_finish_output2
>> 1.02% [kernel] [k] ipt_do_table
>> 0.96% [kernel] [k] inet_gro_receive
>> 0.91% [kernel] [k] mlx5_eq_int
>> 0.88% [kernel] [k] __slab_free.isra.79
>> 0.86% [kernel] [k] __build_skb
>> 0.84% [kernel] [k] page_frag_free
>> 0.76% [kernel] [k] skb_release_data
>> 0.75% [kernel] [k] __netif_receive_skb_core
>> 0.75% [kernel] [k] irq_entries_start
>> 0.71% [kernel] [k] ip_route_input_rcu
>> 0.65% [kernel] [k] vlan_dev_hard_start_xmit
>> 0.56% [kernel] [k] ip_forward
>> 0.56% [kernel] [k] __memcpy
>> 0.52% [kernel] [k] kmem_cache_alloc
>> 0.52% [kernel] [k] kmem_cache_free_bulk
>> 0.49% [kernel] [k] mlx5e_page_release
>> 0.47% [kernel] [k] netif_skb_features
>> 0.47% [kernel] [k] mlx5e_build_rx_skb
>> 0.47% [kernel] [k] dev_hard_start_xmit
>> 0.43% [kernel] [k] __page_pool_put_page
>> 0.43% [kernel] [k] __netif_schedule
>> 0.43% [kernel] [k] mlx5e_xmit
>> 0.41% [kernel] [k] __qdisc_run
>> 0.41% [kernel] [k] validate_xmit_skb.isra.142
>> 0.41% [kernel] [k] swiotlb_unmap_page
>> 0.40% [kernel] [k] inet_lookup_ifaddr_rcu
>> 0.34% [kernel] [k] ip_rcv_core.isra.20.constprop.25
>> 0.34% [kernel] [k] tcp4_gro_receive
>> 0.29% [kernel] [k] _raw_spin_lock_irqsave
>> 0.29% [kernel] [k] napi_consume_skb
>> 0.29% [kernel] [k] skb_gro_receive
>> 0.29% [kernel] [k] ___slab_alloc.isra.80
>> 0.27% [kernel] [k] eth_type_trans
>> 0.26% [kernel] [k] __free_pages_ok
>> 0.26% [kernel] [k] __get_xps_queue_idx
>> 0.24% [kernel] [k] _raw_spin_trylock
>> 0.23% [kernel] [k] __local_bh_enable_ip
>> 0.22% [kernel] [k] pfifo_fast_enqueue
>> 0.21% [kernel] [k] tasklet_action_common.isra.21
>> 0.21% [kernel] [k] sch_direct_xmit
>> 0.21% [kernel] [k] skb_network_protocol
>> 0.21% [kernel] [k] kmem_cache_free
>> 0.20% [kernel] [k] netdev_pick_tx
>> 0.18% [kernel] [k] napi_gro_complete
>> 0.18% [kernel] [k] __sched_text_start
>> 0.18% [kernel] [k] mlx5e_xdp_handle
>> 0.17% [kernel] [k] ip_finish_output
>> 0.16% [kernel] [k] napi_gro_flush
>> 0.16% [kernel] [k] vlan_passthru_hard_header
>> 0.16% [kernel] [k] skb_segment
>> 0.15% [kernel] [k] __alloc_pages_nodemask
>> 0.15% [kernel] [k] mlx5e_features_check
>> 0.15% [kernel] [k] mlx5e_napi_poll
>> 0.15% [kernel] [k] napi_gro_receive
>> 0.14% [kernel] [k] fib_validate_source
>> 0.14% [kernel] [k] _raw_spin_lock_irq
>> 0.14% [kernel] [k] inet_gro_complete
>> 0.14% [kernel] [k] get_partial_node.isra.78
>> 0.13% [kernel] [k] napi_complete_done
>> 0.13% [kernel] [k] ip_rcv_finish_core.isra.17
>> 0.13% [kernel] [k] cmd_exec
>>
>>
>>
>> ethtool -S enp175s0f1
>> NIC statistics:
>> rx_packets: 173730800927
>> rx_bytes: 99827422751332
>> tx_packets: 142532009512
>> tx_bytes: 184633045911222
>> tx_tso_packets: 25989113891
>> tx_tso_bytes: 132933363384458
>> tx_tso_inner_packets: 0
>> tx_tso_inner_bytes: 0
>> tx_added_vlan_packets: 74630239613
>> tx_nop: 2029817748
>> rx_lro_packets: 0
>> rx_lro_bytes: 0
>> rx_ecn_mark: 0
>> rx_removed_vlan_packets: 173730800927
>> rx_csum_unnecessary: 0
>> rx_csum_none: 434357
>> rx_csum_complete: 173730366570
>> rx_csum_unnecessary_inner: 0
>> rx_xdp_drop: 0
>> rx_xdp_redirect: 0
>> rx_xdp_tx_xmit: 0
>> rx_xdp_tx_full: 0
>> rx_xdp_tx_err: 0
>> rx_xdp_tx_cqe: 0
>> tx_csum_none: 38260960853
>> tx_csum_partial: 36369278774
>> tx_csum_partial_inner: 0
>> tx_queue_stopped: 1
>> tx_queue_dropped: 0
>> tx_xmit_more: 748638099
>> tx_recover: 0
>> tx_cqes: 73881645031
>> tx_queue_wake: 1
>> tx_udp_seg_rem: 0
>> tx_cqe_err: 0
>> tx_xdp_xmit: 0
>> tx_xdp_full: 0
>> tx_xdp_err: 0
>> tx_xdp_cqes: 0
>> rx_wqe_err: 0
>> rx_mpwqe_filler_cqes: 0
>> rx_mpwqe_filler_strides: 0
>> rx_buff_alloc_err: 0
>> rx_cqe_compress_blks: 0
>> rx_cqe_compress_pkts: 0
> If this is a pcie bottleneck it might be useful to enable CQE
> compression (to reduce PCIe completion descriptors transactions)
> you should see the above rx_cqe_compress_pkts increasing when enabled.
>
> $ ethtool --set-priv-flags enp175s0f1 rx_cqe_compress on
> $ ethtool --show-priv-flags enp175s0f1
> Private flags for p6p1:
> rx_cqe_moder : on
> cqe_moder : off
> rx_cqe_compress : on
> ...
>
> try this on both interfaces.
Done
ethtool --show-priv-flags enp175s0f1
Private flags for enp175s0f1:
rx_cqe_moder : on
tx_cqe_moder : off
rx_cqe_compress : on
rx_striding_rq : off
rx_no_csum_complete: off
ethtool --show-priv-flags enp175s0f0
Private flags for enp175s0f0:
rx_cqe_moder : on
tx_cqe_moder : off
rx_cqe_compress : on
rx_striding_rq : off
rx_no_csum_complete: off
>
>> rx_page_reuse: 0
>> rx_cache_reuse: 14441066823
>> rx_cache_full: 51126004413
>> rx_cache_empty: 21297344082
>> rx_cache_busy: 51127247487
>> rx_cache_waive: 21298322293
>> rx_congst_umr: 0
>> rx_arfs_err: 0
>> ch_events: 24603119858
>> ch_poll: 25180949074
>> ch_arm: 24480437587
>> ch_aff_change: 75
>> ch_eq_rearm: 0
>> rx_out_of_buffer: 669685
> comparing this to rx_vport_unicast_packets, it is a very small
> percentage of dropped packets due to stalled rx cpu, so rx cpu is not a
> bottleneck, at least for the driver rx rings.
>
>> rx_if_down_packets: 61
>> rx_vport_unicast_packets: 173731641945
>> rx_vport_unicast_bytes: 100522745036693
>> tx_vport_unicast_packets: 142531901313
>> tx_vport_unicast_bytes: 185189071776429
>> rx_vport_multicast_packets: 100360886
>> rx_vport_multicast_bytes: 6639236688
>> tx_vport_multicast_packets: 32837
>> tx_vport_multicast_bytes: 2978810
>> rx_vport_broadcast_packets: 44854
>> rx_vport_broadcast_bytes: 6313510
>> tx_vport_broadcast_packets: 72258
>> tx_vport_broadcast_bytes: 4335480
>> rx_vport_rdma_unicast_packets: 0
>> rx_vport_rdma_unicast_bytes: 0
>> tx_vport_rdma_unicast_packets: 0
>> tx_vport_rdma_unicast_bytes: 0
>> rx_vport_rdma_multicast_packets: 0
>> rx_vport_rdma_multicast_bytes: 0
>> tx_vport_rdma_multicast_packets: 0
>> tx_vport_rdma_multicast_bytes: 0
>> tx_packets_phy: 142532004669
>> rx_packets_phy: 173980375752
>> rx_crc_errors_phy: 0
>> tx_bytes_phy: 185759204762903
>> rx_bytes_phy: 101326109361379
>> tx_multicast_phy: 32837
>> tx_broadcast_phy: 72258
>> rx_multicast_phy: 100360885
>> rx_broadcast_phy: 44854
>> rx_in_range_len_errors_phy: 2
>> rx_out_of_range_len_phy: 0
>> rx_oversize_pkts_phy: 59
>> rx_symbol_err_phy: 0
>> tx_mac_control_phy: 0
>> rx_mac_control_phy: 0
>> rx_unsupported_op_phy: 0
>> rx_pause_ctrl_phy: 0
>> tx_pause_ctrl_phy: 0
>> rx_discards_phy: 148328738
>> tx_discards_phy: 0
>> tx_errors_phy: 0
>> rx_undersize_pkts_phy: 0
>> rx_fragments_phy: 0
>> rx_jabbers_phy: 0
>> rx_64_bytes_phy: 36551843112
>> rx_65_to_127_bytes_phy: 65102131735
>> rx_128_to_255_bytes_phy: 5755731137
>> rx_256_to_511_bytes_phy: 2475619839
>> rx_512_to_1023_bytes_phy: 2826971156
>> rx_1024_to_1518_bytes_phy: 42474023107
>> rx_1519_to_2047_bytes_phy: 18794051270
>> rx_2048_to_4095_bytes_phy: 0
>> rx_4096_to_8191_bytes_phy: 0
>> rx_8192_to_10239_bytes_phy: 0
>> link_down_events_phy: 0
>> rx_pcs_symbol_err_phy: 0
>> rx_corrected_bits_phy: 0
>> rx_pci_signal_integrity: 0
>> tx_pci_signal_integrity: 48
>> rx_prio0_bytes: 101316322498995
>> rx_prio0_packets: 173711151686
>> tx_prio0_bytes: 185759176566814
>> tx_prio0_packets: 142531983704
>> rx_prio1_bytes: 47062768
>> rx_prio1_packets: 228932
>> tx_prio1_bytes: 0
>> tx_prio1_packets: 0
>> rx_prio2_bytes: 12434759
>> rx_prio2_packets: 83773
>> tx_prio2_bytes: 0
>> tx_prio2_packets: 0
>> rx_prio3_bytes: 288843134
>> rx_prio3_packets: 982102
>> tx_prio3_bytes: 0
>> tx_prio3_packets: 0
>> rx_prio4_bytes: 699797236
>> rx_prio4_packets: 8109231
>> tx_prio4_bytes: 0
>> tx_prio4_packets: 0
>> rx_prio5_bytes: 1385386738
>> rx_prio5_packets: 9661187
>> tx_prio5_bytes: 0
>> tx_prio5_packets: 0
>> rx_prio6_bytes: 317092102
>> rx_prio6_packets: 1951538
>> tx_prio6_bytes: 0
>> tx_prio6_packets: 0
>> rx_prio7_bytes: 7015734695
>> rx_prio7_packets: 99847456
>> tx_prio7_bytes: 0
>> tx_prio7_packets: 0
>> module_unplug: 0
>> module_bus_stuck: 0
>> module_high_temp: 0
>> module_bad_shorted: 0
>> ch0_events: 936264703
>> ch0_poll: 963766474
>> ch0_arm: 930246079
>> ch0_aff_change: 0
>> ch0_eq_rearm: 0
>> ch1_events: 869408429
>> ch1_poll: 896099392
>> ch1_arm: 864336861
>> ch1_aff_change: 0
>> ch1_eq_rearm: 0
>> ch2_events: 843345698
>> ch2_poll: 869749522
>> ch2_arm: 838186113
>> ch2_aff_change: 2
>> ch2_eq_rearm: 0
>> ch3_events: 850261340
>> ch3_poll: 876721111
>> ch3_arm: 845295235
>> ch3_aff_change: 3
>> ch3_eq_rearm: 0
>> ch4_events: 974985780
>> ch4_poll: 997781915
>> ch4_arm: 969618250
>> ch4_aff_change: 3
>> ch4_eq_rearm: 0
>> ch5_events: 888559089
>> ch5_poll: 912783615
>> ch5_arm: 883826078
>> ch5_aff_change: 2
>> ch5_eq_rearm: 0
>> ch6_events: 873730730
>> ch6_poll: 899635752
>> ch6_arm: 868677574
>> ch6_aff_change: 4
>> ch6_eq_rearm: 0
>> ch7_events: 873478411
>> ch7_poll: 899216716
>> ch7_arm: 868693645
>> ch7_aff_change: 3
>> ch7_eq_rearm: 0
>> ch8_events: 871900967
>> ch8_poll: 898575518
>> ch8_arm: 866763693
>> ch8_aff_change: 3
>> ch8_eq_rearm: 0
>> ch9_events: 880325565
>> ch9_poll: 904983269
>> ch9_arm: 875643922
>> ch9_aff_change: 2
>> ch9_eq_rearm: 0
>> ch10_events: 889919775
>> ch10_poll: 915335809
>> ch10_arm: 885110225
>> ch10_aff_change: 4
>> ch10_eq_rearm: 0
>> ch11_events: 962709175
>> ch11_poll: 983963451
>> ch11_arm: 958117526
>> ch11_aff_change: 2
>> ch11_eq_rearm: 0
>> ch12_events: 941333837
>> ch12_poll: 964625523
>> ch12_arm: 936409706
>> ch12_aff_change: 2
>> ch12_eq_rearm: 0
>> ch13_events: 914996974
>> ch13_poll: 937441049
>> ch13_arm: 910478393
>> ch13_aff_change: 4
>> ch13_eq_rearm: 0
>> ch14_events: 888050001
>> ch14_poll: 911818008
>> ch14_arm: 883465035
>> ch14_aff_change: 4
>> ch14_eq_rearm: 0
>> ch15_events: 947547704
>> ch15_poll: 969073194
>> ch15_arm: 942686515
>> ch15_aff_change: 4
>> ch15_eq_rearm: 0
>> ch16_events: 825804904
>> ch16_poll: 840630747
>> ch16_arm: 822227488
>> ch16_aff_change: 2
>> ch16_eq_rearm: 0
>> ch17_events: 861673823
>> ch17_poll: 874754041
>> ch17_arm: 858520448
>> ch17_aff_change: 2
>> ch17_eq_rearm: 0
>> ch18_events: 879413440
>> ch18_poll: 893962529
>> ch18_arm: 875983204
>> ch18_aff_change: 4
>> ch18_eq_rearm: 0
>> ch19_events: 896073709
>> ch19_poll: 909216857
>> ch19_arm: 893022121
>> ch19_aff_change: 4
>> ch19_eq_rearm: 0
>> ch20_events: 865188535
>> ch20_poll: 880692345
>> ch20_arm: 861440265
>> ch20_aff_change: 3
>> ch20_eq_rearm: 0
>> ch21_events: 862709303
>> ch21_poll: 878104242
>> ch21_arm: 859041767
>> ch21_aff_change: 2
>> ch21_eq_rearm: 0
>> ch22_events: 887720551
>> ch22_poll: 904122074
>> ch22_arm: 883983794
>> ch22_aff_change: 2
>> ch22_eq_rearm: 0
>> ch23_events: 813355027
>> ch23_poll: 828074467
>> ch23_arm: 809912398
>> ch23_aff_change: 4
>> ch23_eq_rearm: 0
>> ch24_events: 822366675
>> ch24_poll: 839917937
>> ch24_arm: 818422754
>> ch24_aff_change: 2
>> ch24_eq_rearm: 0
>> ch25_events: 826642292
>> ch25_poll: 842630121
>> ch25_arm: 822642618
>> ch25_aff_change: 2
>> ch25_eq_rearm: 0
>> ch26_events: 826392584
>> ch26_poll: 843406973
>> ch26_arm: 822455000
>> ch26_aff_change: 3
>> ch26_eq_rearm: 0
>> ch27_events: 828960899
>> ch27_poll: 843866518
>> ch27_arm: 825230937
>> ch27_aff_change: 3
>> ch27_eq_rearm: 0
>> ch28_events: 7
>> ch28_poll: 7
>> ch28_arm: 7
>> ch28_aff_change: 0
>> ch28_eq_rearm: 0
>> ch29_events: 4
>> ch29_poll: 4
>> ch29_arm: 4
>> ch29_aff_change: 0
>> ch29_eq_rearm: 0
>> ch30_events: 4
>> ch30_poll: 4
>> ch30_arm: 4
>> ch30_aff_change: 0
>> ch30_eq_rearm: 0
>> ch31_events: 4
>> ch31_poll: 4
>> ch31_arm: 4
>> ch31_aff_change: 0
>> ch31_eq_rearm: 0
>> ch32_events: 4
>> ch32_poll: 4
>> ch32_arm: 4
>> ch32_aff_change: 0
>> ch32_eq_rearm: 0
>> ch33_events: 4
>> ch33_poll: 4
>> ch33_arm: 4
>> ch33_aff_change: 0
>> ch33_eq_rearm: 0
>> ch34_events: 4
>> ch34_poll: 4
>> ch34_arm: 4
>> ch34_aff_change: 0
>> ch34_eq_rearm: 0
>> ch35_events: 4
>> ch35_poll: 4
>> ch35_arm: 4
>> ch35_aff_change: 0
>> ch35_eq_rearm: 0
>> ch36_events: 4
>> ch36_poll: 4
>> ch36_arm: 4
>> ch36_aff_change: 0
>> ch36_eq_rearm: 0
>> ch37_events: 4
>> ch37_poll: 4
>> ch37_arm: 4
>> ch37_aff_change: 0
>> ch37_eq_rearm: 0
>> ch38_events: 4
>> ch38_poll: 4
>> ch38_arm: 4
>> ch38_aff_change: 0
>> ch38_eq_rearm: 0
>> ch39_events: 4
>> ch39_poll: 4
>> ch39_arm: 4
>> ch39_aff_change: 0
>> ch39_eq_rearm: 0
>> ch40_events: 4
>> ch40_poll: 4
>> ch40_arm: 4
>> ch40_aff_change: 0
>> ch40_eq_rearm: 0
>> ch41_events: 4
>> ch41_poll: 4
>> ch41_arm: 4
>> ch41_aff_change: 0
>> ch41_eq_rearm: 0
>> ch42_events: 4
>> ch42_poll: 4
>> ch42_arm: 4
>> ch42_aff_change: 0
>> ch42_eq_rearm: 0
>> ch43_events: 4
>> ch43_poll: 4
>> ch43_arm: 4
>> ch43_aff_change: 0
>> ch43_eq_rearm: 0
>> ch44_events: 4
>> ch44_poll: 4
>> ch44_arm: 4
>> ch44_aff_change: 0
>> ch44_eq_rearm: 0
>> ch45_events: 4
>> ch45_poll: 4
>> ch45_arm: 4
>> ch45_aff_change: 0
>> ch45_eq_rearm: 0
>> ch46_events: 4
>> ch46_poll: 4
>> ch46_arm: 4
>> ch46_aff_change: 0
>> ch46_eq_rearm: 0
>> ch47_events: 4
>> ch47_poll: 4
>> ch47_arm: 4
>> ch47_aff_change: 0
>> ch47_eq_rearm: 0
>> ch48_events: 4
>> ch48_poll: 4
>> ch48_arm: 4
>> ch48_aff_change: 0
>> ch48_eq_rearm: 0
>> ch49_events: 4
>> ch49_poll: 4
>> ch49_arm: 4
>> ch49_aff_change: 0
>> ch49_eq_rearm: 0
>> ch50_events: 4
>> ch50_poll: 4
>> ch50_arm: 4
>> ch50_aff_change: 0
>> ch50_eq_rearm: 0
>> ch51_events: 4
>> ch51_poll: 4
>> ch51_arm: 4
>> ch51_aff_change: 0
>> ch51_eq_rearm: 0
>> ch52_events: 4
>> ch52_poll: 4
>> ch52_arm: 4
>> ch52_aff_change: 0
>> ch52_eq_rearm: 0
>> ch53_events: 4
>> ch53_poll: 4
>> ch53_arm: 4
>> ch53_aff_change: 0
>> ch53_eq_rearm: 0
>> ch54_events: 4
>> ch54_poll: 4
>> ch54_arm: 4
>> ch54_aff_change: 0
>> ch54_eq_rearm: 0
>> ch55_events: 4
>> ch55_poll: 4
>> ch55_arm: 4
>> ch55_aff_change: 0
>> ch55_eq_rearm: 0
>> rx0_packets: 7284057433
>> rx0_bytes: 4330611281319
>> rx0_csum_complete: 7283623076
>> rx0_csum_unnecessary: 0
>> rx0_csum_unnecessary_inner: 0
>> rx0_csum_none: 434357
>> rx0_xdp_drop: 0
>> rx0_xdp_redirect: 0
>> rx0_lro_packets: 0
>> rx0_lro_bytes: 0
>> rx0_ecn_mark: 0
>> rx0_removed_vlan_packets: 7284057433
>> rx0_wqe_err: 0
>> rx0_mpwqe_filler_cqes: 0
>> rx0_mpwqe_filler_strides: 0
>> rx0_buff_alloc_err: 0
>> rx0_cqe_compress_blks: 0
>> rx0_cqe_compress_pkts: 0
>> rx0_page_reuse: 0
>> rx0_cache_reuse: 1989731589
>> rx0_cache_full: 28213297
>> rx0_cache_empty: 1624089822
>> rx0_cache_busy: 28213961
>> rx0_cache_waive: 1624083610
>> rx0_congst_umr: 0
>> rx0_arfs_err: 0
>> rx0_xdp_tx_xmit: 0
>> rx0_xdp_tx_full: 0
>> rx0_xdp_tx_err: 0
>> rx0_xdp_tx_cqes: 0
>> rx1_packets: 6691319211
>> rx1_bytes: 3799580210608
>> rx1_csum_complete: 6691319211
>> rx1_csum_unnecessary: 0
>> rx1_csum_unnecessary_inner: 0
>> rx1_csum_none: 0
>> rx1_xdp_drop: 0
>> rx1_xdp_redirect: 0
>> rx1_lro_packets: 0
>> rx1_lro_bytes: 0
>> rx1_ecn_mark: 0
>> rx1_removed_vlan_packets: 6691319211
>> rx1_wqe_err: 0
>> rx1_mpwqe_filler_cqes: 0
>> rx1_mpwqe_filler_strides: 0
>> rx1_buff_alloc_err: 0
>> rx1_cqe_compress_blks: 0
>> rx1_cqe_compress_pkts: 0
>> rx1_page_reuse: 0
>> rx1_cache_reuse: 2270019
>> rx1_cache_full: 3343389331
>> rx1_cache_empty: 6656
>> rx1_cache_busy: 3343389585
>> rx1_cache_waive: 0
>> rx1_congst_umr: 0
>> rx1_arfs_err: 0
>> rx1_xdp_tx_xmit: 0
>> rx1_xdp_tx_full: 0
>> rx1_xdp_tx_err: 0
>> rx1_xdp_tx_cqes: 0
>> rx2_packets: 6618370416
>> rx2_bytes: 3762508364015
>> rx2_csum_complete: 6618370416
>> rx2_csum_unnecessary: 0
>> rx2_csum_unnecessary_inner: 0
>> rx2_csum_none: 0
>> rx2_xdp_drop: 0
>> rx2_xdp_redirect: 0
>> rx2_lro_packets: 0
>> rx2_lro_bytes: 0
>> rx2_ecn_mark: 0
>> rx2_removed_vlan_packets: 6618370416
>> rx2_wqe_err: 0
>> rx2_mpwqe_filler_cqes: 0
>> rx2_mpwqe_filler_strides: 0
>> rx2_buff_alloc_err: 0
>> rx2_cqe_compress_blks: 0
>> rx2_cqe_compress_pkts: 0
>> rx2_page_reuse: 0
>> rx2_cache_reuse: 111419328
>> rx2_cache_full: 1807563903
>> rx2_cache_empty: 1390208158
>> rx2_cache_busy: 1807564378
>> rx2_cache_waive: 1390201722
>> rx2_congst_umr: 0
>> rx2_arfs_err: 0
>> rx2_xdp_tx_xmit: 0
>> rx2_xdp_tx_full: 0
>> rx2_xdp_tx_err: 0
>> rx2_xdp_tx_cqes: 0
>> rx3_packets: 6665308976
>> rx3_bytes: 3828546206006
>> rx3_csum_complete: 6665308976
>> rx3_csum_unnecessary: 0
>> rx3_csum_unnecessary_inner: 0
>> rx3_csum_none: 0
>> rx3_xdp_drop: 0
>> rx3_xdp_redirect: 0
>> rx3_lro_packets: 0
>> rx3_lro_bytes: 0
>> rx3_ecn_mark: 0
>> rx3_removed_vlan_packets: 6665308976
>> rx3_wqe_err: 0
>> rx3_mpwqe_filler_cqes: 0
>> rx3_mpwqe_filler_strides: 0
>> rx3_buff_alloc_err: 0
>> rx3_cqe_compress_blks: 0
>> rx3_cqe_compress_pkts: 0
>> rx3_page_reuse: 0
>> rx3_cache_reuse: 215779091
>> rx3_cache_full: 1720040649
>> rx3_cache_empty: 1396840926
>> rx3_cache_busy: 1720041127
>> rx3_cache_waive: 1396834493
>> rx3_congst_umr: 0
>> rx3_arfs_err: 0
>> rx3_xdp_tx_xmit: 0
>> rx3_xdp_tx_full: 0
>> rx3_xdp_tx_err: 0
>> rx3_xdp_tx_cqes: 0
>> rx4_packets: 6764448165
>> rx4_bytes: 3883101339142
>> rx4_csum_complete: 6764448165
>> rx4_csum_unnecessary: 0
>> rx4_csum_unnecessary_inner: 0
>> rx4_csum_none: 0
>> rx4_xdp_drop: 0
>> rx4_xdp_redirect: 0
>> rx4_lro_packets: 0
>> rx4_lro_bytes: 0
>> rx4_ecn_mark: 0
>> rx4_removed_vlan_packets: 6764448165
>> rx4_wqe_err: 0
>> rx4_mpwqe_filler_cqes: 0
>> rx4_mpwqe_filler_strides: 0
>> rx4_buff_alloc_err: 0
>> rx4_cqe_compress_blks: 0
>> rx4_cqe_compress_pkts: 0
>> rx4_page_reuse: 0
>> rx4_cache_reuse: 1930710653
>> rx4_cache_full: 6490815
>> rx4_cache_empty: 1445028605
>> rx4_cache_busy: 6491478
>> rx4_cache_waive: 1445022392
>> rx4_congst_umr: 0
>> rx4_arfs_err: 0
>> rx4_xdp_tx_xmit: 0
>> rx4_xdp_tx_full: 0
>> rx4_xdp_tx_err: 0
>> rx4_xdp_tx_cqes: 0
>> rx5_packets: 6736853264
>> rx5_bytes: 3925186068552
>> rx5_csum_complete: 6736853264
>> rx5_csum_unnecessary: 0
>> rx5_csum_unnecessary_inner: 0
>> rx5_csum_none: 0
>> rx5_xdp_drop: 0
>> rx5_xdp_redirect: 0
>> rx5_lro_packets: 0
>> rx5_lro_bytes: 0
>> rx5_ecn_mark: 0
>> rx5_removed_vlan_packets: 6736853264
>> rx5_wqe_err: 0
>> rx5_mpwqe_filler_cqes: 0
>> rx5_mpwqe_filler_strides: 0
>> rx5_buff_alloc_err: 0
>> rx5_cqe_compress_blks: 0
>> rx5_cqe_compress_pkts: 0
>> rx5_page_reuse: 0
>> rx5_cache_reuse: 7283914
>> rx5_cache_full: 3361142463
>> rx5_cache_empty: 6656
>> rx5_cache_busy: 3361142718
>> rx5_cache_waive: 0
>> rx5_congst_umr: 0
>> rx5_arfs_err: 0
>> rx5_xdp_tx_xmit: 0
>> rx5_xdp_tx_full: 0
>> rx5_xdp_tx_err: 0
>> rx5_xdp_tx_cqes: 0
>> rx6_packets: 6751588828
>> rx6_bytes: 3860537598885
>> rx6_csum_complete: 6751588828
>> rx6_csum_unnecessary: 0
>> rx6_csum_unnecessary_inner: 0
>> rx6_csum_none: 0
>> rx6_xdp_drop: 0
>> rx6_xdp_redirect: 0
>> rx6_lro_packets: 0
>> rx6_lro_bytes: 0
>> rx6_ecn_mark: 0
>> rx6_removed_vlan_packets: 6751588828
>> rx6_wqe_err: 0
>> rx6_mpwqe_filler_cqes: 0
>> rx6_mpwqe_filler_strides: 0
>> rx6_buff_alloc_err: 0
>> rx6_cqe_compress_blks: 0
>> rx6_cqe_compress_pkts: 0
>> rx6_page_reuse: 0
>> rx6_cache_reuse: 96032126
>> rx6_cache_full: 1857890923
>> rx6_cache_empty: 1421877543
>> rx6_cache_busy: 1857891399
>> rx6_cache_waive: 1421871110
>> rx6_congst_umr: 0
>> rx6_arfs_err: 0
>> rx6_xdp_tx_xmit: 0
>> rx6_xdp_tx_full: 0
>> rx6_xdp_tx_err: 0
>> rx6_xdp_tx_cqes: 0
>> rx7_packets: 6935300074
>> rx7_bytes: 4004713524388
>> rx7_csum_complete: 6935300074
>> rx7_csum_unnecessary: 0
>> rx7_csum_unnecessary_inner: 0
>> rx7_csum_none: 0
>> rx7_xdp_drop: 0
>> rx7_xdp_redirect: 0
>> rx7_lro_packets: 0
>> rx7_lro_bytes: 0
>> rx7_ecn_mark: 0
>> rx7_removed_vlan_packets: 6935300074
>> rx7_wqe_err: 0
>> rx7_mpwqe_filler_cqes: 0
>> rx7_mpwqe_filler_strides: 0
>> rx7_buff_alloc_err: 0
>> rx7_cqe_compress_blks: 0
>> rx7_cqe_compress_pkts: 0
>> rx7_page_reuse: 0
>> rx7_cache_reuse: 17555187
>> rx7_cache_full: 3450094595
>> rx7_cache_empty: 6656
>> rx7_cache_busy: 3450094849
>> rx7_cache_waive: 0
>> rx7_congst_umr: 0
>> rx7_arfs_err: 0
>> rx7_xdp_tx_xmit: 0
>> rx7_xdp_tx_full: 0
>> rx7_xdp_tx_err: 0
>> rx7_xdp_tx_cqes: 0
>> rx8_packets: 6678640094
>> rx8_bytes: 3783722686028
>> rx8_csum_complete: 6678640094
>> rx8_csum_unnecessary: 0
>> rx8_csum_unnecessary_inner: 0
>> rx8_csum_none: 0
>> rx8_xdp_drop: 0
>> rx8_xdp_redirect: 0
>> rx8_lro_packets: 0
>> rx8_lro_bytes: 0
>> rx8_ecn_mark: 0
>> rx8_removed_vlan_packets: 6678640094
>> rx8_wqe_err: 0
>> rx8_mpwqe_filler_cqes: 0
>> rx8_mpwqe_filler_strides: 0
>> rx8_buff_alloc_err: 0
>> rx8_cqe_compress_blks: 0
>> rx8_cqe_compress_pkts: 0
>> rx8_page_reuse: 0
>> rx8_cache_reuse: 71006578
>> rx8_cache_full: 1879380649
>> rx8_cache_empty: 1388938999
>> rx8_cache_busy: 1879381123
>> rx8_cache_waive: 1388932565
>> rx8_congst_umr: 0
>> rx8_arfs_err: 0
>> rx8_xdp_tx_xmit: 0
>> rx8_xdp_tx_full: 0
>> rx8_xdp_tx_err: 0
>> rx8_xdp_tx_cqes: 0
>> rx9_packets: 6709855557
>> rx9_bytes: 3849522227880
>> rx9_csum_complete: 6709855557
>> rx9_csum_unnecessary: 0
>> rx9_csum_unnecessary_inner: 0
>> rx9_csum_none: 0
>> rx9_xdp_drop: 0
>> rx9_xdp_redirect: 0
>> rx9_lro_packets: 0
>> rx9_lro_bytes: 0
>> rx9_ecn_mark: 0
>> rx9_removed_vlan_packets: 6709855557
>> rx9_wqe_err: 0
>> rx9_mpwqe_filler_cqes: 0
>> rx9_mpwqe_filler_strides: 0
>> rx9_buff_alloc_err: 0
>> rx9_cqe_compress_blks: 0
>> rx9_cqe_compress_pkts: 0
>> rx9_page_reuse: 0
>> rx9_cache_reuse: 108980215
>> rx9_cache_full: 1822730121
>> rx9_cache_empty: 1423223623
>> rx9_cache_busy: 1822730594
>> rx9_cache_waive: 1423217187
>> rx9_congst_umr: 0
>> rx9_arfs_err: 0
>> rx9_xdp_tx_xmit: 0
>> rx9_xdp_tx_full: 0
>> rx9_xdp_tx_err: 0
>> rx9_xdp_tx_cqes: 0
>> rx10_packets: 6761861066
>> rx10_bytes: 3816266733385
>> rx10_csum_complete: 6761861066
>> rx10_csum_unnecessary: 0
>> rx10_csum_unnecessary_inner: 0
>> rx10_csum_none: 0
>> rx10_xdp_drop: 0
>> rx10_xdp_redirect: 0
>> rx10_lro_packets: 0
>> rx10_lro_bytes: 0
>> rx10_ecn_mark: 0
>> rx10_removed_vlan_packets: 6761861066
>> rx10_wqe_err: 0
>> rx10_mpwqe_filler_cqes: 0
>> rx10_mpwqe_filler_strides: 0
>> rx10_buff_alloc_err: 0
>> rx10_cqe_compress_blks: 0
>> rx10_cqe_compress_pkts: 0
>> rx10_page_reuse: 0
>> rx10_cache_reuse: 3489300
>> rx10_cache_full: 3377440977
>> rx10_cache_empty: 6656
>> rx10_cache_busy: 3377441216
>> rx10_cache_waive: 0
>> rx10_congst_umr: 0
>> rx10_arfs_err: 0
>> rx10_xdp_tx_xmit: 0
>> rx10_xdp_tx_full: 0
>> rx10_xdp_tx_err: 0
>> rx10_xdp_tx_cqes: 0
>> rx11_packets: 6868113938
>> rx11_bytes: 4048196300710
>> rx11_csum_complete: 6868113938
>> rx11_csum_unnecessary: 0
>> rx11_csum_unnecessary_inner: 0
>> rx11_csum_none: 0
>> rx11_xdp_drop: 0
>> rx11_xdp_redirect: 0
>> rx11_lro_packets: 0
>> rx11_lro_bytes: 0
>> rx11_ecn_mark: 0
>> rx11_removed_vlan_packets: 6868113938
>> rx11_wqe_err: 0
>> rx11_mpwqe_filler_cqes: 0
>> rx11_mpwqe_filler_strides: 0
>> rx11_buff_alloc_err: 0
>> rx11_cqe_compress_blks: 0
>> rx11_cqe_compress_pkts: 0
>> rx11_page_reuse: 0
>> rx11_cache_reuse: 1948516819
>> rx11_cache_full: 17132157
>> rx11_cache_empty: 1468413985
>> rx11_cache_busy: 17132820
>> rx11_cache_waive: 1468407772
>> rx11_congst_umr: 0
>> rx11_arfs_err: 0
>> rx11_xdp_tx_xmit: 0
>> rx11_xdp_tx_full: 0
>> rx11_xdp_tx_err: 0
>> rx11_xdp_tx_cqes: 0
>> rx12_packets: 6742955386
>> rx12_bytes: 3865747629271
>> rx12_csum_complete: 6742955386
>> rx12_csum_unnecessary: 0
>> rx12_csum_unnecessary_inner: 0
>> rx12_csum_none: 0
>> rx12_xdp_drop: 0
>> rx12_xdp_redirect: 0
>> rx12_lro_packets: 0
>> rx12_lro_bytes: 0
>> rx12_ecn_mark: 0
>> rx12_removed_vlan_packets: 6742955386
>> rx12_wqe_err: 0
>> rx12_mpwqe_filler_cqes: 0
>> rx12_mpwqe_filler_strides: 0
>> rx12_buff_alloc_err: 0
>> rx12_cqe_compress_blks: 0
>> rx12_cqe_compress_pkts: 0
>> rx12_page_reuse: 0
>> rx12_cache_reuse: 30809331
>> rx12_cache_full: 3340668106
>> rx12_cache_empty: 6656
>> rx12_cache_busy: 3340668333
>> rx12_cache_waive: 0
>> rx12_congst_umr: 0
>> rx12_arfs_err: 0
>> rx12_xdp_tx_xmit: 0
>> rx12_xdp_tx_full: 0
>> rx12_xdp_tx_err: 0
>> rx12_xdp_tx_cqes: 0
>> rx13_packets: 6707028036
>> rx13_bytes: 3813462190623
>> rx13_csum_complete: 6707028036
>> rx13_csum_unnecessary: 0
>> rx13_csum_unnecessary_inner: 0
>> rx13_csum_none: 0
>> rx13_xdp_drop: 0
>> rx13_xdp_redirect: 0
>> rx13_lro_packets: 0
>> rx13_lro_bytes: 0
>> rx13_ecn_mark: 0
>> rx13_removed_vlan_packets: 6707028036
>> rx13_wqe_err: 0
>> rx13_mpwqe_filler_cqes: 0
>> rx13_mpwqe_filler_strides: 0
>> rx13_buff_alloc_err: 0
>> rx13_cqe_compress_blks: 0
>> rx13_cqe_compress_pkts: 0
>> rx13_page_reuse: 0
>> rx13_cache_reuse: 14951053
>> rx13_cache_full: 3338562710
>> rx13_cache_empty: 6656
>> rx13_cache_busy: 3338562963
>> rx13_cache_waive: 0
>> rx13_congst_umr: 0
>> rx13_arfs_err: 0
>> rx13_xdp_tx_xmit: 0
>> rx13_xdp_tx_full: 0
>> rx13_xdp_tx_err: 0
>> rx13_xdp_tx_cqes: 0
>> rx14_packets: 6737074410
>> rx14_bytes: 3868905276119
>> rx14_csum_complete: 6737074410
>> rx14_csum_unnecessary: 0
>> rx14_csum_unnecessary_inner: 0
>> rx14_csum_none: 0
>> rx14_xdp_drop: 0
>> rx14_xdp_redirect: 0
>> rx14_lro_packets: 0
>> rx14_lro_bytes: 0
>> rx14_ecn_mark: 0
>> rx14_removed_vlan_packets: 6737074410
>> rx14_wqe_err: 0
>> rx14_mpwqe_filler_cqes: 0
>> rx14_mpwqe_filler_strides: 0
>> rx14_buff_alloc_err: 0
>> rx14_cqe_compress_blks: 0
>> rx14_cqe_compress_pkts: 0
>> rx14_page_reuse: 0
>> rx14_cache_reuse: 967799432
>> rx14_cache_full: 982704312
>> rx14_cache_empty: 1418039639
>> rx14_cache_busy: 982704789
>> rx14_cache_waive: 1418033206
>> rx14_congst_umr: 0
>> rx14_arfs_err: 0
>> rx14_xdp_tx_xmit: 0
>> rx14_xdp_tx_full: 0
>> rx14_xdp_tx_err: 0
>> rx14_xdp_tx_cqes: 0
>> rx15_packets: 6641887441
>> rx15_bytes: 3742874400402
>> rx15_csum_complete: 6641887441
>> rx15_csum_unnecessary: 0
>> rx15_csum_unnecessary_inner: 0
>> rx15_csum_none: 0
>> rx15_xdp_drop: 0
>> rx15_xdp_redirect: 0
>> rx15_lro_packets: 0
>> rx15_lro_bytes: 0
>> rx15_ecn_mark: 0
>> rx15_removed_vlan_packets: 6641887441
>> rx15_wqe_err: 0
>> rx15_mpwqe_filler_cqes: 0
>> rx15_mpwqe_filler_strides: 0
>> rx15_buff_alloc_err: 0
>> rx15_cqe_compress_blks: 0
>> rx15_cqe_compress_pkts: 0
>> rx15_page_reuse: 0
>> rx15_cache_reuse: 1920227538
>> rx15_cache_full: 19386129
>> rx15_cache_empty: 1381335137
>> rx15_cache_busy: 19387693
>> rx15_cache_waive: 1381329825
>> rx15_congst_umr: 0
>> rx15_arfs_err: 0
>> rx15_xdp_tx_xmit: 0
>> rx15_xdp_tx_full: 0
>> rx15_xdp_tx_err: 0
>> rx15_xdp_tx_cqes: 0
>> rx16_packets: 5420472874
>> rx16_bytes: 3079293332581
>> rx16_csum_complete: 5420472874
>> rx16_csum_unnecessary: 0
>> rx16_csum_unnecessary_inner: 0
>> rx16_csum_none: 0
>> rx16_xdp_drop: 0
>> rx16_xdp_redirect: 0
>> rx16_lro_packets: 0
>> rx16_lro_bytes: 0
>> rx16_ecn_mark: 0
>> rx16_removed_vlan_packets: 5420472874
>> rx16_wqe_err: 0
>> rx16_mpwqe_filler_cqes: 0
>> rx16_mpwqe_filler_strides: 0
>> rx16_buff_alloc_err: 0
>> rx16_cqe_compress_blks: 0
>> rx16_cqe_compress_pkts: 0
>> rx16_page_reuse: 0
>> rx16_cache_reuse: 2361079
>> rx16_cache_full: 2707875103
>> rx16_cache_empty: 6656
>> rx16_cache_busy: 2707875349
>> rx16_cache_waive: 0
>> rx16_congst_umr: 0
>> rx16_arfs_err: 0
>> rx16_xdp_tx_xmit: 0
>> rx16_xdp_tx_full: 0
>> rx16_xdp_tx_err: 0
>> rx16_xdp_tx_cqes: 0
>> rx17_packets: 5428380986
>> rx17_bytes: 3080981893118
>> rx17_csum_complete: 5428380986
>> rx17_csum_unnecessary: 0
>> rx17_csum_unnecessary_inner: 0
>> rx17_csum_none: 0
>> rx17_xdp_drop: 0
>> rx17_xdp_redirect: 0
>> rx17_lro_packets: 0
>> rx17_lro_bytes: 0
>> rx17_ecn_mark: 0
>> rx17_removed_vlan_packets: 5428380986
>> rx17_wqe_err: 0
>> rx17_mpwqe_filler_cqes: 0
>> rx17_mpwqe_filler_strides: 0
>> rx17_buff_alloc_err: 0
>> rx17_cqe_compress_blks: 0
>> rx17_cqe_compress_pkts: 0
>> rx17_page_reuse: 0
>> rx17_cache_reuse: 1552266402
>> rx17_cache_full: 5947505
>> rx17_cache_empty: 1155981856
>> rx17_cache_busy: 5948870
>> rx17_cache_waive: 1155976345
>> rx17_congst_umr: 0
>> rx17_arfs_err: 0
>> rx17_xdp_tx_xmit: 0
>> rx17_xdp_tx_full: 0
>> rx17_xdp_tx_err: 0
>> rx17_xdp_tx_cqes: 0
>> rx18_packets: 5529118410
>> rx18_bytes: 3254749573833
>> rx18_csum_complete: 5529118410
>> rx18_csum_unnecessary: 0
>> rx18_csum_unnecessary_inner: 0
>> rx18_csum_none: 0
>> rx18_xdp_drop: 0
>> rx18_xdp_redirect: 0
>> rx18_lro_packets: 0
>> rx18_lro_bytes: 0
>> rx18_ecn_mark: 0
>> rx18_removed_vlan_packets: 5529118410
>> rx18_wqe_err: 0
>> rx18_mpwqe_filler_cqes: 0
>> rx18_mpwqe_filler_strides: 0
>> rx18_buff_alloc_err: 0
>> rx18_cqe_compress_blks: 0
>> rx18_cqe_compress_pkts: 0
>> rx18_page_reuse: 0
>> rx18_cache_reuse: 67438840
>> rx18_cache_full: 1536718472
>> rx18_cache_empty: 1160408072
>> rx18_cache_busy: 1536718932
>> rx18_cache_waive: 1160401638
>> rx18_congst_umr: 0
>> rx18_arfs_err: 0
>> rx18_xdp_tx_xmit: 0
>> rx18_xdp_tx_full: 0
>> rx18_xdp_tx_err: 0
>> rx18_xdp_tx_cqes: 0
>> rx19_packets: 5449932653
>> rx19_bytes: 3148726579411
>> rx19_csum_complete: 5449932653
>> rx19_csum_unnecessary: 0
>> rx19_csum_unnecessary_inner: 0
>> rx19_csum_none: 0
>> rx19_xdp_drop: 0
>> rx19_xdp_redirect: 0
>> rx19_lro_packets: 0
>> rx19_lro_bytes: 0
>> rx19_ecn_mark: 0
>> rx19_removed_vlan_packets: 5449932653
>> rx19_wqe_err: 0
>> rx19_mpwqe_filler_cqes: 0
>> rx19_mpwqe_filler_strides: 0
>> rx19_buff_alloc_err: 0
>> rx19_cqe_compress_blks: 0
>> rx19_cqe_compress_pkts: 0
>> rx19_page_reuse: 0
>> rx19_cache_reuse: 1537841743
>> rx19_cache_full: 9920960
>> rx19_cache_empty: 1177208938
>> rx19_cache_busy: 9922299
>> rx19_cache_waive: 1177203401
>> rx19_congst_umr: 0
>> rx19_arfs_err: 0
>> rx19_xdp_tx_xmit: 0
>> rx19_xdp_tx_full: 0
>> rx19_xdp_tx_err: 0
>> rx19_xdp_tx_cqes: 0
>> rx20_packets: 5407910071
>> rx20_bytes: 3123560861922
>> rx20_csum_complete: 5407910071
>> rx20_csum_unnecessary: 0
>> rx20_csum_unnecessary_inner: 0
>> rx20_csum_none: 0
>> rx20_xdp_drop: 0
>> rx20_xdp_redirect: 0
>> rx20_lro_packets: 0
>> rx20_lro_bytes: 0
>> rx20_ecn_mark: 0
>> rx20_removed_vlan_packets: 5407910071
>> rx20_wqe_err: 0
>> rx20_mpwqe_filler_cqes: 0
>> rx20_mpwqe_filler_strides: 0
>> rx20_buff_alloc_err: 0
>> rx20_cqe_compress_blks: 0
>> rx20_cqe_compress_pkts: 0
>> rx20_page_reuse: 0
>> rx20_cache_reuse: 10255209
>> rx20_cache_full: 2693699571
>> rx20_cache_empty: 6656
>> rx20_cache_busy: 2693699823
>> rx20_cache_waive: 0
>> rx20_congst_umr: 0
>> rx20_arfs_err: 0
>> rx20_xdp_tx_xmit: 0
>> rx20_xdp_tx_full: 0
>> rx20_xdp_tx_err: 0
>> rx20_xdp_tx_cqes: 0
>> rx21_packets: 5417498508
>> rx21_bytes: 3131335892379
>> rx21_csum_complete: 5417498508
>> rx21_csum_unnecessary: 0
>> rx21_csum_unnecessary_inner: 0
>> rx21_csum_none: 0
>> rx21_xdp_drop: 0
>> rx21_xdp_redirect: 0
>> rx21_lro_packets: 0
>> rx21_lro_bytes: 0
>> rx21_ecn_mark: 0
>> rx21_removed_vlan_packets: 5417498508
>> rx21_wqe_err: 0
>> rx21_mpwqe_filler_cqes: 0
>> rx21_mpwqe_filler_strides: 0
>> rx21_buff_alloc_err: 0
>> rx21_cqe_compress_blks: 0
>> rx21_cqe_compress_pkts: 0
>> rx21_page_reuse: 0
>> rx21_cache_reuse: 192662917
>> rx21_cache_full: 1374120417
>> rx21_cache_empty: 1141972100
>> rx21_cache_busy: 1374120891
>> rx21_cache_waive: 1141965665
>> rx21_congst_umr: 0
>> rx21_arfs_err: 0
>> rx21_xdp_tx_xmit: 0
>> rx21_xdp_tx_full: 0
>> rx21_xdp_tx_err: 0
>> rx21_xdp_tx_cqes: 0
>> rx22_packets: 5613634706
>> rx22_bytes: 3240055099058
>> rx22_csum_complete: 5613634706
>> rx22_csum_unnecessary: 0
>> rx22_csum_unnecessary_inner: 0
>> rx22_csum_none: 0
>> rx22_xdp_drop: 0
>> rx22_xdp_redirect: 0
>> rx22_lro_packets: 0
>> rx22_lro_bytes: 0
>> rx22_ecn_mark: 0
>> rx22_removed_vlan_packets: 5613634706
>> rx22_wqe_err: 0
>> rx22_mpwqe_filler_cqes: 0
>> rx22_mpwqe_filler_strides: 0
>> rx22_buff_alloc_err: 0
>> rx22_cqe_compress_blks: 0
>> rx22_cqe_compress_pkts: 0
>> rx22_page_reuse: 0
>> rx22_cache_reuse: 12161531
>> rx22_cache_full: 2794655567
>> rx22_cache_empty: 6656
>> rx22_cache_busy: 2794655821
>> rx22_cache_waive: 0
>> rx22_congst_umr: 0
>> rx22_arfs_err: 0
>> rx22_xdp_tx_xmit: 0
>> rx22_xdp_tx_full: 0
>> rx22_xdp_tx_err: 0
>> rx22_xdp_tx_cqes: 0
>> rx23_packets: 5389977167
>> rx23_bytes: 3054270771559
>> rx23_csum_complete: 5389977167
>> rx23_csum_unnecessary: 0
>> rx23_csum_unnecessary_inner: 0
>> rx23_csum_none: 0
>> rx23_xdp_drop: 0
>> rx23_xdp_redirect: 0
>> rx23_lro_packets: 0
>> rx23_lro_bytes: 0
>> rx23_ecn_mark: 0
>> rx23_removed_vlan_packets: 5389977167
>> rx23_wqe_err: 0
>> rx23_mpwqe_filler_cqes: 0
>> rx23_mpwqe_filler_strides: 0
>> rx23_buff_alloc_err: 0
>> rx23_cqe_compress_blks: 0
>> rx23_cqe_compress_pkts: 0
>> rx23_page_reuse: 0
>> rx23_cache_reuse: 709328
>> rx23_cache_full: 2694279000
>> rx23_cache_empty: 6656
>> rx23_cache_busy: 2694279252
>> rx23_cache_waive: 0
>> rx23_congst_umr: 0
>> rx23_arfs_err: 0
>> rx23_xdp_tx_xmit: 0
>> rx23_xdp_tx_full: 0
>> rx23_xdp_tx_err: 0
>> rx23_xdp_tx_cqes: 0
>> rx24_packets: 5547561932
>> rx24_bytes: 3166602453443
>> rx24_csum_complete: 5547561932
>> rx24_csum_unnecessary: 0
>> rx24_csum_unnecessary_inner: 0
>> rx24_csum_none: 0
>> rx24_xdp_drop: 0
>> rx24_xdp_redirect: 0
>> rx24_lro_packets: 0
>> rx24_lro_bytes: 0
>> rx24_ecn_mark: 0
>> rx24_removed_vlan_packets: 5547561932
>> rx24_wqe_err: 0
>> rx24_mpwqe_filler_cqes: 0
>> rx24_mpwqe_filler_strides: 0
>> rx24_buff_alloc_err: 0
>> rx24_cqe_compress_blks: 0
>> rx24_cqe_compress_pkts: 0
>> rx24_page_reuse: 0
>> rx24_cache_reuse: 57885119
>> rx24_cache_full: 1529450077
>> rx24_cache_empty: 1186451948
>> rx24_cache_busy: 1529450553
>> rx24_cache_waive: 1186445515
>> rx24_congst_umr: 0
>> rx24_arfs_err: 0
>> rx24_xdp_tx_xmit: 0
>> rx24_xdp_tx_full: 0
>> rx24_xdp_tx_err: 0
>> rx24_xdp_tx_cqes: 0
>> rx25_packets: 5414569326
>> rx25_bytes: 3184757708091
>> rx25_csum_complete: 5414569326
>> rx25_csum_unnecessary: 0
>> rx25_csum_unnecessary_inner: 0
>> rx25_csum_none: 0
>> rx25_xdp_drop: 0
>> rx25_xdp_redirect: 0
>> rx25_lro_packets: 0
>> rx25_lro_bytes: 0
>> rx25_ecn_mark: 0
>> rx25_removed_vlan_packets: 5414569326
>> rx25_wqe_err: 0
>> rx25_mpwqe_filler_cqes: 0
>> rx25_mpwqe_filler_strides: 0
>> rx25_buff_alloc_err: 0
>> rx25_cqe_compress_blks: 0
>> rx25_cqe_compress_pkts: 0
>> rx25_page_reuse: 0
>> rx25_cache_reuse: 5080853
>> rx25_cache_full: 2702203555
>> rx25_cache_empty: 6656
>> rx25_cache_busy: 2702203807
>> rx25_cache_waive: 0
>> rx25_congst_umr: 0
>> rx25_arfs_err: 0
>> rx25_xdp_tx_xmit: 0
>> rx25_xdp_tx_full: 0
>> rx25_xdp_tx_err: 0
>> rx25_xdp_tx_cqes: 0
>> rx26_packets: 5479972151
>> rx26_bytes: 3110642276239
>> rx26_csum_complete: 5479972151
>> rx26_csum_unnecessary: 0
>> rx26_csum_unnecessary_inner: 0
>> rx26_csum_none: 0
>> rx26_xdp_drop: 0
>> rx26_xdp_redirect: 0
>> rx26_lro_packets: 0
>> rx26_lro_bytes: 0
>> rx26_ecn_mark: 0
>> rx26_removed_vlan_packets: 5479972151
>> rx26_wqe_err: 0
>> rx26_mpwqe_filler_cqes: 0
>> rx26_mpwqe_filler_strides: 0
>> rx26_buff_alloc_err: 0
>> rx26_cqe_compress_blks: 0
>> rx26_cqe_compress_pkts: 0
>> rx26_page_reuse: 0
>> rx26_cache_reuse: 26543335
>> rx26_cache_full: 2713442485
>> rx26_cache_empty: 6656
>> rx26_cache_busy: 2713442737
>> rx26_cache_waive: 0
>> rx26_congst_umr: 0
>> rx26_arfs_err: 0
>> rx26_xdp_tx_xmit: 0
>> rx26_xdp_tx_full: 0
>> rx26_xdp_tx_err: 0
>> rx26_xdp_tx_cqes: 0
>> rx27_packets: 5337113900
>> rx27_bytes: 3068966906075
>> rx27_csum_complete: 5337113900
>> rx27_csum_unnecessary: 0
>> rx27_csum_unnecessary_inner: 0
>> rx27_csum_none: 0
>> rx27_xdp_drop: 0
>> rx27_xdp_redirect: 0
>> rx27_lro_packets: 0
>> rx27_lro_bytes: 0
>> rx27_ecn_mark: 0
>> rx27_removed_vlan_packets: 5337113900
>> rx27_wqe_err: 0
>> rx27_mpwqe_filler_cqes: 0
>> rx27_mpwqe_filler_strides: 0
>> rx27_buff_alloc_err: 0
>> rx27_cqe_compress_blks: 0
>> rx27_cqe_compress_pkts: 0
>> rx27_page_reuse: 0
>> rx27_cache_reuse: 1539298962
>> rx27_cache_full: 10861919
>> rx27_cache_empty: 1117173179
>> rx27_cache_busy: 12091463
>> rx27_cache_waive: 1118395847
>> rx27_congst_umr: 0
>> rx27_arfs_err: 0
>> rx27_xdp_tx_xmit: 0
>> rx27_xdp_tx_full: 0
>> rx27_xdp_tx_err: 0
>> rx27_xdp_tx_cqes: 0
>> rx28_packets: 0
>> rx28_bytes: 0
>> rx28_csum_complete: 0
>> rx28_csum_unnecessary: 0
>> rx28_csum_unnecessary_inner: 0
>> rx28_csum_none: 0
>> rx28_xdp_drop: 0
>> rx28_xdp_redirect: 0
>> rx28_lro_packets: 0
>> rx28_lro_bytes: 0
>> rx28_ecn_mark: 0
>> rx28_removed_vlan_packets: 0
>> rx28_wqe_err: 0
>> rx28_mpwqe_filler_cqes: 0
>> rx28_mpwqe_filler_strides: 0
>> rx28_buff_alloc_err: 0
>> rx28_cqe_compress_blks: 0
>> rx28_cqe_compress_pkts: 0
>> rx28_page_reuse: 0
>> rx28_cache_reuse: 0
>> rx28_cache_full: 0
>> rx28_cache_empty: 2560
>> rx28_cache_busy: 0
>> rx28_cache_waive: 0
>> rx28_congst_umr: 0
>> rx28_arfs_err: 0
>> rx28_xdp_tx_xmit: 0
>> rx28_xdp_tx_full: 0
>> rx28_xdp_tx_err: 0
>> rx28_xdp_tx_cqes: 0
>> rx29_packets: 0
>> rx29_bytes: 0
>> rx29_csum_complete: 0
>> rx29_csum_unnecessary: 0
>> rx29_csum_unnecessary_inner: 0
>> rx29_csum_none: 0
>> rx29_xdp_drop: 0
>> rx29_xdp_redirect: 0
>> rx29_lro_packets: 0
>> rx29_lro_bytes: 0
>> rx29_ecn_mark: 0
>> rx29_removed_vlan_packets: 0
>> rx29_wqe_err: 0
>> rx29_mpwqe_filler_cqes: 0
>> rx29_mpwqe_filler_strides: 0
>> rx29_buff_alloc_err: 0
>> rx29_cqe_compress_blks: 0
>> rx29_cqe_compress_pkts: 0
>> rx29_page_reuse: 0
>> rx29_cache_reuse: 0
>> rx29_cache_full: 0
>> rx29_cache_empty: 2560
>> rx29_cache_busy: 0
>> rx29_cache_waive: 0
>> rx29_congst_umr: 0
>> rx29_arfs_err: 0
>> rx29_xdp_tx_xmit: 0
>> rx29_xdp_tx_full: 0
>> rx29_xdp_tx_err: 0
>> rx29_xdp_tx_cqes: 0
>> rx30_packets: 0
>> rx30_bytes: 0
>> rx30_csum_complete: 0
>> rx30_csum_unnecessary: 0
>> rx30_csum_unnecessary_inner: 0
>> rx30_csum_none: 0
>> rx30_xdp_drop: 0
>> rx30_xdp_redirect: 0
>> rx30_lro_packets: 0
>> rx30_lro_bytes: 0
>> rx30_ecn_mark: 0
>> rx30_removed_vlan_packets: 0
>> rx30_wqe_err: 0
>> rx30_mpwqe_filler_cqes: 0
>> rx30_mpwqe_filler_strides: 0
>> rx30_buff_alloc_err: 0
>> rx30_cqe_compress_blks: 0
>> rx30_cqe_compress_pkts: 0
>> rx30_page_reuse: 0
>> rx30_cache_reuse: 0
>> rx30_cache_full: 0
>> rx30_cache_empty: 2560
>> rx30_cache_busy: 0
>> rx30_cache_waive: 0
>> rx30_congst_umr: 0
>> rx30_arfs_err: 0
>> rx30_xdp_tx_xmit: 0
>> rx30_xdp_tx_full: 0
>> rx30_xdp_tx_err: 0
>> rx30_xdp_tx_cqes: 0
>> rx31_packets: 0
>> rx31_bytes: 0
>> rx31_csum_complete: 0
>> rx31_csum_unnecessary: 0
>> rx31_csum_unnecessary_inner: 0
>> rx31_csum_none: 0
>> rx31_xdp_drop: 0
>> rx31_xdp_redirect: 0
>> rx31_lro_packets: 0
>> rx31_lro_bytes: 0
>> rx31_ecn_mark: 0
>> rx31_removed_vlan_packets: 0
>> rx31_wqe_err: 0
>> rx31_mpwqe_filler_cqes: 0
>> rx31_mpwqe_filler_strides: 0
>> rx31_buff_alloc_err: 0
>> rx31_cqe_compress_blks: 0
>> rx31_cqe_compress_pkts: 0
>> rx31_page_reuse: 0
>> rx31_cache_reuse: 0
>> rx31_cache_full: 0
>> rx31_cache_empty: 2560
>> rx31_cache_busy: 0
>> rx31_cache_waive: 0
>> rx31_congst_umr: 0
>> rx31_arfs_err: 0
>> rx31_xdp_tx_xmit: 0
>> rx31_xdp_tx_full: 0
>> rx31_xdp_tx_err: 0
>> rx31_xdp_tx_cqes: 0
>> rx32_packets: 0
>> rx32_bytes: 0
>> rx32_csum_complete: 0
>> rx32_csum_unnecessary: 0
>> rx32_csum_unnecessary_inner: 0
>> rx32_csum_none: 0
>> rx32_xdp_drop: 0
>> rx32_xdp_redirect: 0
>> rx32_lro_packets: 0
>> rx32_lro_bytes: 0
>> rx32_ecn_mark: 0
>> rx32_removed_vlan_packets: 0
>> rx32_wqe_err: 0
>> rx32_mpwqe_filler_cqes: 0
>> rx32_mpwqe_filler_strides: 0
>> rx32_buff_alloc_err: 0
>> rx32_cqe_compress_blks: 0
>> rx32_cqe_compress_pkts: 0
>> rx32_page_reuse: 0
>> rx32_cache_reuse: 0
>> rx32_cache_full: 0
>> rx32_cache_empty: 2560
>> rx32_cache_busy: 0
>> rx32_cache_waive: 0
>> rx32_congst_umr: 0
>> rx32_arfs_err: 0
>> rx32_xdp_tx_xmit: 0
>> rx32_xdp_tx_full: 0
>> rx32_xdp_tx_err: 0
>> rx32_xdp_tx_cqes: 0
>> rx33_packets: 0
>> rx33_bytes: 0
>> rx33_csum_complete: 0
>> rx33_csum_unnecessary: 0
>> rx33_csum_unnecessary_inner: 0
>> rx33_csum_none: 0
>> rx33_xdp_drop: 0
>> rx33_xdp_redirect: 0
>> rx33_lro_packets: 0
>> rx33_lro_bytes: 0
>> rx33_ecn_mark: 0
>> rx33_removed_vlan_packets: 0
>> rx33_wqe_err: 0
>> rx33_mpwqe_filler_cqes: 0
>> rx33_mpwqe_filler_strides: 0
>> rx33_buff_alloc_err: 0
>> rx33_cqe_compress_blks: 0
>> rx33_cqe_compress_pkts: 0
>> rx33_page_reuse: 0
>> rx33_cache_reuse: 0
>> rx33_cache_full: 0
>> rx33_cache_empty: 2560
>> rx33_cache_busy: 0
>> rx33_cache_waive: 0
>> rx33_congst_umr: 0
>> rx33_arfs_err: 0
>> rx33_xdp_tx_xmit: 0
>> rx33_xdp_tx_full: 0
>> rx33_xdp_tx_err: 0
>> rx33_xdp_tx_cqes: 0
>> rx34_packets: 0
>> rx34_bytes: 0
>> rx34_csum_complete: 0
>> rx34_csum_unnecessary: 0
>> rx34_csum_unnecessary_inner: 0
>> rx34_csum_none: 0
>> rx34_xdp_drop: 0
>> rx34_xdp_redirect: 0
>> rx34_lro_packets: 0
>> rx34_lro_bytes: 0
>> rx34_ecn_mark: 0
>> rx34_removed_vlan_packets: 0
>> rx34_wqe_err: 0
>> rx34_mpwqe_filler_cqes: 0
>> rx34_mpwqe_filler_strides: 0
>> rx34_buff_alloc_err: 0
>> rx34_cqe_compress_blks: 0
>> rx34_cqe_compress_pkts: 0
>> rx34_page_reuse: 0
>> rx34_cache_reuse: 0
>> rx34_cache_full: 0
>> rx34_cache_empty: 2560
>> rx34_cache_busy: 0
>> rx34_cache_waive: 0
>> rx34_congst_umr: 0
>> rx34_arfs_err: 0
>> rx34_xdp_tx_xmit: 0
>> rx34_xdp_tx_full: 0
>> rx34_xdp_tx_err: 0
>> rx34_xdp_tx_cqes: 0
>> rx35_packets: 0
>> rx35_bytes: 0
>> rx35_csum_complete: 0
>> rx35_csum_unnecessary: 0
>> rx35_csum_unnecessary_inner: 0
>> rx35_csum_none: 0
>> rx35_xdp_drop: 0
>> rx35_xdp_redirect: 0
>> rx35_lro_packets: 0
>> rx35_lro_bytes: 0
>> rx35_ecn_mark: 0
>> rx35_removed_vlan_packets: 0
>> rx35_wqe_err: 0
>> rx35_mpwqe_filler_cqes: 0
>> rx35_mpwqe_filler_strides: 0
>> rx35_buff_alloc_err: 0
>> rx35_cqe_compress_blks: 0
>> rx35_cqe_compress_pkts: 0
>> rx35_page_reuse: 0
>> rx35_cache_reuse: 0
>> rx35_cache_full: 0
>> rx35_cache_empty: 2560
>> rx35_cache_busy: 0
>> rx35_cache_waive: 0
>> rx35_congst_umr: 0
>> rx35_arfs_err: 0
>> rx35_xdp_tx_xmit: 0
>> rx35_xdp_tx_full: 0
>> rx35_xdp_tx_err: 0
>> rx35_xdp_tx_cqes: 0
>> rx36_packets: 0
>> rx36_bytes: 0
>> rx36_csum_complete: 0
>> rx36_csum_unnecessary: 0
>> rx36_csum_unnecessary_inner: 0
>> rx36_csum_none: 0
>> rx36_xdp_drop: 0
>> rx36_xdp_redirect: 0
>> rx36_lro_packets: 0
>> rx36_lro_bytes: 0
>> rx36_ecn_mark: 0
>> rx36_removed_vlan_packets: 0
>> rx36_wqe_err: 0
>> rx36_mpwqe_filler_cqes: 0
>> rx36_mpwqe_filler_strides: 0
>> rx36_buff_alloc_err: 0
>> rx36_cqe_compress_blks: 0
>> rx36_cqe_compress_pkts: 0
>> rx36_page_reuse: 0
>> rx36_cache_reuse: 0
>> rx36_cache_full: 0
>> rx36_cache_empty: 2560
>> rx36_cache_busy: 0
>> rx36_cache_waive: 0
>> rx36_congst_umr: 0
>> rx36_arfs_err: 0
>> rx36_xdp_tx_xmit: 0
>> rx36_xdp_tx_full: 0
>> rx36_xdp_tx_err: 0
>> rx36_xdp_tx_cqes: 0
>> rx37_packets: 0
>> rx37_bytes: 0
>> rx37_csum_complete: 0
>> rx37_csum_unnecessary: 0
>> rx37_csum_unnecessary_inner: 0
>> rx37_csum_none: 0
>> rx37_xdp_drop: 0
>> rx37_xdp_redirect: 0
>> rx37_lro_packets: 0
>> rx37_lro_bytes: 0
>> rx37_ecn_mark: 0
>> rx37_removed_vlan_packets: 0
>> rx37_wqe_err: 0
>> rx37_mpwqe_filler_cqes: 0
>> rx37_mpwqe_filler_strides: 0
>> rx37_buff_alloc_err: 0
>> rx37_cqe_compress_blks: 0
>> rx37_cqe_compress_pkts: 0
>> rx37_page_reuse: 0
>> rx37_cache_reuse: 0
>> rx37_cache_full: 0
>> rx37_cache_empty: 2560
>> rx37_cache_busy: 0
>> rx37_cache_waive: 0
>> rx37_congst_umr: 0
>> rx37_arfs_err: 0
>> rx37_xdp_tx_xmit: 0
>> rx37_xdp_tx_full: 0
>> rx37_xdp_tx_err: 0
>> rx37_xdp_tx_cqes: 0
>> rx38_packets: 0
>> rx38_bytes: 0
>> rx38_csum_complete: 0
>> rx38_csum_unnecessary: 0
>> rx38_csum_unnecessary_inner: 0
>> rx38_csum_none: 0
>> rx38_xdp_drop: 0
>> rx38_xdp_redirect: 0
>> rx38_lro_packets: 0
>> rx38_lro_bytes: 0
>> rx38_ecn_mark: 0
>> rx38_removed_vlan_packets: 0
>> rx38_wqe_err: 0
>> rx38_mpwqe_filler_cqes: 0
>> rx38_mpwqe_filler_strides: 0
>> rx38_buff_alloc_err: 0
>> rx38_cqe_compress_blks: 0
>> rx38_cqe_compress_pkts: 0
>> rx38_page_reuse: 0
>> rx38_cache_reuse: 0
>> rx38_cache_full: 0
>> rx38_cache_empty: 2560
>> rx38_cache_busy: 0
>> rx38_cache_waive: 0
>> rx38_congst_umr: 0
>> rx38_arfs_err: 0
>> rx38_xdp_tx_xmit: 0
>> rx38_xdp_tx_full: 0
>> rx38_xdp_tx_err: 0
>> rx38_xdp_tx_cqes: 0
>> rx39_packets: 0
>> rx39_bytes: 0
>> rx39_csum_complete: 0
>> rx39_csum_unnecessary: 0
>> rx39_csum_unnecessary_inner: 0
>> rx39_csum_none: 0
>> rx39_xdp_drop: 0
>> rx39_xdp_redirect: 0
>> rx39_lro_packets: 0
>> rx39_lro_bytes: 0
>> rx39_ecn_mark: 0
>> rx39_removed_vlan_packets: 0
>> rx39_wqe_err: 0
>> rx39_mpwqe_filler_cqes: 0
>> rx39_mpwqe_filler_strides: 0
>> rx39_buff_alloc_err: 0
>> rx39_cqe_compress_blks: 0
>> rx39_cqe_compress_pkts: 0
>> rx39_page_reuse: 0
>> rx39_cache_reuse: 0
>> rx39_cache_full: 0
>> rx39_cache_empty: 2560
>> rx39_cache_busy: 0
>> rx39_cache_waive: 0
>> rx39_congst_umr: 0
>> rx39_arfs_err: 0
>> rx39_xdp_tx_xmit: 0
>> rx39_xdp_tx_full: 0
>> rx39_xdp_tx_err: 0
>> rx39_xdp_tx_cqes: 0
>> rx40_packets: 0
>> rx40_bytes: 0
>> rx40_csum_complete: 0
>> rx40_csum_unnecessary: 0
>> rx40_csum_unnecessary_inner: 0
>> rx40_csum_none: 0
>> rx40_xdp_drop: 0
>> rx40_xdp_redirect: 0
>> rx40_lro_packets: 0
>> rx40_lro_bytes: 0
>> rx40_ecn_mark: 0
>> rx40_removed_vlan_packets: 0
>> rx40_wqe_err: 0
>> rx40_mpwqe_filler_cqes: 0
>> rx40_mpwqe_filler_strides: 0
>> rx40_buff_alloc_err: 0
>> rx40_cqe_compress_blks: 0
>> rx40_cqe_compress_pkts: 0
>> rx40_page_reuse: 0
>> rx40_cache_reuse: 0
>> rx40_cache_full: 0
>> rx40_cache_empty: 2560
>> rx40_cache_busy: 0
>> rx40_cache_waive: 0
>> rx40_congst_umr: 0
>> rx40_arfs_err: 0
>> rx40_xdp_tx_xmit: 0
>> rx40_xdp_tx_full: 0
>> rx40_xdp_tx_err: 0
>> rx40_xdp_tx_cqes: 0
>> rx41_packets: 0
>> rx41_bytes: 0
>> rx41_csum_complete: 0
>> rx41_csum_unnecessary: 0
>> rx41_csum_unnecessary_inner: 0
>> rx41_csum_none: 0
>> rx41_xdp_drop: 0
>> rx41_xdp_redirect: 0
>> rx41_lro_packets: 0
>> rx41_lro_bytes: 0
>> rx41_ecn_mark: 0
>> rx41_removed_vlan_packets: 0
>> rx41_wqe_err: 0
>> rx41_mpwqe_filler_cqes: 0
>> rx41_mpwqe_filler_strides: 0
>> rx41_buff_alloc_err: 0
>> rx41_cqe_compress_blks: 0
>> rx41_cqe_compress_pkts: 0
>> rx41_page_reuse: 0
>> rx41_cache_reuse: 0
>> rx41_cache_full: 0
>> rx41_cache_empty: 2560
>> rx41_cache_busy: 0
>> rx41_cache_waive: 0
>> rx41_congst_umr: 0
>> rx41_arfs_err: 0
>> rx41_xdp_tx_xmit: 0
>> rx41_xdp_tx_full: 0
>> rx41_xdp_tx_err: 0
>> rx41_xdp_tx_cqes: 0
>> rx42_packets: 0
>> rx42_bytes: 0
>> rx42_csum_complete: 0
>> rx42_csum_unnecessary: 0
>> rx42_csum_unnecessary_inner: 0
>> rx42_csum_none: 0
>> rx42_xdp_drop: 0
>> rx42_xdp_redirect: 0
>> rx42_lro_packets: 0
>> rx42_lro_bytes: 0
>> rx42_ecn_mark: 0
>> rx42_removed_vlan_packets: 0
>> rx42_wqe_err: 0
>> rx42_mpwqe_filler_cqes: 0
>> rx42_mpwqe_filler_strides: 0
>> rx42_buff_alloc_err: 0
>> rx42_cqe_compress_blks: 0
>> rx42_cqe_compress_pkts: 0
>> rx42_page_reuse: 0
>> rx42_cache_reuse: 0
>> rx42_cache_full: 0
>> rx42_cache_empty: 2560
>> rx42_cache_busy: 0
>> rx42_cache_waive: 0
>> rx42_congst_umr: 0
>> rx42_arfs_err: 0
>> rx42_xdp_tx_xmit: 0
>> rx42_xdp_tx_full: 0
>> rx42_xdp_tx_err: 0
>> rx42_xdp_tx_cqes: 0
>> rx43_packets: 0
>> rx43_bytes: 0
>> rx43_csum_complete: 0
>> rx43_csum_unnecessary: 0
>> rx43_csum_unnecessary_inner: 0
>> rx43_csum_none: 0
>> rx43_xdp_drop: 0
>> rx43_xdp_redirect: 0
>> rx43_lro_packets: 0
>> rx43_lro_bytes: 0
>> rx43_ecn_mark: 0
>> rx43_removed_vlan_packets: 0
>> rx43_wqe_err: 0
>> rx43_mpwqe_filler_cqes: 0
>> rx43_mpwqe_filler_strides: 0
>> rx43_buff_alloc_err: 0
>> rx43_cqe_compress_blks: 0
>> rx43_cqe_compress_pkts: 0
>> rx43_page_reuse: 0
>> rx43_cache_reuse: 0
>> rx43_cache_full: 0
>> rx43_cache_empty: 2560
>> rx43_cache_busy: 0
>> rx43_cache_waive: 0
>> rx43_congst_umr: 0
>> rx43_arfs_err: 0
>> rx43_xdp_tx_xmit: 0
>> rx43_xdp_tx_full: 0
>> rx43_xdp_tx_err: 0
>> rx43_xdp_tx_cqes: 0
>> rx44_packets: 0
>> rx44_bytes: 0
>> rx44_csum_complete: 0
>> rx44_csum_unnecessary: 0
>> rx44_csum_unnecessary_inner: 0
>> rx44_csum_none: 0
>> rx44_xdp_drop: 0
>> rx44_xdp_redirect: 0
>> rx44_lro_packets: 0
>> rx44_lro_bytes: 0
>> rx44_ecn_mark: 0
>> rx44_removed_vlan_packets: 0
>> rx44_wqe_err: 0
>> rx44_mpwqe_filler_cqes: 0
>> rx44_mpwqe_filler_strides: 0
>> rx44_buff_alloc_err: 0
>> rx44_cqe_compress_blks: 0
>> rx44_cqe_compress_pkts: 0
>> rx44_page_reuse: 0
>> rx44_cache_reuse: 0
>> rx44_cache_full: 0
>> rx44_cache_empty: 2560
>> rx44_cache_busy: 0
>> rx44_cache_waive: 0
>> rx44_congst_umr: 0
>> rx44_arfs_err: 0
>> rx44_xdp_tx_xmit: 0
>> rx44_xdp_tx_full: 0
>> rx44_xdp_tx_err: 0
>> rx44_xdp_tx_cqes: 0
>> rx45_packets: 0
>> rx45_bytes: 0
>> rx45_csum_complete: 0
>> rx45_csum_unnecessary: 0
>> rx45_csum_unnecessary_inner: 0
>> rx45_csum_none: 0
>> rx45_xdp_drop: 0
>> rx45_xdp_redirect: 0
>> rx45_lro_packets: 0
>> rx45_lro_bytes: 0
>> rx45_ecn_mark: 0
>> rx45_removed_vlan_packets: 0
>> rx45_wqe_err: 0
>> rx45_mpwqe_filler_cqes: 0
>> rx45_mpwqe_filler_strides: 0
>> rx45_buff_alloc_err: 0
>> rx45_cqe_compress_blks: 0
>> rx45_cqe_compress_pkts: 0
>> rx45_page_reuse: 0
>> rx45_cache_reuse: 0
>> rx45_cache_full: 0
>> rx45_cache_empty: 2560
>> rx45_cache_busy: 0
>> rx45_cache_waive: 0
>> rx45_congst_umr: 0
>> rx45_arfs_err: 0
>> rx45_xdp_tx_xmit: 0
>> rx45_xdp_tx_full: 0
>> rx45_xdp_tx_err: 0
>> rx45_xdp_tx_cqes: 0
>> rx46_packets: 0
>> rx46_bytes: 0
>> rx46_csum_complete: 0
>> rx46_csum_unnecessary: 0
>> rx46_csum_unnecessary_inner: 0
>> rx46_csum_none: 0
>> rx46_xdp_drop: 0
>> rx46_xdp_redirect: 0
>> rx46_lro_packets: 0
>> rx46_lro_bytes: 0
>> rx46_ecn_mark: 0
>> rx46_removed_vlan_packets: 0
>> rx46_wqe_err: 0
>> rx46_mpwqe_filler_cqes: 0
>> rx46_mpwqe_filler_strides: 0
>> rx46_buff_alloc_err: 0
>> rx46_cqe_compress_blks: 0
>> rx46_cqe_compress_pkts: 0
>> rx46_page_reuse: 0
>> rx46_cache_reuse: 0
>> rx46_cache_full: 0
>> rx46_cache_empty: 2560
>> rx46_cache_busy: 0
>> rx46_cache_waive: 0
>> rx46_congst_umr: 0
>> rx46_arfs_err: 0
>> rx46_xdp_tx_xmit: 0
>> rx46_xdp_tx_full: 0
>> rx46_xdp_tx_err: 0
>> rx46_xdp_tx_cqes: 0
>> rx47_packets: 0
>> rx47_bytes: 0
>> rx47_csum_complete: 0
>> rx47_csum_unnecessary: 0
>> rx47_csum_unnecessary_inner: 0
>> rx47_csum_none: 0
>> rx47_xdp_drop: 0
>> rx47_xdp_redirect: 0
>> rx47_lro_packets: 0
>> rx47_lro_bytes: 0
>> rx47_ecn_mark: 0
>> rx47_removed_vlan_packets: 0
>> rx47_wqe_err: 0
>> rx47_mpwqe_filler_cqes: 0
>> rx47_mpwqe_filler_strides: 0
>> rx47_buff_alloc_err: 0
>> rx47_cqe_compress_blks: 0
>> rx47_cqe_compress_pkts: 0
>> rx47_page_reuse: 0
>> rx47_cache_reuse: 0
>> rx47_cache_full: 0
>> rx47_cache_empty: 2560
>> rx47_cache_busy: 0
>> rx47_cache_waive: 0
>> rx47_congst_umr: 0
>> rx47_arfs_err: 0
>> rx47_xdp_tx_xmit: 0
>> rx47_xdp_tx_full: 0
>> rx47_xdp_tx_err: 0
>> rx47_xdp_tx_cqes: 0
>> rx48_packets: 0
>> rx48_bytes: 0
>> rx48_csum_complete: 0
>> rx48_csum_unnecessary: 0
>> rx48_csum_unnecessary_inner: 0
>> rx48_csum_none: 0
>> rx48_xdp_drop: 0
>> rx48_xdp_redirect: 0
>> rx48_lro_packets: 0
>> rx48_lro_bytes: 0
>> rx48_ecn_mark: 0
>> rx48_removed_vlan_packets: 0
>> rx48_wqe_err: 0
>> rx48_mpwqe_filler_cqes: 0
>> rx48_mpwqe_filler_strides: 0
>> rx48_buff_alloc_err: 0
>> rx48_cqe_compress_blks: 0
>> rx48_cqe_compress_pkts: 0
>> rx48_page_reuse: 0
>> rx48_cache_reuse: 0
>> rx48_cache_full: 0
>> rx48_cache_empty: 2560
>> rx48_cache_busy: 0
>> rx48_cache_waive: 0
>> rx48_congst_umr: 0
>> rx48_arfs_err: 0
>> rx48_xdp_tx_xmit: 0
>> rx48_xdp_tx_full: 0
>> rx48_xdp_tx_err: 0
>> rx48_xdp_tx_cqes: 0
>> rx49_packets: 0
>> rx49_bytes: 0
>> rx49_csum_complete: 0
>> rx49_csum_unnecessary: 0
>> rx49_csum_unnecessary_inner: 0
>> rx49_csum_none: 0
>> rx49_xdp_drop: 0
>> rx49_xdp_redirect: 0
>> rx49_lro_packets: 0
>> rx49_lro_bytes: 0
>> rx49_ecn_mark: 0
>> rx49_removed_vlan_packets: 0
>> rx49_wqe_err: 0
>> rx49_mpwqe_filler_cqes: 0
>> rx49_mpwqe_filler_strides: 0
>> rx49_buff_alloc_err: 0
>> rx49_cqe_compress_blks: 0
>> rx49_cqe_compress_pkts: 0
>> rx49_page_reuse: 0
>> rx49_cache_reuse: 0
>> rx49_cache_full: 0
>> rx49_cache_empty: 2560
>> rx49_cache_busy: 0
>> rx49_cache_waive: 0
>> rx49_congst_umr: 0
>> rx49_arfs_err: 0
>> rx49_xdp_tx_xmit: 0
>> rx49_xdp_tx_full: 0
>> rx49_xdp_tx_err: 0
>> rx49_xdp_tx_cqes: 0
>> rx50_packets: 0
>> rx50_bytes: 0
>> rx50_csum_complete: 0
>> rx50_csum_unnecessary: 0
>> rx50_csum_unnecessary_inner: 0
>> rx50_csum_none: 0
>> rx50_xdp_drop: 0
>> rx50_xdp_redirect: 0
>> rx50_lro_packets: 0
>> rx50_lro_bytes: 0
>> rx50_ecn_mark: 0
>> rx50_removed_vlan_packets: 0
>> rx50_wqe_err: 0
>> rx50_mpwqe_filler_cqes: 0
>> rx50_mpwqe_filler_strides: 0
>> rx50_buff_alloc_err: 0
>> rx50_cqe_compress_blks: 0
>> rx50_cqe_compress_pkts: 0
>> rx50_page_reuse: 0
>> rx50_cache_reuse: 0
>> rx50_cache_full: 0
>> rx50_cache_empty: 2560
>> rx50_cache_busy: 0
>> rx50_cache_waive: 0
>> rx50_congst_umr: 0
>> rx50_arfs_err: 0
>> rx50_xdp_tx_xmit: 0
>> rx50_xdp_tx_full: 0
>> rx50_xdp_tx_err: 0
>> rx50_xdp_tx_cqes: 0
>> rx51_packets: 0
>> rx51_bytes: 0
>> rx51_csum_complete: 0
>> rx51_csum_unnecessary: 0
>> rx51_csum_unnecessary_inner: 0
>> rx51_csum_none: 0
>> rx51_xdp_drop: 0
>> rx51_xdp_redirect: 0
>> rx51_lro_packets: 0
>> rx51_lro_bytes: 0
>> rx51_ecn_mark: 0
>> rx51_removed_vlan_packets: 0
>> rx51_wqe_err: 0
>> rx51_mpwqe_filler_cqes: 0
>> rx51_mpwqe_filler_strides: 0
>> rx51_buff_alloc_err: 0
>> rx51_cqe_compress_blks: 0
>> rx51_cqe_compress_pkts: 0
>> rx51_page_reuse: 0
>> rx51_cache_reuse: 0
>> rx51_cache_full: 0
>> rx51_cache_empty: 2560
>> rx51_cache_busy: 0
>> rx51_cache_waive: 0
>> rx51_congst_umr: 0
>> rx51_arfs_err: 0
>> rx51_xdp_tx_xmit: 0
>> rx51_xdp_tx_full: 0
>> rx51_xdp_tx_err: 0
>> rx51_xdp_tx_cqes: 0
>> rx52_packets: 0
>> rx52_bytes: 0
>> rx52_csum_complete: 0
>> rx52_csum_unnecessary: 0
>> rx52_csum_unnecessary_inner: 0
>> rx52_csum_none: 0
>> rx52_xdp_drop: 0
>> rx52_xdp_redirect: 0
>> rx52_lro_packets: 0
>> rx52_lro_bytes: 0
>> rx52_ecn_mark: 0
>> rx52_removed_vlan_packets: 0
>> rx52_wqe_err: 0
>> rx52_mpwqe_filler_cqes: 0
>> rx52_mpwqe_filler_strides: 0
>> rx52_buff_alloc_err: 0
>> rx52_cqe_compress_blks: 0
>> rx52_cqe_compress_pkts: 0
>> rx52_page_reuse: 0
>> rx52_cache_reuse: 0
>> rx52_cache_full: 0
>> rx52_cache_empty: 2560
>> rx52_cache_busy: 0
>> rx52_cache_waive: 0
>> rx52_congst_umr: 0
>> rx52_arfs_err: 0
>> rx52_xdp_tx_xmit: 0
>> rx52_xdp_tx_full: 0
>> rx52_xdp_tx_err: 0
>> rx52_xdp_tx_cqes: 0
>> rx53_packets: 0
>> rx53_bytes: 0
>> rx53_csum_complete: 0
>> rx53_csum_unnecessary: 0
>> rx53_csum_unnecessary_inner: 0
>> rx53_csum_none: 0
>> rx53_xdp_drop: 0
>> rx53_xdp_redirect: 0
>> rx53_lro_packets: 0
>> rx53_lro_bytes: 0
>> rx53_ecn_mark: 0
>> rx53_removed_vlan_packets: 0
>> rx53_wqe_err: 0
>> rx53_mpwqe_filler_cqes: 0
>> rx53_mpwqe_filler_strides: 0
>> rx53_buff_alloc_err: 0
>> rx53_cqe_compress_blks: 0
>> rx53_cqe_compress_pkts: 0
>> rx53_page_reuse: 0
>> rx53_cache_reuse: 0
>> rx53_cache_full: 0
>> rx53_cache_empty: 2560
>> rx53_cache_busy: 0
>> rx53_cache_waive: 0
>> rx53_congst_umr: 0
>> rx53_arfs_err: 0
>> rx53_xdp_tx_xmit: 0
>> rx53_xdp_tx_full: 0
>> rx53_xdp_tx_err: 0
>> rx53_xdp_tx_cqes: 0
>> rx54_packets: 0
>> rx54_bytes: 0
>> rx54_csum_complete: 0
>> rx54_csum_unnecessary: 0
>> rx54_csum_unnecessary_inner: 0
>> rx54_csum_none: 0
>> rx54_xdp_drop: 0
>> rx54_xdp_redirect: 0
>> rx54_lro_packets: 0
>> rx54_lro_bytes: 0
>> rx54_ecn_mark: 0
>> rx54_removed_vlan_packets: 0
>> rx54_wqe_err: 0
>> rx54_mpwqe_filler_cqes: 0
>> rx54_mpwqe_filler_strides: 0
>> rx54_buff_alloc_err: 0
>> rx54_cqe_compress_blks: 0
>> rx54_cqe_compress_pkts: 0
>> rx54_page_reuse: 0
>> rx54_cache_reuse: 0
>> rx54_cache_full: 0
>> rx54_cache_empty: 2560
>> rx54_cache_busy: 0
>> rx54_cache_waive: 0
>> rx54_congst_umr: 0
>> rx54_arfs_err: 0
>> rx54_xdp_tx_xmit: 0
>> rx54_xdp_tx_full: 0
>> rx54_xdp_tx_err: 0
>> rx54_xdp_tx_cqes: 0
>> rx55_packets: 0
>> rx55_bytes: 0
>> rx55_csum_complete: 0
>> rx55_csum_unnecessary: 0
>> rx55_csum_unnecessary_inner: 0
>> rx55_csum_none: 0
>> rx55_xdp_drop: 0
>> rx55_xdp_redirect: 0
>> rx55_lro_packets: 0
>> rx55_lro_bytes: 0
>> rx55_ecn_mark: 0
>> rx55_removed_vlan_packets: 0
>> rx55_wqe_err: 0
>> rx55_mpwqe_filler_cqes: 0
>> rx55_mpwqe_filler_strides: 0
>> rx55_buff_alloc_err: 0
>> rx55_cqe_compress_blks: 0
>> rx55_cqe_compress_pkts: 0
>> rx55_page_reuse: 0
>> rx55_cache_reuse: 0
>> rx55_cache_full: 0
>> rx55_cache_empty: 2560
>> rx55_cache_busy: 0
>> rx55_cache_waive: 0
>> rx55_congst_umr: 0
>> rx55_arfs_err: 0
>> rx55_xdp_tx_xmit: 0
>> rx55_xdp_tx_full: 0
>> rx55_xdp_tx_err: 0
>> rx55_xdp_tx_cqes: 0
>> tx0_packets: 5868971166
>> tx0_bytes: 7384241881537
>> tx0_tso_packets: 1005089669
>> tx0_tso_bytes: 5138882499687
>> tx0_tso_inner_packets: 0
>> tx0_tso_inner_bytes: 0
>> tx0_csum_partial: 1405330470
>> tx0_csum_partial_inner: 0
>> tx0_added_vlan_packets: 3247061022
>> tx0_nop: 83925216
>> tx0_csum_none: 1841730552
>> tx0_stopped: 0
>> tx0_dropped: 0
>> tx0_xmit_more: 29664303
>> tx0_recover: 0
>> tx0_cqes: 3217398842
>> tx0_wake: 0
>> tx0_cqe_err: 0
>> tx1_packets: 5599378674
>> tx1_bytes: 7272236466962
>> tx1_tso_packets: 1024612268
>> tx1_tso_bytes: 5244192050917
>> tx1_tso_inner_packets: 0
>> tx1_tso_inner_bytes: 0
>> tx1_csum_partial: 1438007932
>> tx1_csum_partial_inner: 0
>> tx1_added_vlan_packets: 2919765857
>> tx1_nop: 79661231
>> tx1_csum_none: 1481757925
>> tx1_stopped: 0
>> tx1_dropped: 0
>> tx1_xmit_more: 29485355
>> tx1_recover: 0
>> tx1_cqes: 2890282176
>> tx1_wake: 0
>> tx1_cqe_err: 0
>> tx2_packets: 5413821094
>> tx2_bytes: 7033951631334
>> tx2_tso_packets: 1002868589
>> tx2_tso_bytes: 5089549008985
>> tx2_tso_inner_packets: 0
>> tx2_tso_inner_bytes: 0
>> tx2_csum_partial: 1404186175
>> tx2_csum_partial_inner: 0
>> tx2_added_vlan_packets: 2822670460
>> tx2_nop: 77115408
>> tx2_csum_none: 1418484285
>> tx2_stopped: 0
>> tx2_dropped: 0
>> tx2_xmit_more: 29321129
>> tx2_recover: 0
>> tx2_cqes: 2793351019
>> tx2_wake: 0
>> tx2_cqe_err: 0
>> tx3_packets: 5479609727
>> tx3_bytes: 7116904107659
>> tx3_tso_packets: 1002992639
>> tx3_tso_bytes: 5154225081979
>> tx3_tso_inner_packets: 0
>> tx3_tso_inner_bytes: 0
>> tx3_csum_partial: 1415739849
>> tx3_csum_partial_inner: 0
>> tx3_added_vlan_packets: 2842823811
>> tx3_nop: 78060813
>> tx3_csum_none: 1427083971
>> tx3_stopped: 0
>> tx3_dropped: 0
>> tx3_xmit_more: 28575040
>> tx3_recover: 0
>> tx3_cqes: 2814250785
>> tx3_wake: 0
>> tx3_cqe_err: 0
>> tx4_packets: 5508297397
>> tx4_bytes: 7127659369902
>> tx4_tso_packets: 1007356432
>> tx4_tso_bytes: 5145975736034
>> tx4_tso_inner_packets: 0
>> tx4_tso_inner_bytes: 0
>> tx4_csum_partial: 1411271000
>> tx4_csum_partial_inner: 0
>> tx4_added_vlan_packets: 2882086825
>> tx4_nop: 78433610
>> tx4_csum_none: 1470815825
>> tx4_stopped: 0
>> tx4_dropped: 0
>> tx4_xmit_more: 28632444
>> tx4_recover: 0
>> tx4_cqes: 2853456464
>> tx4_wake: 0
>> tx4_cqe_err: 0
>> tx5_packets: 5513864156
>> tx5_bytes: 7165864145517
>> tx5_tso_packets: 1014046485
>> tx5_tso_bytes: 5192635614477
>> tx5_tso_inner_packets: 0
>> tx5_tso_inner_bytes: 0
>> tx5_csum_partial: 1420810473
>> tx5_csum_partial_inner: 0
>> tx5_added_vlan_packets: 2861370556
>> tx5_nop: 78481355
>> tx5_csum_none: 1440560083
>> tx5_stopped: 0
>> tx5_dropped: 0
>> tx5_xmit_more: 28222467
>> tx5_recover: 0
>> tx5_cqes: 2833149758
>> tx5_wake: 0
>> tx5_cqe_err: 0
>> tx6_packets: 5560724761
>> tx6_bytes: 7210309972086
>> tx6_tso_packets: 994050514
>> tx6_tso_bytes: 5171393741595
>> tx6_tso_inner_packets: 0
>> tx6_tso_inner_bytes: 0
>> tx6_csum_partial: 1414303265
>> tx6_csum_partial_inner: 0
>> tx6_added_vlan_packets: 2905794177
>> tx6_nop: 79353318
>> tx6_csum_none: 1491490912
>> tx6_stopped: 0
>> tx6_dropped: 0
>> tx6_xmit_more: 31246664
>> tx6_recover: 0
>> tx6_cqes: 2874549217
>> tx6_wake: 0
>> tx6_cqe_err: 0
>> tx7_packets: 5557594170
>> tx7_bytes: 7223138778685
>> tx7_tso_packets: 1013475396
>> tx7_tso_bytes: 5241530065484
>> tx7_tso_inner_packets: 0
>> tx7_tso_inner_bytes: 0
>> tx7_csum_partial: 1438604314
>> tx7_csum_partial_inner: 0
>> tx7_added_vlan_packets: 2873917552
>> tx7_nop: 79057059
>> tx7_csum_none: 1435313239
>> tx7_stopped: 0
>> tx7_dropped: 0
>> tx7_xmit_more: 29258761
>> tx7_recover: 0
>> tx7_cqes: 2844660578
>> tx7_wake: 0
>> tx7_cqe_err: 0
>> tx8_packets: 5521254733
>> tx8_bytes: 7208043146297
>> tx8_tso_packets: 1014670801
>> tx8_tso_bytes: 5185842447246
>> tx8_tso_inner_packets: 0
>> tx8_tso_inner_bytes: 0
>> tx8_csum_partial: 1431631562
>> tx8_csum_partial_inner: 0
>> tx8_added_vlan_packets: 2872641129
>> tx8_nop: 78545776
>> tx8_csum_none: 1441009567
>> tx8_stopped: 0
>> tx8_dropped: 0
>> tx8_xmit_more: 29106291
>> tx8_recover: 0
>> tx8_cqes: 2843536748
>> tx8_wake: 0
>> tx8_cqe_err: 0
>> tx9_packets: 5528889957
>> tx9_bytes: 7191793816058
>> tx9_tso_packets: 1015955476
>> tx9_tso_bytes: 5207232047828
>> tx9_tso_inner_packets: 0
>> tx9_tso_inner_bytes: 0
>> tx9_csum_partial: 1421266796
>> tx9_csum_partial_inner: 0
>> tx9_added_vlan_packets: 2869523921
>> tx9_nop: 78586218
>> tx9_csum_none: 1448257125
>> tx9_stopped: 0
>> tx9_dropped: 0
>> tx9_xmit_more: 29483347
>> tx9_recover: 0
>> tx9_cqes: 2840042245
>> tx9_wake: 0
>> tx9_cqe_err: 0
>> tx10_packets: 5556351222
>> tx10_bytes: 7254798330757
>> tx10_tso_packets: 1028554460
>> tx10_tso_bytes: 5246179615774
>> tx10_tso_inner_packets: 0
>> tx10_tso_inner_bytes: 0
>> tx10_csum_partial: 1430459021
>> tx10_csum_partial_inner: 0
>> tx10_added_vlan_packets: 2881683382
>> tx10_nop: 79139584
>> tx10_csum_none: 1451224361
>> tx10_stopped: 0
>> tx10_dropped: 0
>> tx10_xmit_more: 29217190
>> tx10_recover: 0
>> tx10_cqes: 2852467898
>> tx10_wake: 0
>> tx10_cqe_err: 0
>> tx11_packets: 5455631854
>> tx11_bytes: 7061121713772
>> tx11_tso_packets: 992133383
>> tx11_tso_bytes: 5089419722682
>> tx11_tso_inner_packets: 0
>> tx11_tso_inner_bytes: 0
>> tx11_csum_partial: 1395542033
>> tx11_csum_partial_inner: 0
>> tx11_added_vlan_packets: 2852589093
>> tx11_nop: 77799857
>> tx11_csum_none: 1457047060
>> tx11_stopped: 0
>> tx11_dropped: 0
>> tx11_xmit_more: 29559927
>> tx11_recover: 0
>> tx11_cqes: 2823031110
>> tx11_wake: 0
>> tx11_cqe_err: 0
>> tx12_packets: 5488286808
>> tx12_bytes: 7137087569303
>> tx12_tso_packets: 1006435537
>> tx12_tso_bytes: 5163371416750
>> tx12_tso_inner_packets: 0
>> tx12_tso_inner_bytes: 0
>> tx12_csum_partial: 1414799411
>> tx12_csum_partial_inner: 0
>> tx12_added_vlan_packets: 2841679543
>> tx12_nop: 78387039
>> tx12_csum_none: 1426880132
>> tx12_stopped: 0
>> tx12_dropped: 0
>> tx12_xmit_more: 28607526
>> tx12_recover: 0
>> tx12_cqes: 2813073557
>> tx12_wake: 0
>> tx12_cqe_err: 0
>> tx13_packets: 5594132290
>> tx13_bytes: 7251106284829
>> tx13_tso_packets: 1035172061
>> tx13_tso_bytes: 5251200286298
>> tx13_tso_inner_packets: 0
>> tx13_tso_inner_bytes: 0
>> tx13_csum_partial: 1443665981
>> tx13_csum_partial_inner: 0
>> tx13_added_vlan_packets: 2916604799
>> tx13_nop: 79670465
>> tx13_csum_none: 1472938818
>> tx13_stopped: 0
>> tx13_dropped: 0
>> tx13_xmit_more: 27797067
>> tx13_recover: 0
>> tx13_cqes: 2888809352
>> tx13_wake: 0
>> tx13_cqe_err: 0
>> tx14_packets: 5548790952
>> tx14_bytes: 7194211868411
>> tx14_tso_packets: 1021015561
>> tx14_tso_bytes: 5231483708869
>> tx14_tso_inner_packets: 0
>> tx14_tso_inner_bytes: 0
>> tx14_csum_partial: 1427711576
>> tx14_csum_partial_inner: 0
>> tx14_added_vlan_packets: 2875288572
>> tx14_nop: 78900224
>> tx14_csum_none: 1447576996
>> tx14_stopped: 0
>> tx14_dropped: 0
>> tx14_xmit_more: 30003496
>> tx14_recover: 0
>> tx14_cqes: 2845286732
>> tx14_wake: 0
>> tx14_cqe_err: 0
>> tx15_packets: 5609310963
>> tx15_bytes: 7271380831798
>> tx15_tso_packets: 1027830118
>> tx15_tso_bytes: 5229697431506
>> tx15_tso_inner_packets: 0
>> tx15_tso_inner_bytes: 0
>> tx15_csum_partial: 1429209941
>> tx15_csum_partial_inner: 0
>> tx15_added_vlan_packets: 2940315402
>> tx15_nop: 79950883
>> tx15_csum_none: 1511105462
>> tx15_stopped: 0
>> tx15_dropped: 0
>> tx15_xmit_more: 28820740
>> tx15_recover: 0
>> tx15_cqes: 2911496633
>> tx15_wake: 0
>> tx15_cqe_err: 0
>> tx16_packets: 4465363036
>> tx16_bytes: 5769771803704
>> tx16_tso_packets: 817101913
>> tx16_tso_bytes: 4180172833814
>> tx16_tso_inner_packets: 0
>> tx16_tso_inner_bytes: 0
>> tx16_csum_partial: 1136731404
>> tx16_csum_partial_inner: 0
>> tx16_added_vlan_packets: 2332178232
>> tx16_nop: 63458573
>> tx16_csum_none: 1195446828
>> tx16_stopped: 0
>> tx16_dropped: 0
>> tx16_xmit_more: 23756254
>> tx16_recover: 0
>> tx16_cqes: 2308423025
>> tx16_wake: 0
>> tx16_cqe_err: 0
>> tx17_packets: 4380386348
>> tx17_bytes: 5708702994526
>> tx17_tso_packets: 813638023
>> tx17_tso_bytes: 4130806014947
>> tx17_tso_inner_packets: 0
>> tx17_tso_inner_bytes: 0
>> tx17_csum_partial: 1133007164
>> tx17_csum_partial_inner: 0
>> tx17_added_vlan_packets: 2277314787
>> tx17_nop: 62377372
>> tx17_csum_none: 1144307623
>> tx17_stopped: 0
>> tx17_dropped: 0
>> tx17_xmit_more: 23731361
>> tx17_recover: 0
>> tx17_cqes: 2253584638
>> tx17_wake: 0
>> tx17_cqe_err: 0
>> tx18_packets: 4450359743
>> tx18_bytes: 5758968674820
>> tx18_tso_packets: 815791601
>> tx18_tso_bytes: 4179942688909
>> tx18_tso_inner_packets: 0
>> tx18_tso_inner_bytes: 0
>> tx18_csum_partial: 1137649257
>> tx18_csum_partial_inner: 0
>> tx18_added_vlan_packets: 2314556550
>> tx18_nop: 63271085
>> tx18_csum_none: 1176907293
>> tx18_stopped: 0
>> tx18_dropped: 0
>> tx18_xmit_more: 23055770
>> tx18_recover: 0
>> tx18_cqes: 2291501928
>> tx18_wake: 0
>> tx18_cqe_err: 0
>> tx19_packets: 4596064378
>> tx19_bytes: 5916675706535
>> tx19_tso_packets: 825788649
>> tx19_tso_bytes: 4208046929921
>> tx19_tso_inner_packets: 0
>> tx19_tso_inner_bytes: 0
>> tx19_csum_partial: 1150666569
>> tx19_csum_partial_inner: 0
>> tx19_added_vlan_packets: 2450567026
>> tx19_nop: 65468504
>> tx19_csum_none: 1299900457
>> tx19_stopped: 0
>> tx19_dropped: 0
>> tx19_xmit_more: 23846250
>> tx19_recover: 0
>> tx19_cqes: 2426722127
>> tx19_wake: 0
>> tx19_cqe_err: 0
>> tx20_packets: 4424935388
>> tx20_bytes: 5757631205901
>> tx20_tso_packets: 804875006
>> tx20_tso_bytes: 4156262736109
>> tx20_tso_inner_packets: 0
>> tx20_tso_inner_bytes: 0
>> tx20_csum_partial: 1134144916
>> tx20_csum_partial_inner: 0
>> tx20_added_vlan_packets: 2294839665
>> tx20_nop: 63023986
>> tx20_csum_none: 1160694749
>> tx20_stopped: 0
>> tx20_dropped: 0
>> tx20_xmit_more: 23393201
>> tx20_recover: 0
>> tx20_cqes: 2271447623
>> tx20_wake: 0
>> tx20_cqe_err: 0
>> tx21_packets: 4595062285
>> tx21_bytes: 5958671993467
>> tx21_tso_packets: 821936215
>> tx21_tso_bytes: 4187977870684
>> tx21_tso_inner_packets: 0
>> tx21_tso_inner_bytes: 0
>> tx21_csum_partial: 1143339787
>> tx21_csum_partial_inner: 0
>> tx21_added_vlan_packets: 2457167412
>> tx21_nop: 65697763
>> tx21_csum_none: 1313827625
>> tx21_stopped: 0
>> tx21_dropped: 0
>> tx21_xmit_more: 23858345
>> tx21_recover: 0
>> tx21_cqes: 2433310348
>> tx21_wake: 0
>> tx21_cqe_err: 0
>> tx22_packets: 4664446513
>> tx22_bytes: 5931429292082
>> tx22_tso_packets: 814457881
>> tx22_tso_bytes: 4148607956533
>> tx22_tso_inner_packets: 0
>> tx22_tso_inner_bytes: 0
>> tx22_csum_partial: 1127284783
>> tx22_csum_partial_inner: 0
>> tx22_added_vlan_packets: 2548650146
>> tx22_nop: 66299909
>> tx22_csum_none: 1421365363
>> tx22_stopped: 0
>> tx22_dropped: 0
>> tx22_xmit_more: 23800911
>> tx22_recover: 0
>> tx22_cqes: 2524850415
>> tx22_wake: 0
>> tx22_cqe_err: 0
>> tx23_packets: 4416221747
>> tx23_bytes: 5721472587985
>> tx23_tso_packets: 823538520
>> tx23_tso_bytes: 4163520218617
>> tx23_tso_inner_packets: 0
>> tx23_tso_inner_bytes: 0
>> tx23_csum_partial: 1135996006
>> tx23_csum_partial_inner: 0
>> tx23_added_vlan_packets: 2292404120
>> tx23_nop: 62709432
>> tx23_csum_none: 1156408114
>> tx23_stopped: 0
>> tx23_dropped: 0
>> tx23_xmit_more: 22299889
>> tx23_recover: 0
>> tx23_cqes: 2270105487
>> tx23_wake: 0
>> tx23_cqe_err: 0
>> tx24_packets: 4420014824
>> tx24_bytes: 5740767318521
>> tx24_tso_packets: 820838072
>> tx24_tso_bytes: 4183722948422
>> tx24_tso_inner_packets: 0
>> tx24_tso_inner_bytes: 0
>> tx24_csum_partial: 1138070059
>> tx24_csum_partial_inner: 0
>> tx24_added_vlan_packets: 2289043946
>> tx24_nop: 62797341
>> tx24_csum_none: 1150973887
>> tx24_stopped: 0
>> tx24_dropped: 0
>> tx24_xmit_more: 22744690
>> tx24_recover: 0
>> tx24_cqes: 2266300568
>> tx24_wake: 0
>> tx24_cqe_err: 0
>> tx25_packets: 4413225545
>> tx25_bytes: 5716162617155
>> tx25_tso_packets: 808274341
>> tx25_tso_bytes: 4138408857714
>> tx25_tso_inner_packets: 0
>> tx25_tso_inner_bytes: 0
>> tx25_csum_partial: 1134587898
>> tx25_csum_partial_inner: 0
>> tx25_added_vlan_packets: 2297149310
>> tx25_nop: 62958238
>> tx25_csum_none: 1162561412
>> tx25_stopped: 0
>> tx25_dropped: 0
>> tx25_xmit_more: 24463552
>> tx25_recover: 0
>> tx25_cqes: 2272686971
>> tx25_wake: 0
>> tx25_cqe_err: 0
>> tx26_packets: 4524907591
>> tx26_bytes: 5865394280699
>> tx26_tso_packets: 807270022
>> tx26_tso_bytes: 4148754705317
>> tx26_tso_inner_packets: 0
>> tx26_tso_inner_bytes: 0
>> tx26_csum_partial: 1130306933
>> tx26_csum_partial_inner: 0
>> tx26_added_vlan_packets: 2402682460
>> tx26_nop: 64474322
>> tx26_csum_none: 1272375527
>> tx26_stopped: 1
>> tx26_dropped: 0
>> tx26_xmit_more: 23316186
>> tx26_recover: 0
>> tx26_cqes: 2379367502
>> tx26_wake: 1
>> tx26_cqe_err: 0
>> tx27_packets: 4376114969
>> tx27_bytes: 5683551238304
>> tx27_tso_packets: 809344829
>> tx27_tso_bytes: 4124331859270
>> tx27_tso_inner_packets: 0
>> tx27_tso_inner_bytes: 0
>> tx27_csum_partial: 1124954937
>> tx27_csum_partial_inner: 0
>> tx27_added_vlan_packets: 2267871300
>> tx27_nop: 62213214
>> tx27_csum_none: 1142916363
>> tx27_stopped: 0
>> tx27_dropped: 0
>> tx27_xmit_more: 23369974
>> tx27_recover: 0
>> tx27_cqes: 2244502686
>> tx27_wake: 0
>> tx27_cqe_err: 0
>> tx28_packets: 3
>> tx28_bytes: 266
>> tx28_tso_packets: 0
>> tx28_tso_bytes: 0
>> tx28_tso_inner_packets: 0
>> tx28_tso_inner_bytes: 0
>> tx28_csum_partial: 0
>> tx28_csum_partial_inner: 0
>> tx28_added_vlan_packets: 0
>> tx28_nop: 0
>> tx28_csum_none: 3
>> tx28_stopped: 0
>> tx28_dropped: 0
>> tx28_xmit_more: 0
>> tx28_recover: 0
>> tx28_cqes: 3
>> tx28_wake: 0
>> tx28_cqe_err: 0
>> tx29_packets: 0
>> tx29_bytes: 0
>> tx29_tso_packets: 0
>> tx29_tso_bytes: 0
>> tx29_tso_inner_packets: 0
>> tx29_tso_inner_bytes: 0
>> tx29_csum_partial: 0
>> tx29_csum_partial_inner: 0
>> tx29_added_vlan_packets: 0
>> tx29_nop: 0
>> tx29_csum_none: 0
>> tx29_stopped: 0
>> tx29_dropped: 0
>> tx29_xmit_more: 0
>> tx29_recover: 0
>> tx29_cqes: 0
>> tx29_wake: 0
>> tx29_cqe_err: 0
>> tx30_packets: 0
>> tx30_bytes: 0
>> tx30_tso_packets: 0
>> tx30_tso_bytes: 0
>> tx30_tso_inner_packets: 0
>> tx30_tso_inner_bytes: 0
>> tx30_csum_partial: 0
>> tx30_csum_partial_inner: 0
>> tx30_added_vlan_packets: 0
>> tx30_nop: 0
>> tx30_csum_none: 0
>> tx30_stopped: 0
>> tx30_dropped: 0
>> tx30_xmit_more: 0
>> tx30_recover: 0
>> tx30_cqes: 0
>> tx30_wake: 0
>> tx30_cqe_err: 0
>> tx31_packets: 0
>> tx31_bytes: 0
>> tx31_tso_packets: 0
>> tx31_tso_bytes: 0
>> tx31_tso_inner_packets: 0
>> tx31_tso_inner_bytes: 0
>> tx31_csum_partial: 0
>> tx31_csum_partial_inner: 0
>> tx31_added_vlan_packets: 0
>> tx31_nop: 0
>> tx31_csum_none: 0
>> tx31_stopped: 0
>> tx31_dropped: 0
>> tx31_xmit_more: 0
>> tx31_recover: 0
>> tx31_cqes: 0
>> tx31_wake: 0
>> tx31_cqe_err: 0
>> tx32_packets: 0
>> tx32_bytes: 0
>> tx32_tso_packets: 0
>> tx32_tso_bytes: 0
>> tx32_tso_inner_packets: 0
>> tx32_tso_inner_bytes: 0
>> tx32_csum_partial: 0
>> tx32_csum_partial_inner: 0
>> tx32_added_vlan_packets: 0
>> tx32_nop: 0
>> tx32_csum_none: 0
>> tx32_stopped: 0
>> tx32_dropped: 0
>> tx32_xmit_more: 0
>> tx32_recover: 0
>> tx32_cqes: 0
>> tx32_wake: 0
>> tx32_cqe_err: 0
>> tx33_packets: 0
>> tx33_bytes: 0
>> tx33_tso_packets: 0
>> tx33_tso_bytes: 0
>> tx33_tso_inner_packets: 0
>> tx33_tso_inner_bytes: 0
>> tx33_csum_partial: 0
>> tx33_csum_partial_inner: 0
>> tx33_added_vlan_packets: 0
>> tx33_nop: 0
>> tx33_csum_none: 0
>> tx33_stopped: 0
>> tx33_dropped: 0
>> tx33_xmit_more: 0
>> tx33_recover: 0
>> tx33_cqes: 0
>> tx33_wake: 0
>> tx33_cqe_err: 0
>> tx34_packets: 0
>> tx34_bytes: 0
>> tx34_tso_packets: 0
>> tx34_tso_bytes: 0
>> tx34_tso_inner_packets: 0
>> tx34_tso_inner_bytes: 0
>> tx34_csum_partial: 0
>> tx34_csum_partial_inner: 0
>> tx34_added_vlan_packets: 0
>> tx34_nop: 0
>> tx34_csum_none: 0
>> tx34_stopped: 0
>> tx34_dropped: 0
>> tx34_xmit_more: 0
>> tx34_recover: 0
>> tx34_cqes: 0
>> tx34_wake: 0
>> tx34_cqe_err: 0
>> tx35_packets: 0
>> tx35_bytes: 0
>> tx35_tso_packets: 0
>> tx35_tso_bytes: 0
>> tx35_tso_inner_packets: 0
>> tx35_tso_inner_bytes: 0
>> tx35_csum_partial: 0
>> tx35_csum_partial_inner: 0
>> tx35_added_vlan_packets: 0
>> tx35_nop: 0
>> tx35_csum_none: 0
>> tx35_stopped: 0
>> tx35_dropped: 0
>> tx35_xmit_more: 0
>> tx35_recover: 0
>> tx35_cqes: 0
>> tx35_wake: 0
>> tx35_cqe_err: 0
>> tx36_packets: 0
>> tx36_bytes: 0
>> tx36_tso_packets: 0
>> tx36_tso_bytes: 0
>> tx36_tso_inner_packets: 0
>> tx36_tso_inner_bytes: 0
>> tx36_csum_partial: 0
>> tx36_csum_partial_inner: 0
>> tx36_added_vlan_packets: 0
>> tx36_nop: 0
>> tx36_csum_none: 0
>> tx36_stopped: 0
>> tx36_dropped: 0
>> tx36_xmit_more: 0
>> tx36_recover: 0
>> tx36_cqes: 0
>> tx36_wake: 0
>> tx36_cqe_err: 0
>> tx37_packets: 0
>> tx37_bytes: 0
>> tx37_tso_packets: 0
>> tx37_tso_bytes: 0
>> tx37_tso_inner_packets: 0
>> tx37_tso_inner_bytes: 0
>> tx37_csum_partial: 0
>> tx37_csum_partial_inner: 0
>> tx37_added_vlan_packets: 0
>> tx37_nop: 0
>> tx37_csum_none: 0
>> tx37_stopped: 0
>> tx37_dropped: 0
>> tx37_xmit_more: 0
>> tx37_recover: 0
>> tx37_cqes: 0
>> tx37_wake: 0
>> tx37_cqe_err: 0
>> tx38_packets: 0
>> tx38_bytes: 0
>> tx38_tso_packets: 0
>> tx38_tso_bytes: 0
>> tx38_tso_inner_packets: 0
>> tx38_tso_inner_bytes: 0
>> tx38_csum_partial: 0
>> tx38_csum_partial_inner: 0
>> tx38_added_vlan_packets: 0
>> tx38_nop: 0
>> tx38_csum_none: 0
>> tx38_stopped: 0
>> tx38_dropped: 0
>> tx38_xmit_more: 0
>> tx38_recover: 0
>> tx38_cqes: 0
>> tx38_wake: 0
>> tx38_cqe_err: 0
>> tx39_packets: 0
>> tx39_bytes: 0
>> tx39_tso_packets: 0
>> tx39_tso_bytes: 0
>> tx39_tso_inner_packets: 0
>> tx39_tso_inner_bytes: 0
>> tx39_csum_partial: 0
>> tx39_csum_partial_inner: 0
>> tx39_added_vlan_packets: 0
>> tx39_nop: 0
>> tx39_csum_none: 0
>> tx39_stopped: 0
>> tx39_dropped: 0
>> tx39_xmit_more: 0
>> tx39_recover: 0
>> tx39_cqes: 0
>> tx39_wake: 0
>> tx39_cqe_err: 0
>> tx40_packets: 0
>> tx40_bytes: 0
>> tx40_tso_packets: 0
>> tx40_tso_bytes: 0
>> tx40_tso_inner_packets: 0
>> tx40_tso_inner_bytes: 0
>> tx40_csum_partial: 0
>> tx40_csum_partial_inner: 0
>> tx40_added_vlan_packets: 0
>> tx40_nop: 0
>> tx40_csum_none: 0
>> tx40_stopped: 0
>> tx40_dropped: 0
>> tx40_xmit_more: 0
>> tx40_recover: 0
>> tx40_cqes: 0
>> tx40_wake: 0
>> tx40_cqe_err: 0
>> tx41_packets: 0
>> tx41_bytes: 0
>> tx41_tso_packets: 0
>> tx41_tso_bytes: 0
>> tx41_tso_inner_packets: 0
>> tx41_tso_inner_bytes: 0
>> tx41_csum_partial: 0
>> tx41_csum_partial_inner: 0
>> tx41_added_vlan_packets: 0
>> tx41_nop: 0
>> tx41_csum_none: 0
>> tx41_stopped: 0
>> tx41_dropped: 0
>> tx41_xmit_more: 0
>> tx41_recover: 0
>> tx41_cqes: 0
>> tx41_wake: 0
>> tx41_cqe_err: 0
>> tx42_packets: 0
>> tx42_bytes: 0
>> tx42_tso_packets: 0
>> tx42_tso_bytes: 0
>> tx42_tso_inner_packets: 0
>> tx42_tso_inner_bytes: 0
>> tx42_csum_partial: 0
>> tx42_csum_partial_inner: 0
>> tx42_added_vlan_packets: 0
>> tx42_nop: 0
>> tx42_csum_none: 0
>> tx42_stopped: 0
>> tx42_dropped: 0
>> tx42_xmit_more: 0
>> tx42_recover: 0
>> tx42_cqes: 0
>> tx42_wake: 0
>> tx42_cqe_err: 0
>> tx43_packets: 0
>> tx43_bytes: 0
>> tx43_tso_packets: 0
>> tx43_tso_bytes: 0
>> tx43_tso_inner_packets: 0
>> tx43_tso_inner_bytes: 0
>> tx43_csum_partial: 0
>> tx43_csum_partial_inner: 0
>> tx43_added_vlan_packets: 0
>> tx43_nop: 0
>> tx43_csum_none: 0
>> tx43_stopped: 0
>> tx43_dropped: 0
>> tx43_xmit_more: 0
>> tx43_recover: 0
>> tx43_cqes: 0
>> tx43_wake: 0
>> tx43_cqe_err: 0
>> tx44_packets: 0
>> tx44_bytes: 0
>> tx44_tso_packets: 0
>> tx44_tso_bytes: 0
>> tx44_tso_inner_packets: 0
>> tx44_tso_inner_bytes: 0
>> tx44_csum_partial: 0
>> tx44_csum_partial_inner: 0
>> tx44_added_vlan_packets: 0
>> tx44_nop: 0
>> tx44_csum_none: 0
>> tx44_stopped: 0
>> tx44_dropped: 0
>> tx44_xmit_more: 0
>> tx44_recover: 0
>> tx44_cqes: 0
>> tx44_wake: 0
>> tx44_cqe_err: 0
>> tx45_packets: 0
>> tx45_bytes: 0
>> tx45_tso_packets: 0
>> tx45_tso_bytes: 0
>> tx45_tso_inner_packets: 0
>> tx45_tso_inner_bytes: 0
>> tx45_csum_partial: 0
>> tx45_csum_partial_inner: 0
>> tx45_added_vlan_packets: 0
>> tx45_nop: 0
>> tx45_csum_none: 0
>> tx45_stopped: 0
>> tx45_dropped: 0
>> tx45_xmit_more: 0
>> tx45_recover: 0
>> tx45_cqes: 0
>> tx45_wake: 0
>> tx45_cqe_err: 0
>> tx46_packets: 0
>> tx46_bytes: 0
>> tx46_tso_packets: 0
>> tx46_tso_bytes: 0
>> tx46_tso_inner_packets: 0
>> tx46_tso_inner_bytes: 0
>> tx46_csum_partial: 0
>> tx46_csum_partial_inner: 0
>> tx46_added_vlan_packets: 0
>> tx46_nop: 0
>> tx46_csum_none: 0
>> tx46_stopped: 0
>> tx46_dropped: 0
>> tx46_xmit_more: 0
>> tx46_recover: 0
>> tx46_cqes: 0
>> tx46_wake: 0
>> tx46_cqe_err: 0
>> tx47_packets: 0
>> tx47_bytes: 0
>> tx47_tso_packets: 0
>> tx47_tso_bytes: 0
>> tx47_tso_inner_packets: 0
>> tx47_tso_inner_bytes: 0
>> tx47_csum_partial: 0
>> tx47_csum_partial_inner: 0
>> tx47_added_vlan_packets: 0
>> tx47_nop: 0
>> tx47_csum_none: 0
>> tx47_stopped: 0
>> tx47_dropped: 0
>> tx47_xmit_more: 0
>> tx47_recover: 0
>> tx47_cqes: 0
>> tx47_wake: 0
>> tx47_cqe_err: 0
>> tx48_packets: 0
>> tx48_bytes: 0
>> tx48_tso_packets: 0
>> tx48_tso_bytes: 0
>> tx48_tso_inner_packets: 0
>> tx48_tso_inner_bytes: 0
>> tx48_csum_partial: 0
>> tx48_csum_partial_inner: 0
>> tx48_added_vlan_packets: 0
>> tx48_nop: 0
>> tx48_csum_none: 0
>> tx48_stopped: 0
>> tx48_dropped: 0
>> tx48_xmit_more: 0
>> tx48_recover: 0
>> tx48_cqes: 0
>> tx48_wake: 0
>> tx48_cqe_err: 0
>> tx49_packets: 0
>> tx49_bytes: 0
>> tx49_tso_packets: 0
>> tx49_tso_bytes: 0
>> tx49_tso_inner_packets: 0
>> tx49_tso_inner_bytes: 0
>> tx49_csum_partial: 0
>> tx49_csum_partial_inner: 0
>> tx49_added_vlan_packets: 0
>> tx49_nop: 0
>> tx49_csum_none: 0
>> tx49_stopped: 0
>> tx49_dropped: 0
>> tx49_xmit_more: 0
>> tx49_recover: 0
>> tx49_cqes: 0
>> tx49_wake: 0
>> tx49_cqe_err: 0
>> tx50_packets: 0
>> tx50_bytes: 0
>> tx50_tso_packets: 0
>> tx50_tso_bytes: 0
>> tx50_tso_inner_packets: 0
>> tx50_tso_inner_bytes: 0
>> tx50_csum_partial: 0
>> tx50_csum_partial_inner: 0
>> tx50_added_vlan_packets: 0
>> tx50_nop: 0
>> tx50_csum_none: 0
>> tx50_stopped: 0
>> tx50_dropped: 0
>> tx50_xmit_more: 0
>> tx50_recover: 0
>> tx50_cqes: 0
>> tx50_wake: 0
>> tx50_cqe_err: 0
>> tx51_packets: 0
>> tx51_bytes: 0
>> tx51_tso_packets: 0
>> tx51_tso_bytes: 0
>> tx51_tso_inner_packets: 0
>> tx51_tso_inner_bytes: 0
>> tx51_csum_partial: 0
>> tx51_csum_partial_inner: 0
>> tx51_added_vlan_packets: 0
>> tx51_nop: 0
>> tx51_csum_none: 0
>> tx51_stopped: 0
>> tx51_dropped: 0
>> tx51_xmit_more: 0
>> tx51_recover: 0
>> tx51_cqes: 0
>> tx51_wake: 0
>> tx51_cqe_err: 0
>> tx52_packets: 0
>> tx52_bytes: 0
>> tx52_tso_packets: 0
>> tx52_tso_bytes: 0
>> tx52_tso_inner_packets: 0
>> tx52_tso_inner_bytes: 0
>> tx52_csum_partial: 0
>> tx52_csum_partial_inner: 0
>> tx52_added_vlan_packets: 0
>> tx52_nop: 0
>> tx52_csum_none: 0
>> tx52_stopped: 0
>> tx52_dropped: 0
>> tx52_xmit_more: 0
>> tx52_recover: 0
>> tx52_cqes: 0
>> tx52_wake: 0
>> tx52_cqe_err: 0
>> tx53_packets: 0
>> tx53_bytes: 0
>> tx53_tso_packets: 0
>> tx53_tso_bytes: 0
>> tx53_tso_inner_packets: 0
>> tx53_tso_inner_bytes: 0
>> tx53_csum_partial: 0
>> tx53_csum_partial_inner: 0
>> tx53_added_vlan_packets: 0
>> tx53_nop: 0
>> tx53_csum_none: 0
>> tx53_stopped: 0
>> tx53_dropped: 0
>> tx53_xmit_more: 0
>> tx53_recover: 0
>> tx53_cqes: 0
>> tx53_wake: 0
>> tx53_cqe_err: 0
>> tx54_packets: 0
>> tx54_bytes: 0
>> tx54_tso_packets: 0
>> tx54_tso_bytes: 0
>> tx54_tso_inner_packets: 0
>> tx54_tso_inner_bytes: 0
>> tx54_csum_partial: 0
>> tx54_csum_partial_inner: 0
>> tx54_added_vlan_packets: 0
>> tx54_nop: 0
>> tx54_csum_none: 0
>> tx54_stopped: 0
>> tx54_dropped: 0
>> tx54_xmit_more: 0
>> tx54_recover: 0
>> tx54_cqes: 0
>> tx54_wake: 0
>> tx54_cqe_err: 0
>> tx55_packets: 0
>> tx55_bytes: 0
>> tx55_tso_packets: 0
>> tx55_tso_bytes: 0
>> tx55_tso_inner_packets: 0
>> tx55_tso_inner_bytes: 0
>> tx55_csum_partial: 0
>> tx55_csum_partial_inner: 0
>> tx55_added_vlan_packets: 0
>> tx55_nop: 0
>> tx55_csum_none: 0
>> tx55_stopped: 0
>> tx55_dropped: 0
>> tx55_xmit_more: 0
>> tx55_recover: 0
>> tx55_cqes: 0
>> tx55_wake: 0
>> tx55_cqe_err: 0
>> tx0_xdp_xmit: 0
>> tx0_xdp_full: 0
>> tx0_xdp_err: 0
>> tx0_xdp_cqes: 0
>> tx1_xdp_xmit: 0
>> tx1_xdp_full: 0
>> tx1_xdp_err: 0
>> tx1_xdp_cqes: 0
>> tx2_xdp_xmit: 0
>> tx2_xdp_full: 0
>> tx2_xdp_err: 0
>> tx2_xdp_cqes: 0
>> tx3_xdp_xmit: 0
>> tx3_xdp_full: 0
>> tx3_xdp_err: 0
>> tx3_xdp_cqes: 0
>> tx4_xdp_xmit: 0
>> tx4_xdp_full: 0
>> tx4_xdp_err: 0
>> tx4_xdp_cqes: 0
>> tx5_xdp_xmit: 0
>> tx5_xdp_full: 0
>> tx5_xdp_err: 0
>> tx5_xdp_cqes: 0
>> tx6_xdp_xmit: 0
>> tx6_xdp_full: 0
>> tx6_xdp_err: 0
>> tx6_xdp_cqes: 0
>> tx7_xdp_xmit: 0
>> tx7_xdp_full: 0
>> tx7_xdp_err: 0
>> tx7_xdp_cqes: 0
>> tx8_xdp_xmit: 0
>> tx8_xdp_full: 0
>> tx8_xdp_err: 0
>> tx8_xdp_cqes: 0
>> tx9_xdp_xmit: 0
>> tx9_xdp_full: 0
>> tx9_xdp_err: 0
>> tx9_xdp_cqes: 0
>> tx10_xdp_xmit: 0
>> tx10_xdp_full: 0
>> tx10_xdp_err: 0
>> tx10_xdp_cqes: 0
>> tx11_xdp_xmit: 0
>> tx11_xdp_full: 0
>> tx11_xdp_err: 0
>> tx11_xdp_cqes: 0
>> tx12_xdp_xmit: 0
>> tx12_xdp_full: 0
>> tx12_xdp_err: 0
>> tx12_xdp_cqes: 0
>> tx13_xdp_xmit: 0
>> tx13_xdp_full: 0
>> tx13_xdp_err: 0
>> tx13_xdp_cqes: 0
>> tx14_xdp_xmit: 0
>> tx14_xdp_full: 0
>> tx14_xdp_err: 0
>> tx14_xdp_cqes: 0
>> tx15_xdp_xmit: 0
>> tx15_xdp_full: 0
>> tx15_xdp_err: 0
>> tx15_xdp_cqes: 0
>> tx16_xdp_xmit: 0
>> tx16_xdp_full: 0
>> tx16_xdp_err: 0
>> tx16_xdp_cqes: 0
>> tx17_xdp_xmit: 0
>> tx17_xdp_full: 0
>> tx17_xdp_err: 0
>> tx17_xdp_cqes: 0
>> tx18_xdp_xmit: 0
>> tx18_xdp_full: 0
>> tx18_xdp_err: 0
>> tx18_xdp_cqes: 0
>> tx19_xdp_xmit: 0
>> tx19_xdp_full: 0
>> tx19_xdp_err: 0
>> tx19_xdp_cqes: 0
>> tx20_xdp_xmit: 0
>> tx20_xdp_full: 0
>> tx20_xdp_err: 0
>> tx20_xdp_cqes: 0
>> tx21_xdp_xmit: 0
>> tx21_xdp_full: 0
>> tx21_xdp_err: 0
>> tx21_xdp_cqes: 0
>> tx22_xdp_xmit: 0
>> tx22_xdp_full: 0
>> tx22_xdp_err: 0
>> tx22_xdp_cqes: 0
>> tx23_xdp_xmit: 0
>> tx23_xdp_full: 0
>> tx23_xdp_err: 0
>> tx23_xdp_cqes: 0
>> tx24_xdp_xmit: 0
>> tx24_xdp_full: 0
>> tx24_xdp_err: 0
>> tx24_xdp_cqes: 0
>> tx25_xdp_xmit: 0
>> tx25_xdp_full: 0
>> tx25_xdp_err: 0
>> tx25_xdp_cqes: 0
>> tx26_xdp_xmit: 0
>> tx26_xdp_full: 0
>> tx26_xdp_err: 0
>> tx26_xdp_cqes: 0
>> tx27_xdp_xmit: 0
>> tx27_xdp_full: 0
>> tx27_xdp_err: 0
>> tx27_xdp_cqes: 0
>> tx28_xdp_xmit: 0
>> tx28_xdp_full: 0
>> tx28_xdp_err: 0
>> tx28_xdp_cqes: 0
>> tx29_xdp_xmit: 0
>> tx29_xdp_full: 0
>> tx29_xdp_err: 0
>> tx29_xdp_cqes: 0
>> tx30_xdp_xmit: 0
>> tx30_xdp_full: 0
>> tx30_xdp_err: 0
>> tx30_xdp_cqes: 0
>> tx31_xdp_xmit: 0
>> tx31_xdp_full: 0
>> tx31_xdp_err: 0
>> tx31_xdp_cqes: 0
>> tx32_xdp_xmit: 0
>> tx32_xdp_full: 0
>> tx32_xdp_err: 0
>> tx32_xdp_cqes: 0
>> tx33_xdp_xmit: 0
>> tx33_xdp_full: 0
>> tx33_xdp_err: 0
>> tx33_xdp_cqes: 0
>> tx34_xdp_xmit: 0
>> tx34_xdp_full: 0
>> tx34_xdp_err: 0
>> tx34_xdp_cqes: 0
>> tx35_xdp_xmit: 0
>> tx35_xdp_full: 0
>> tx35_xdp_err: 0
>> tx35_xdp_cqes: 0
>> tx36_xdp_xmit: 0
>> tx36_xdp_full: 0
>> tx36_xdp_err: 0
>> tx36_xdp_cqes: 0
>> tx37_xdp_xmit: 0
>> tx37_xdp_full: 0
>> tx37_xdp_err: 0
>> tx37_xdp_cqes: 0
>> tx38_xdp_xmit: 0
>> tx38_xdp_full: 0
>> tx38_xdp_err: 0
>> tx38_xdp_cqes: 0
>> tx39_xdp_xmit: 0
>> tx39_xdp_full: 0
>> tx39_xdp_err: 0
>> tx39_xdp_cqes: 0
>> tx40_xdp_xmit: 0
>> tx40_xdp_full: 0
>> tx40_xdp_err: 0
>> tx40_xdp_cqes: 0
>> tx41_xdp_xmit: 0
>> tx41_xdp_full: 0
>> tx41_xdp_err: 0
>> tx41_xdp_cqes: 0
>> tx42_xdp_xmit: 0
>> tx42_xdp_full: 0
>> tx42_xdp_err: 0
>> tx42_xdp_cqes: 0
>> tx43_xdp_xmit: 0
>> tx43_xdp_full: 0
>> tx43_xdp_err: 0
>> tx43_xdp_cqes: 0
>> tx44_xdp_xmit: 0
>> tx44_xdp_full: 0
>> tx44_xdp_err: 0
>> tx44_xdp_cqes: 0
>> tx45_xdp_xmit: 0
>> tx45_xdp_full: 0
>> tx45_xdp_err: 0
>> tx45_xdp_cqes: 0
>> tx46_xdp_xmit: 0
>> tx46_xdp_full: 0
>> tx46_xdp_err: 0
>> tx46_xdp_cqes: 0
>> tx47_xdp_xmit: 0
>> tx47_xdp_full: 0
>> tx47_xdp_err: 0
>> tx47_xdp_cqes: 0
>> tx48_xdp_xmit: 0
>> tx48_xdp_full: 0
>> tx48_xdp_err: 0
>> tx48_xdp_cqes: 0
>> tx49_xdp_xmit: 0
>> tx49_xdp_full: 0
>> tx49_xdp_err: 0
>> tx49_xdp_cqes: 0
>> tx50_xdp_xmit: 0
>> tx50_xdp_full: 0
>> tx50_xdp_err: 0
>> tx50_xdp_cqes: 0
>> tx51_xdp_xmit: 0
>> tx51_xdp_full: 0
>> tx51_xdp_err: 0
>> tx51_xdp_cqes: 0
>> tx52_xdp_xmit: 0
>> tx52_xdp_full: 0
>> tx52_xdp_err: 0
>> tx52_xdp_cqes: 0
>> tx53_xdp_xmit: 0
>> tx53_xdp_full: 0
>> tx53_xdp_err: 0
>> tx53_xdp_cqes: 0
>> tx54_xdp_xmit: 0
>> tx54_xdp_full: 0
>> tx54_xdp_err: 0
>> tx54_xdp_cqes: 0
>> tx55_xdp_xmit: 0
>> tx55_xdp_full: 0
>> tx55_xdp_err: 0
>> tx55_xdp_cqes: 0
>>
>> ethtool -S enp175s0f0
>> NIC statistics:
>> rx_packets: 141574897253
>> rx_bytes: 184445040406258
>> tx_packets: 172569543894
>> tx_bytes: 99486882076365
>> tx_tso_packets: 9367664195
>> tx_tso_bytes: 56435233992948
>> tx_tso_inner_packets: 0
>> tx_tso_inner_bytes: 0
>> tx_added_vlan_packets: 141297671626
>> tx_nop: 2102916272
>> rx_lro_packets: 0
>> rx_lro_bytes: 0
>> rx_ecn_mark: 0
>> rx_removed_vlan_packets: 141574897252
>> rx_csum_unnecessary: 0
>> rx_csum_none: 23135854
>> rx_csum_complete: 141551761398
>> rx_csum_unnecessary_inner: 0
>> rx_xdp_drop: 0
>> rx_xdp_redirect: 0
>> rx_xdp_tx_xmit: 0
>> rx_xdp_tx_full: 0
>> rx_xdp_tx_err: 0
>> rx_xdp_tx_cqe: 0
>> tx_csum_none: 127934791664
> It is a good idea to look into this, tx is not requesting hw tx
> csumming for a lot of packets, maybe you are wasting a lot of cpu on
> calculating csum, or maybe this is just the rx csum complete..
>
>> tx_csum_partial: 13362879974
>> tx_csum_partial_inner: 0
>> tx_queue_stopped: 232561
> TX queues are stalling, could be an indentation for the pcie
> bottelneck.
>
>> tx_queue_dropped: 0
>> tx_xmit_more: 1266021946
>> tx_recover: 0
>> tx_cqes: 140031716469
>> tx_queue_wake: 232561
>> tx_udp_seg_rem: 0
>> tx_cqe_err: 0
>> tx_xdp_xmit: 0
>> tx_xdp_full: 0
>> tx_xdp_err: 0
>> tx_xdp_cqes: 0
>> rx_wqe_err: 0
>> rx_mpwqe_filler_cqes: 0
>> rx_mpwqe_filler_strides: 0
>> rx_buff_alloc_err: 0
>> rx_cqe_compress_blks: 0
>> rx_cqe_compress_pkts: 0
>> rx_page_reuse: 0
>> rx_cache_reuse: 16625975793
>> rx_cache_full: 54161465914
>> rx_cache_empty: 258048
>> rx_cache_busy: 54161472735
>> rx_cache_waive: 0
>> rx_congst_umr: 0
>> rx_arfs_err: 0
>> ch_events: 40572621887
>> ch_poll: 40885650979
>> ch_arm: 40429276692
>> ch_aff_change: 0
>> ch_eq_rearm: 0
>> rx_out_of_buffer: 2791690
>> rx_if_down_packets: 74
>> rx_vport_unicast_packets: 141843476308
>> rx_vport_unicast_bytes: 185421265403318
>> tx_vport_unicast_packets: 172569484005
>> tx_vport_unicast_bytes: 100019940094298
>> rx_vport_multicast_packets: 85122935
>> rx_vport_multicast_bytes: 5761316431
>> tx_vport_multicast_packets: 6452
>> tx_vport_multicast_bytes: 643540
>> rx_vport_broadcast_packets: 22423624
>> rx_vport_broadcast_bytes: 1390127090
>> tx_vport_broadcast_packets: 22024
>> tx_vport_broadcast_bytes: 1321440
>> rx_vport_rdma_unicast_packets: 0
>> rx_vport_rdma_unicast_bytes: 0
>> tx_vport_rdma_unicast_packets: 0
>> tx_vport_rdma_unicast_bytes: 0
>> rx_vport_rdma_multicast_packets: 0
>> rx_vport_rdma_multicast_bytes: 0
>> tx_vport_rdma_multicast_packets: 0
>> tx_vport_rdma_multicast_bytes: 0
>> tx_packets_phy: 172569501577
>> rx_packets_phy: 142871314588
>> rx_crc_errors_phy: 0
>> tx_bytes_phy: 100710212814151
>> rx_bytes_phy: 187209224289564
>> tx_multicast_phy: 6452
>> tx_broadcast_phy: 22024
>> rx_multicast_phy: 85122933
>> rx_broadcast_phy: 22423623
>> rx_in_range_len_errors_phy: 2
>> rx_out_of_range_len_phy: 0
>> rx_oversize_pkts_phy: 0
>> rx_symbol_err_phy: 0
>> tx_mac_control_phy: 0
>> rx_mac_control_phy: 0
>> rx_unsupported_op_phy: 0
>> rx_pause_ctrl_phy: 0
>> tx_pause_ctrl_phy: 0
>> rx_discards_phy: 920161423
> Ok, this port seem to be suffering more, RX is congested, maybe due to
> the pcie bottleneck.
Yes this side is receiving more traffic - second port is +10G more tx
>> tx_discards_phy: 0
>> tx_errors_phy: 0
>> rx_undersize_pkts_phy: 0
>> rx_fragments_phy: 0
>> rx_jabbers_phy: 0
>> rx_64_bytes_phy: 412006326
>> rx_65_to_127_bytes_phy: 11934371453
>> rx_128_to_255_bytes_phy: 3415281165
>> rx_256_to_511_bytes_phy: 2072955511
>> rx_512_to_1023_bytes_phy: 2415393005
>> rx_1024_to_1518_bytes_phy: 72182391608
>> rx_1519_to_2047_bytes_phy: 50438902587
>> rx_2048_to_4095_bytes_phy: 0
>> rx_4096_to_8191_bytes_phy: 0
>> rx_8192_to_10239_bytes_phy: 0
>> link_down_events_phy: 0
>> rx_pcs_symbol_err_phy: 0
>> rx_corrected_bits_phy: 0
>> rx_pci_signal_integrity: 0
>> tx_pci_signal_integrity: 48
>> rx_prio0_bytes: 186709842592642
>> rx_prio0_packets: 141481966007
>> tx_prio0_bytes: 100710171118138
>> tx_prio0_packets: 172569437949
>> rx_prio1_bytes: 492288152326
>> rx_prio1_packets: 385996045
>> tx_prio1_bytes: 0
>> tx_prio1_packets: 0
>> rx_prio2_bytes: 22119952
>> rx_prio2_packets: 70788
>> tx_prio2_bytes: 0
>> tx_prio2_packets: 0
>> rx_prio3_bytes: 546141102
>> rx_prio3_packets: 681608
>> tx_prio3_bytes: 0
>> tx_prio3_packets: 0
>> rx_prio4_bytes: 14665067
>> rx_prio4_packets: 29486
>> tx_prio4_bytes: 0
>> tx_prio4_packets: 0
>> rx_prio5_bytes: 158862504
>> rx_prio5_packets: 965307
>> tx_prio5_bytes: 0
>> tx_prio5_packets: 0
>> rx_prio6_bytes: 669337783
>> rx_prio6_packets: 1475775
>> tx_prio6_bytes: 0
>> tx_prio6_packets: 0
>> rx_prio7_bytes: 5623481349
>> rx_prio7_packets: 79926412
>> tx_prio7_bytes: 0
>> tx_prio7_packets: 0
>> module_unplug: 0
>> module_bus_stuck: 0
>> module_high_temp: 0
>> module_bad_shorted: 0
>> ch0_events: 1446162630
>> ch0_poll: 1463312972
>> ch0_arm: 1440728278
>> ch0_aff_change: 0
>> ch0_eq_rearm: 0
>> ch1_events: 1384301405
>> ch1_poll: 1399210915
>> ch1_arm: 1378636486
>> ch1_aff_change: 0
>> ch1_eq_rearm: 0
>> ch2_events: 1382788887
>> ch2_poll: 1397231470
>> ch2_arm: 1377058116
>> ch2_aff_change: 0
>> ch2_eq_rearm: 0
>> ch3_events: 1461956995
>> ch3_poll: 1475553146
>> ch3_arm: 1456571625
>> ch3_aff_change: 0
>> ch3_eq_rearm: 0
>> ch4_events: 1497359109
>> ch4_poll: 1511021037
>> ch4_arm: 1491733757
>> ch4_aff_change: 0
>> ch4_eq_rearm: 0
>> ch5_events: 1387736262
>> ch5_poll: 1400964615
>> ch5_arm: 1382382834
>> ch5_aff_change: 0
>> ch5_eq_rearm: 0
>> ch6_events: 1376772405
>> ch6_poll: 1390851449
>> ch6_arm: 1371551764
>> ch6_aff_change: 0
>> ch6_eq_rearm: 0
>> ch7_events: 1431271514
>> ch7_poll: 1445049729
>> ch7_arm: 1425753718
>> ch7_aff_change: 0
>> ch7_eq_rearm: 0
>> ch8_events: 1426976374
>> ch8_poll: 1439938692
>> ch8_arm: 1421392984
>> ch8_aff_change: 0
>> ch8_eq_rearm: 0
>> ch9_events: 1456160031
>> ch9_poll: 1468922870
>> ch9_arm: 1450930446
>> ch9_aff_change: 0
>> ch9_eq_rearm: 0
>> ch10_events: 1443640165
>> ch10_poll: 1456812203
>> ch10_arm: 1438425101
>> ch10_aff_change: 0
>> ch10_eq_rearm: 0
>> ch11_events: 1381104776
>> ch11_poll: 1393811057
>> ch11_arm: 1376059326
>> ch11_aff_change: 0
>> ch11_eq_rearm: 0
>> ch12_events: 1365223276
>> ch12_poll: 1378406059
>> ch12_arm: 1359950494
>> ch12_aff_change: 0
>> ch12_eq_rearm: 0
>> ch13_events: 1421622259
>> ch13_poll: 1434670996
>> ch13_arm: 1416241801
>> ch13_aff_change: 0
>> ch13_eq_rearm: 0
>> ch14_events: 1379084590
>> ch14_poll: 1392425015
>> ch14_arm: 1373675179
>> ch14_aff_change: 0
>> ch14_eq_rearm: 0
>> ch15_events: 1531217338
>> ch15_poll: 1543353833
>> ch15_arm: 1526350453
>> ch15_aff_change: 0
>> ch15_eq_rearm: 0
>> ch16_events: 1460469776
>> ch16_poll: 1467995928
>> ch16_arm: 1456010194
>> ch16_aff_change: 0
>> ch16_eq_rearm: 0
>> ch17_events: 1494067670
>> ch17_poll: 1500856680
>> ch17_arm: 1489232674
>> ch17_aff_change: 0
>> ch17_eq_rearm: 0
>> ch18_events: 1530126866
>> ch18_poll: 1537293620
>> ch18_arm: 1525476123
>> ch18_aff_change: 0
>> ch18_eq_rearm: 0
>> ch19_events: 1499526149
>> ch19_poll: 1506789309
>> ch19_arm: 1495161602
>> ch19_aff_change: 0
>> ch19_eq_rearm: 0
>> ch20_events: 1451479763
>> ch20_poll: 1459767921
>> ch20_arm: 1446360801
>> ch20_aff_change: 0
>> ch20_eq_rearm: 0
>> ch21_events: 1521413613
>> ch21_poll: 1529345146
>> ch21_arm: 1517229314
>> ch21_aff_change: 0
>> ch21_eq_rearm: 0
>> ch22_events: 1471950045
>> ch22_poll: 1479746764
>> ch22_arm: 1467681629
>> ch22_aff_change: 0
>> ch22_eq_rearm: 0
>> ch23_events: 1502968393
>> ch23_poll: 1510419909
>> ch23_arm: 1498168438
>> ch23_aff_change: 0
>> ch23_eq_rearm: 0
>> ch24_events: 1473451639
>> ch24_poll: 1482606899
>> ch24_arm: 1468212489
>> ch24_aff_change: 0
>> ch24_eq_rearm: 0
>> ch25_events: 1440399182
>> ch25_poll: 1448897475
>> ch25_arm: 1435044786
>> ch25_aff_change: 0
>> ch25_eq_rearm: 0
>> ch26_events: 1436831565
>> ch26_poll: 1445485731
>> ch26_arm: 1431827527
>> ch26_aff_change: 0
>> ch26_eq_rearm: 0
>> ch27_events: 1516560621
>> ch27_poll: 1524911010
>> ch27_arm: 1511430164
>> ch27_aff_change: 0
>> ch27_eq_rearm: 0
>> ch28_events: 4
>> ch28_poll: 4
>> ch28_arm: 4
>> ch28_aff_change: 0
>> ch28_eq_rearm: 0
>> ch29_events: 6
>> ch29_poll: 6
>> ch29_arm: 6
>> ch29_aff_change: 0
>> ch29_eq_rearm: 0
>> ch30_events: 4
>> ch30_poll: 4
>> ch30_arm: 4
>> ch30_aff_change: 0
>> ch30_eq_rearm: 0
>> ch31_events: 4
>> ch31_poll: 4
>> ch31_arm: 4
>> ch31_aff_change: 0
>> ch31_eq_rearm: 0
>> ch32_events: 4
>> ch32_poll: 4
>> ch32_arm: 4
>> ch32_aff_change: 0
>> ch32_eq_rearm: 0
>> ch33_events: 4
>> ch33_poll: 4
>> ch33_arm: 4
>> ch33_aff_change: 0
>> ch33_eq_rearm: 0
>> ch34_events: 4
>> ch34_poll: 4
>> ch34_arm: 4
>> ch34_aff_change: 0
>> ch34_eq_rearm: 0
>> ch35_events: 4
>> ch35_poll: 4
>> ch35_arm: 4
>> ch35_aff_change: 0
>> ch35_eq_rearm: 0
>> ch36_events: 4
>> ch36_poll: 4
>> ch36_arm: 4
>> ch36_aff_change: 0
>> ch36_eq_rearm: 0
>> ch37_events: 4
>> ch37_poll: 4
>> ch37_arm: 4
>> ch37_aff_change: 0
>> ch37_eq_rearm: 0
>> ch38_events: 4
>> ch38_poll: 4
>> ch38_arm: 4
>> ch38_aff_change: 0
>> ch38_eq_rearm: 0
>> ch39_events: 4
>> ch39_poll: 4
>> ch39_arm: 4
>> ch39_aff_change: 0
>> ch39_eq_rearm: 0
>> ch40_events: 4
>> ch40_poll: 4
>> ch40_arm: 4
>> ch40_aff_change: 0
>> ch40_eq_rearm: 0
>> ch41_events: 4
>> ch41_poll: 4
>> ch41_arm: 4
>> ch41_aff_change: 0
>> ch41_eq_rearm: 0
>> ch42_events: 4
>> ch42_poll: 4
>> ch42_arm: 4
>> ch42_aff_change: 0
>> ch42_eq_rearm: 0
>> ch43_events: 4
>> ch43_poll: 4
>> ch43_arm: 4
>> ch43_aff_change: 0
>> ch43_eq_rearm: 0
>> ch44_events: 4
>> ch44_poll: 4
>> ch44_arm: 4
>> ch44_aff_change: 0
>> ch44_eq_rearm: 0
>> ch45_events: 4
>> ch45_poll: 4
>> ch45_arm: 4
>> ch45_aff_change: 0
>> ch45_eq_rearm: 0
>> ch46_events: 4
>> ch46_poll: 4
>> ch46_arm: 4
>> ch46_aff_change: 0
>> ch46_eq_rearm: 0
>> ch47_events: 4
>> ch47_poll: 4
>> ch47_arm: 4
>> ch47_aff_change: 0
>> ch47_eq_rearm: 0
>> ch48_events: 4
>> ch48_poll: 4
>> ch48_arm: 4
>> ch48_aff_change: 0
>> ch48_eq_rearm: 0
>> ch49_events: 4
>> ch49_poll: 4
>> ch49_arm: 4
>> ch49_aff_change: 0
>> ch49_eq_rearm: 0
>> ch50_events: 4
>> ch50_poll: 4
>> ch50_arm: 4
>> ch50_aff_change: 0
>> ch50_eq_rearm: 0
>> ch51_events: 4
>> ch51_poll: 4
>> ch51_arm: 4
>> ch51_aff_change: 0
>> ch51_eq_rearm: 0
>> ch52_events: 4
>> ch52_poll: 4
>> ch52_arm: 4
>> ch52_aff_change: 0
>> ch52_eq_rearm: 0
>> ch53_events: 4
>> ch53_poll: 4
>> ch53_arm: 4
>> ch53_aff_change: 0
>> ch53_eq_rearm: 0
>> ch54_events: 4
>> ch54_poll: 4
>> ch54_arm: 4
>> ch54_aff_change: 0
>> ch54_eq_rearm: 0
>> ch55_events: 4
>> ch55_poll: 4
>> ch55_arm: 4
>> ch55_aff_change: 0
>> ch55_eq_rearm: 0
>> rx0_packets: 5861448653
>> rx0_bytes: 7389128595728
>> rx0_csum_complete: 5838312798
>> rx0_csum_unnecessary: 0
>> rx0_csum_unnecessary_inner: 0
>> rx0_csum_none: 23135855
>> rx0_xdp_drop: 0
>> rx0_xdp_redirect: 0
>> rx0_lro_packets: 0
>> rx0_lro_bytes: 0
>> rx0_ecn_mark: 0
>> rx0_removed_vlan_packets: 5861448653
>> rx0_wqe_err: 0
>> rx0_mpwqe_filler_cqes: 0
>> rx0_mpwqe_filler_strides: 0
>> rx0_buff_alloc_err: 0
>> rx0_cqe_compress_blks: 0
>> rx0_cqe_compress_pkts: 0
>> rx0_page_reuse: 0
>> rx0_cache_reuse: 2559
>> rx0_cache_full: 2930721512
>> rx0_cache_empty: 6656
>> rx0_cache_busy: 2930721765
>> rx0_cache_waive: 0
>> rx0_congst_umr: 0
>> rx0_arfs_err: 0
>> rx0_xdp_tx_xmit: 0
>> rx0_xdp_tx_full: 0
>> rx0_xdp_tx_err: 0
>> rx0_xdp_tx_cqes: 0
>> rx1_packets: 5550585106
>> rx1_bytes: 7255635262803
>> rx1_csum_complete: 5550585106
>> rx1_csum_unnecessary: 0
>> rx1_csum_unnecessary_inner: 0
>> rx1_csum_none: 0
>> rx1_xdp_drop: 0
>> rx1_xdp_redirect: 0
>> rx1_lro_packets: 0
>> rx1_lro_bytes: 0
>> rx1_ecn_mark: 0
>> rx1_removed_vlan_packets: 5550585106
>> rx1_wqe_err: 0
>> rx1_mpwqe_filler_cqes: 0
>> rx1_mpwqe_filler_strides: 0
>> rx1_buff_alloc_err: 0
>> rx1_cqe_compress_blks: 0
>> rx1_cqe_compress_pkts: 0
>> rx1_page_reuse: 0
>> rx1_cache_reuse: 2918845
>> rx1_cache_full: 2772373453
>> rx1_cache_empty: 6656
>> rx1_cache_busy: 2772373707
>> rx1_cache_waive: 0
>> rx1_congst_umr: 0
>> rx1_arfs_err: 0
>> rx1_xdp_tx_xmit: 0
>> rx1_xdp_tx_full: 0
>> rx1_xdp_tx_err: 0
>> rx1_xdp_tx_cqes: 0
>> rx2_packets: 5383874739
>> rx2_bytes: 7031545423967
>> rx2_csum_complete: 5383874739
>> rx2_csum_unnecessary: 0
>> rx2_csum_unnecessary_inner: 0
>> rx2_csum_none: 0
>> rx2_xdp_drop: 0
>> rx2_xdp_redirect: 0
>> rx2_lro_packets: 0
>> rx2_lro_bytes: 0
>> rx2_ecn_mark: 0
>> rx2_removed_vlan_packets: 5383874739
>> rx2_wqe_err: 0
>> rx2_mpwqe_filler_cqes: 0
>> rx2_mpwqe_filler_strides: 0
>> rx2_buff_alloc_err: 0
>> rx2_cqe_compress_blks: 0
>> rx2_cqe_compress_pkts: 0
>> rx2_page_reuse: 0
>> rx2_cache_reuse: 2173370
>> rx2_cache_full: 2689763744
>> rx2_cache_empty: 6656
>> rx2_cache_busy: 2689763998
>> rx2_cache_waive: 0
>> rx2_congst_umr: 0
>> rx2_arfs_err: 0
>> rx2_xdp_tx_xmit: 0
>> rx2_xdp_tx_full: 0
>> rx2_xdp_tx_err: 0
>> rx2_xdp_tx_cqes: 0
>> rx3_packets: 5456494012
>> rx3_bytes: 7120241119485
>> rx3_csum_complete: 5456494012
>> rx3_csum_unnecessary: 0
>> rx3_csum_unnecessary_inner: 0
>> rx3_csum_none: 0
>> rx3_xdp_drop: 0
>> rx3_xdp_redirect: 0
>> rx3_lro_packets: 0
>> rx3_lro_bytes: 0
>> rx3_ecn_mark: 0
>> rx3_removed_vlan_packets: 5456494012
>> rx3_wqe_err: 0
>> rx3_mpwqe_filler_cqes: 0
>> rx3_mpwqe_filler_strides: 0
>> rx3_buff_alloc_err: 0
>> rx3_cqe_compress_blks: 0
>> rx3_cqe_compress_pkts: 0
>> rx3_page_reuse: 0
>> rx3_cache_reuse: 2120123
>> rx3_cache_full: 2726126628
>> rx3_cache_empty: 6656
>> rx3_cache_busy: 2726126881
>> rx3_cache_waive: 0
>> rx3_congst_umr: 0
>> rx3_arfs_err: 0
>> rx3_xdp_tx_xmit: 0
>> rx3_xdp_tx_full: 0
>> rx3_xdp_tx_err: 0
>> rx3_xdp_tx_cqes: 0
>> rx4_packets: 5475216251
>> rx4_bytes: 7123129170196
>> rx4_csum_complete: 5475216251
>> rx4_csum_unnecessary: 0
>> rx4_csum_unnecessary_inner: 0
>> rx4_csum_none: 0
>> rx4_xdp_drop: 0
>> rx4_xdp_redirect: 0
>> rx4_lro_packets: 0
>> rx4_lro_bytes: 0
>> rx4_ecn_mark: 0
>> rx4_removed_vlan_packets: 5475216251
>> rx4_wqe_err: 0
>> rx4_mpwqe_filler_cqes: 0
>> rx4_mpwqe_filler_strides: 0
>> rx4_buff_alloc_err: 0
>> rx4_cqe_compress_blks: 0
>> rx4_cqe_compress_pkts: 0
>> rx4_page_reuse: 0
>> rx4_cache_reuse: 2668296355
>> rx4_cache_full: 69311549
>> rx4_cache_empty: 6656
>> rx4_cache_busy: 69311769
>> rx4_cache_waive: 0
>> rx4_congst_umr: 0
>> rx4_arfs_err: 0
>> rx4_xdp_tx_xmit: 0
>> rx4_xdp_tx_full: 0
>> rx4_xdp_tx_err: 0
>> rx4_xdp_tx_cqes: 0
>> rx5_packets: 5474372232
>> rx5_bytes: 7159146801926
>> rx5_csum_complete: 5474372232
>> rx5_csum_unnecessary: 0
>> rx5_csum_unnecessary_inner: 0
>> rx5_csum_none: 0
>> rx5_xdp_drop: 0
>> rx5_xdp_redirect: 0
>> rx5_lro_packets: 0
>> rx5_lro_bytes: 0
>> rx5_ecn_mark: 0
>> rx5_removed_vlan_packets: 5474372232
>> rx5_wqe_err: 0
>> rx5_mpwqe_filler_cqes: 0
>> rx5_mpwqe_filler_strides: 0
>> rx5_buff_alloc_err: 0
>> rx5_cqe_compress_blks: 0
>> rx5_cqe_compress_pkts: 0
>> rx5_page_reuse: 0
>> rx5_cache_reuse: 626187
>> rx5_cache_full: 2736559674
>> rx5_cache_empty: 6656
>> rx5_cache_busy: 2736559929
>> rx5_cache_waive: 0
>> rx5_congst_umr: 0
>> rx5_arfs_err: 0
>> rx5_xdp_tx_xmit: 0
>> rx5_xdp_tx_full: 0
>> rx5_xdp_tx_err: 0
>> rx5_xdp_tx_cqes: 0
>> rx6_packets: 5533622456
>> rx6_bytes: 7207308809081
>> rx6_csum_complete: 5533622456
>> rx6_csum_unnecessary: 0
>> rx6_csum_unnecessary_inner: 0
>> rx6_csum_none: 0
>> rx6_xdp_drop: 0
>> rx6_xdp_redirect: 0
>> rx6_lro_packets: 0
>> rx6_lro_bytes: 0
>> rx6_ecn_mark: 0
>> rx6_removed_vlan_packets: 5533622456
>> rx6_wqe_err: 0
>> rx6_mpwqe_filler_cqes: 0
>> rx6_mpwqe_filler_strides: 0
>> rx6_buff_alloc_err: 0
>> rx6_cqe_compress_blks: 0
>> rx6_cqe_compress_pkts: 0
>> rx6_page_reuse: 0
>> rx6_cache_reuse: 2325217
>> rx6_cache_full: 2764485756
>> rx6_cache_empty: 6656
>> rx6_cache_busy: 2764486011
>> rx6_cache_waive: 0
>> rx6_congst_umr: 0
>> rx6_arfs_err: 0
>> rx6_xdp_tx_xmit: 0
>> rx6_xdp_tx_full: 0
>> rx6_xdp_tx_err: 0
>> rx6_xdp_tx_cqes: 0
>> rx7_packets: 5533901822
>> rx7_bytes: 7227441240536
>> rx7_csum_complete: 5533901822
>> rx7_csum_unnecessary: 0
>> rx7_csum_unnecessary_inner: 0
>> rx7_csum_none: 0
>> rx7_xdp_drop: 0
>> rx7_xdp_redirect: 0
>> rx7_lro_packets: 0
>> rx7_lro_bytes: 0
>> rx7_ecn_mark: 0
>> rx7_removed_vlan_packets: 5533901822
>> rx7_wqe_err: 0
>> rx7_mpwqe_filler_cqes: 0
>> rx7_mpwqe_filler_strides: 0
>> rx7_buff_alloc_err: 0
>> rx7_cqe_compress_blks: 0
>> rx7_cqe_compress_pkts: 0
>> rx7_page_reuse: 0
>> rx7_cache_reuse: 2372505
>> rx7_cache_full: 2764578151
>> rx7_cache_empty: 6656
>> rx7_cache_busy: 2764578403
>> rx7_cache_waive: 0
>> rx7_congst_umr: 0
>> rx7_arfs_err: 0
>> rx7_xdp_tx_xmit: 0
>> rx7_xdp_tx_full: 0
>> rx7_xdp_tx_err: 0
>> rx7_xdp_tx_cqes: 0
>> rx8_packets: 5485670137
>> rx8_bytes: 7203339989013
>> rx8_csum_complete: 5485670137
>> rx8_csum_unnecessary: 0
>> rx8_csum_unnecessary_inner: 0
>> rx8_csum_none: 0
>> rx8_xdp_drop: 0
>> rx8_xdp_redirect: 0
>> rx8_lro_packets: 0
>> rx8_lro_bytes: 0
>> rx8_ecn_mark: 0
>> rx8_removed_vlan_packets: 5485670137
>> rx8_wqe_err: 0
>> rx8_mpwqe_filler_cqes: 0
>> rx8_mpwqe_filler_strides: 0
>> rx8_buff_alloc_err: 0
>> rx8_cqe_compress_blks: 0
>> rx8_cqe_compress_pkts: 0
>> rx8_page_reuse: 0
>> rx8_cache_reuse: 7522232
>> rx8_cache_full: 2735312581
>> rx8_cache_empty: 6656
>> rx8_cache_busy: 2735312836
>> rx8_cache_waive: 0
>> rx8_congst_umr: 0
>> rx8_arfs_err: 0
>> rx8_xdp_tx_xmit: 0
>> rx8_xdp_tx_full: 0
>> rx8_xdp_tx_err: 0
>> rx8_xdp_tx_cqes: 0
>> rx9_packets: 5482212354
>> rx9_bytes: 7169663341718
>> rx9_csum_complete: 5482212354
>> rx9_csum_unnecessary: 0
>> rx9_csum_unnecessary_inner: 0
>> rx9_csum_none: 0
>> rx9_xdp_drop: 0
>> rx9_xdp_redirect: 0
>> rx9_lro_packets: 0
>> rx9_lro_bytes: 0
>> rx9_ecn_mark: 0
>> rx9_removed_vlan_packets: 5482212354
>> rx9_wqe_err: 0
>> rx9_mpwqe_filler_cqes: 0
>> rx9_mpwqe_filler_strides: 0
>> rx9_buff_alloc_err: 0
>> rx9_cqe_compress_blks: 0
>> rx9_cqe_compress_pkts: 0
>> rx9_page_reuse: 0
>> rx9_cache_reuse: 37279961
>> rx9_cache_full: 2703825961
>> rx9_cache_empty: 6656
>> rx9_cache_busy: 2703826215
>> rx9_cache_waive: 0
>> rx9_congst_umr: 0
>> rx9_arfs_err: 0
>> rx9_xdp_tx_xmit: 0
>> rx9_xdp_tx_full: 0
>> rx9_xdp_tx_err: 0
>> rx9_xdp_tx_cqes: 0
>> rx10_packets: 5524679952
>> rx10_bytes: 7248301275181
>> rx10_csum_complete: 5524679952
>> rx10_csum_unnecessary: 0
>> rx10_csum_unnecessary_inner: 0
>> rx10_csum_none: 0
>> rx10_xdp_drop: 0
>> rx10_xdp_redirect: 0
>> rx10_lro_packets: 0
>> rx10_lro_bytes: 0
>> rx10_ecn_mark: 0
>> rx10_removed_vlan_packets: 5524679952
>> rx10_wqe_err: 0
>> rx10_mpwqe_filler_cqes: 0
>> rx10_mpwqe_filler_strides: 0
>> rx10_buff_alloc_err: 0
>> rx10_cqe_compress_blks: 0
>> rx10_cqe_compress_pkts: 0
>> rx10_page_reuse: 0
>> rx10_cache_reuse: 2049666
>> rx10_cache_full: 2760290055
>> rx10_cache_empty: 6656
>> rx10_cache_busy: 2760290310
>> rx10_cache_waive: 0
>> rx10_congst_umr: 0
>> rx10_arfs_err: 0
>> rx10_xdp_tx_xmit: 0
>> rx10_xdp_tx_full: 0
>> rx10_xdp_tx_err: 0
>> rx10_xdp_tx_cqes: 0
>> rx11_packets: 5394633545
>> rx11_bytes: 7033509636092
>> rx11_csum_complete: 5394633545
>> rx11_csum_unnecessary: 0
>> rx11_csum_unnecessary_inner: 0
>> rx11_csum_none: 0
>> rx11_xdp_drop: 0
>> rx11_xdp_redirect: 0
>> rx11_lro_packets: 0
>> rx11_lro_bytes: 0
>> rx11_ecn_mark: 0
>> rx11_removed_vlan_packets: 5394633545
>> rx11_wqe_err: 0
>> rx11_mpwqe_filler_cqes: 0
>> rx11_mpwqe_filler_strides: 0
>> rx11_buff_alloc_err: 0
>> rx11_cqe_compress_blks: 0
>> rx11_cqe_compress_pkts: 0
>> rx11_page_reuse: 0
>> rx11_cache_reuse: 2617466268
>> rx11_cache_full: 79850284
>> rx11_cache_empty: 6656
>> rx11_cache_busy: 79850504
>> rx11_cache_waive: 0
>> rx11_congst_umr: 0
>> rx11_arfs_err: 0
>> rx11_xdp_tx_xmit: 0
>> rx11_xdp_tx_full: 0
>> rx11_xdp_tx_err: 0
>> rx11_xdp_tx_cqes: 0
>> rx12_packets: 5458907385
>> rx12_bytes: 7134867867515
>> rx12_csum_complete: 5458907385
>> rx12_csum_unnecessary: 0
>> rx12_csum_unnecessary_inner: 0
>> rx12_csum_none: 0
>> rx12_xdp_drop: 0
>> rx12_xdp_redirect: 0
>> rx12_lro_packets: 0
>> rx12_lro_bytes: 0
>> rx12_ecn_mark: 0
>> rx12_removed_vlan_packets: 5458907385
>> rx12_wqe_err: 0
>> rx12_mpwqe_filler_cqes: 0
>> rx12_mpwqe_filler_strides: 0
>> rx12_buff_alloc_err: 0
>> rx12_cqe_compress_blks: 0
>> rx12_cqe_compress_pkts: 0
>> rx12_page_reuse: 0
>> rx12_cache_reuse: 2650214169
>> rx12_cache_full: 79239303
>> rx12_cache_empty: 6656
>> rx12_cache_busy: 79239523
>> rx12_cache_waive: 0
>> rx12_congst_umr: 0
>> rx12_arfs_err: 0
>> rx12_xdp_tx_xmit: 0
>> rx12_xdp_tx_full: 0
>> rx12_xdp_tx_err: 0
>> rx12_xdp_tx_cqes: 0
>> rx13_packets: 5549932912
>> rx13_bytes: 7232548705586
>> rx13_csum_complete: 5549932912
>> rx13_csum_unnecessary: 0
>> rx13_csum_unnecessary_inner: 0
>> rx13_csum_none: 0
>> rx13_xdp_drop: 0
>> rx13_xdp_redirect: 0
>> rx13_lro_packets: 0
>> rx13_lro_bytes: 0
>> rx13_ecn_mark: 0
>> rx13_removed_vlan_packets: 5549932912
>> rx13_wqe_err: 0
>> rx13_mpwqe_filler_cqes: 0
>> rx13_mpwqe_filler_strides: 0
>> rx13_buff_alloc_err: 0
>> rx13_cqe_compress_blks: 0
>> rx13_cqe_compress_pkts: 0
>> rx13_page_reuse: 0
>> rx13_cache_reuse: 2417696
>> rx13_cache_full: 2772548505
>> rx13_cache_empty: 6656
>> rx13_cache_busy: 2772548760
>> rx13_cache_waive: 0
>> rx13_congst_umr: 0
>> rx13_arfs_err: 0
>> rx13_xdp_tx_xmit: 0
>> rx13_xdp_tx_full: 0
>> rx13_xdp_tx_err: 0
>> rx13_xdp_tx_cqes: 0
>> rx14_packets: 5517712329
>> rx14_bytes: 7192111965227
>> rx14_csum_complete: 5517712329
>> rx14_csum_unnecessary: 0
>> rx14_csum_unnecessary_inner: 0
>> rx14_csum_none: 0
>> rx14_xdp_drop: 0
>> rx14_xdp_redirect: 0
>> rx14_lro_packets: 0
>> rx14_lro_bytes: 0
>> rx14_ecn_mark: 0
>> rx14_removed_vlan_packets: 5517712329
>> rx14_wqe_err: 0
>> rx14_mpwqe_filler_cqes: 0
>> rx14_mpwqe_filler_strides: 0
>> rx14_buff_alloc_err: 0
>> rx14_cqe_compress_blks: 0
>> rx14_cqe_compress_pkts: 0
>> rx14_page_reuse: 0
>> rx14_cache_reuse: 1830206
>> rx14_cache_full: 2757025703
>> rx14_cache_empty: 6656
>> rx14_cache_busy: 2757025958
>> rx14_cache_waive: 0
>> rx14_congst_umr: 0
>> rx14_arfs_err: 0
>> rx14_xdp_tx_xmit: 0
>> rx14_xdp_tx_full: 0
>> rx14_xdp_tx_err: 0
>> rx14_xdp_tx_cqes: 0
>> rx15_packets: 5578343373
>> rx15_bytes: 7268484501219
>> rx15_csum_complete: 5578343373
>> rx15_csum_unnecessary: 0
>> rx15_csum_unnecessary_inner: 0
>> rx15_csum_none: 0
>> rx15_xdp_drop: 0
>> rx15_xdp_redirect: 0
>> rx15_lro_packets: 0
>> rx15_lro_bytes: 0
>> rx15_ecn_mark: 0
>> rx15_removed_vlan_packets: 5578343373
>> rx15_wqe_err: 0
>> rx15_mpwqe_filler_cqes: 0
>> rx15_mpwqe_filler_strides: 0
>> rx15_buff_alloc_err: 0
>> rx15_cqe_compress_blks: 0
>> rx15_cqe_compress_pkts: 0
>> rx15_page_reuse: 0
>> rx15_cache_reuse: 2317165
>> rx15_cache_full: 2786854266
>> rx15_cache_empty: 6656
>> rx15_cache_busy: 2786854519
>> rx15_cache_waive: 0
>> rx15_congst_umr: 0
>> rx15_arfs_err: 0
>> rx15_xdp_tx_xmit: 0
>> rx15_xdp_tx_full: 0
>> rx15_xdp_tx_err: 0
>> rx15_xdp_tx_cqes: 0
>> rx16_packets: 4435773951
>> rx16_bytes: 5766665272007
>> rx16_csum_complete: 4435773951
>> rx16_csum_unnecessary: 0
>> rx16_csum_unnecessary_inner: 0
>> rx16_csum_none: 0
>> rx16_xdp_drop: 0
>> rx16_xdp_redirect: 0
>> rx16_lro_packets: 0
>> rx16_lro_bytes: 0
>> rx16_ecn_mark: 0
>> rx16_removed_vlan_packets: 4435773951
>> rx16_wqe_err: 0
>> rx16_mpwqe_filler_cqes: 0
>> rx16_mpwqe_filler_strides: 0
>> rx16_buff_alloc_err: 0
>> rx16_cqe_compress_blks: 0
>> rx16_cqe_compress_pkts: 0
>> rx16_page_reuse: 0
>> rx16_cache_reuse: 2033793
>> rx16_cache_full: 2215852927
>> rx16_cache_empty: 6656
>> rx16_cache_busy: 2215853179
>> rx16_cache_waive: 0
>> rx16_congst_umr: 0
>> rx16_arfs_err: 0
>> rx16_xdp_tx_xmit: 0
>> rx16_xdp_tx_full: 0
>> rx16_xdp_tx_err: 0
>> rx16_xdp_tx_cqes: 0
>> rx17_packets: 4344087587
>> rx17_bytes: 5695006496323
>> rx17_csum_complete: 4344087587
>> rx17_csum_unnecessary: 0
>> rx17_csum_unnecessary_inner: 0
>> rx17_csum_none: 0
>> rx17_xdp_drop: 0
>> rx17_xdp_redirect: 0
>> rx17_lro_packets: 0
>> rx17_lro_bytes: 0
>> rx17_ecn_mark: 0
>> rx17_removed_vlan_packets: 4344087587
>> rx17_wqe_err: 0
>> rx17_mpwqe_filler_cqes: 0
>> rx17_mpwqe_filler_strides: 0
>> rx17_buff_alloc_err: 0
>> rx17_cqe_compress_blks: 0
>> rx17_cqe_compress_pkts: 0
>> rx17_page_reuse: 0
>> rx17_cache_reuse: 2652127
>> rx17_cache_full: 2169391411
>> rx17_cache_empty: 6656
>> rx17_cache_busy: 2169391665
>> rx17_cache_waive: 0
>> rx17_congst_umr: 0
>> rx17_arfs_err: 0
>> rx17_xdp_tx_xmit: 0
>> rx17_xdp_tx_full: 0
>> rx17_xdp_tx_err: 0
>> rx17_xdp_tx_cqes: 0
>> rx18_packets: 4407422804
>> rx18_bytes: 5741134634177
>> rx18_csum_complete: 4407422804
>> rx18_csum_unnecessary: 0
>> rx18_csum_unnecessary_inner: 0
>> rx18_csum_none: 0
>> rx18_xdp_drop: 0
>> rx18_xdp_redirect: 0
>> rx18_lro_packets: 0
>> rx18_lro_bytes: 0
>> rx18_ecn_mark: 0
>> rx18_removed_vlan_packets: 4407422804
>> rx18_wqe_err: 0
>> rx18_mpwqe_filler_cqes: 0
>> rx18_mpwqe_filler_strides: 0
>> rx18_buff_alloc_err: 0
>> rx18_cqe_compress_blks: 0
>> rx18_cqe_compress_pkts: 0
>> rx18_page_reuse: 0
>> rx18_cache_reuse: 2156080239
>> rx18_cache_full: 47630941
>> rx18_cache_empty: 6656
>> rx18_cache_busy: 47631161
>> rx18_cache_waive: 0
>> rx18_congst_umr: 0
>> rx18_arfs_err: 0
>> rx18_xdp_tx_xmit: 0
>> rx18_xdp_tx_full: 0
>> rx18_xdp_tx_err: 0
>> rx18_xdp_tx_cqes: 0
>> rx19_packets: 4545554180
>> rx19_bytes: 5905277503466
>> rx19_csum_complete: 4545554180
>> rx19_csum_unnecessary: 0
>> rx19_csum_unnecessary_inner: 0
>> rx19_csum_none: 0
>> rx19_xdp_drop: 0
>> rx19_xdp_redirect: 0
>> rx19_lro_packets: 0
>> rx19_lro_bytes: 0
>> rx19_ecn_mark: 0
>> rx19_removed_vlan_packets: 4545554180
>> rx19_wqe_err: 0
>> rx19_mpwqe_filler_cqes: 0
>> rx19_mpwqe_filler_strides: 0
>> rx19_buff_alloc_err: 0
>> rx19_cqe_compress_blks: 0
>> rx19_cqe_compress_pkts: 0
>> rx19_page_reuse: 0
>> rx19_cache_reuse: 11112455
>> rx19_cache_full: 2261664379
>> rx19_cache_empty: 6656
>> rx19_cache_busy: 2261664601
>> rx19_cache_waive: 0
>> rx19_congst_umr: 0
>> rx19_arfs_err: 0
>> rx19_xdp_tx_xmit: 0
>> rx19_xdp_tx_full: 0
>> rx19_xdp_tx_err: 0
>> rx19_xdp_tx_cqes: 0
>> rx20_packets: 4397428553
>> rx20_bytes: 5757329184301
>> rx20_csum_complete: 4397428553
>> rx20_csum_unnecessary: 0
>> rx20_csum_unnecessary_inner: 0
>> rx20_csum_none: 0
>> rx20_xdp_drop: 0
>> rx20_xdp_redirect: 0
>> rx20_lro_packets: 0
>> rx20_lro_bytes: 0
>> rx20_ecn_mark: 0
>> rx20_removed_vlan_packets: 4397428553
>> rx20_wqe_err: 0
>> rx20_mpwqe_filler_cqes: 0
>> rx20_mpwqe_filler_strides: 0
>> rx20_buff_alloc_err: 0
>> rx20_cqe_compress_blks: 0
>> rx20_cqe_compress_pkts: 0
>> rx20_page_reuse: 0
>> rx20_cache_reuse: 2168116995
>> rx20_cache_full: 30597061
>> rx20_cache_empty: 6656
>> rx20_cache_busy: 30597281
>> rx20_cache_waive: 0
>> rx20_congst_umr: 0
>> rx20_arfs_err: 0
>> rx20_xdp_tx_xmit: 0
>> rx20_xdp_tx_full: 0
>> rx20_xdp_tx_err: 0
>> rx20_xdp_tx_cqes: 0
>> rx21_packets: 4552564821
>> rx21_bytes: 5944840329249
>> rx21_csum_complete: 4552564821
>> rx21_csum_unnecessary: 0
>> rx21_csum_unnecessary_inner: 0
>> rx21_csum_none: 0
>> rx21_xdp_drop: 0
>> rx21_xdp_redirect: 0
>> rx21_lro_packets: 0
>> rx21_lro_bytes: 0
>> rx21_ecn_mark: 0
>> rx21_removed_vlan_packets: 4552564821
>> rx21_wqe_err: 0
>> rx21_mpwqe_filler_cqes: 0
>> rx21_mpwqe_filler_strides: 0
>> rx21_buff_alloc_err: 0
>> rx21_cqe_compress_blks: 0
>> rx21_cqe_compress_pkts: 0
>> rx21_page_reuse: 0
>> rx21_cache_reuse: 2295681
>> rx21_cache_full: 2273986474
>> rx21_cache_empty: 6656
>> rx21_cache_busy: 2273986727
>> rx21_cache_waive: 0
>> rx21_congst_umr: 0
>> rx21_arfs_err: 0
>> rx21_xdp_tx_xmit: 0
>> rx21_xdp_tx_full: 0
>> rx21_xdp_tx_err: 0
>> rx21_xdp_tx_cqes: 0
>> rx22_packets: 4629499740
>> rx22_bytes: 5924206566499
>> rx22_csum_complete: 4629499740
>> rx22_csum_unnecessary: 0
>> rx22_csum_unnecessary_inner: 0
>> rx22_csum_none: 0
>> rx22_xdp_drop: 0
>> rx22_xdp_redirect: 0
>> rx22_lro_packets: 0
>> rx22_lro_bytes: 0
>> rx22_ecn_mark: 0
>> rx22_removed_vlan_packets: 4629499740
>> rx22_wqe_err: 0
>> rx22_mpwqe_filler_cqes: 0
>> rx22_mpwqe_filler_strides: 0
>> rx22_buff_alloc_err: 0
>> rx22_cqe_compress_blks: 0
>> rx22_cqe_compress_pkts: 0
>> rx22_page_reuse: 0
>> rx22_cache_reuse: 1407527
>> rx22_cache_full: 2313342088
>> rx22_cache_empty: 6656
>> rx22_cache_busy: 2313342341
>> rx22_cache_waive: 0
>> rx22_congst_umr: 0
>> rx22_arfs_err: 0
>> rx22_xdp_tx_xmit: 0
>> rx22_xdp_tx_full: 0
>> rx22_xdp_tx_err: 0
>> rx22_xdp_tx_cqes: 0
>> rx23_packets: 4387124505
>> rx23_bytes: 5718118678470
>> rx23_csum_complete: 4387124505
>> rx23_csum_unnecessary: 0
>> rx23_csum_unnecessary_inner: 0
>> rx23_csum_none: 0
>> rx23_xdp_drop: 0
>> rx23_xdp_redirect: 0
>> rx23_lro_packets: 0
>> rx23_lro_bytes: 0
>> rx23_ecn_mark: 0
>> rx23_removed_vlan_packets: 4387124505
>> rx23_wqe_err: 0
>> rx23_mpwqe_filler_cqes: 0
>> rx23_mpwqe_filler_strides: 0
>> rx23_buff_alloc_err: 0
>> rx23_cqe_compress_blks: 0
>> rx23_cqe_compress_pkts: 0
>> rx23_page_reuse: 0
>> rx23_cache_reuse: 2013280
>> rx23_cache_full: 2191548717
>> rx23_cache_empty: 6656
>> rx23_cache_busy: 2191548972
>> rx23_cache_waive: 0
>> rx23_congst_umr: 0
>> rx23_arfs_err: 0
>> rx23_xdp_tx_xmit: 0
>> rx23_xdp_tx_full: 0
>> rx23_xdp_tx_err: 0
>> rx23_xdp_tx_cqes: 0
>> rx24_packets: 4398791634
>> rx24_bytes: 5744875564632
>> rx24_csum_complete: 4398791634
>> rx24_csum_unnecessary: 0
>> rx24_csum_unnecessary_inner: 0
>> rx24_csum_none: 0
>> rx24_xdp_drop: 0
>> rx24_xdp_redirect: 0
>> rx24_lro_packets: 0
>> rx24_lro_bytes: 0
>> rx24_ecn_mark: 0
>> rx24_removed_vlan_packets: 4398791634
>> rx24_wqe_err: 0
>> rx24_mpwqe_filler_cqes: 0
>> rx24_mpwqe_filler_strides: 0
>> rx24_buff_alloc_err: 0
>> rx24_cqe_compress_blks: 0
>> rx24_cqe_compress_pkts: 0
>> rx24_page_reuse: 0
>> rx24_cache_reuse: 2143926100
>> rx24_cache_full: 55469496
>> rx24_cache_empty: 6656
>> rx24_cache_busy: 55469716
>> rx24_cache_waive: 0
>> rx24_congst_umr: 0
>> rx24_arfs_err: 0
>> rx24_xdp_tx_xmit: 0
>> rx24_xdp_tx_full: 0
>> rx24_xdp_tx_err: 0
>> rx24_xdp_tx_cqes: 0
>> rx25_packets: 4377204935
>> rx25_bytes: 5710369124105
>> rx25_csum_complete: 4377204935
>> rx25_csum_unnecessary: 0
>> rx25_csum_unnecessary_inner: 0
>> rx25_csum_none: 0
>> rx25_xdp_drop: 0
>> rx25_xdp_redirect: 0
>> rx25_lro_packets: 0
>> rx25_lro_bytes: 0
>> rx25_ecn_mark: 0
>> rx25_removed_vlan_packets: 4377204935
>> rx25_wqe_err: 0
>> rx25_mpwqe_filler_cqes: 0
>> rx25_mpwqe_filler_strides: 0
>> rx25_buff_alloc_err: 0
>> rx25_cqe_compress_blks: 0
>> rx25_cqe_compress_pkts: 0
>> rx25_page_reuse: 0
>> rx25_cache_reuse: 2132658660
>> rx25_cache_full: 55943584
>> rx25_cache_empty: 6656
>> rx25_cache_busy: 55943804
>> rx25_cache_waive: 0
>> rx25_congst_umr: 0
>> rx25_arfs_err: 0
>> rx25_xdp_tx_xmit: 0
>> rx25_xdp_tx_full: 0
>> rx25_xdp_tx_err: 0
>> rx25_xdp_tx_cqes: 0
>> rx26_packets: 4496003688
>> rx26_bytes: 5862180715503
>> rx26_csum_complete: 4496003688
>> rx26_csum_unnecessary: 0
>> rx26_csum_unnecessary_inner: 0
>> rx26_csum_none: 0
>> rx26_xdp_drop: 0
>> rx26_xdp_redirect: 0
>> rx26_lro_packets: 0
>> rx26_lro_bytes: 0
>> rx26_ecn_mark: 0
>> rx26_removed_vlan_packets: 4496003688
>> rx26_wqe_err: 0
>> rx26_mpwqe_filler_cqes: 0
>> rx26_mpwqe_filler_strides: 0
>> rx26_buff_alloc_err: 0
>> rx26_cqe_compress_blks: 0
>> rx26_cqe_compress_pkts: 0
>> rx26_page_reuse: 0
>> rx26_cache_reuse: 8
>> rx26_cache_full: 2248001581
>> rx26_cache_empty: 6656
>> rx26_cache_busy: 2248001836
>> rx26_cache_waive: 0
>> rx26_congst_umr: 0
>> rx26_arfs_err: 0
>> rx26_xdp_tx_xmit: 0
>> rx26_xdp_tx_full: 0
>> rx26_xdp_tx_err: 0
>> rx26_xdp_tx_cqes: 0
>> rx27_packets: 4341849333
>> rx27_bytes: 5678653545018
>> rx27_csum_complete: 4341849333
>> rx27_csum_unnecessary: 0
>> rx27_csum_unnecessary_inner: 0
>> rx27_csum_none: 0
>> rx27_xdp_drop: 0
>> rx27_xdp_redirect: 0
>> rx27_lro_packets: 0
>> rx27_lro_bytes: 0
>> rx27_ecn_mark: 0
>> rx27_removed_vlan_packets: 4341849333
>> rx27_wqe_err: 0
>> rx27_mpwqe_filler_cqes: 0
>> rx27_mpwqe_filler_strides: 0
>> rx27_buff_alloc_err: 0
>> rx27_cqe_compress_blks: 0
>> rx27_cqe_compress_pkts: 0
>> rx27_page_reuse: 0
>> rx27_cache_reuse: 1748188
>> rx27_cache_full: 2169176223
>> rx27_cache_empty: 6656
>> rx27_cache_busy: 2169176476
>> rx27_cache_waive: 0
>> rx27_congst_umr: 0
>> rx27_arfs_err: 0
>> rx27_xdp_tx_xmit: 0
>> rx27_xdp_tx_full: 0
>> rx27_xdp_tx_err: 0
>> rx27_xdp_tx_cqes: 0
>> rx28_packets: 0
>> rx28_bytes: 0
>> rx28_csum_complete: 0
>> rx28_csum_unnecessary: 0
>> rx28_csum_unnecessary_inner: 0
>> rx28_csum_none: 0
>> rx28_xdp_drop: 0
>> rx28_xdp_redirect: 0
>> rx28_lro_packets: 0
>> rx28_lro_bytes: 0
>> rx28_ecn_mark: 0
>> rx28_removed_vlan_packets: 0
>> rx28_wqe_err: 0
>> rx28_mpwqe_filler_cqes: 0
>> rx28_mpwqe_filler_strides: 0
>> rx28_buff_alloc_err: 0
>> rx28_cqe_compress_blks: 0
>> rx28_cqe_compress_pkts: 0
>> rx28_page_reuse: 0
>> rx28_cache_reuse: 0
>> rx28_cache_full: 0
>> rx28_cache_empty: 2560
>> rx28_cache_busy: 0
>> rx28_cache_waive: 0
>> rx28_congst_umr: 0
>> rx28_arfs_err: 0
>> rx28_xdp_tx_xmit: 0
>> rx28_xdp_tx_full: 0
>> rx28_xdp_tx_err: 0
>> rx28_xdp_tx_cqes: 0
>> rx29_packets: 0
>> rx29_bytes: 0
>> rx29_csum_complete: 0
>> rx29_csum_unnecessary: 0
>> rx29_csum_unnecessary_inner: 0
>> rx29_csum_none: 0
>> rx29_xdp_drop: 0
>> rx29_xdp_redirect: 0
>> rx29_lro_packets: 0
>> rx29_lro_bytes: 0
>> rx29_ecn_mark: 0
>> rx29_removed_vlan_packets: 0
>> rx29_wqe_err: 0
>> rx29_mpwqe_filler_cqes: 0
>> rx29_mpwqe_filler_strides: 0
>> rx29_buff_alloc_err: 0
>> rx29_cqe_compress_blks: 0
>> rx29_cqe_compress_pkts: 0
>> rx29_page_reuse: 0
>> rx29_cache_reuse: 0
>> rx29_cache_full: 0
>> rx29_cache_empty: 2560
>> rx29_cache_busy: 0
>> rx29_cache_waive: 0
>> rx29_congst_umr: 0
>> rx29_arfs_err: 0
>> rx29_xdp_tx_xmit: 0
>> rx29_xdp_tx_full: 0
>> rx29_xdp_tx_err: 0
>> rx29_xdp_tx_cqes: 0
>> rx30_packets: 0
>> rx30_bytes: 0
>> rx30_csum_complete: 0
>> rx30_csum_unnecessary: 0
>> rx30_csum_unnecessary_inner: 0
>> rx30_csum_none: 0
>> rx30_xdp_drop: 0
>> rx30_xdp_redirect: 0
>> rx30_lro_packets: 0
>> rx30_lro_bytes: 0
>> rx30_ecn_mark: 0
>> rx30_removed_vlan_packets: 0
>> rx30_wqe_err: 0
>> rx30_mpwqe_filler_cqes: 0
>> rx30_mpwqe_filler_strides: 0
>> rx30_buff_alloc_err: 0
>> rx30_cqe_compress_blks: 0
>> rx30_cqe_compress_pkts: 0
>> rx30_page_reuse: 0
>> rx30_cache_reuse: 0
>> rx30_cache_full: 0
>> rx30_cache_empty: 2560
>> rx30_cache_busy: 0
>> rx30_cache_waive: 0
>> rx30_congst_umr: 0
>> rx30_arfs_err: 0
>> rx30_xdp_tx_xmit: 0
>> rx30_xdp_tx_full: 0
>> rx30_xdp_tx_err: 0
>> rx30_xdp_tx_cqes: 0
>> rx31_packets: 0
>> rx31_bytes: 0
>> rx31_csum_complete: 0
>> rx31_csum_unnecessary: 0
>> rx31_csum_unnecessary_inner: 0
>> rx31_csum_none: 0
>> rx31_xdp_drop: 0
>> rx31_xdp_redirect: 0
>> rx31_lro_packets: 0
>> rx31_lro_bytes: 0
>> rx31_ecn_mark: 0
>> rx31_removed_vlan_packets: 0
>> rx31_wqe_err: 0
>> rx31_mpwqe_filler_cqes: 0
>> rx31_mpwqe_filler_strides: 0
>> rx31_buff_alloc_err: 0
>> rx31_cqe_compress_blks: 0
>> rx31_cqe_compress_pkts: 0
>> rx31_page_reuse: 0
>> rx31_cache_reuse: 0
>> rx31_cache_full: 0
>> rx31_cache_empty: 2560
>> rx31_cache_busy: 0
>> rx31_cache_waive: 0
>> rx31_congst_umr: 0
>> rx31_arfs_err: 0
>> rx31_xdp_tx_xmit: 0
>> rx31_xdp_tx_full: 0
>> rx31_xdp_tx_err: 0
>> rx31_xdp_tx_cqes: 0
>> rx32_packets: 0
>> rx32_bytes: 0
>> rx32_csum_complete: 0
>> rx32_csum_unnecessary: 0
>> rx32_csum_unnecessary_inner: 0
>> rx32_csum_none: 0
>> rx32_xdp_drop: 0
>> rx32_xdp_redirect: 0
>> rx32_lro_packets: 0
>> rx32_lro_bytes: 0
>> rx32_ecn_mark: 0
>> rx32_removed_vlan_packets: 0
>> rx32_wqe_err: 0
>> rx32_mpwqe_filler_cqes: 0
>> rx32_mpwqe_filler_strides: 0
>> rx32_buff_alloc_err: 0
>> rx32_cqe_compress_blks: 0
>> rx32_cqe_compress_pkts: 0
>> rx32_page_reuse: 0
>> rx32_cache_reuse: 0
>> rx32_cache_full: 0
>> rx32_cache_empty: 2560
>> rx32_cache_busy: 0
>> rx32_cache_waive: 0
>> rx32_congst_umr: 0
>> rx32_arfs_err: 0
>> rx32_xdp_tx_xmit: 0
>> rx32_xdp_tx_full: 0
>> rx32_xdp_tx_err: 0
>> rx32_xdp_tx_cqes: 0
>> rx33_packets: 0
>> rx33_bytes: 0
>> rx33_csum_complete: 0
>> rx33_csum_unnecessary: 0
>> rx33_csum_unnecessary_inner: 0
>> rx33_csum_none: 0
>> rx33_xdp_drop: 0
>> rx33_xdp_redirect: 0
>> rx33_lro_packets: 0
>> rx33_lro_bytes: 0
>> rx33_ecn_mark: 0
>> rx33_removed_vlan_packets: 0
>> rx33_wqe_err: 0
>> rx33_mpwqe_filler_cqes: 0
>> rx33_mpwqe_filler_strides: 0
>> rx33_buff_alloc_err: 0
>> rx33_cqe_compress_blks: 0
>> rx33_cqe_compress_pkts: 0
>> rx33_page_reuse: 0
>> rx33_cache_reuse: 0
>> rx33_cache_full: 0
>> rx33_cache_empty: 2560
>> rx33_cache_busy: 0
>> rx33_cache_waive: 0
>> rx33_congst_umr: 0
>> rx33_arfs_err: 0
>> rx33_xdp_tx_xmit: 0
>> rx33_xdp_tx_full: 0
>> rx33_xdp_tx_err: 0
>> rx33_xdp_tx_cqes: 0
>> rx34_packets: 0
>> rx34_bytes: 0
>> rx34_csum_complete: 0
>> rx34_csum_unnecessary: 0
>> rx34_csum_unnecessary_inner: 0
>> rx34_csum_none: 0
>> rx34_xdp_drop: 0
>> rx34_xdp_redirect: 0
>> rx34_lro_packets: 0
>> rx34_lro_bytes: 0
>> rx34_ecn_mark: 0
>> rx34_removed_vlan_packets: 0
>> rx34_wqe_err: 0
>> rx34_mpwqe_filler_cqes: 0
>> rx34_mpwqe_filler_strides: 0
>> rx34_buff_alloc_err: 0
>> rx34_cqe_compress_blks: 0
>> rx34_cqe_compress_pkts: 0
>> rx34_page_reuse: 0
>> rx34_cache_reuse: 0
>> rx34_cache_full: 0
>> rx34_cache_empty: 2560
>> rx34_cache_busy: 0
>> rx34_cache_waive: 0
>> rx34_congst_umr: 0
>> rx34_arfs_err: 0
>> rx34_xdp_tx_xmit: 0
>> rx34_xdp_tx_full: 0
>> rx34_xdp_tx_err: 0
>> rx34_xdp_tx_cqes: 0
>> rx35_packets: 0
>> rx35_bytes: 0
>> rx35_csum_complete: 0
>> rx35_csum_unnecessary: 0
>> rx35_csum_unnecessary_inner: 0
>> rx35_csum_none: 0
>> rx35_xdp_drop: 0
>> rx35_xdp_redirect: 0
>> rx35_lro_packets: 0
>> rx35_lro_bytes: 0
>> rx35_ecn_mark: 0
>> rx35_removed_vlan_packets: 0
>> rx35_wqe_err: 0
>> rx35_mpwqe_filler_cqes: 0
>> rx35_mpwqe_filler_strides: 0
>> rx35_buff_alloc_err: 0
>> rx35_cqe_compress_blks: 0
>> rx35_cqe_compress_pkts: 0
>> rx35_page_reuse: 0
>> rx35_cache_reuse: 0
>> rx35_cache_full: 0
>> rx35_cache_empty: 2560
>> rx35_cache_busy: 0
>> rx35_cache_waive: 0
>> rx35_congst_umr: 0
>> rx35_arfs_err: 0
>> rx35_xdp_tx_xmit: 0
>> rx35_xdp_tx_full: 0
>> rx35_xdp_tx_err: 0
>> rx35_xdp_tx_cqes: 0
>> rx36_packets: 0
>> rx36_bytes: 0
>> rx36_csum_complete: 0
>> rx36_csum_unnecessary: 0
>> rx36_csum_unnecessary_inner: 0
>> rx36_csum_none: 0
>> rx36_xdp_drop: 0
>> rx36_xdp_redirect: 0
>> rx36_lro_packets: 0
>> rx36_lro_bytes: 0
>> rx36_ecn_mark: 0
>> rx36_removed_vlan_packets: 0
>> rx36_wqe_err: 0
>> rx36_mpwqe_filler_cqes: 0
>> rx36_mpwqe_filler_strides: 0
>> rx36_buff_alloc_err: 0
>> rx36_cqe_compress_blks: 0
>> rx36_cqe_compress_pkts: 0
>> rx36_page_reuse: 0
>> rx36_cache_reuse: 0
>> rx36_cache_full: 0
>> rx36_cache_empty: 2560
>> rx36_cache_busy: 0
>> rx36_cache_waive: 0
>> rx36_congst_umr: 0
>> rx36_arfs_err: 0
>> rx36_xdp_tx_xmit: 0
>> rx36_xdp_tx_full: 0
>> rx36_xdp_tx_err: 0
>> rx36_xdp_tx_cqes: 0
>> rx37_packets: 0
>> rx37_bytes: 0
>> rx37_csum_complete: 0
>> rx37_csum_unnecessary: 0
>> rx37_csum_unnecessary_inner: 0
>> rx37_csum_none: 0
>> rx37_xdp_drop: 0
>> rx37_xdp_redirect: 0
>> rx37_lro_packets: 0
>> rx37_lro_bytes: 0
>> rx37_ecn_mark: 0
>> rx37_removed_vlan_packets: 0
>> rx37_wqe_err: 0
>> rx37_mpwqe_filler_cqes: 0
>> rx37_mpwqe_filler_strides: 0
>> rx37_buff_alloc_err: 0
>> rx37_cqe_compress_blks: 0
>> rx37_cqe_compress_pkts: 0
>> rx37_page_reuse: 0
>> rx37_cache_reuse: 0
>> rx37_cache_full: 0
>> rx37_cache_empty: 2560
>> rx37_cache_busy: 0
>> rx37_cache_waive: 0
>> rx37_congst_umr: 0
>> rx37_arfs_err: 0
>> rx37_xdp_tx_xmit: 0
>> rx37_xdp_tx_full: 0
>> rx37_xdp_tx_err: 0
>> rx37_xdp_tx_cqes: 0
>> rx38_packets: 0
>> rx38_bytes: 0
>> rx38_csum_complete: 0
>> rx38_csum_unnecessary: 0
>> rx38_csum_unnecessary_inner: 0
>> rx38_csum_none: 0
>> rx38_xdp_drop: 0
>> rx38_xdp_redirect: 0
>> rx38_lro_packets: 0
>> rx38_lro_bytes: 0
>> rx38_ecn_mark: 0
>> rx38_removed_vlan_packets: 0
>> rx38_wqe_err: 0
>> rx38_mpwqe_filler_cqes: 0
>> rx38_mpwqe_filler_strides: 0
>> rx38_buff_alloc_err: 0
>> rx38_cqe_compress_blks: 0
>> rx38_cqe_compress_pkts: 0
>> rx38_page_reuse: 0
>> rx38_cache_reuse: 0
>> rx38_cache_full: 0
>> rx38_cache_empty: 2560
>> rx38_cache_busy: 0
>> rx38_cache_waive: 0
>> rx38_congst_umr: 0
>> rx38_arfs_err: 0
>> rx38_xdp_tx_xmit: 0
>> rx38_xdp_tx_full: 0
>> rx38_xdp_tx_err: 0
>> rx38_xdp_tx_cqes: 0
>> rx39_packets: 0
>> rx39_bytes: 0
>> rx39_csum_complete: 0
>> rx39_csum_unnecessary: 0
>> rx39_csum_unnecessary_inner: 0
>> rx39_csum_none: 0
>> rx39_xdp_drop: 0
>> rx39_xdp_redirect: 0
>> rx39_lro_packets: 0
>> rx39_lro_bytes: 0
>> rx39_ecn_mark: 0
>> rx39_removed_vlan_packets: 0
>> rx39_wqe_err: 0
>> rx39_mpwqe_filler_cqes: 0
>> rx39_mpwqe_filler_strides: 0
>> rx39_buff_alloc_err: 0
>> rx39_cqe_compress_blks: 0
>> rx39_cqe_compress_pkts: 0
>> rx39_page_reuse: 0
>> rx39_cache_reuse: 0
>> rx39_cache_full: 0
>> rx39_cache_empty: 2560
>> rx39_cache_busy: 0
>> rx39_cache_waive: 0
>> rx39_congst_umr: 0
>> rx39_arfs_err: 0
>> rx39_xdp_tx_xmit: 0
>> rx39_xdp_tx_full: 0
>> rx39_xdp_tx_err: 0
>> rx39_xdp_tx_cqes: 0
>> rx40_packets: 0
>> rx40_bytes: 0
>> rx40_csum_complete: 0
>> rx40_csum_unnecessary: 0
>> rx40_csum_unnecessary_inner: 0
>> rx40_csum_none: 0
>> rx40_xdp_drop: 0
>> rx40_xdp_redirect: 0
>> rx40_lro_packets: 0
>> rx40_lro_bytes: 0
>> rx40_ecn_mark: 0
>> rx40_removed_vlan_packets: 0
>> rx40_wqe_err: 0
>> rx40_mpwqe_filler_cqes: 0
>> rx40_mpwqe_filler_strides: 0
>> rx40_buff_alloc_err: 0
>> rx40_cqe_compress_blks: 0
>> rx40_cqe_compress_pkts: 0
>> rx40_page_reuse: 0
>> rx40_cache_reuse: 0
>> rx40_cache_full: 0
>> rx40_cache_empty: 2560
>> rx40_cache_busy: 0
>> rx40_cache_waive: 0
>> rx40_congst_umr: 0
>> rx40_arfs_err: 0
>> rx40_xdp_tx_xmit: 0
>> rx40_xdp_tx_full: 0
>> rx40_xdp_tx_err: 0
>> rx40_xdp_tx_cqes: 0
>> rx41_packets: 0
>> rx41_bytes: 0
>> rx41_csum_complete: 0
>> rx41_csum_unnecessary: 0
>> rx41_csum_unnecessary_inner: 0
>> rx41_csum_none: 0
>> rx41_xdp_drop: 0
>> rx41_xdp_redirect: 0
>> rx41_lro_packets: 0
>> rx41_lro_bytes: 0
>> rx41_ecn_mark: 0
>> rx41_removed_vlan_packets: 0
>> rx41_wqe_err: 0
>> rx41_mpwqe_filler_cqes: 0
>> rx41_mpwqe_filler_strides: 0
>> rx41_buff_alloc_err: 0
>> rx41_cqe_compress_blks: 0
>> rx41_cqe_compress_pkts: 0
>> rx41_page_reuse: 0
>> rx41_cache_reuse: 0
>> rx41_cache_full: 0
>> rx41_cache_empty: 2560
>> rx41_cache_busy: 0
>> rx41_cache_waive: 0
>> rx41_congst_umr: 0
>> rx41_arfs_err: 0
>> rx41_xdp_tx_xmit: 0
>> rx41_xdp_tx_full: 0
>> rx41_xdp_tx_err: 0
>> rx41_xdp_tx_cqes: 0
>> rx42_packets: 0
>> rx42_bytes: 0
>> rx42_csum_complete: 0
>> rx42_csum_unnecessary: 0
>> rx42_csum_unnecessary_inner: 0
>> rx42_csum_none: 0
>> rx42_xdp_drop: 0
>> rx42_xdp_redirect: 0
>> rx42_lro_packets: 0
>> rx42_lro_bytes: 0
>> rx42_ecn_mark: 0
>> rx42_removed_vlan_packets: 0
>> rx42_wqe_err: 0
>> rx42_mpwqe_filler_cqes: 0
>> rx42_mpwqe_filler_strides: 0
>> rx42_buff_alloc_err: 0
>> rx42_cqe_compress_blks: 0
>> rx42_cqe_compress_pkts: 0
>> rx42_page_reuse: 0
>> rx42_cache_reuse: 0
>> rx42_cache_full: 0
>> rx42_cache_empty: 2560
>> rx42_cache_busy: 0
>> rx42_cache_waive: 0
>> rx42_congst_umr: 0
>> rx42_arfs_err: 0
>> rx42_xdp_tx_xmit: 0
>> rx42_xdp_tx_full: 0
>> rx42_xdp_tx_err: 0
>> rx42_xdp_tx_cqes: 0
>> rx43_packets: 0
>> rx43_bytes: 0
>> rx43_csum_complete: 0
>> rx43_csum_unnecessary: 0
>> rx43_csum_unnecessary_inner: 0
>> rx43_csum_none: 0
>> rx43_xdp_drop: 0
>> rx43_xdp_redirect: 0
>> rx43_lro_packets: 0
>> rx43_lro_bytes: 0
>> rx43_ecn_mark: 0
>> rx43_removed_vlan_packets: 0
>> rx43_wqe_err: 0
>> rx43_mpwqe_filler_cqes: 0
>> rx43_mpwqe_filler_strides: 0
>> rx43_buff_alloc_err: 0
>> rx43_cqe_compress_blks: 0
>> rx43_cqe_compress_pkts: 0
>> rx43_page_reuse: 0
>> rx43_cache_reuse: 0
>> rx43_cache_full: 0
>> rx43_cache_empty: 2560
>> rx43_cache_busy: 0
>> rx43_cache_waive: 0
>> rx43_congst_umr: 0
>> rx43_arfs_err: 0
>> rx43_xdp_tx_xmit: 0
>> rx43_xdp_tx_full: 0
>> rx43_xdp_tx_err: 0
>> rx43_xdp_tx_cqes: 0
>> rx44_packets: 0
>> rx44_bytes: 0
>> rx44_csum_complete: 0
>> rx44_csum_unnecessary: 0
>> rx44_csum_unnecessary_inner: 0
>> rx44_csum_none: 0
>> rx44_xdp_drop: 0
>> rx44_xdp_redirect: 0
>> rx44_lro_packets: 0
>> rx44_lro_bytes: 0
>> rx44_ecn_mark: 0
>> rx44_removed_vlan_packets: 0
>> rx44_wqe_err: 0
>> rx44_mpwqe_filler_cqes: 0
>> rx44_mpwqe_filler_strides: 0
>> rx44_buff_alloc_err: 0
>> rx44_cqe_compress_blks: 0
>> rx44_cqe_compress_pkts: 0
>> rx44_page_reuse: 0
>> rx44_cache_reuse: 0
>> rx44_cache_full: 0
>> rx44_cache_empty: 2560
>> rx44_cache_busy: 0
>> rx44_cache_waive: 0
>> rx44_congst_umr: 0
>> rx44_arfs_err: 0
>> rx44_xdp_tx_xmit: 0
>> rx44_xdp_tx_full: 0
>> rx44_xdp_tx_err: 0
>> rx44_xdp_tx_cqes: 0
>> rx45_packets: 0
>> rx45_bytes: 0
>> rx45_csum_complete: 0
>> rx45_csum_unnecessary: 0
>> rx45_csum_unnecessary_inner: 0
>> rx45_csum_none: 0
>> rx45_xdp_drop: 0
>> rx45_xdp_redirect: 0
>> rx45_lro_packets: 0
>> rx45_lro_bytes: 0
>> rx45_ecn_mark: 0
>> rx45_removed_vlan_packets: 0
>> rx45_wqe_err: 0
>> rx45_mpwqe_filler_cqes: 0
>> rx45_mpwqe_filler_strides: 0
>> rx45_buff_alloc_err: 0
>> rx45_cqe_compress_blks: 0
>> rx45_cqe_compress_pkts: 0
>> rx45_page_reuse: 0
>> rx45_cache_reuse: 0
>> rx45_cache_full: 0
>> rx45_cache_empty: 2560
>> rx45_cache_busy: 0
>> rx45_cache_waive: 0
>> rx45_congst_umr: 0
>> rx45_arfs_err: 0
>> rx45_xdp_tx_xmit: 0
>> rx45_xdp_tx_full: 0
>> rx45_xdp_tx_err: 0
>> rx45_xdp_tx_cqes: 0
>> rx46_packets: 0
>> rx46_bytes: 0
>> rx46_csum_complete: 0
>> rx46_csum_unnecessary: 0
>> rx46_csum_unnecessary_inner: 0
>> rx46_csum_none: 0
>> rx46_xdp_drop: 0
>> rx46_xdp_redirect: 0
>> rx46_lro_packets: 0
>> rx46_lro_bytes: 0
>> rx46_ecn_mark: 0
>> rx46_removed_vlan_packets: 0
>> rx46_wqe_err: 0
>> rx46_mpwqe_filler_cqes: 0
>> rx46_mpwqe_filler_strides: 0
>> rx46_buff_alloc_err: 0
>> rx46_cqe_compress_blks: 0
>> rx46_cqe_compress_pkts: 0
>> rx46_page_reuse: 0
>> rx46_cache_reuse: 0
>> rx46_cache_full: 0
>> rx46_cache_empty: 2560
>> rx46_cache_busy: 0
>> rx46_cache_waive: 0
>> rx46_congst_umr: 0
>> rx46_arfs_err: 0
>> rx46_xdp_tx_xmit: 0
>> rx46_xdp_tx_full: 0
>> rx46_xdp_tx_err: 0
>> rx46_xdp_tx_cqes: 0
>> rx47_packets: 0
>> rx47_bytes: 0
>> rx47_csum_complete: 0
>> rx47_csum_unnecessary: 0
>> rx47_csum_unnecessary_inner: 0
>> rx47_csum_none: 0
>> rx47_xdp_drop: 0
>> rx47_xdp_redirect: 0
>> rx47_lro_packets: 0
>> rx47_lro_bytes: 0
>> rx47_ecn_mark: 0
>> rx47_removed_vlan_packets: 0
>> rx47_wqe_err: 0
>> rx47_mpwqe_filler_cqes: 0
>> rx47_mpwqe_filler_strides: 0
>> rx47_buff_alloc_err: 0
>> rx47_cqe_compress_blks: 0
>> rx47_cqe_compress_pkts: 0
>> rx47_page_reuse: 0
>> rx47_cache_reuse: 0
>> rx47_cache_full: 0
>> rx47_cache_empty: 2560
>> rx47_cache_busy: 0
>> rx47_cache_waive: 0
>> rx47_congst_umr: 0
>> rx47_arfs_err: 0
>> rx47_xdp_tx_xmit: 0
>> rx47_xdp_tx_full: 0
>> rx47_xdp_tx_err: 0
>> rx47_xdp_tx_cqes: 0
>> rx48_packets: 0
>> rx48_bytes: 0
>> rx48_csum_complete: 0
>> rx48_csum_unnecessary: 0
>> rx48_csum_unnecessary_inner: 0
>> rx48_csum_none: 0
>> rx48_xdp_drop: 0
>> rx48_xdp_redirect: 0
>> rx48_lro_packets: 0
>> rx48_lro_bytes: 0
>> rx48_ecn_mark: 0
>> rx48_removed_vlan_packets: 0
>> rx48_wqe_err: 0
>> rx48_mpwqe_filler_cqes: 0
>> rx48_mpwqe_filler_strides: 0
>> rx48_buff_alloc_err: 0
>> rx48_cqe_compress_blks: 0
>> rx48_cqe_compress_pkts: 0
>> rx48_page_reuse: 0
>> rx48_cache_reuse: 0
>> rx48_cache_full: 0
>> rx48_cache_empty: 2560
>> rx48_cache_busy: 0
>> rx48_cache_waive: 0
>> rx48_congst_umr: 0
>> rx48_arfs_err: 0
>> rx48_xdp_tx_xmit: 0
>> rx48_xdp_tx_full: 0
>> rx48_xdp_tx_err: 0
>> rx48_xdp_tx_cqes: 0
>> rx49_packets: 0
>> rx49_bytes: 0
>> rx49_csum_complete: 0
>> rx49_csum_unnecessary: 0
>> rx49_csum_unnecessary_inner: 0
>> rx49_csum_none: 0
>> rx49_xdp_drop: 0
>> rx49_xdp_redirect: 0
>> rx49_lro_packets: 0
>> rx49_lro_bytes: 0
>> rx49_ecn_mark: 0
>> rx49_removed_vlan_packets: 0
>> rx49_wqe_err: 0
>> rx49_mpwqe_filler_cqes: 0
>> rx49_mpwqe_filler_strides: 0
>> rx49_buff_alloc_err: 0
>> rx49_cqe_compress_blks: 0
>> rx49_cqe_compress_pkts: 0
>> rx49_page_reuse: 0
>> rx49_cache_reuse: 0
>> rx49_cache_full: 0
>> rx49_cache_empty: 2560
>> rx49_cache_busy: 0
>> rx49_cache_waive: 0
>> rx49_congst_umr: 0
>> rx49_arfs_err: 0
>> rx49_xdp_tx_xmit: 0
>> rx49_xdp_tx_full: 0
>> rx49_xdp_tx_err: 0
>> rx49_xdp_tx_cqes: 0
>> rx50_packets: 0
>> rx50_bytes: 0
>> rx50_csum_complete: 0
>> rx50_csum_unnecessary: 0
>> rx50_csum_unnecessary_inner: 0
>> rx50_csum_none: 0
>> rx50_xdp_drop: 0
>> rx50_xdp_redirect: 0
>> rx50_lro_packets: 0
>> rx50_lro_bytes: 0
>> rx50_ecn_mark: 0
>> rx50_removed_vlan_packets: 0
>> rx50_wqe_err: 0
>> rx50_mpwqe_filler_cqes: 0
>> rx50_mpwqe_filler_strides: 0
>> rx50_buff_alloc_err: 0
>> rx50_cqe_compress_blks: 0
>> rx50_cqe_compress_pkts: 0
>> rx50_page_reuse: 0
>> rx50_cache_reuse: 0
>> rx50_cache_full: 0
>> rx50_cache_empty: 2560
>> rx50_cache_busy: 0
>> rx50_cache_waive: 0
>> rx50_congst_umr: 0
>> rx50_arfs_err: 0
>> rx50_xdp_tx_xmit: 0
>> rx50_xdp_tx_full: 0
>> rx50_xdp_tx_err: 0
>> rx50_xdp_tx_cqes: 0
>> rx51_packets: 0
>> rx51_bytes: 0
>> rx51_csum_complete: 0
>> rx51_csum_unnecessary: 0
>> rx51_csum_unnecessary_inner: 0
>> rx51_csum_none: 0
>> rx51_xdp_drop: 0
>> rx51_xdp_redirect: 0
>> rx51_lro_packets: 0
>> rx51_lro_bytes: 0
>> rx51_ecn_mark: 0
>> rx51_removed_vlan_packets: 0
>> rx51_wqe_err: 0
>> rx51_mpwqe_filler_cqes: 0
>> rx51_mpwqe_filler_strides: 0
>> rx51_buff_alloc_err: 0
>> rx51_cqe_compress_blks: 0
>> rx51_cqe_compress_pkts: 0
>> rx51_page_reuse: 0
>> rx51_cache_reuse: 0
>> rx51_cache_full: 0
>> rx51_cache_empty: 2560
>> rx51_cache_busy: 0
>> rx51_cache_waive: 0
>> rx51_congst_umr: 0
>> rx51_arfs_err: 0
>> rx51_xdp_tx_xmit: 0
>> rx51_xdp_tx_full: 0
>> rx51_xdp_tx_err: 0
>> rx51_xdp_tx_cqes: 0
>> rx52_packets: 0
>> rx52_bytes: 0
>> rx52_csum_complete: 0
>> rx52_csum_unnecessary: 0
>> rx52_csum_unnecessary_inner: 0
>> rx52_csum_none: 0
>> rx52_xdp_drop: 0
>> rx52_xdp_redirect: 0
>> rx52_lro_packets: 0
>> rx52_lro_bytes: 0
>> rx52_ecn_mark: 0
>> rx52_removed_vlan_packets: 0
>> rx52_wqe_err: 0
>> rx52_mpwqe_filler_cqes: 0
>> rx52_mpwqe_filler_strides: 0
>> rx52_buff_alloc_err: 0
>> rx52_cqe_compress_blks: 0
>> rx52_cqe_compress_pkts: 0
>> rx52_page_reuse: 0
>> rx52_cache_reuse: 0
>> rx52_cache_full: 0
>> rx52_cache_empty: 2560
>> rx52_cache_busy: 0
>> rx52_cache_waive: 0
>> rx52_congst_umr: 0
>> rx52_arfs_err: 0
>> rx52_xdp_tx_xmit: 0
>> rx52_xdp_tx_full: 0
>> rx52_xdp_tx_err: 0
>> rx52_xdp_tx_cqes: 0
>> rx53_packets: 0
>> rx53_bytes: 0
>> rx53_csum_complete: 0
>> rx53_csum_unnecessary: 0
>> rx53_csum_unnecessary_inner: 0
>> rx53_csum_none: 0
>> rx53_xdp_drop: 0
>> rx53_xdp_redirect: 0
>> rx53_lro_packets: 0
>> rx53_lro_bytes: 0
>> rx53_ecn_mark: 0
>> rx53_removed_vlan_packets: 0
>> rx53_wqe_err: 0
>> rx53_mpwqe_filler_cqes: 0
>> rx53_mpwqe_filler_strides: 0
>> rx53_buff_alloc_err: 0
>> rx53_cqe_compress_blks: 0
>> rx53_cqe_compress_pkts: 0
>> rx53_page_reuse: 0
>> rx53_cache_reuse: 0
>> rx53_cache_full: 0
>> rx53_cache_empty: 2560
>> rx53_cache_busy: 0
>> rx53_cache_waive: 0
>> rx53_congst_umr: 0
>> rx53_arfs_err: 0
>> rx53_xdp_tx_xmit: 0
>> rx53_xdp_tx_full: 0
>> rx53_xdp_tx_err: 0
>> rx53_xdp_tx_cqes: 0
>> rx54_packets: 0
>> rx54_bytes: 0
>> rx54_csum_complete: 0
>> rx54_csum_unnecessary: 0
>> rx54_csum_unnecessary_inner: 0
>> rx54_csum_none: 0
>> rx54_xdp_drop: 0
>> rx54_xdp_redirect: 0
>> rx54_lro_packets: 0
>> rx54_lro_bytes: 0
>> rx54_ecn_mark: 0
>> rx54_removed_vlan_packets: 0
>> rx54_wqe_err: 0
>> rx54_mpwqe_filler_cqes: 0
>> rx54_mpwqe_filler_strides: 0
>> rx54_buff_alloc_err: 0
>> rx54_cqe_compress_blks: 0
>> rx54_cqe_compress_pkts: 0
>> rx54_page_reuse: 0
>> rx54_cache_reuse: 0
>> rx54_cache_full: 0
>> rx54_cache_empty: 2560
>> rx54_cache_busy: 0
>> rx54_cache_waive: 0
>> rx54_congst_umr: 0
>> rx54_arfs_err: 0
>> rx54_xdp_tx_xmit: 0
>> rx54_xdp_tx_full: 0
>> rx54_xdp_tx_err: 0
>> rx54_xdp_tx_cqes: 0
>> rx55_packets: 0
>> rx55_bytes: 0
>> rx55_csum_complete: 0
>> rx55_csum_unnecessary: 0
>> rx55_csum_unnecessary_inner: 0
>> rx55_csum_none: 0
>> rx55_xdp_drop: 0
>> rx55_xdp_redirect: 0
>> rx55_lro_packets: 0
>> rx55_lro_bytes: 0
>> rx55_ecn_mark: 0
>> rx55_removed_vlan_packets: 0
>> rx55_wqe_err: 0
>> rx55_mpwqe_filler_cqes: 0
>> rx55_mpwqe_filler_strides: 0
>> rx55_buff_alloc_err: 0
>> rx55_cqe_compress_blks: 0
>> rx55_cqe_compress_pkts: 0
>> rx55_page_reuse: 0
>> rx55_cache_reuse: 0
>> rx55_cache_full: 0
>> rx55_cache_empty: 2560
>> rx55_cache_busy: 0
>> rx55_cache_waive: 0
>> rx55_congst_umr: 0
>> rx55_arfs_err: 0
>> rx55_xdp_tx_xmit: 0
>> rx55_xdp_tx_full: 0
>> rx55_xdp_tx_err: 0
>> rx55_xdp_tx_cqes: 0
>> tx0_packets: 6019477917
>> tx0_bytes: 3445238940825
>> tx0_tso_packets: 311304622
>> tx0_tso_bytes: 1897094773213
>> tx0_tso_inner_packets: 0
>> tx0_tso_inner_bytes: 0
>> tx0_csum_partial: 457981794
>> tx0_csum_partial_inner: 0
>> tx0_added_vlan_packets: 4965567654
>> tx0_nop: 72290329
>> tx0_csum_none: 4507585860
>> tx0_stopped: 9118
>> tx0_dropped: 0
>> tx0_xmit_more: 51651593
>> tx0_recover: 0
>> tx0_cqes: 4913918402
>> tx0_wake: 9118
>> tx0_cqe_err: 0
>> tx1_packets: 5700413414
>> tx1_bytes: 3340870662350
>> tx1_tso_packets: 318201557
>> tx1_tso_bytes: 1915233462303
>> tx1_tso_inner_packets: 0
>> tx1_tso_inner_bytes: 0
>> tx1_csum_partial: 461736722
>> tx1_csum_partial_inner: 0
>> tx1_added_vlan_packets: 4638708749
>> tx1_nop: 70061796
>> tx1_csum_none: 4176972027
>> tx1_stopped: 9248
>> tx1_dropped: 0
>> tx1_xmit_more: 39531959
>> tx1_recover: 0
>> tx1_cqes: 4599179178
>> tx1_wake: 9248
>> tx1_cqe_err: 0
>> tx2_packets: 5795960848
>> tx2_bytes: 3394876820271
>> tx2_tso_packets: 322935065
>> tx2_tso_bytes: 1910825901109
>> tx2_tso_inner_packets: 0
>> tx2_tso_inner_bytes: 0
>> tx2_csum_partial: 460747092
>> tx2_csum_partial_inner: 0
>> tx2_added_vlan_packets: 4743705654
>> tx2_nop: 72722430
>> tx2_csum_none: 4282958562
>> tx2_stopped: 8938
>> tx2_dropped: 0
>> tx2_xmit_more: 44084718
>> tx2_recover: 0
>> tx2_cqes: 4699623410
>> tx2_wake: 8938
>> tx2_cqe_err: 0
>> tx3_packets: 5580215878
>> tx3_bytes: 3191677257787
>> tx3_tso_packets: 305771141
>> tx3_tso_bytes: 1823265793476
>> tx3_tso_inner_packets: 0
>> tx3_tso_inner_bytes: 0
>> tx3_csum_partial: 434976070
>> tx3_csum_partial_inner: 0
>> tx3_added_vlan_packets: 4569899956
>> tx3_nop: 68184348
>> tx3_csum_none: 4134923886
>> tx3_stopped: 8383
>> tx3_dropped: 0
>> tx3_xmit_more: 41940375
>> tx3_recover: 0
>> tx3_cqes: 4527961924
>> tx3_wake: 8383
>> tx3_cqe_err: 0
>> tx4_packets: 6795007068
>> tx4_bytes: 3963890025270
>> tx4_tso_packets: 358437617
>> tx4_tso_bytes: 2154747995355
>> tx4_tso_inner_packets: 0
>> tx4_tso_inner_bytes: 0
>> tx4_csum_partial: 504764524
>> tx4_csum_partial_inner: 0
>> tx4_added_vlan_packets: 5602510191
>> tx4_nop: 81345604
>> tx4_csum_none: 5097745667
>> tx4_stopped: 10248
>> tx4_dropped: 0
>> tx4_xmit_more: 49068571
>> tx4_recover: 0
>> tx4_cqes: 5553444276
>> tx4_wake: 10248
>> tx4_cqe_err: 0
>> tx5_packets: 6408089261
>> tx5_bytes: 3676275848279
>> tx5_tso_packets: 345129329
>> tx5_tso_bytes: 2108447877473
>> tx5_tso_inner_packets: 0
>> tx5_tso_inner_bytes: 0
>> tx5_csum_partial: 494705894
>> tx5_csum_partial_inner: 0
>> tx5_added_vlan_packets: 5235998343
>> tx5_nop: 77694627
>> tx5_csum_none: 4741292449
>> tx5_stopped: 46
>> tx5_dropped: 0
>> tx5_xmit_more: 46675831
>> tx5_recover: 0
>> tx5_cqes: 5189323550
>> tx5_wake: 46
>> tx5_cqe_err: 0
>> tx6_packets: 6382289663
>> tx6_bytes: 3670991418150
>> tx6_tso_packets: 342927826
>> tx6_tso_bytes: 2075049679904
>> tx6_tso_inner_packets: 0
>> tx6_tso_inner_bytes: 0
>> tx6_csum_partial: 490369221
>> tx6_csum_partial_inner: 0
>> tx6_added_vlan_packets: 5232144528
>> tx6_nop: 77391246
>> tx6_csum_none: 4741775307
>> tx6_stopped: 10823
>> tx6_dropped: 0
>> tx6_xmit_more: 44487607
>> tx6_recover: 0
>> tx6_cqes: 5187659877
>> tx6_wake: 10823
>> tx6_cqe_err: 0
>> tx7_packets: 6456378284
>> tx7_bytes: 3758013320518
>> tx7_tso_packets: 350958294
>> tx7_tso_bytes: 2126833408524
>> tx7_tso_inner_packets: 0
>> tx7_tso_inner_bytes: 0
>> tx7_csum_partial: 501804109
>> tx7_csum_partial_inner: 0
>> tx7_added_vlan_packets: 5275635204
>> tx7_nop: 79010883
>> tx7_csum_none: 4773831096
>> tx7_stopped: 14684
>> tx7_dropped: 0
>> tx7_xmit_more: 44447469
>> tx7_recover: 0
>> tx7_cqes: 5231191770
>> tx7_wake: 14684
>> tx7_cqe_err: 0
>> tx8_packets: 6401799768
>> tx8_bytes: 3681210808766
>> tx8_tso_packets: 342878228
>> tx8_tso_bytes: 2089688012191
>> tx8_tso_inner_packets: 0
>> tx8_tso_inner_bytes: 0
>> tx8_csum_partial: 494865145
>> tx8_csum_partial_inner: 0
>> tx8_added_vlan_packets: 5242288908
>> tx8_nop: 77250910
>> tx8_csum_none: 4747423763
>> tx8_stopped: 2
>> tx8_dropped: 0
>> tx8_xmit_more: 44191737
>> tx8_recover: 0
>> tx8_cqes: 5198098454
>> tx8_wake: 2
>> tx8_cqe_err: 0
>> tx9_packets: 6632882888
>> tx9_bytes: 3820110338309
>> tx9_tso_packets: 354189056
>> tx9_tso_bytes: 2187883597128
>> tx9_tso_inner_packets: 0
>> tx9_tso_inner_bytes: 0
>> tx9_csum_partial: 511108218
>> tx9_csum_partial_inner: 0
>> tx9_added_vlan_packets: 5413836353
>> tx9_nop: 80560668
>> tx9_csum_none: 4902728135
>> tx9_stopped: 9091
>> tx9_dropped: 0
>> tx9_xmit_more: 54501293
>> tx9_recover: 0
>> tx9_cqes: 5359337638
>> tx9_wake: 9091
>> tx9_cqe_err: 0
>> tx10_packets: 6421786406
>> tx10_bytes: 3692798413429
>> tx10_tso_packets: 346878943
>> tx10_tso_bytes: 2111921062110
>> tx10_tso_inner_packets: 0
>> tx10_tso_inner_bytes: 0
>> tx10_csum_partial: 494356645
>> tx10_csum_partial_inner: 0
>> tx10_added_vlan_packets: 5248274374
>> tx10_nop: 77922624
>> tx10_csum_none: 4753917730
>> tx10_stopped: 9617
>> tx10_dropped: 0
>> tx10_xmit_more: 44473939
>> tx10_recover: 0
>> tx10_cqes: 5203802547
>> tx10_wake: 9617
>> tx10_cqe_err: 0
>> tx11_packets: 6406750938
>> tx11_bytes: 3660343565126
>> tx11_tso_packets: 355917271
>> tx11_tso_bytes: 2130812246956
>> tx11_tso_inner_packets: 0
>> tx11_tso_inner_bytes: 0
>> tx11_csum_partial: 500336369
>> tx11_csum_partial_inner: 0
>> tx11_added_vlan_packets: 5228267547
>> tx11_nop: 78906315
>> tx11_csum_none: 4727931178
>> tx11_stopped: 9607
>> tx11_dropped: 0
>> tx11_xmit_more: 40041492
>> tx11_recover: 0
>> tx11_cqes: 5188228290
>> tx11_wake: 9607
>> tx11_cqe_err: 0
>> tx12_packets: 6422347846
>> tx12_bytes: 3718772753227
>> tx12_tso_packets: 355397223
>> tx12_tso_bytes: 2162614059758
>> tx12_tso_inner_packets: 0
>> tx12_tso_inner_bytes: 0
>> tx12_csum_partial: 511437844
>> tx12_csum_partial_inner: 0
>> tx12_added_vlan_packets: 5221373746
>> tx12_nop: 78866779
>> tx12_csum_none: 4709935902
>> tx12_stopped: 10280
>> tx12_dropped: 0
>> tx12_xmit_more: 42189399
>> tx12_recover: 0
>> tx12_cqes: 5179187154
>> tx12_wake: 10280
>> tx12_cqe_err: 0
>> tx13_packets: 6429383816
>> tx13_bytes: 3725679445046
>> tx13_tso_packets: 360934759
>> tx13_tso_bytes: 2148016411436
>> tx13_tso_inner_packets: 0
>> tx13_tso_inner_bytes: 0
>> tx13_csum_partial: 505245849
>> tx13_csum_partial_inner: 0
>> tx13_added_vlan_packets: 5240267441
>> tx13_nop: 80295637
>> tx13_csum_none: 4735021592
>> tx13_stopped: 84
>> tx13_dropped: 0
>> tx13_xmit_more: 43118045
>> tx13_recover: 0
>> tx13_cqes: 5197150348
>> tx13_wake: 84
>> tx13_cqe_err: 0
>> tx14_packets: 6375279148
>> tx14_bytes: 3624267203336
>> tx14_tso_packets: 344388148
>> tx14_tso_bytes: 2094966273548
>> tx14_tso_inner_packets: 0
>> tx14_tso_inner_bytes: 0
>> tx14_csum_partial: 494129407
>> tx14_csum_partial_inner: 0
>> tx14_added_vlan_packets: 5210749337
>> tx14_nop: 77280615
>> tx14_csum_none: 4716619930
>> tx14_stopped: 13057
>> tx14_dropped: 0
>> tx14_xmit_more: 40849682
>> tx14_recover: 0
>> tx14_cqes: 5169902694
>> tx14_wake: 13057
>> tx14_cqe_err: 0
>> tx15_packets: 6489306520
>> tx15_bytes: 3775716194795
>> tx15_tso_packets: 368716406
>> tx15_tso_bytes: 2165876423354
>> tx15_tso_inner_packets: 0
>> tx15_tso_inner_bytes: 0
>> tx15_csum_partial: 509887864
>> tx15_csum_partial_inner: 0
>> tx15_added_vlan_packets: 5296767390
>> tx15_nop: 80803468
>> tx15_csum_none: 4786879529
>> tx15_stopped: 1
>> tx15_dropped: 0
>> tx15_xmit_more: 46979676
>> tx15_recover: 0
>> tx15_cqes: 5249789328
>> tx15_wake: 1
>> tx15_cqe_err: 0
>> tx16_packets: 6559857761
>> tx16_bytes: 3724080573905
>> tx16_tso_packets: 350864176
>> tx16_tso_bytes: 2099634006033
>> tx16_tso_inner_packets: 0
>> tx16_tso_inner_bytes: 0
>> tx16_csum_partial: 489397232
>> tx16_csum_partial_inner: 0
>> tx16_added_vlan_packets: 5398869334
>> tx16_nop: 79046075
>> tx16_csum_none: 4909472106
>> tx16_stopped: 4480
>> tx16_dropped: 0
>> tx16_xmit_more: 47273286
>> tx16_recover: 0
>> tx16_cqes: 5351598315
>> tx16_wake: 4480
>> tx16_cqe_err: 0
>> tx17_packets: 6358711533
>> tx17_bytes: 3650180865573
>> tx17_tso_packets: 350723136
>> tx17_tso_bytes: 2109426587128
>> tx17_tso_inner_packets: 0
>> tx17_tso_inner_bytes: 0
>> tx17_csum_partial: 494719487
>> tx17_csum_partial_inner: 0
>> tx17_added_vlan_packets: 5190068796
>> tx17_nop: 77285612
>> tx17_csum_none: 4695349309
>> tx17_stopped: 10443
>> tx17_dropped: 0
>> tx17_xmit_more: 45582108
>> tx17_recover: 0
>> tx17_cqes: 5144489363
>> tx17_wake: 10443
>> tx17_cqe_err: 0
>> tx18_packets: 6655328437
>> tx18_bytes: 3801768461807
>> tx18_tso_packets: 356516373
>> tx18_tso_bytes: 2164829247550
>> tx18_tso_inner_packets: 0
>> tx18_tso_inner_bytes: 0
>> tx18_csum_partial: 500508446
>> tx18_csum_partial_inner: 0
>> tx18_added_vlan_packets: 5454166840
>> tx18_nop: 80423007
>> tx18_csum_none: 4953658394
>> tx18_stopped: 14760
>> tx18_dropped: 0
>> tx18_xmit_more: 50837465
>> tx18_recover: 0
>> tx18_cqes: 5403332553
>> tx18_wake: 14760
>> tx18_cqe_err: 0
>> tx19_packets: 6408680611
>> tx19_bytes: 3644119934372
>> tx19_tso_packets: 350727530
>> tx19_tso_bytes: 2089896715365
>> tx19_tso_inner_packets: 0
>> tx19_tso_inner_bytes: 0
>> tx19_csum_partial: 486536490
>> tx19_csum_partial_inner: 0
>> tx19_added_vlan_packets: 5255839020
>> tx19_nop: 78525198
>> tx19_csum_none: 4769302530
>> tx19_stopped: 8614
>> tx19_dropped: 0
>> tx19_xmit_more: 43605232
>> tx19_recover: 0
>> tx19_cqes: 5212236833
>> tx19_wake: 8614
>> tx19_cqe_err: 0
>> tx20_packets: 5609275141
>> tx20_bytes: 3187279031581
>> tx20_tso_packets: 298609303
>> tx20_tso_bytes: 1794382229379
>> tx20_tso_inner_packets: 0
>> tx20_tso_inner_bytes: 0
>> tx20_csum_partial: 430691178
>> tx20_csum_partial_inner: 0
>> tx20_added_vlan_packets: 4616844286
>> tx20_nop: 67450040
>> tx20_csum_none: 4186153108
>> tx20_stopped: 9099
>> tx20_dropped: 0
>> tx20_xmit_more: 42040991
>> tx20_recover: 0
>> tx20_cqes: 4574805846
>> tx20_wake: 9099
>> tx20_cqe_err: 0
>> tx21_packets: 5641621183
>> tx21_bytes: 3279282331124
>> tx21_tso_packets: 311297057
>> tx21_tso_bytes: 1875735401012
>> tx21_tso_inner_packets: 0
>> tx21_tso_inner_bytes: 0
>> tx21_csum_partial: 444333894
>> tx21_csum_partial_inner: 0
>> tx21_added_vlan_packets: 4603527701
>> tx21_nop: 68857983
>> tx21_csum_none: 4159193807
>> tx21_stopped: 10082
>> tx21_dropped: 0
>> tx21_xmit_more: 43988081
>> tx21_recover: 0
>> tx21_cqes: 4559542410
>> tx21_wake: 10082
>> tx21_cqe_err: 0
>> tx22_packets: 5822168288
>> tx22_bytes: 3452026726862
>> tx22_tso_packets: 308230791
>> tx22_tso_bytes: 1859686450671
>> tx22_tso_inner_packets: 0
>> tx22_tso_inner_bytes: 0
>> tx22_csum_partial: 442751518
>> tx22_csum_partial_inner: 0
>> tx22_added_vlan_packets: 4792100335
>> tx22_nop: 70631706
>> tx22_csum_none: 4349348817
>> tx22_stopped: 9355
>> tx22_dropped: 0
>> tx22_xmit_more: 45165994
>> tx22_recover: 0
>> tx22_cqes: 4746936601
>> tx22_wake: 9355
>> tx22_cqe_err: 0
>> tx23_packets: 5664896066
>> tx23_bytes: 3207724186946
>> tx23_tso_packets: 300418757
>> tx23_tso_bytes: 1794180478679
>> tx23_tso_inner_packets: 0
>> tx23_tso_inner_bytes: 0
>> tx23_csum_partial: 429898848
>> tx23_csum_partial_inner: 0
>> tx23_added_vlan_packets: 4674317320
>> tx23_nop: 67899896
>> tx23_csum_none: 4244418472
>> tx23_stopped: 11684
>> tx23_dropped: 0
>> tx23_xmit_more: 43351132
>> tx23_recover: 0
>> tx23_cqes: 4630969028
>> tx23_wake: 11684
>> tx23_cqe_err: 0
>> tx24_packets: 5663326601
>> tx24_bytes: 3250127095110
>> tx24_tso_packets: 301327422
>> tx24_tso_bytes: 1831260534157
>> tx24_tso_inner_packets: 0
>> tx24_tso_inner_bytes: 0
>> tx24_csum_partial: 438757312
>> tx24_csum_partial_inner: 0
>> tx24_added_vlan_packets: 4646014986
>> tx24_nop: 68431153
>> tx24_csum_none: 4207257674
>> tx24_stopped: 9240
>> tx24_dropped: 0
>> tx24_xmit_more: 47699542
>> tx24_recover: 0
>> tx24_cqes: 4598317913
>> tx24_wake: 9240
>> tx24_cqe_err: 0
>> tx25_packets: 5703883962
>> tx25_bytes: 3291856915695
>> tx25_tso_packets: 308900318
>> tx25_tso_bytes: 1855516128386
>> tx25_tso_inner_packets: 0
>> tx25_tso_inner_bytes: 0
>> tx25_csum_partial: 444753744
>> tx25_csum_partial_inner: 0
>> tx25_added_vlan_packets: 4676528924
>> tx25_nop: 69230967
>> tx25_csum_none: 4231775180
>> tx25_stopped: 1140
>> tx25_dropped: 0
>> tx25_xmit_more: 40819195
>> tx25_recover: 0
>> tx25_cqes: 4635710966
>> tx25_wake: 1140
>> tx25_cqe_err: 0
>> tx26_packets: 5803495984
>> tx26_bytes: 3413564272139
>> tx26_tso_packets: 319986230
>> tx26_tso_bytes: 1929042839677
>> tx26_tso_inner_packets: 0
>> tx26_tso_inner_bytes: 0
>> tx26_csum_partial: 464771163
>> tx26_csum_partial_inner: 0
>> tx26_added_vlan_packets: 4734767280
>> tx26_nop: 71345080
>> tx26_csum_none: 4269996117
>> tx26_stopped: 10972
>> tx26_dropped: 0
>> tx26_xmit_more: 43793424
>> tx26_recover: 0
>> tx26_cqes: 4690976400
>> tx26_wake: 10972
>> tx26_cqe_err: 0
>> tx27_packets: 5960955343
>> tx27_bytes: 3444156164526
>> tx27_tso_packets: 325099639
>> tx27_tso_bytes: 1928378678784
>> tx27_tso_inner_packets: 0
>> tx27_tso_inner_bytes: 0
>> tx27_csum_partial: 467310289
>> tx27_csum_partial_inner: 0
>> tx27_added_vlan_packets: 4888651368
>> tx27_nop: 73201664
>> tx27_csum_none: 4421341079
>> tx27_stopped: 9465
>> tx27_dropped: 0
>> tx27_xmit_more: 53632121
>> tx27_recover: 0
>> tx27_cqes: 4835021398
>> tx27_wake: 9465
>> tx27_cqe_err: 0
>> tx28_packets: 0
>> tx28_bytes: 0
>> tx28_tso_packets: 0
>> tx28_tso_bytes: 0
>> tx28_tso_inner_packets: 0
>> tx28_tso_inner_bytes: 0
>> tx28_csum_partial: 0
>> tx28_csum_partial_inner: 0
>> tx28_added_vlan_packets: 0
>> tx28_nop: 0
>> tx28_csum_none: 0
>> tx28_stopped: 0
>> tx28_dropped: 0
>> tx28_xmit_more: 0
>> tx28_recover: 0
>> tx28_cqes: 0
>> tx28_wake: 0
>> tx28_cqe_err: 0
>> tx29_packets: 3
>> tx29_bytes: 266
>> tx29_tso_packets: 0
>> tx29_tso_bytes: 0
>> tx29_tso_inner_packets: 0
>> tx29_tso_inner_bytes: 0
>> tx29_csum_partial: 0
>> tx29_csum_partial_inner: 0
>> tx29_added_vlan_packets: 0
>> tx29_nop: 0
>> tx29_csum_none: 3
>> tx29_stopped: 0
>> tx29_dropped: 0
>> tx29_xmit_more: 1
>> tx29_recover: 0
>> tx29_cqes: 2
>> tx29_wake: 0
>> tx29_cqe_err: 0
>> tx30_packets: 0
>> tx30_bytes: 0
>> tx30_tso_packets: 0
>> tx30_tso_bytes: 0
>> tx30_tso_inner_packets: 0
>> tx30_tso_inner_bytes: 0
>> tx30_csum_partial: 0
>> tx30_csum_partial_inner: 0
>> tx30_added_vlan_packets: 0
>> tx30_nop: 0
>> tx30_csum_none: 0
>> tx30_stopped: 0
>> tx30_dropped: 0
>> tx30_xmit_more: 0
>> tx30_recover: 0
>> tx30_cqes: 0
>> tx30_wake: 0
>> tx30_cqe_err: 0
>> tx31_packets: 0
>> tx31_bytes: 0
>> tx31_tso_packets: 0
>> tx31_tso_bytes: 0
>> tx31_tso_inner_packets: 0
>> tx31_tso_inner_bytes: 0
>> tx31_csum_partial: 0
>> tx31_csum_partial_inner: 0
>> tx31_added_vlan_packets: 0
>> tx31_nop: 0
>> tx31_csum_none: 0
>> tx31_stopped: 0
>> tx31_dropped: 0
>> tx31_xmit_more: 0
>> tx31_recover: 0
>> tx31_cqes: 0
>> tx31_wake: 0
>> tx31_cqe_err: 0
>> tx32_packets: 0
>> tx32_bytes: 0
>> tx32_tso_packets: 0
>> tx32_tso_bytes: 0
>> tx32_tso_inner_packets: 0
>> tx32_tso_inner_bytes: 0
>> tx32_csum_partial: 0
>> tx32_csum_partial_inner: 0
>> tx32_added_vlan_packets: 0
>> tx32_nop: 0
>> tx32_csum_none: 0
>> tx32_stopped: 0
>> tx32_dropped: 0
>> tx32_xmit_more: 0
>> tx32_recover: 0
>> tx32_cqes: 0
>> tx32_wake: 0
>> tx32_cqe_err: 0
>> tx33_packets: 0
>> tx33_bytes: 0
>> tx33_tso_packets: 0
>> tx33_tso_bytes: 0
>> tx33_tso_inner_packets: 0
>> tx33_tso_inner_bytes: 0
>> tx33_csum_partial: 0
>> tx33_csum_partial_inner: 0
>> tx33_added_vlan_packets: 0
>> tx33_nop: 0
>> tx33_csum_none: 0
>> tx33_stopped: 0
>> tx33_dropped: 0
>> tx33_xmit_more: 0
>> tx33_recover: 0
>> tx33_cqes: 0
>> tx33_wake: 0
>> tx33_cqe_err: 0
>> tx34_packets: 0
>> tx34_bytes: 0
>> tx34_tso_packets: 0
>> tx34_tso_bytes: 0
>> tx34_tso_inner_packets: 0
>> tx34_tso_inner_bytes: 0
>> tx34_csum_partial: 0
>> tx34_csum_partial_inner: 0
>> tx34_added_vlan_packets: 0
>> tx34_nop: 0
>> tx34_csum_none: 0
>> tx34_stopped: 0
>> tx34_dropped: 0
>> tx34_xmit_more: 0
>> tx34_recover: 0
>> tx34_cqes: 0
>> tx34_wake: 0
>> tx34_cqe_err: 0
>> tx35_packets: 0
>> tx35_bytes: 0
>> tx35_tso_packets: 0
>> tx35_tso_bytes: 0
>> tx35_tso_inner_packets: 0
>> tx35_tso_inner_bytes: 0
>> tx35_csum_partial: 0
>> tx35_csum_partial_inner: 0
>> tx35_added_vlan_packets: 0
>> tx35_nop: 0
>> tx35_csum_none: 0
>> tx35_stopped: 0
>> tx35_dropped: 0
>> tx35_xmit_more: 0
>> tx35_recover: 0
>> tx35_cqes: 0
>> tx35_wake: 0
>> tx35_cqe_err: 0
>> tx36_packets: 0
>> tx36_bytes: 0
>> tx36_tso_packets: 0
>> tx36_tso_bytes: 0
>> tx36_tso_inner_packets: 0
>> tx36_tso_inner_bytes: 0
>> tx36_csum_partial: 0
>> tx36_csum_partial_inner: 0
>> tx36_added_vlan_packets: 0
>> tx36_nop: 0
>> tx36_csum_none: 0
>> tx36_stopped: 0
>> tx36_dropped: 0
>> tx36_xmit_more: 0
>> tx36_recover: 0
>> tx36_cqes: 0
>> tx36_wake: 0
>> tx36_cqe_err: 0
>> tx37_packets: 0
>> tx37_bytes: 0
>> tx37_tso_packets: 0
>> tx37_tso_bytes: 0
>> tx37_tso_inner_packets: 0
>> tx37_tso_inner_bytes: 0
>> tx37_csum_partial: 0
>> tx37_csum_partial_inner: 0
>> tx37_added_vlan_packets: 0
>> tx37_nop: 0
>> tx37_csum_none: 0
>> tx37_stopped: 0
>> tx37_dropped: 0
>> tx37_xmit_more: 0
>> tx37_recover: 0
>> tx37_cqes: 0
>> tx37_wake: 0
>> tx37_cqe_err: 0
>> tx38_packets: 0
>> tx38_bytes: 0
>> tx38_tso_packets: 0
>> tx38_tso_bytes: 0
>> tx38_tso_inner_packets: 0
>> tx38_tso_inner_bytes: 0
>> tx38_csum_partial: 0
>> tx38_csum_partial_inner: 0
>> tx38_added_vlan_packets: 0
>> tx38_nop: 0
>> tx38_csum_none: 0
>> tx38_stopped: 0
>> tx38_dropped: 0
>> tx38_xmit_more: 0
>> tx38_recover: 0
>> tx38_cqes: 0
>> tx38_wake: 0
>> tx38_cqe_err: 0
>> tx39_packets: 0
>> tx39_bytes: 0
>> tx39_tso_packets: 0
>> tx39_tso_bytes: 0
>> tx39_tso_inner_packets: 0
>> tx39_tso_inner_bytes: 0
>> tx39_csum_partial: 0
>> tx39_csum_partial_inner: 0
>> tx39_added_vlan_packets: 0
>> tx39_nop: 0
>> tx39_csum_none: 0
>> tx39_stopped: 0
>> tx39_dropped: 0
>> tx39_xmit_more: 0
>> tx39_recover: 0
>> tx39_cqes: 0
>> tx39_wake: 0
>> tx39_cqe_err: 0
>> tx40_packets: 0
>> tx40_bytes: 0
>> tx40_tso_packets: 0
>> tx40_tso_bytes: 0
>> tx40_tso_inner_packets: 0
>> tx40_tso_inner_bytes: 0
>> tx40_csum_partial: 0
>> tx40_csum_partial_inner: 0
>> tx40_added_vlan_packets: 0
>> tx40_nop: 0
>> tx40_csum_none: 0
>> tx40_stopped: 0
>> tx40_dropped: 0
>> tx40_xmit_more: 0
>> tx40_recover: 0
>> tx40_cqes: 0
>> tx40_wake: 0
>> tx40_cqe_err: 0
>> tx41_packets: 0
>> tx41_bytes: 0
>> tx41_tso_packets: 0
>> tx41_tso_bytes: 0
>> tx41_tso_inner_packets: 0
>> tx41_tso_inner_bytes: 0
>> tx41_csum_partial: 0
>> tx41_csum_partial_inner: 0
>> tx41_added_vlan_packets: 0
>> tx41_nop: 0
>> tx41_csum_none: 0
>> tx41_stopped: 0
>> tx41_dropped: 0
>> tx41_xmit_more: 0
>> tx41_recover: 0
>> tx41_cqes: 0
>> tx41_wake: 0
>> tx41_cqe_err: 0
>> tx42_packets: 0
>> tx42_bytes: 0
>> tx42_tso_packets: 0
>> tx42_tso_bytes: 0
>> tx42_tso_inner_packets: 0
>> tx42_tso_inner_bytes: 0
>> tx42_csum_partial: 0
>> tx42_csum_partial_inner: 0
>> tx42_added_vlan_packets: 0
>> tx42_nop: 0
>> tx42_csum_none: 0
>> tx42_stopped: 0
>> tx42_dropped: 0
>> tx42_xmit_more: 0
>> tx42_recover: 0
>> tx42_cqes: 0
>> tx42_wake: 0
>> tx42_cqe_err: 0
>> tx43_packets: 0
>> tx43_bytes: 0
>> tx43_tso_packets: 0
>> tx43_tso_bytes: 0
>> tx43_tso_inner_packets: 0
>> tx43_tso_inner_bytes: 0
>> tx43_csum_partial: 0
>> tx43_csum_partial_inner: 0
>> tx43_added_vlan_packets: 0
>> tx43_nop: 0
>> tx43_csum_none: 0
>> tx43_stopped: 0
>> tx43_dropped: 0
>> tx43_xmit_more: 0
>> tx43_recover: 0
>> tx43_cqes: 0
>> tx43_wake: 0
>> tx43_cqe_err: 0
>> tx44_packets: 0
>> tx44_bytes: 0
>> tx44_tso_packets: 0
>> tx44_tso_bytes: 0
>> tx44_tso_inner_packets: 0
>> tx44_tso_inner_bytes: 0
>> tx44_csum_partial: 0
>> tx44_csum_partial_inner: 0
>> tx44_added_vlan_packets: 0
>> tx44_nop: 0
>> tx44_csum_none: 0
>> tx44_stopped: 0
>> tx44_dropped: 0
>> tx44_xmit_more: 0
>> tx44_recover: 0
>> tx44_cqes: 0
>> tx44_wake: 0
>> tx44_cqe_err: 0
>> tx45_packets: 0
>> tx45_bytes: 0
>> tx45_tso_packets: 0
>> tx45_tso_bytes: 0
>> tx45_tso_inner_packets: 0
>> tx45_tso_inner_bytes: 0
>> tx45_csum_partial: 0
>> tx45_csum_partial_inner: 0
>> tx45_added_vlan_packets: 0
>> tx45_nop: 0
>> tx45_csum_none: 0
>> tx45_stopped: 0
>> tx45_dropped: 0
>> tx45_xmit_more: 0
>> tx45_recover: 0
>> tx45_cqes: 0
>> tx45_wake: 0
>> tx45_cqe_err: 0
>> tx46_packets: 0
>> tx46_bytes: 0
>> tx46_tso_packets: 0
>> tx46_tso_bytes: 0
>> tx46_tso_inner_packets: 0
>> tx46_tso_inner_bytes: 0
>> tx46_csum_partial: 0
>> tx46_csum_partial_inner: 0
>> tx46_added_vlan_packets: 0
>> tx46_nop: 0
>> tx46_csum_none: 0
>> tx46_stopped: 0
>> tx46_dropped: 0
>> tx46_xmit_more: 0
>> tx46_recover: 0
>> tx46_cqes: 0
>> tx46_wake: 0
>> tx46_cqe_err: 0
>> tx47_packets: 0
>> tx47_bytes: 0
>> tx47_tso_packets: 0
>> tx47_tso_bytes: 0
>> tx47_tso_inner_packets: 0
>> tx47_tso_inner_bytes: 0
>> tx47_csum_partial: 0
>> tx47_csum_partial_inner: 0
>> tx47_added_vlan_packets: 0
>> tx47_nop: 0
>> tx47_csum_none: 0
>> tx47_stopped: 0
>> tx47_dropped: 0
>> tx47_xmit_more: 0
>> tx47_recover: 0
>> tx47_cqes: 0
>> tx47_wake: 0
>> tx47_cqe_err: 0
>> tx48_packets: 0
>> tx48_bytes: 0
>> tx48_tso_packets: 0
>> tx48_tso_bytes: 0
>> tx48_tso_inner_packets: 0
>> tx48_tso_inner_bytes: 0
>> tx48_csum_partial: 0
>> tx48_csum_partial_inner: 0
>> tx48_added_vlan_packets: 0
>> tx48_nop: 0
>> tx48_csum_none: 0
>> tx48_stopped: 0
>> tx48_dropped: 0
>> tx48_xmit_more: 0
>> tx48_recover: 0
>> tx48_cqes: 0
>> tx48_wake: 0
>> tx48_cqe_err: 0
>> tx49_packets: 0
>> tx49_bytes: 0
>> tx49_tso_packets: 0
>> tx49_tso_bytes: 0
>> tx49_tso_inner_packets: 0
>> tx49_tso_inner_bytes: 0
>> tx49_csum_partial: 0
>> tx49_csum_partial_inner: 0
>> tx49_added_vlan_packets: 0
>> tx49_nop: 0
>> tx49_csum_none: 0
>> tx49_stopped: 0
>> tx49_dropped: 0
>> tx49_xmit_more: 0
>> tx49_recover: 0
>> tx49_cqes: 0
>> tx49_wake: 0
>> tx49_cqe_err: 0
>> tx50_packets: 0
>> tx50_bytes: 0
>> tx50_tso_packets: 0
>> tx50_tso_bytes: 0
>> tx50_tso_inner_packets: 0
>> tx50_tso_inner_bytes: 0
>> tx50_csum_partial: 0
>> tx50_csum_partial_inner: 0
>> tx50_added_vlan_packets: 0
>> tx50_nop: 0
>> tx50_csum_none: 0
>> tx50_stopped: 0
>> tx50_dropped: 0
>> tx50_xmit_more: 0
>> tx50_recover: 0
>> tx50_cqes: 0
>> tx50_wake: 0
>> tx50_cqe_err: 0
>> tx51_packets: 0
>> tx51_bytes: 0
>> tx51_tso_packets: 0
>> tx51_tso_bytes: 0
>> tx51_tso_inner_packets: 0
>> tx51_tso_inner_bytes: 0
>> tx51_csum_partial: 0
>> tx51_csum_partial_inner: 0
>> tx51_added_vlan_packets: 0
>> tx51_nop: 0
>> tx51_csum_none: 0
>> tx51_stopped: 0
>> tx51_dropped: 0
>> tx51_xmit_more: 0
>> tx51_recover: 0
>> tx51_cqes: 0
>> tx51_wake: 0
>> tx51_cqe_err: 0
>> tx52_packets: 0
>> tx52_bytes: 0
>> tx52_tso_packets: 0
>> tx52_tso_bytes: 0
>> tx52_tso_inner_packets: 0
>> tx52_tso_inner_bytes: 0
>> tx52_csum_partial: 0
>> tx52_csum_partial_inner: 0
>> tx52_added_vlan_packets: 0
>> tx52_nop: 0
>> tx52_csum_none: 0
>> tx52_stopped: 0
>> tx52_dropped: 0
>> tx52_xmit_more: 0
>> tx52_recover: 0
>> tx52_cqes: 0
>> tx52_wake: 0
>> tx52_cqe_err: 0
>> tx53_packets: 0
>> tx53_bytes: 0
>> tx53_tso_packets: 0
>> tx53_tso_bytes: 0
>> tx53_tso_inner_packets: 0
>> tx53_tso_inner_bytes: 0
>> tx53_csum_partial: 0
>> tx53_csum_partial_inner: 0
>> tx53_added_vlan_packets: 0
>> tx53_nop: 0
>> tx53_csum_none: 0
>> tx53_stopped: 0
>> tx53_dropped: 0
>> tx53_xmit_more: 0
>> tx53_recover: 0
>> tx53_cqes: 0
>> tx53_wake: 0
>> tx53_cqe_err: 0
>> tx54_packets: 0
>> tx54_bytes: 0
>> tx54_tso_packets: 0
>> tx54_tso_bytes: 0
>> tx54_tso_inner_packets: 0
>> tx54_tso_inner_bytes: 0
>> tx54_csum_partial: 0
>> tx54_csum_partial_inner: 0
>> tx54_added_vlan_packets: 0
>> tx54_nop: 0
>> tx54_csum_none: 0
>> tx54_stopped: 0
>> tx54_dropped: 0
>> tx54_xmit_more: 0
>> tx54_recover: 0
>> tx54_cqes: 0
>> tx54_wake: 0
>> tx54_cqe_err: 0
>> tx55_packets: 0
>> tx55_bytes: 0
>> tx55_tso_packets: 0
>> tx55_tso_bytes: 0
>> tx55_tso_inner_packets: 0
>> tx55_tso_inner_bytes: 0
>> tx55_csum_partial: 0
>> tx55_csum_partial_inner: 0
>> tx55_added_vlan_packets: 0
>> tx55_nop: 0
>> tx55_csum_none: 0
>> tx55_stopped: 0
>> tx55_dropped: 0
>> tx55_xmit_more: 0
>> tx55_recover: 0
>> tx55_cqes: 0
>> tx55_wake: 0
>> tx55_cqe_err: 0
>> tx0_xdp_xmit: 0
>> tx0_xdp_full: 0
>> tx0_xdp_err: 0
>> tx0_xdp_cqes: 0
>> tx1_xdp_xmit: 0
>> tx1_xdp_full: 0
>> tx1_xdp_err: 0
>> tx1_xdp_cqes: 0
>> tx2_xdp_xmit: 0
>> tx2_xdp_full: 0
>> tx2_xdp_err: 0
>> tx2_xdp_cqes: 0
>> tx3_xdp_xmit: 0
>> tx3_xdp_full: 0
>> tx3_xdp_err: 0
>> tx3_xdp_cqes: 0
>> tx4_xdp_xmit: 0
>> tx4_xdp_full: 0
>> tx4_xdp_err: 0
>> tx4_xdp_cqes: 0
>> tx5_xdp_xmit: 0
>> tx5_xdp_full: 0
>> tx5_xdp_err: 0
>> tx5_xdp_cqes: 0
>> tx6_xdp_xmit: 0
>> tx6_xdp_full: 0
>> tx6_xdp_err: 0
>> tx6_xdp_cqes: 0
>> tx7_xdp_xmit: 0
>> tx7_xdp_full: 0
>> tx7_xdp_err: 0
>> tx7_xdp_cqes: 0
>> tx8_xdp_xmit: 0
>> tx8_xdp_full: 0
>> tx8_xdp_err: 0
>> tx8_xdp_cqes: 0
>> tx9_xdp_xmit: 0
>> tx9_xdp_full: 0
>> tx9_xdp_err: 0
>> tx9_xdp_cqes: 0
>> tx10_xdp_xmit: 0
>> tx10_xdp_full: 0
>> tx10_xdp_err: 0
>> tx10_xdp_cqes: 0
>> tx11_xdp_xmit: 0
>> tx11_xdp_full: 0
>> tx11_xdp_err: 0
>> tx11_xdp_cqes: 0
>> tx12_xdp_xmit: 0
>> tx12_xdp_full: 0
>> tx12_xdp_err: 0
>> tx12_xdp_cqes: 0
>> tx13_xdp_xmit: 0
>> tx13_xdp_full: 0
>> tx13_xdp_err: 0
>> tx13_xdp_cqes: 0
>> tx14_xdp_xmit: 0
>> tx14_xdp_full: 0
>> tx14_xdp_err: 0
>> tx14_xdp_cqes: 0
>> tx15_xdp_xmit: 0
>> tx15_xdp_full: 0
>> tx15_xdp_err: 0
>> tx15_xdp_cqes: 0
>> tx16_xdp_xmit: 0
>> tx16_xdp_full: 0
>> tx16_xdp_err: 0
>> tx16_xdp_cqes: 0
>> tx17_xdp_xmit: 0
>> tx17_xdp_full: 0
>> tx17_xdp_err: 0
>> tx17_xdp_cqes: 0
>> tx18_xdp_xmit: 0
>> tx18_xdp_full: 0
>> tx18_xdp_err: 0
>> tx18_xdp_cqes: 0
>> tx19_xdp_xmit: 0
>> tx19_xdp_full: 0
>> tx19_xdp_err: 0
>> tx19_xdp_cqes: 0
>> tx20_xdp_xmit: 0
>> tx20_xdp_full: 0
>> tx20_xdp_err: 0
>> tx20_xdp_cqes: 0
>> tx21_xdp_xmit: 0
>> tx21_xdp_full: 0
>> tx21_xdp_err: 0
>> tx21_xdp_cqes: 0
>> tx22_xdp_xmit: 0
>> tx22_xdp_full: 0
>> tx22_xdp_err: 0
>> tx22_xdp_cqes: 0
>> tx23_xdp_xmit: 0
>> tx23_xdp_full: 0
>> tx23_xdp_err: 0
>> tx23_xdp_cqes: 0
>> tx24_xdp_xmit: 0
>> tx24_xdp_full: 0
>> tx24_xdp_err: 0
>> tx24_xdp_cqes: 0
>> tx25_xdp_xmit: 0
>> tx25_xdp_full: 0
>> tx25_xdp_err: 0
>> tx25_xdp_cqes: 0
>> tx26_xdp_xmit: 0
>> tx26_xdp_full: 0
>> tx26_xdp_err: 0
>> tx26_xdp_cqes: 0
>> tx27_xdp_xmit: 0
>> tx27_xdp_full: 0
>> tx27_xdp_err: 0
>> tx27_xdp_cqes: 0
>> tx28_xdp_xmit: 0
>> tx28_xdp_full: 0
>> tx28_xdp_err: 0
>> tx28_xdp_cqes: 0
>> tx29_xdp_xmit: 0
>> tx29_xdp_full: 0
>> tx29_xdp_err: 0
>> tx29_xdp_cqes: 0
>> tx30_xdp_xmit: 0
>> tx30_xdp_full: 0
>> tx30_xdp_err: 0
>> tx30_xdp_cqes: 0
>> tx31_xdp_xmit: 0
>> tx31_xdp_full: 0
>> tx31_xdp_err: 0
>> tx31_xdp_cqes: 0
>> tx32_xdp_xmit: 0
>> tx32_xdp_full: 0
>> tx32_xdp_err: 0
>> tx32_xdp_cqes: 0
>> tx33_xdp_xmit: 0
>> tx33_xdp_full: 0
>> tx33_xdp_err: 0
>> tx33_xdp_cqes: 0
>> tx34_xdp_xmit: 0
>> tx34_xdp_full: 0
>> tx34_xdp_err: 0
>> tx34_xdp_cqes: 0
>> tx35_xdp_xmit: 0
>> tx35_xdp_full: 0
>> tx35_xdp_err: 0
>> tx35_xdp_cqes: 0
>> tx36_xdp_xmit: 0
>> tx36_xdp_full: 0
>> tx36_xdp_err: 0
>> tx36_xdp_cqes: 0
>> tx37_xdp_xmit: 0
>> tx37_xdp_full: 0
>> tx37_xdp_err: 0
>> tx37_xdp_cqes: 0
>> tx38_xdp_xmit: 0
>> tx38_xdp_full: 0
>> tx38_xdp_err: 0
>> tx38_xdp_cqes: 0
>> tx39_xdp_xmit: 0
>> tx39_xdp_full: 0
>> tx39_xdp_err: 0
>> tx39_xdp_cqes: 0
>> tx40_xdp_xmit: 0
>> tx40_xdp_full: 0
>> tx40_xdp_err: 0
>> tx40_xdp_cqes: 0
>> tx41_xdp_xmit: 0
>> tx41_xdp_full: 0
>> tx41_xdp_err: 0
>> tx41_xdp_cqes: 0
>> tx42_xdp_xmit: 0
>> tx42_xdp_full: 0
>> tx42_xdp_err: 0
>> tx42_xdp_cqes: 0
>> tx43_xdp_xmit: 0
>> tx43_xdp_full: 0
>> tx43_xdp_err: 0
>> tx43_xdp_cqes: 0
>> tx44_xdp_xmit: 0
>> tx44_xdp_full: 0
>> tx44_xdp_err: 0
>> tx44_xdp_cqes: 0
>> tx45_xdp_xmit: 0
>> tx45_xdp_full: 0
>> tx45_xdp_err: 0
>> tx45_xdp_cqes: 0
>> tx46_xdp_xmit: 0
>> tx46_xdp_full: 0
>> tx46_xdp_err: 0
>> tx46_xdp_cqes: 0
>> tx47_xdp_xmit: 0
>> tx47_xdp_full: 0
>> tx47_xdp_err: 0
>> tx47_xdp_cqes: 0
>> tx48_xdp_xmit: 0
>> tx48_xdp_full: 0
>> tx48_xdp_err: 0
>> tx48_xdp_cqes: 0
>> tx49_xdp_xmit: 0
>> tx49_xdp_full: 0
>> tx49_xdp_err: 0
>> tx49_xdp_cqes: 0
>> tx50_xdp_xmit: 0
>> tx50_xdp_full: 0
>> tx50_xdp_err: 0
>> tx50_xdp_cqes: 0
>> tx51_xdp_xmit: 0
>> tx51_xdp_full: 0
>> tx51_xdp_err: 0
>> tx51_xdp_cqes: 0
>> tx52_xdp_xmit: 0
>> tx52_xdp_full: 0
>> tx52_xdp_err: 0
>> tx52_xdp_cqes: 0
>> tx53_xdp_xmit: 0
>> tx53_xdp_full: 0
>> tx53_xdp_err: 0
>> tx53_xdp_cqes: 0
>> tx54_xdp_xmit: 0
>> tx54_xdp_full: 0
>> tx54_xdp_err: 0
>> tx54_xdp_cqes: 0
>> tx55_xdp_xmit: 0
>> tx55_xdp_full: 0
>> tx55_xdp_err: 0
>> tx55_xdp_cqes: 0
>>
>>
>> mpstat -P ALL 1 10
>> Average: CPU %usr %nice %sys %iowait %irq %soft
>> %steal
>> %guest %gnice %idle
>> Average: all 0.04 0.00 6.94 0.02 0.00 32.00
>> 0.00 0.00 0.00 61.00
>> Average: 0 0.00 0.00 1.20 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 98.80
>> Average: 1 0.00 0.00 2.30 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 97.70
>> Average: 2 0.10 0.00 0.00 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 99.90
>> Average: 3 0.10 0.00 1.50 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 98.40
>> Average: 4 0.50 0.00 2.50 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 97.00
>> Average: 5 0.00 0.00 0.00 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 100.00
>> Average: 6 0.90 0.00 10.20 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 88.90
>> Average: 7 0.00 0.00 0.00 1.40 0.00 0.00
>> 0.00
>> 0.00 0.00 98.60
>> Average: 8 0.00 0.00 0.00 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 100.00
>> Average: 9 0.00 0.00 0.00 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 100.00
>> Average: 10 0.00 0.00 0.00 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 100.00
>> Average: 11 0.00 0.00 0.00 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 100.00
>> Average: 12 0.00 0.00 0.00 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 100.00
>> Average: 13 0.00 0.00 0.00 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 100.00
>> Average: 14 0.00 0.00 12.99 0.00 0.00 62.64
>> 0.00 0.00 0.00 24.38
>> Average: 15 0.00 0.00 12.70 0.00 0.00 63.40
>> 0.00 0.00 0.00 23.90
>> Average: 16 0.00 0.00 11.20 0.00 0.00 66.40
>> 0.00 0.00 0.00 22.40
>> Average: 17 0.00 0.00 16.60 0.00 0.00 52.10
>> 0.00 0.00 0.00 31.30
>> Average: 18 0.00 0.00 13.90 0.00 0.00 61.20
>> 0.00 0.00 0.00 24.90
>> Average: 19 0.00 0.00 9.99 0.00 0.00 70.33
>> 0.00 0.00 0.00 19.68
>> Average: 20 0.00 0.00 9.00 0.00 0.00 73.00
>> 0.00 0.00 0.00 18.00
>> Average: 21 0.00 0.00 8.70 0.00 0.00 73.90
>> 0.00 0.00 0.00 17.40
>> Average: 22 0.00 0.00 15.42 0.00 0.00 58.56
>> 0.00 0.00 0.00 26.03
>> Average: 23 0.00 0.00 10.81 0.00 0.00 71.67
>> 0.00 0.00 0.00 17.52
>> Average: 24 0.00 0.00 10.00 0.00 0.00 71.80
>> 0.00 0.00 0.00 18.20
>> Average: 25 0.00 0.00 11.19 0.00 0.00 71.13
>> 0.00 0.00 0.00 17.68
>> Average: 26 0.00 0.00 11.00 0.00 0.00 70.80
>> 0.00 0.00 0.00 18.20
>> Average: 27 0.00 0.00 10.01 0.00 0.00 69.57
>> 0.00 0.00 0.00 20.42
> The numa cores are not at 100% util, you have around 20% of idle on
> each one.
Yes - no 100% cpu - but the difference between 80% and 100% is like push
aditional 1-2Gbit/s
>
>> Average: 28 0.00 0.00 0.00 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 100.00
>> Average: 29 0.00 0.00 0.00 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 100.00
>> Average: 30 0.00 0.00 0.00 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 100.00
>> Average: 31 0.00 0.00 0.00 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 100.00
>> Average: 32 0.00 0.00 0.00 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 100.00
>> Average: 33 0.00 0.00 3.90 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 96.10
>> Average: 34 0.00 0.00 0.00 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 100.00
>> Average: 35 0.00 0.00 0.00 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 100.00
>> Average: 36 0.10 0.00 0.20 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 99.70
>> Average: 37 0.20 0.00 0.30 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 99.50
>> Average: 38 0.00 0.00 0.00 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 100.00
>> Average: 39 0.00 0.00 2.60 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 97.40
>> Average: 40 0.00 0.00 0.90 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 99.10
>> Average: 41 0.10 0.00 0.50 0.00 0.00 0.00
>> 0.00
>> 0.00 0.00 99.40
>> Average: 42 0.00 0.00 9.91 0.00 0.00 70.67
>> 0.00 0.00 0.00 19.42
>> Average: 43 0.00 0.00 15.90 0.00 0.00 57.50
>> 0.00 0.00 0.00 26.60
>> Average: 44 0.00 0.00 12.20 0.00 0.00 66.20
>> 0.00 0.00 0.00 21.60
>> Average: 45 0.00 0.00 12.00 0.00 0.00 67.50
>> 0.00 0.00 0.00 20.50
>> Average: 46 0.00 0.00 12.90 0.00 0.00 65.50
>> 0.00 0.00 0.00 21.60
>> Average: 47 0.00 0.00 14.59 0.00 0.00 60.84
>> 0.00 0.00 0.00 24.58
>> Average: 48 0.00 0.00 13.59 0.00 0.00 61.74
>> 0.00 0.00 0.00 24.68
>> Average: 49 0.00 0.00 18.36 0.00 0.00 53.29
>> 0.00 0.00 0.00 28.34
>> Average: 50 0.00 0.00 15.32 0.00 0.00 58.86
>> 0.00 0.00 0.00 25.83
>> Average: 51 0.00 0.00 17.60 0.00 0.00 55.20
>> 0.00 0.00 0.00 27.20
>> Average: 52 0.00 0.00 15.92 0.00 0.00 56.06
>> 0.00 0.00 0.00 28.03
>> Average: 53 0.00 0.00 13.00 0.00 0.00 62.30
>> 0.00 0.00 0.00 24.70
>> Average: 54 0.00 0.00 13.20 0.00 0.00 61.50
>> 0.00 0.00 0.00 25.30
>> Average: 55 0.00 0.00 14.59 0.00 0.00 58.64
>> 0.00 0.00 0.00 26.77
>>
>>
>> ethtool -k enp175s0f0
>> Features for enp175s0f0:
>> rx-checksumming: on
>> tx-checksumming: on
>> tx-checksum-ipv4: on
>> tx-checksum-ip-generic: off [fixed]
>> tx-checksum-ipv6: on
>> tx-checksum-fcoe-crc: off [fixed]
>> tx-checksum-sctp: off [fixed]
>> scatter-gather: on
>> tx-scatter-gather: on
>> tx-scatter-gather-fraglist: off [fixed]
>> tcp-segmentation-offload: on
>> tx-tcp-segmentation: on
>> tx-tcp-ecn-segmentation: off [fixed]
>> tx-tcp-mangleid-segmentation: off
>> tx-tcp6-segmentation: on
>> udp-fragmentation-offload: off
>> generic-segmentation-offload: on
>> generic-receive-offload: on
>> large-receive-offload: off [fixed]
>> rx-vlan-offload: on
>> tx-vlan-offload: on
>> ntuple-filters: off
>> receive-hashing: on
>> highdma: on [fixed]
>> rx-vlan-filter: on
>> vlan-challenged: off [fixed]
>> tx-lockless: off [fixed]
>> netns-local: off [fixed]
>> tx-gso-robust: off [fixed]
>> tx-fcoe-segmentation: off [fixed]
>> tx-gre-segmentation: on
>> tx-gre-csum-segmentation: on
>> tx-ipxip4-segmentation: off [fixed]
>> tx-ipxip6-segmentation: off [fixed]
>> tx-udp_tnl-segmentation: on
>> tx-udp_tnl-csum-segmentation: on
>> tx-gso-partial: on
>> tx-sctp-segmentation: off [fixed]
>> tx-esp-segmentation: off [fixed]
>> tx-udp-segmentation: on
>> fcoe-mtu: off [fixed]
>> tx-nocache-copy: off
>> loopback: off [fixed]
>> rx-fcs: off
>> rx-all: off
>> tx-vlan-stag-hw-insert: on
>> rx-vlan-stag-hw-parse: off [fixed]
>> rx-vlan-stag-filter: on [fixed]
>> l2-fwd-offload: off [fixed]
>> hw-tc-offload: off
>> esp-hw-offload: off [fixed]
>> esp-tx-csum-hw-offload: off [fixed]
>> rx-udp_tunnel-port-offload: on
>> tls-hw-tx-offload: off [fixed]
>> tls-hw-rx-offload: off [fixed]
>> rx-gro-hw: off [fixed]
>> tls-hw-record: off [fixed]
>>
>> ethtool -c enp175s0f0
>> Coalesce parameters for enp175s0f0:
>> Adaptive RX: off TX: on
>> stats-block-usecs: 0
>> sample-interval: 0
>> pkt-rate-low: 0
>> pkt-rate-high: 0
>> dmac: 32703
>>
>> rx-usecs: 256
>> rx-frames: 128
>> rx-usecs-irq: 0
>> rx-frames-irq: 0
>>
>> tx-usecs: 8
>> tx-frames: 128
>> tx-usecs-irq: 0
>> tx-frames-irq: 0
>>
>> rx-usecs-low: 0
>> rx-frame-low: 0
>> tx-usecs-low: 0
>> tx-frame-low: 0
>>
>> rx-usecs-high: 0
>> rx-frame-high: 0
>> tx-usecs-high: 0
>> tx-frame-high: 0
>>
>> ethtool -g enp175s0f0
>> Ring parameters for enp175s0f0:
>> Pre-set maximums:
>> RX: 8192
>> RX Mini: 0
>> RX Jumbo: 0
>> TX: 8192
>> Current hardware settings:
>> RX: 4096
>> RX Mini: 0
>> RX Jumbo: 0
>> TX: 4096
>>
>>
>>
>>
>>
>>
Also changed a little coalesce params - and best for this config are:
ethtool -c enp175s0f0
Coalesce parameters for enp175s0f0:
Adaptive RX: off TX: off
stats-block-usecs: 0
sample-interval: 0
pkt-rate-low: 0
pkt-rate-high: 0
dmac: 32573
rx-usecs: 40
rx-frames: 128
rx-usecs-irq: 0
rx-frames-irq: 0
tx-usecs: 8
tx-frames: 8
tx-usecs-irq: 0
tx-frames-irq: 0
rx-usecs-low: 0
rx-frame-low: 0
tx-usecs-low: 0
tx-frame-low: 0
rx-usecs-high: 0
rx-frame-high: 0
tx-usecs-high: 0
tx-frame-high: 0
Less drops on RX side - and more pps in overall forwarded.
^ permalink raw reply
* Re: [RFC PATCH] lib: Introduce generic __cmpxchg_u64() and use it where needed
From: Paul E. McKenney @ 2018-11-01 20:29 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Eric Dumazet, Trond Myklebust, mark.rutland@arm.com,
linux-kernel@vger.kernel.org, ralf@linux-mips.org,
jlayton@kernel.org, linuxppc-dev@lists.ozlabs.org,
bfields@fieldses.org, linux-mips@linux-mips.org,
linux@roeck-us.net, linux-nfs@vger.kernel.org,
akpm@linux-foundation.org, will.deacon@arm.com,
boqun.feng@gmail.com, paul.burton@mips.com,
"anna.schumaker@netapp.com" <an
In-Reply-To: <20181101172739.GA3196@hirez.programming.kicks-ass.net>
On Thu, Nov 01, 2018 at 06:27:39PM +0100, Peter Zijlstra wrote:
> On Thu, Nov 01, 2018 at 06:14:32PM +0100, Peter Zijlstra wrote:
> > > This reminds me of this sooooo silly patch :/
> > >
> > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=adb03115f4590baa280ddc440a8eff08a6be0cb7
>
> You'd probably want to write it like so; +- some ordering stuff, that
> code didn't look like it really needs the memory barriers implied by
> these, but I didn't look too hard.
The atomic_fetch_add() API would need to be propagated out to the other
architectures, correct?
Thanx, Paul
> diff --git a/net/ipv4/route.c b/net/ipv4/route.c
> index c0a9d26c06ce..11deb1d7e96b 100644
> --- a/net/ipv4/route.c
> +++ b/net/ipv4/route.c
> @@ -485,16 +485,10 @@ u32 ip_idents_reserve(u32 hash, int segs)
> u32 now = (u32)jiffies;
> u32 new, delta = 0;
>
> - if (old != now && cmpxchg(p_tstamp, old, now) == old)
> + if (old != now && try_cmpxchg(p_tstamp, &old, now))
> delta = prandom_u32_max(now - old);
>
> - /* Do not use atomic_add_return() as it makes UBSAN unhappy */
> - do {
> - old = (u32)atomic_read(p_id);
> - new = old + delta + segs;
> - } while (atomic_cmpxchg(p_id, old, new) != old);
> -
> - return new - segs;
> + return atomic_fetch_add(segs + delta, p_id) + delta;
> }
> EXPORT_SYMBOL(ip_idents_reserve);
>
>
^ permalink raw reply
* [PATCH net] net: hns3: Fix for out-of-bounds access when setting pfc back pressure
From: Yunsheng Lin @ 2018-11-01 12:12 UTC (permalink / raw)
To: davem
Cc: huangdaode, xuwei5, liguozhu, Yisen.Zhuang, john.garry, linuxarm,
yisen.zhuang, salil.mehta, lipeng321, netdev, linux-kernel
The vport should be initialized to hdev->vport for each bp group,
otherwise it will cause out-of-bounds access and bp setting not
correct problem.
[ 35.254124] BUG: KASAN: slab-out-of-bounds in hclge_pause_setup_hw+0x2a0/0x3f8 [hclge]
[ 35.254126] Read of size 2 at addr ffff803b6651581a by task kworker/0:1/14
[ 35.254132] CPU: 0 PID: 14 Comm: kworker/0:1 Not tainted 4.19.0-rc7-hulk+ #85
[ 35.254133] Hardware name: Huawei D06/D06, BIOS Hisilicon D06 UEFI RC0 - B052 (V0.52) 09/14/2018
[ 35.254141] Workqueue: events work_for_cpu_fn
[ 35.254144] Call trace:
[ 35.254147] dump_backtrace+0x0/0x2f0
[ 35.254149] show_stack+0x24/0x30
[ 35.254154] dump_stack+0x110/0x184
[ 35.254157] print_address_description+0x168/0x2b0
[ 35.254160] kasan_report+0x184/0x310
[ 35.254162] __asan_load2+0x7c/0xa0
[ 35.254170] hclge_pause_setup_hw+0x2a0/0x3f8 [hclge]
[ 35.254177] hclge_tm_init_hw+0x794/0x9f0 [hclge]
[ 35.254184] hclge_tm_schd_init+0x48/0x58 [hclge]
[ 35.254191] hclge_init_ae_dev+0x778/0x1168 [hclge]
[ 35.254196] hnae3_register_ae_dev+0x14c/0x298 [hnae3]
[ 35.254206] hns3_probe+0x88/0xa8 [hns3]
[ 35.254210] local_pci_probe+0x7c/0xf0
[ 35.254212] work_for_cpu_fn+0x34/0x50
[ 35.254214] process_one_work+0x4d4/0xa38
[ 35.254216] worker_thread+0x55c/0x8d8
[ 35.254219] kthread+0x1b0/0x1b8
[ 35.254222] ret_from_fork+0x10/0x1c
[ 35.254224] The buggy address belongs to the page:
[ 35.254228] page:ffff7e00ed994400 count:1 mapcount:0 mapping:0000000000000000 index:0x0 compound_mapcount: 0
[ 35.273835] flags: 0xfffff8000008000(head)
[ 35.282007] raw: 0fffff8000008000 dead000000000100 dead000000000200 0000000000000000
[ 35.282010] raw: 0000000000000000 0000000000000000 00000001ffffffff 0000000000000000
[ 35.282012] page dumped because: kasan: bad access detected
[ 35.282014] Memory state around the buggy address:
[ 35.282017] ffff803b66515700: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
[ 35.282019] ffff803b66515780: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
[ 35.282021] >ffff803b66515800: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
[ 35.282022] ^
[ 35.282024] ffff803b66515880: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
[ 35.282026] ffff803b66515900: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
[ 35.282028] ==================================================================
[ 35.282029] Disabling lock debugging due to kernel taint
[ 35.282747] hclge driver initialization finished.
Fixes: 67bf2541f4b9 ("net: hns3: Fixes the back pressure setting when sriov is enabled")
Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
---
drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c
index aa5cb98..0c9c6ae 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c
@@ -1168,11 +1168,12 @@ static int hclge_pfc_setup_hw(struct hclge_dev *hdev)
*/
static int hclge_bp_setup_hw(struct hclge_dev *hdev, u8 tc)
{
- struct hclge_vport *vport = hdev->vport;
u32 i, k, qs_bitmap;
int ret;
for (i = 0; i < HCLGE_BP_GRP_NUM; i++) {
+ struct hclge_vport *vport = hdev->vport;
+
qs_bitmap = 0;
for (k = 0; k < hdev->num_alloc_vport; k++) {
--
2.8.1
^ permalink raw reply related
* Re: [PATCH v3 1/2] kretprobe: produce sane stack traces
From: Aleksa Sarai @ 2018-11-01 21:13 UTC (permalink / raw)
To: Masami Hiramatsu
Cc: Naveen N. Rao, Anil S Keshavamurthy, David S. Miller,
Jonathan Corbet, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Alexander Shishkin, Jiri Olsa,
Namhyung Kim, Steven Rostedt, Shuah Khan, Alexei Starovoitov,
Daniel Borkmann, Brendan Gregg, Christian Brauner, Aleksa Sarai,
netdev, linux-doc, linux-kernel
In-Reply-To: <20181102002039.8f22c10fa47cae75fa709165@kernel.org>
[-- Attachment #1: Type: text/plain, Size: 2107 bytes --]
On 2018-11-02, Masami Hiramatsu <mhiramat@kernel.org> wrote:
> Please split the test case as an independent patch.
Will do. Should the Documentation/ change also be a separate patch?
> > new file mode 100644
> > index 000000000000..03146c6a1a3c
> > --- /dev/null
> > +++ b/tools/testing/selftests/ftrace/test.d/kprobe/kretprobe_stacktrace.tc
> > @@ -0,0 +1,25 @@
> > +#!/bin/sh
> > +# SPDX-License-Identifier: GPL-2.0+
> > +# description: Kretprobe dynamic event with a stacktrace
> > +
> > +[ -f kprobe_events ] || exit_unsupported # this is configurable
> > +
> > +echo 0 > events/enable
> > +echo 1 > options/stacktrace
> > +
> > +echo 'r:teststackprobe sched_fork $retval' > kprobe_events
> > +grep teststackprobe kprobe_events
> > +test -d events/kprobes/teststackprobe
>
> Hmm, what happen if we have 2 or more kretprobes on same stack?
> It seems you just save stack in pre_handler, but that stack can already
> includes another kretprobe's trampline address...
Yeah, you're quite right...
My first instinct was to do something awful like iterate over the set of
"kretprobe_instance"s with ->task == current, and then correct
kretprobe_trampoline entries using ->ret_addr. (I think this would be
correct because each task can only be in one context at once, and in
order to get to a particular kretprobe all of your caller kretprobes
were inserted before you and any sibling or later kretprobe_instances
will have been removed. But I might be insanely wrong on this front.)
However (as I noted in the other thread), there is a problem where
kretprobe_trampoline actually stops the unwinder in its tracks and thus
you only get the first kretprobe_trampoline. This is something I'm going
to look into some more (despite not having made progress on it last
time) since now it's something that actually needs to be fixed (and
as I mentioned in the other thread, show_stack() actually works on x86
in this context unlike the other stack_trace users).
--
Aleksa Sarai
Senior Software Engineer (Containers)
SUSE Linux GmbH
<https://www.cyphar.com/>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
* [PULL] vhost: cleanups and fixes
From: Michael S. Tsirkin @ 2018-11-01 21:19 UTC (permalink / raw)
To: Linus Torvalds
Cc: lenaic, mhocko, bijan.mottahedeh, kvm, mst, netdev, liang.z.li,
linux-kernel, virtualization, stefanha, joe, akpm, mhocko,
torvalds
The following changes since commit 84df9525b0c27f3ebc2ebb1864fa62a97fdedb7d:
Linux 4.19 (2018-10-22 07:37:37 +0100)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git tags/for_linus
for you to fetch changes up to 79f800b2e76923cd8ce0aa659cb5c019d9643bc9:
MAINTAINERS: remove reference to bogus vsock file (2018-10-24 21:16:14 -0400)
----------------------------------------------------------------
virtio, vhost: fixes, tweaks
virtio balloon page hinting support
vhost scsi control queue
misc fixes.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
----------------------------------------------------------------
Bijan Mottahedeh (3):
vhost/scsi: Respond to control queue operations
vhost/scsi: Extract common handling code from control queue handler
vhost/scsi: Use common handling code in request queue handler
Greg Edwards (1):
vhost/scsi: truncate T10 PI iov_iter to prot_bytes
Lénaïc Huard (1):
kvm_config: add CONFIG_VIRTIO_MENU
Stefan Hajnoczi (1):
MAINTAINERS: remove reference to bogus vsock file
Wei Wang (3):
virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT
mm/page_poison: expose page_poisoning_enabled to kernel modules
virtio-balloon: VIRTIO_BALLOON_F_PAGE_POISON
MAINTAINERS | 1 -
drivers/vhost/scsi.c | 426 ++++++++++++++++++++++++++++--------
drivers/virtio/virtio_balloon.c | 380 +++++++++++++++++++++++++++++---
include/uapi/linux/virtio_balloon.h | 8 +
kernel/configs/kvm_guest.config | 1 +
mm/page_poison.c | 6 +
6 files changed, 688 insertions(+), 134 deletions(-)
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [PATCH v1 net] net: dsa: microchip: initialize mutex before use
From: Andrew Lunn @ 2018-11-01 12:17 UTC (permalink / raw)
To: Tristram.Ha
Cc: David S. Miller, Florian Fainelli, Pavel Machek, UNGLinuxDriver,
netdev
In-Reply-To: <1541040548-13465-1-git-send-email-Tristram.Ha@microchip.com>
On Wed, Oct 31, 2018 at 07:49:08PM -0700, Tristram.Ha@microchip.com wrote:
> From: Tristram Ha <Tristram.Ha@microchip.com>
>
> Initialize mutex before use. Avoid kernel complaint when
> CONFIG_DEBUG_LOCK_ALLOC is enabled.
>
> Fixes: b987e98e50ab90e5 ("dsa: add DSA switch driver for Microchip KSZ9477")
> Signed-off-by: Tristram Ha <Tristram.Ha@microchip.com>
Reviewed-by: Andrew Lunn <andrew@lunn.ch>
Andrew
^ permalink raw reply
* [PATCH net] bonding/802.3ad: fix link_failure_count tracking
From: Jarod Wilson @ 2018-11-01 21:22 UTC (permalink / raw)
To: linux-kernel
Cc: Jarod Wilson, Mahesh Bandewar, David S . Miller, netdev, stable
Commit 4d2c0cda07448ea6980f00102dc3964eb25e241c set slave->link to
BOND_LINK_DOWN for 802.3ad bonds whenever invalid speed/duplex values
were read, to fix a problem with slaves getting into weird states, but
in the process, broke tracking of link failures, as going straight to
BOND_LINK_DOWN when a link is indeed down (cable pulled, switch rebooted)
means we broke out of bond_miimon_inspect()'s BOND_LINK_DOWN case because
!link_state was already true, we never incremented commit, and never got
a chance to call bond_miimon_commit(), where slave->link_failure_count
would be incremented. I believe the simple fix here is to mark the slave
as BOND_LINK_FAIL, and let bond_miimon_inspect() transition the link from
_FAIL to either _UP or _DOWN, and in the latter case, we now get proper
incrementing of link_failure_count again.
Fixes: 4d2c0cda07448ea6980f00102dc3964eb25e241c
CC: Mahesh Bandewar <maheshb@google.com>
CC: David S. Miller <davem@davemloft.net>
CC: netdev@vger.kernel.org
CC: stable@vger.kernel.org
Signed-off-by: Jarod Wilson <jarod@redhat.com>
---
drivers/net/bonding/bond_main.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
index ffa37adb7681..333387f1f1fe 100644
--- a/drivers/net/bonding/bond_main.c
+++ b/drivers/net/bonding/bond_main.c
@@ -3112,13 +3112,13 @@ static int bond_slave_netdev_event(unsigned long event,
case NETDEV_CHANGE:
/* For 802.3ad mode only:
* Getting invalid Speed/Duplex values here will put slave
- * in weird state. So mark it as link-down for the time
+ * in weird state. So mark it as link-fail for the time
* being and let link-monitoring (miimon) set it right when
* correct speeds/duplex are available.
*/
if (bond_update_speed_duplex(slave) &&
BOND_MODE(bond) == BOND_MODE_8023AD)
- slave->link = BOND_LINK_DOWN;
+ slave->link = BOND_LINK_FAIL;
if (BOND_MODE(bond) == BOND_MODE_8023AD)
bond_3ad_adapter_speed_duplex_changed(slave);
--
2.16.1
^ permalink raw reply related
* Re: [RFC PATCH] lib: Introduce generic __cmpxchg_u64() and use it where needed
From: Peter Zijlstra @ 2018-11-01 21:38 UTC (permalink / raw)
To: Paul E. McKenney
Cc: Eric Dumazet, Trond Myklebust, mark.rutland@arm.com,
linux-kernel@vger.kernel.org, ralf@linux-mips.org,
jlayton@kernel.org, linuxppc-dev@lists.ozlabs.org,
bfields@fieldses.org, linux-mips@linux-mips.org,
linux@roeck-us.net, linux-nfs@vger.kernel.org,
akpm@linux-foundation.org, will.deacon@arm.com,
boqun.feng@gmail.com, paul.burton@mips.com,
"anna.schumaker@netapp.com" <an
In-Reply-To: <20181101202910.GB4170@linux.ibm.com>
On Thu, Nov 01, 2018 at 01:29:10PM -0700, Paul E. McKenney wrote:
> On Thu, Nov 01, 2018 at 06:27:39PM +0100, Peter Zijlstra wrote:
> > On Thu, Nov 01, 2018 at 06:14:32PM +0100, Peter Zijlstra wrote:
> > > > This reminds me of this sooooo silly patch :/
> > > >
> > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=adb03115f4590baa280ddc440a8eff08a6be0cb7
> >
> > You'd probably want to write it like so; +- some ordering stuff, that
> > code didn't look like it really needs the memory barriers implied by
> > these, but I didn't look too hard.
>
> The atomic_fetch_add() API would need to be propagated out to the other
> architectures, correct?
Like these commits I did like 2 years ago ? :-)
$ git log --oneline 6dc25876cdb1...1f51dee7ca74
6dc25876cdb1 locking/atomic, arch/xtensa: Implement atomic_fetch_{add,sub,and,or,xor}()
a8bcccaba162 locking/atomic, arch/x86: Implement atomic{,64}_fetch_{add,sub,and,or,xor}()
1af5de9af138 locking/atomic, arch/tile: Implement atomic{,64}_fetch_{add,sub,and,or,xor}()
3a1adb23a52c locking/atomic, arch/sparc: Implement atomic{,64}_fetch_{add,sub,and,or,xor}()
7d9794e75237 locking/atomic, arch/sh: Implement atomic_fetch_{add,sub,and,or,xor}()
56fefbbc3f13 locking/atomic, arch/s390: Implement atomic{,64}_fetch_{add,sub,and,or,xor}()
a28cc7bbe8e3 locking/atomic, arch/powerpc: Implement atomic{,64}_fetch_{add,sub,and,or,xor}{,_relaxed,_acquire,_release}()
e5857a6ed600 locking/atomic, arch/parisc: Implement atomic{,64}_fetch_{add,sub,and,or,xor}()
f8d638e28d7c locking/atomic, arch/mn10300: Implement atomic_fetch_{add,sub,and,or,xor}()
4edac529eb62 locking/atomic, arch/mips: Implement atomic{,64}_fetch_{add,sub,and,or,xor}()
e898eb27ffd8 locking/atomic, arch/metag: Implement atomic_fetch_{add,sub,and,or,xor}()
e39d88ea3ce4 locking/atomic, arch/m68k: Implement atomic_fetch_{add,sub,and,or,xor}()
f64937052303 locking/atomic, arch/m32r: Implement atomic_fetch_{add,sub,and,or,xor}()
cc102507fac7 locking/atomic, arch/ia64: Implement atomic{,64}_fetch_{add,sub,and,or,xor}()
4be7dd393515 locking/atomic, arch/hexagon: Implement atomic_fetch_{add,sub,and,or,xor}()
0c074cbc3309 locking/atomic, arch/h8300: Implement atomic_fetch_{add,sub,and,or,xor}()
d9c730281617 locking/atomic, arch/frv: Implement atomic{,64}_fetch_{add,sub,and,or,xor}()
e87fc0ec0705 locking/atomic, arch/blackfin: Implement atomic_fetch_{add,sub,and,or,xor}()
1a6eafacd481 locking/atomic, arch/avr32: Implement atomic_fetch_{add,sub,and,or,xor}()
2efe95fe6952 locking/atomic, arch/arm64: Implement atomic{,64}_fetch_{add,sub,and,andnot,or,xor}{,_relaxed,_acquire,_release}() for LSE instructions
6822a84dd4e3 locking/atomic, arch/arm64: Generate LSE non-return cases using common macros
e490f9b1d3b4 locking/atomic, arch/arm64: Implement atomic{,64}_fetch_{add,sub,and,andnot,or,xor}{,_relaxed,_acquire,_release}()
6da068c1beba locking/atomic, arch/arm: Implement atomic{,64}_fetch_{add,sub,and,andnot,or,xor}{,_relaxed,_acquire,_release}()
fbffe892e525 locking/atomic, arch/arc: Implement atomic_fetch_{add,sub,and,andnot,or,xor}()
^ permalink raw reply
* Re: [RFC PATCH] lib: Introduce generic __cmpxchg_u64() and use it where needed
From: Peter Zijlstra @ 2018-11-01 21:45 UTC (permalink / raw)
To: Dmitry Vyukov
Cc: Paul E. McKenney, Trond Myklebust, mark.rutland@arm.com,
linux-kernel@vger.kernel.org, ralf@linux-mips.org,
jlayton@kernel.org, linuxppc-dev@lists.ozlabs.org,
bfields@fieldses.org, linux-mips@linux-mips.org,
linux@roeck-us.net, linux-nfs@vger.kernel.org,
akpm@linux-foundation.org, will.deacon@arm.com,
boqun.feng@gmail.com, paul.burton@mips.com,
"anna.schumaker@netapp.com
In-Reply-To: <CACT4Y+aC45BtS88DXarn3A+LV2RRRsPQoSs_3_DnKjU4O3AMHQ@mail.gmail.com>
On Thu, Nov 01, 2018 at 06:46:50PM +0100, Dmitry Vyukov wrote:
> If there is a warning that we don't want to see at all, then we can
> disable it. It supposed to be a useful tool, rather than a thing in
> itself that lives own life. We already I think removed 1 particularly
> noisy warning and made another optional via a config.
> But the thing with overflows is that, even if it's defined, it's not
> necessary the intended behavior. For example, take allocation size
> calculation done via unsigned size_t. If it overflows it does not help
> if C defines result or not, it still gives a user controlled write
> primitive. We've seen similar cases with timeout/deadline calculation
> in kernel, we really don't want it to just wrap modulo-2, right. Some
> user-space projects even test with unsigned overflow warnings or
> implicit truncation warnings, which are formally legal, but frequently
> bugs.
Sure; but then don't call it UB.
If we want to have an additional integer over/underflow checker (ideally
with a gcc plugin that has explicit annotations like __wrap to make it
go away) that is fine; and it can be done on unsigned and signed.
^ permalink raw reply
* Re: [PATCH net] rtnetlink: invoke 'cb->done' destructor before 'cb->args' reset
From: Alexey Kodanev @ 2018-11-01 13:11 UTC (permalink / raw)
To: David Ahern, netdev; +Cc: David Miller
In-Reply-To: <50d88abb-bfba-6c70-af68-7bac60a0d4b1@gmail.com>
On 10/31/2018 08:35 PM, David Ahern wrote:
> On 10/31/18 10:55 AM, David Ahern wrote:
>> I think the simplest fix for 4.20 is to break the loop if ret is non-0 -
>> restore the previous behavior.
>
> that is the only recourse. It has to bail if ret is non-0. Do you want
> to send a patch with that fix?
>
I see, and inet6_dump_fib() cleanups fib6_walker if ret is zero. Will send the fix.
^ permalink raw reply
* RE: [PATCH iproute2-next] rdma: Refresh help section of resource information
From: Steve Wise @ 2018-11-01 13:22 UTC (permalink / raw)
To: 'Leon Romanovsky', 'David Ahern'
Cc: 'Leon Romanovsky', 'netdev',
'RDMA mailing list', 'Stephen Hemminger'
In-Reply-To: <20181101083505.25152-1-leon@kernel.org>
> -----Original Message-----
> From: Leon Romanovsky <leon@kernel.org>
> Sent: Thursday, November 1, 2018 3:35 AM
> To: David Ahern <dsahern@gmail.com>
> Cc: Leon Romanovsky <leonro@mellanox.com>; netdev
> <netdev@vger.kernel.org>; RDMA mailing list <linux-rdma@vger.kernel.org>;
> Stephen Hemminger <stephen@networkplumber.org>; Steve Wise
> <swise@opengridcomputing.com>
> Subject: [PATCH iproute2-next] rdma: Refresh help section of resource
> information
>
> From: Leon Romanovsky <leonro@mellanox.com>
>
> After commit 4060e4c0d257 ("rdma: Add PD resource tracking
> information"), the resource information shows PDs and MRs,
> but help pages didn't fully reflect it.
>
> Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Oops. Thanks. Looks fine.
Reviewed-by: Steve Wise <swise@opengridcomputing.com>
^ permalink raw reply
* Re: [PATCH net] rtnetlink: invoke 'cb->done' destructor before 'cb->args' reset
From: Alexey Kodanev @ 2018-11-01 13:42 UTC (permalink / raw)
To: David Ahern, netdev; +Cc: David Miller
In-Reply-To: <dbd48fde-3a7f-2800-4317-4eab2ecfd41d@oracle.com>
On 11/01/2018 04:11 PM, Alexey Kodanev wrote:
> On 10/31/2018 08:35 PM, David Ahern wrote:
>> On 10/31/18 10:55 AM, David Ahern wrote:
>>> I think the simplest fix for 4.20 is to break the loop if ret is non-0 -
>>> restore the previous behavior.
>>
>> that is the only recourse. It has to bail if ret is non-0. Do you want
>> to send a patch with that fix?
>>
>
> I see, and inet6_dump_fib() cleanups fib6_walker if ret is zero. Will send the fix.
Can it happen that inet6_dump_fib() returns skb->len (0) in the below cases?
* if (arg.filter.flags & RTM_F_CLONED)
return skb->len;
...
w = (void *)cb->args[2];
if (!w) {
...
w = kzalloc(...)
...
* if (arg.filter.table_id) {
...
if (!tb) {
if (arg.filter.dump_all_families)
return skb->len;
Would it be safer to add "res = skb->len; goto out;" instead of "return skb->len;"
so that it can call fib6_dump_end() for "res <= 0"? Or use cb->data instead of
cb->args?
^ permalink raw reply
* Re: [PATCH v1 net] net: dsa: microchip: initialize mutex before use
From: Pavel Machek @ 2018-11-01 13:47 UTC (permalink / raw)
To: Andrew Lunn
Cc: Tristram.Ha, David S. Miller, Florian Fainelli, UNGLinuxDriver,
netdev
In-Reply-To: <20181101121719.GA7652@lunn.ch>
[-- Attachment #1: Type: text/plain, Size: 634 bytes --]
On Thu 2018-11-01 13:17:19, Andrew Lunn wrote:
> On Wed, Oct 31, 2018 at 07:49:08PM -0700, Tristram.Ha@microchip.com wrote:
> > From: Tristram Ha <Tristram.Ha@microchip.com>
> >
> > Initialize mutex before use. Avoid kernel complaint when
> > CONFIG_DEBUG_LOCK_ALLOC is enabled.
> >
> > Fixes: b987e98e50ab90e5 ("dsa: add DSA switch driver for Microchip KSZ9477")
> > Signed-off-by: Tristram Ha <Tristram.Ha@microchip.com>
Reviewed-by: Pavel Machek <pavel@ucw.cz>
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
^ permalink raw reply
* Re: [PATCH net-next 5/6] net/ncsi: Reset channel state in ncsi_start_dev()
From: Samuel Mendoza-Jonas @ 2018-11-01 22:53 UTC (permalink / raw)
To: Justin.Lee1, netdev; +Cc: davem, linux-kernel, openbmc
In-Reply-To: <96158616f0774e3d9fae0e99bb16e605@AUSX13MPS306.AMER.DELL.COM>
On Tue, 2018-10-30 at 21:26 +0000, Justin.Lee1@Dell.com wrote:
> > +int ncsi_reset_dev(struct ncsi_dev *nd)
> > +{
> > + struct ncsi_dev_priv *ndp = TO_NCSI_DEV_PRIV(nd);
> > + struct ncsi_channel *nc, *active;
> > + struct ncsi_package *np;
> > + unsigned long flags;
> > + bool enabled;
> > + int state;
> > +
> > + active = NULL;
> > + NCSI_FOR_EACH_PACKAGE(ndp, np) {
> > + NCSI_FOR_EACH_CHANNEL(np, nc) {
> > + spin_lock_irqsave(&nc->lock, flags);
> > + enabled = nc->monitor.enabled;
> > + state = nc->state;
> > + spin_unlock_irqrestore(&nc->lock, flags);
> > +
> > + if (enabled)
> > + ncsi_stop_channel_monitor(nc);
> > + if (state == NCSI_CHANNEL_ACTIVE) {
> > + active = nc;
> > + break;
>
> Is the original intention to process the channel one by one?
> If it is the case, there are two loops and we might need to use
> "goto found" instead.
Yes we'll need to break out of the package loop here as well.
>
> > + }
> > + }
> > + }
> > +
>
> found: ?
>
> > + if (!active) {
> > + /* Done */
> > + spin_lock_irqsave(&ndp->lock, flags);
> > + ndp->flags &= ~NCSI_DEV_RESET;
> > + spin_unlock_irqrestore(&ndp->lock, flags);
> > + return ncsi_choose_active_channel(ndp);
> > + }
> > +
> > + spin_lock_irqsave(&ndp->lock, flags);
> > + ndp->flags |= NCSI_DEV_RESET;
> > + ndp->active_channel = active;
> > + ndp->active_package = active->package;
> > + spin_unlock_irqrestore(&ndp->lock, flags);
> > +
> > + nd->state = ncsi_dev_state_suspend;
> > + schedule_work(&ndp->work);
> > + return 0;
> > +}
>
> Also similar issue in ncsi_choose_active_channel() function below.
>
> > @@ -916,32 +1045,49 @@ static int ncsi_choose_active_channel(struct ncsi_dev_priv *ndp)
> >
> > ncm = &nc->modes[NCSI_MODE_LINK];
> > if (ncm->data[2] & 0x1) {
> > - spin_unlock_irqrestore(&nc->lock, flags);
> > found = nc;
> > - goto out;
> > + with_link = true;
> > }
> >
> > - spin_unlock_irqrestore(&nc->lock, flags);
> > + /* If multi_channel is enabled configure all valid
> > + * channels whether or not they currently have link
> > + * so they will have AENs enabled.
> > + */
> > + if (with_link || np->multi_channel) {
>
> I notice that there is a case that we will misconfigure the interface.
> For example below, multi-channel is not enable for package 1.
> But we enable the channel for ncsi2 below (package 1 channel 0) as that interface is the first
> channel for that package with link.
I don't think I see the issue here; multi-channel is not set on package
1, but both channels are in the channel whitelist. Channel 0 is
configured since it's the first found on package 1, and channel 1 is not
since channel 0 is already found. Are you expecting something different?
>
> cat /sys/kernel/debug/ncsi_protocol/ncsi_device_
> IFIDX IFNAME NAME PID CID RX TX MP MC WP WC PC CS PS LS RU CR NQ HA
> =====================================================================
> 2 eth2 ncsi0 000 000 1 1 1 1 1 1 0 2 1 1 1 1 0 1
> 2 eth2 ncsi1 000 001 1 0 1 1 1 1 0 2 1 1 1 1 0 1
> 2 eth2 ncsi2 001 000 1 0 1 0 1 1 0 2 1 1 1 1 0 1
> 2 eth2 ncsi3 001 001 0 0 1 0 1 1 0 1 0 1 1 1 0 1
> =====================================================================
> MP: Multi-mode Package WP: Whitelist Package
> MC: Multi-mode Channel WC: Whitelist Channel
> PC: Primary Channel CS: Channel State IA/A/IV 1/2/3
> PS: Poll Status LS: Link Status
> RU: Running CR: Carrier OK
> NQ: Queue Stopped HA: Hardware Arbitration
>
> I temporally change to the following to avoid that.
> if ((with_link &&
> !np->multi_channel &&
> list_empty(&ndp->channel_queue)) || np->multi_channel) {
>
> > + spin_lock_irqsave(&ndp->lock, flags);
> > + list_add_tail_rcu(&nc->link,
> > + &ndp->channel_queue);
> > + spin_unlock_irqrestore(&ndp->lock, flags);
> > +
> > + netdev_dbg(ndp->ndev.dev,
> > + "NCSI: Channel %u added to queue (link %s)\n",
> > + nc->id,
> > + ncm->data[2] & 0x1 ? "up" : "down");
> > + }
> > +
> > + spin_unlock_irqrestore(&nc->lock, cflags);
> > +
> > + if (with_link && !np->multi_channel)
> > + break;
>
> Similar issue here. As we are using break, so each package will configure one active TX.
>
I believe this is handled properly in ncsi_channel_is_tx() in the most
recent revision.
> > }
> > + if (with_link && !ndp->multi_package)
> > + break;
> > }
> >
> > - if (!found) {
> > + if (list_empty(&ndp->channel_queue) && found) {
> > + netdev_info(ndp->ndev.dev,
> > + "NCSI: No channel with link found, configuring channel %u\n",
> > + found->id);
> > + spin_lock_irqsave(&ndp->lock, flags);
> > + list_add_tail_rcu(&found->link, &ndp->channel_queue);
> > + spin_unlock_irqrestore(&ndp->lock, flags);
> > + } else if (!found) {
> > netdev_warn(ndp->ndev.dev,
> > - "NCSI: No channel found with link\n");
> > + "NCSI: No channel found to configure!\n");
> > ncsi_report_link(ndp, true);
> > return -ENODEV;
> > }
>
> Also, for deselect package handler function, do we want to set to inactive here?
> If we just change the state, the cached data still keeps the old value. If the new
> ncsi_reset_dev() function is handling one by one, can we skip this part?
Technically yes we could skip the state change here since
ncsi_reset_dev() will have already done it. However if we send a DP
command via some other means then it is probably best to ensure we treat
all channels on that package as inactive.
>
> static int ncsi_rsp_handler_dp(struct ncsi_request *nr)
> {
> struct ncsi_rsp_pkt *rsp;
> struct ncsi_dev_priv *ndp = nr->ndp;
> struct ncsi_package *np;
> struct ncsi_channel *nc;
> unsigned long flags;
>
> /* Find the package */
> rsp = (struct ncsi_rsp_pkt *)skb_network_header(nr->rsp);
> ncsi_find_package_and_channel(ndp, rsp->rsp.common.channel,
> &np, NULL);
> if (!np)
> return -ENODEV;
>
> /* Change state of all channels attached to the package */
> NCSI_FOR_EACH_CHANNEL(np, nc) {
> spin_lock_irqsave(&nc->lock, flags);
> nc->state = NCSI_CHANNEL_INACTIVE;
>
> spin_unlock_irqrestore(&nc->lock, flags);
> }
>
> return 0;
> }
>
>
^ permalink raw reply
* Re: Kernel 4.19 network performance - forwarding/routing normal users traffic
From: Paweł Staszewski @ 2018-11-01 13:52 UTC (permalink / raw)
To: Jesper Dangaard Brouer, David Ahern; +Cc: netdev, Yoel Caspersen
In-Reply-To: <20181101115522.10b0dd0a@redhat.com>
W dniu 01.11.2018 o 11:55, Jesper Dangaard Brouer pisze:
> On Wed, 31 Oct 2018 21:37:16 -0600 David Ahern <dsahern@gmail.com> wrote:
>
>> This is mainly a forwarding use case? Seems so based on the perf report.
>> I suspect forwarding with XDP would show pretty good improvement.
> Yes, significant performance improvements.
>
> Notice Davids talk: "Leveraging Kernel Tables with XDP"
> http://vger.kernel.org/lpc-networking2018.html#session-1
It will be rly interesting
> It looks like that you are doing "pure" IP-routing, without any
> iptables conntrack stuff (from your perf report data). That will
> actually be a really good use-case for accelerating this with XDP.
Yes pure IP routing
iptables used only for some local input filtering.
>
> I want you to understand the philosophy behind how David and I want
> people to leverage XDP. Think of XDP as a software offload layer for
> the kernel network stack. Setup and use Linux kernel network stack, but
> accelerate parts of it with XDP, e.g. the route FIB lookup.
>
> Sample code avail here:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/samples/bpf/xdp_fwd_kern.c
I can try some tests on same hw but testlab configuration - will give it
a try :)
> (I do warn, what we just found a bug/crash in setup+tairdown for the
> mlx5 driver you are using, that we/mlx _will_ fix soon)
Ok
>
>
>> You need the vlan changes I have queued up though.
> I know Yoel will be very interested in those changes too! I've
> convinced Yoel to write an XDP program for his Border Network Gateway
> (BNG) production system[1], and his is a heavy VLAN user. And the plan
> is to Open Source this when he have-something-working.
>
> [1] https://www.version2.dk/blog/software-router-del-5-linux-bng-1086060
Ok - for now i need to split traffic into two separate 100G ports placed
in two different x16 pciexpress slots to check if the problem is mainly
caused by no more pciex x16 bandwidth available.
^ permalink raw reply
* [PATCH net 1/2] net: bcmgenet: protect stop from timeout
From: Florian Fainelli @ 2018-11-01 22:55 UTC (permalink / raw)
To: netdev; +Cc: jaedon.shin, opendmb, Florian Fainelli, David S. Miller,
open list
In-Reply-To: <20181101225538.18632-1-f.fainelli@gmail.com>
From: Doug Berger <opendmb@gmail.com>
A timing hazard exists when the network interface is stopped that
allows a watchdog timeout to be processed by a separate core in
parallel. This creates the potential for the timeout handler to
wake the queues while the driver is shutting down, or access
registers after their clocks have been removed.
The more common case is that the watchdog timeout will produce a
warning message which doesn't lead to a crash. The chances of this
are greatly increased by the fact that bcmgenet_netif_stop stops
the transmit queues which can easily precipitate a watchdog time-
out because of stale trans_start data in the queues.
This commit corrects the behavior by ensuring that the watchdog
timeout is disabled before enterring bcmgenet_netif_stop. There
are currently only two users of the bcmgenet_netif_stop function:
close and suspend.
The close case already handles the issue by exiting the RUNNING
state before invoking the driver close service.
The suspend case now performs the netif_device_detach to exit the
PRESENT state before the call to bcmgenet_netif_stop rather than
after it.
These behaviors prevent any future scheduling of the driver timeout
service during the window. The netif_tx_stop_all_queues function
in bcmgenet_netif_stop is replaced with netif_tx_disable to ensure
synchronization with any transmit or timeout threads that may
already be executing on other cores.
For symmetry, the netif_device_attach call upon resume is moved to
after the call to bcmgenet_netif_start. Since it wakes the transmit
queues it is not necessary to invoke netif_tx_start_all_queues from
bcmgenet_netif_start so it is moved into the driver open service.
Fixes: 1c1008c793fa ("net: bcmgenet: add main driver file")
Signed-off-by: Doug Berger <opendmb@gmail.com>
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
---
drivers/net/ethernet/broadcom/genet/bcmgenet.c | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.c b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
index 20c1681bb1af..2d6f090bf644 100644
--- a/drivers/net/ethernet/broadcom/genet/bcmgenet.c
+++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
@@ -2855,7 +2855,6 @@ static void bcmgenet_netif_start(struct net_device *dev)
umac_enable_set(priv, CMD_TX_EN | CMD_RX_EN, true);
- netif_tx_start_all_queues(dev);
bcmgenet_enable_tx_napi(priv);
/* Monitor link interrupts now */
@@ -2937,6 +2936,8 @@ static int bcmgenet_open(struct net_device *dev)
bcmgenet_netif_start(dev);
+ netif_tx_start_all_queues(dev);
+
return 0;
err_irq1:
@@ -2958,7 +2959,7 @@ static void bcmgenet_netif_stop(struct net_device *dev)
struct bcmgenet_priv *priv = netdev_priv(dev);
bcmgenet_disable_tx_napi(priv);
- netif_tx_stop_all_queues(dev);
+ netif_tx_disable(dev);
/* Disable MAC receive */
umac_enable_set(priv, CMD_RX_EN, false);
@@ -3620,13 +3621,13 @@ static int bcmgenet_suspend(struct device *d)
if (!netif_running(dev))
return 0;
+ netif_device_detach(dev);
+
bcmgenet_netif_stop(dev);
if (!device_may_wakeup(d))
phy_suspend(dev->phydev);
- netif_device_detach(dev);
-
/* Prepare the device for Wake-on-LAN and switch to the slow clock */
if (device_may_wakeup(d) && priv->wolopts) {
ret = bcmgenet_power_down(priv, GENET_POWER_WOL_MAGIC);
@@ -3700,8 +3701,6 @@ static int bcmgenet_resume(struct device *d)
/* Always enable ring 16 - descriptor ring */
bcmgenet_enable_dma(priv, dma_ctrl);
- netif_device_attach(dev);
-
if (!device_may_wakeup(d))
phy_resume(dev->phydev);
@@ -3710,6 +3709,8 @@ static int bcmgenet_resume(struct device *d)
bcmgenet_netif_start(dev);
+ netif_device_attach(dev);
+
return 0;
out_clk_disable:
--
2.17.1
^ permalink raw reply related
* [PATCH net 2/2] net: systemport: Protect stop from timeout
From: Florian Fainelli @ 2018-11-01 22:55 UTC (permalink / raw)
To: netdev; +Cc: jaedon.shin, opendmb, Florian Fainelli, David S. Miller,
open list
In-Reply-To: <20181101225538.18632-1-f.fainelli@gmail.com>
A timing hazard exists when the network interface is stopped that
allows a watchdog timeout to be processed by a separate core in
parallel. This creates the potential for the timeout handler to
wake the queues while the driver is shutting down, or access
registers after their clocks have been removed.
The more common case is that the watchdog timeout will produce a
warning message which doesn't lead to a crash. The chances of this
are greatly increased by the fact that bcm_sysport_netif_stop stops
the transmit queues which can easily precipitate a watchdog time-
out because of stale trans_start data in the queues.
This commit corrects the behavior by ensuring that the watchdog
timeout is disabled before enterring bcm_sysport_netif_stop. There
are currently only two users of the bcm_sysport_netif_stop function:
close and suspend.
The close case already handles the issue by exiting the RUNNING
state before invoking the driver close service.
The suspend case now performs the netif_device_detach to exit the
PRESENT state before the call to bcm_sysport_netif_stop rather than
after it.
These behaviors prevent any future scheduling of the driver timeout
service during the window. The netif_tx_stop_all_queues function
in bcm_sysport_netif_stop is replaced with netif_tx_disable to ensure
synchronization with any transmit or timeout threads that may
already be executing on other cores.
For symmetry, the netif_device_attach call upon resume is moved to
after the call to bcm_sysport_netif_start. Since it wakes the transmit
queues it is not necessary to invoke netif_tx_start_all_queues from
bcm_sysport_netif_start so it is moved into the driver open service.
Fixes: 40755a0fce17 ("net: systemport: add suspend and resume support")
Fixes: 80105befdb4b ("net: systemport: add Broadcom SYSTEMPORT Ethernet MAC driver")
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
---
drivers/net/ethernet/broadcom/bcmsysport.c | 15 +++++++--------
1 file changed, 7 insertions(+), 8 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bcmsysport.c b/drivers/net/ethernet/broadcom/bcmsysport.c
index 4122553e224b..0e2d99c737e3 100644
--- a/drivers/net/ethernet/broadcom/bcmsysport.c
+++ b/drivers/net/ethernet/broadcom/bcmsysport.c
@@ -1902,9 +1902,6 @@ static void bcm_sysport_netif_start(struct net_device *dev)
intrl2_1_mask_clear(priv, 0xffffffff);
else
intrl2_0_mask_clear(priv, INTRL2_0_TDMA_MBDONE_MASK);
-
- /* Last call before we start the real business */
- netif_tx_start_all_queues(dev);
}
static void rbuf_init(struct bcm_sysport_priv *priv)
@@ -2048,6 +2045,8 @@ static int bcm_sysport_open(struct net_device *dev)
bcm_sysport_netif_start(dev);
+ netif_tx_start_all_queues(dev);
+
return 0;
out_clear_rx_int:
@@ -2071,7 +2070,7 @@ static void bcm_sysport_netif_stop(struct net_device *dev)
struct bcm_sysport_priv *priv = netdev_priv(dev);
/* stop all software from updating hardware */
- netif_tx_stop_all_queues(dev);
+ netif_tx_disable(dev);
napi_disable(&priv->napi);
cancel_work_sync(&priv->dim.dim.work);
phy_stop(dev->phydev);
@@ -2658,12 +2657,12 @@ static int __maybe_unused bcm_sysport_suspend(struct device *d)
if (!netif_running(dev))
return 0;
+ netif_device_detach(dev);
+
bcm_sysport_netif_stop(dev);
phy_suspend(dev->phydev);
- netif_device_detach(dev);
-
/* Disable UniMAC RX */
umac_enable_set(priv, CMD_RX_EN, 0);
@@ -2746,8 +2745,6 @@ static int __maybe_unused bcm_sysport_resume(struct device *d)
goto out_free_rx_ring;
}
- netif_device_attach(dev);
-
/* RX pipe enable */
topctrl_writel(priv, 0, RX_FLUSH_CNTL);
@@ -2788,6 +2785,8 @@ static int __maybe_unused bcm_sysport_resume(struct device *d)
bcm_sysport_netif_start(dev);
+ netif_device_attach(dev);
+
return 0;
out_free_rx_ring:
--
2.17.1
^ permalink raw reply related
* Re: [PULL] vhost: cleanups and fixes
From: Linus Torvalds @ 2018-11-01 23:06 UTC (permalink / raw)
To: Kees Cook
Cc: mst, kvm, virtualization, netdev, Linux Kernel Mailing List,
Andrew Morton, bijan.mottahedeh, gedwards, joe, lenaic,
liang.z.li, mhocko, mhocko, stefanha, wei.w.wang
In-Reply-To: <CAGXu5jJ0HgV2qN=wohEgro6ixqXHOHBTsvS5a9Dcpz8gxVo3bA@mail.gmail.com>
On Thu, Nov 1, 2018 at 4:00 PM Kees Cook <keescook@chromium.org> wrote:
>
> + memset(&rsp, 0, sizeof(rsp));
> + rsp.response = VIRTIO_SCSI_S_FUNCTION_REJECTED;
> + resp = vq->iov[out].iov_base;
> + ret = __copy_to_user(resp, &rsp, sizeof(rsp));
>
> Is it actually safe to trust that iov_base has passed an earlier
> access_ok() check here? Why not just use copy_to_user() instead?
Good point.
We really should have removed those double-underscore things ages ago.
Also, apart from the address, what about the size? Wouldn't it be
better to use copy_to_iter() rather than implement it badly by hand?
Linus
^ permalink raw reply
* Re: [net 1/8] igb: shorten maximum PHC timecounter update interval
From: Miroslav Lichvar @ 2018-11-01 14:12 UTC (permalink / raw)
To: Jeff Kirsher
Cc: davem, netdev, nhorman, sassmann, Jacob Keller, Richard Cochran
In-Reply-To: <20181031194254.16417-2-jeffrey.t.kirsher@intel.com>
On Wed, Oct 31, 2018 at 12:42:47PM -0700, Jeff Kirsher wrote:
> From: Miroslav Lichvar <mlichvar@redhat.com>
>
> The timecounter needs to be updated at least once per ~550 seconds in
> order to avoid a 40-bit SYSTIM timestamp to be misinterpreted as an old
> timestamp.
>
> Since commit 500462a9d ("timers: Switch to a non-cascading wheel"),
> scheduling of delayed work seems to be less accurate and a requested
> delay of 540 seconds may actually be longer than 550 seconds. Shorten
> the delay to 480 seconds to be sure the timecounter is updated in time.
It looks like this is the v1 of the patch. There was a v2 I sent on
Oct 26, which made the interval even shorter. I can send a separate
patch for that change.
--
Miroslav Lichvar
^ permalink raw reply
* Re: [PULL] vhost: cleanups and fixes
From: Michael S. Tsirkin @ 2018-11-01 23:38 UTC (permalink / raw)
To: Kees Cook
Cc: lenaic, Michal Hocko, bijan.mottahedeh, KVM, Network Development,
liang.z.li, LKML, virtualization, Stefan Hajnoczi, Joe Perches,
Andrew Morton, Michal Hocko, Linus Torvalds
In-Reply-To: <CAGXu5jJ0HgV2qN=wohEgro6ixqXHOHBTsvS5a9Dcpz8gxVo3bA@mail.gmail.com>
On Thu, Nov 01, 2018 at 04:00:23PM -0700, Kees Cook wrote:
> On Thu, Nov 1, 2018 at 2:19 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> > The following changes since commit 84df9525b0c27f3ebc2ebb1864fa62a97fdedb7d:
> >
> > Linux 4.19 (2018-10-22 07:37:37 +0100)
> >
> > are available in the Git repository at:
> >
> > git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git tags/for_linus
> >
> > for you to fetch changes up to 79f800b2e76923cd8ce0aa659cb5c019d9643bc9:
> >
> > MAINTAINERS: remove reference to bogus vsock file (2018-10-24 21:16:14 -0400)
> >
> > ----------------------------------------------------------------
> > virtio, vhost: fixes, tweaks
> >
> > virtio balloon page hinting support
> > vhost scsi control queue
> >
> > misc fixes.
> >
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> >
> > ----------------------------------------------------------------
> > Bijan Mottahedeh (3):
> > vhost/scsi: Respond to control queue operations
>
> +static void
> +vhost_scsi_send_tmf_resp(struct vhost_scsi *vs,
> + struct vhost_virtqueue *vq,
> + int head, unsigned int out)
> +{
> + struct virtio_scsi_ctrl_tmf_resp __user *resp;
> + struct virtio_scsi_ctrl_tmf_resp rsp;
> + int ret;
> +
> + pr_debug("%s\n", __func__);
> + memset(&rsp, 0, sizeof(rsp));
> + rsp.response = VIRTIO_SCSI_S_FUNCTION_REJECTED;
> + resp = vq->iov[out].iov_base;
> + ret = __copy_to_user(resp, &rsp, sizeof(rsp));
>
> Is it actually safe to trust that iov_base has passed an earlier
> access_ok() check here? Why not just use copy_to_user() instead?
>
> -Kees
I am not sure copy_to_user will do the right thing here, because all
this runs in context of a kernel thread. We do need access_ok which
takes place way earlier in context of the task.
Another reason it is safe is because the address is not
coming from userspace at all.
> > vhost/scsi: Extract common handling code from control queue handler
> > vhost/scsi: Use common handling code in request queue handler
> >
> > Greg Edwards (1):
> > vhost/scsi: truncate T10 PI iov_iter to prot_bytes
> >
> > Lénaïc Huard (1):
> > kvm_config: add CONFIG_VIRTIO_MENU
> >
> > Stefan Hajnoczi (1):
> > MAINTAINERS: remove reference to bogus vsock file
> >
> > Wei Wang (3):
> > virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT
> > mm/page_poison: expose page_poisoning_enabled to kernel modules
> > virtio-balloon: VIRTIO_BALLOON_F_PAGE_POISON
> >
> > MAINTAINERS | 1 -
> > drivers/vhost/scsi.c | 426 ++++++++++++++++++++++++++++--------
> > drivers/virtio/virtio_balloon.c | 380 +++++++++++++++++++++++++++++---
> > include/uapi/linux/virtio_balloon.h | 8 +
> > kernel/configs/kvm_guest.config | 1 +
> > mm/page_poison.c | 6 +
> > 6 files changed, 688 insertions(+), 134 deletions(-)
>
>
>
> --
> Kees Cook
^ permalink raw reply
* Re: [PULL] vhost: cleanups and fixes
From: Michael S. Tsirkin @ 2018-11-01 23:55 UTC (permalink / raw)
To: Linus Torvalds
Cc: Kees Cook, kvm, virtualization, netdev, Linux Kernel Mailing List,
Andrew Morton, bijan.mottahedeh, gedwards, joe, lenaic,
liang.z.li, mhocko, mhocko, stefanha, wei.w.wang
In-Reply-To: <CAHk-=wicvws38JqzVF6oNEZ0jGzP6RecR6yAGtyzX3AkoJ321g@mail.gmail.com>
On Thu, Nov 01, 2018 at 04:06:19PM -0700, Linus Torvalds wrote:
> On Thu, Nov 1, 2018 at 4:00 PM Kees Cook <keescook@chromium.org> wrote:
> >
> > + memset(&rsp, 0, sizeof(rsp));
> > + rsp.response = VIRTIO_SCSI_S_FUNCTION_REJECTED;
> > + resp = vq->iov[out].iov_base;
> > + ret = __copy_to_user(resp, &rsp, sizeof(rsp));
> >
> > Is it actually safe to trust that iov_base has passed an earlier
> > access_ok() check here? Why not just use copy_to_user() instead?
>
> Good point.
>
> We really should have removed those double-underscore things ages ago.
Well in case of vhost there are a bunch of reasons to keep them.
One is that all access_ok checks take place
way earlier in context of the owner task. Result is saved
and then used for access repeatedly. Skipping reding access_ok twice
did seem to give a small speedup in the past.
In fact I am looking
into switching some of the uses to unsafe_put_user/unsafe_get_user
after doing something like barrier_nospec after the
access_ok checks. Seems to give a measureable speedup.
Another is that the double underscore things actually can be done
in softirq context if you do preempt_disable/preempt_enable.
IIUC Jason's looking into using that to cut down the latency
for when the access is very small.
> Also, apart from the address, what about the size? Wouldn't it be
> better to use copy_to_iter() rather than implement it badly by hand?
>
> Linus
Generally size is checked when we retrieve the iov but I will recheck
this case and reply here.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox