Re: Kernel 4.19 network performance - forwarding/routing normal users traffic

[Jesper Dangaard Brouer <brouer@redhat.com>]

On Sat, 10 Nov 2018 22:53:53 +0100 Paweł Staszewski <pstaszewski@itcare.pl> wrote:

> Now im messing with ring configuration for connectx5 nics.
> And after reading that paper:
> https://netdevconf.org/2.1/slides/apr6/network-performance/04-amir-RX_and_TX_bulking_v2.pdf
>

Do notice that some of the ideas in that slide deck, was never
implemented. But they are still on my todo list ;-).

Notice how that it show that TX bulking is very important, but based on
your ethtool_stats.pl, I can see that not much TX bulking is happening
in your case.  This is indicated via the xmit_more counters.

 Ethtool(enp175s0) stat:    2630 (     2,630) <= tx_xmit_more /sec
 Ethtool(enp175s0) stat: 4956995 ( 4,956,995) <= tx_packets /sec

And the per queue levels are also avail:

 Ethtool(enp175s0) stat: 184845 ( 184,845) <= tx7_packets /sec
 Ethtool(enp175s0) stat:     78 (      78) <= tx7_xmit_more /sec

This means that you are doing too many doorbell's to the NIC hardware
at TX time, which I worry could be what cause the NIC and PCIe hardware
not to operate at optimal speeds.

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer