Focusing the XDP project

[Jesper Dangaard Brouer <brouer@redhat.com>]

First thing to bring in order for the XDP project:

  RX batching is missing.

I don't want to discuss packet page-sizes or multi-port forwarding,
before we have established the most fundamental principal that all
other solution use; RX batching.

Without building in RX batching, from the beginning/now, the XDP
architecture have lost.  As adding features and capabilities, will
just lead us back to the exact same performance problems as before!


Today we already have the 64 packets NAPI budget, but we are not
taking advantage of this. For XDP as long as eBPF always return
XDP_DROP or XDP_TX, then we (falsely) experience the effect of bulking
(as code fits within the icache) and see huge perf boosts.

The initial principal of bulking/batching packets to amortize per
packet costs.  The next step is just as important: Lookup table sizes
(FIB) kills performance again. The solution is implementing a smart
table lookup scheme that prefetch hash table key-cells and afterwards
prefetch data-cells, based on the RX batch of packets.  Notice VPP
revolves around similar tricks, and why it beats DPDK, and why it
scales with 1Millon routes.

I hope I've made it very clear where the focus for XDP should be.
This involves implementing what I call RX-stages in the drivers. While
doing that we can figure out the most optimal data structure for
packet batching.
 I know Saeed is already working on RX-stages for mlx5, and I've tested
the initial version of his patch, and the results are excellent.

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer