Re: [PATCH RFC 0/3] DPDK ethdev callback support

[Bruce Richardson <bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>]

On Mon, Dec 22, 2014 at 10:31:57AM -0800, Stephen Hemminger wrote:
> On Mon, 22 Dec 2014 16:47:21 +0000
> Bruce Richardson <bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:
> 
> > This RFC is for a small addition to the ethdev library, to add in support for
> > callbacks at the RX and TX stages. This allows packet processing to be done on
> > packets before they get returned to applications using rte_eth_rx_burst call.
> > 
> > Use case: the first use case for this is to enable a consistent set of 
> > packets mbufs to be received by applications irrespective of the NIC used
> > to receive those. For example, for a port type that does not support RSS,
> > a callback on RX can be configured to calculate a hash in software. 
> > Similarly, this mechanism can be used to add other information to mbufs
> > as they are received, such as timestamps or sequence numbers, without cluttering
> > up the main packet processing path with checks for whether packets have these
> > fields filled in or not.
> > A second use case is ease of intrumenting existing code. The example application 
> > shows how combining a timestamp insertion callback on RX can be paired with a 
> > latency calculation callback on TX to easily instrument any application for
> > packet latency.
> > A third use case is to potentially extend existing NIC capabilities beyond
> > what is currently supported. For example, where flow director capabilities
> > can match up to a certain limit of flows - in the thousands, in the case of
> > NICs using the ixgbe driver - a callback can extend this to potentially
> > millions of flows by using a software hash table lookup inline for packets
> > that missing the hardware lookup filters. It would all appear transparent 
> > to the packet handling code in the main application.
> > 
> > Future extensions: in future the ethdev library can be extended to provide
> > a standard set of callbacks for use by drivers. 
> > 
> > For now this patch set is RFC and still needs additional work for creating
> > a remove function for callbacks and to add in additional testing code.
> > Since this adds in new code into the critical data path, I have run some
> > performance tests using testpmd with the ixgbe vector drivers (i.e. the
> > fastest, fast-path we have :-) ). Performance drops due to this patch
> > seems minimal to non-existant, rough tests on my system indicate a drop
> > of perhaps 1%.
> > 
> > All feedback welcome.
> > 
> > Bruce Richardson (3):
> >   ethdev: rename callbacks field to intr_cbs
> >   ethdev: Add in data rxtx callback support
> >   examples: example showing use of callbacks.
> > 
> >  app/test/virtual_pmd.c                 |   2 +-
> >  examples/rxtx_callbacks/Makefile       |  57 +++++++++
> >  examples/rxtx_callbacks/basicfwd.c     | 222 +++++++++++++++++++++++++++++++++
> >  examples/rxtx_callbacks/basicfwd.h     |  46 +++++++
> >  lib/librte_ether/rte_ethdev.c          | 103 +++++++++++++--
> >  lib/librte_ether/rte_ethdev.h          | 125 ++++++++++++++++++-
> >  lib/librte_pmd_bond/rte_eth_bond_api.c |   2 +-
> >  7 files changed, 543 insertions(+), 14 deletions(-)
> >  create mode 100644 examples/rxtx_callbacks/Makefile
> >  create mode 100644 examples/rxtx_callbacks/basicfwd.c
> >  create mode 100644 examples/rxtx_callbacks/basicfwd.h
> > 
> 
> What about SMP safety? The callback list is not thread safe.
> Do you plan to start integrating with an RCU framework like userspace RCU?

I hadn't considered multiple CPUs adding removing callbacks simultaneously. I'll
look into it.

/Bruce