Re: [WIP] net+mlx4: auto doorbell

[Jesper Dangaard Brouer <brouer@redhat.com>]

On Wed, 30 Nov 2016 18:27:45 +0200
Saeed Mahameed <saeedm@dev.mellanox.co.il> wrote:

> >> All in all, this is risky business :),  the right way to go is to
> >> force the upper layer to use xmit-more and delay doorbells/use bulking
> >> but from the same context (xmit routine).  For example see
> >> Achiad's suggestion (attached in Jesper's response), he used stop
> >> queue to force the stack to queue up packets (TX bulking)
> >> which would set xmit-more and will use the next completion to
> >> release the "stopped" ring TXQ rather than hit the doorbell on
> >> behalf of it.  
> >
> > Well, you depend on having a higher level queue like a qdisc.
> >
> > Some users do not use a qdisc.
> > If you stop the queue, they no longer can send anything -> drops.
> >

You do have a point that stopping the device might not be the best way
to create a push-back (to allow stack queue packets).

 netif_tx_stop_queue() / __QUEUE_STATE_DRV_XOFF


> In this case, i think they should implement their own bulking (pktgen
> is not a good example) but XDP can predict if it has more packets to
> xmit  as long as all of them fall in the same NAPI cycle.
> Others should try and do the same.

I actually agree with Saeed here.

Maybe we can come up with another __QUEUE_STATE_xxx that informs the
upper layer what the driver is doing.  Then users not using a qdisc can
use this indication (like the qdisc could).  (qdisc-bypass users already
check the QUEUE_STATE flags e.g. via netif_xmit_frozen_or_drv_stopped).

My main objection is that this is a driver local optimization.  By not
involving the upper layers, the netstack looses the ability to amortize
it's cost, as it still does per packet handling.

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer