Re: [RFC PATCH v2 02/11] netdev: implement netlink api to bind dma-buf to netdevice

[Jakub Kicinski <kuba@kernel.org>]

On Mon, 21 Aug 2023 20:38:09 -0400 Willem de Bruijn wrote:
> > Are you talking about HW devices, or virt? I thought most HW made
> > in the last 10 years should be able to take down individual queues :o  
> 
> That's great. This is currently mostly encapsulated device-wide behind
> ndo_close, with code looping over all rx rings, say.
> 
> Taking a look at one driver, bnxt, it indeed has a per-ring
> communication exchange with the device, in hwrm_ring_free_send_msg
> ("/* Flush rings and disable interrupts */"), which is called before
> the other normal steps: napi disable, dma unmap, posted mem free,
> irq_release, napi delete and ring mem free.
> 
> This is what you meant? The issue I was unsure of was quiescing the
> device immediately, i.e., that hwrm_ring_free_send_msg.

Yes, and I recall we had similar APIs at Netronome for the NFP.
I haven't see it in MS specs myself but I wouldn't be surprised if 
they required it..

There's a bit of an unknown in how well all of this actually works,
as the FW/HW paths were not exercised outside of RDMA and potentially
other proprietary stuff.

> I guess this means that this could all be structured on a per-queue
> basis rather than from ndo_close. Would be a significant change to
> many drivers, I'd imagine.

Yes, it definitely is. The question is how much of it do we require
from Mina before merging the mem provider work. I'd really like to
avoid the "delegate all the work to the driver" approach that AF_XDP
has taken, which is what I'm afraid we'll end up with if we push too
hard for a full set of APIs from the start.