Re: [RFC PATCH v2 02/11] netdev: implement netlink api to bind dma-buf to netdevice

[David Ahern <dsahern@kernel.org>]

On 8/18/23 3:52 PM, Mina Almasry wrote:
> The sticking points are:
> 1. From David: this proposal doesn't give an application the ability
> to flush an rx queue, which means that we have to rely on a driver
> reset that affects all queues to refill the rx queue buffers.

Generically, the design needs to be able to flush (or invalidate) all
references to the dma-buf once the process no longer "owns" it.

> 2. From Jakub: the uAPI and implementation here needs to be in line
> with his general direction & extensible to apply to existing use cases
> `ethtool -L/-G`, etc.

I think this is a bit more open ended given the openness of the netdev
netlink API. i.e., managing a H/W queue (create, delete, stop / flush,
associate a page_pool) could be done through this API.

> 
> AFAIU this is what I need to do in the next version:
> 
> 1. The uAPI will be changed such that it will either re-configure an
> existing queue to bind it to the dma-buf, or allocate a new queue
> bound to the dma-buf (not sure which is better at the moment). Either

1. API to manage a page-pool (create, delete, update).

2. API to add and remove a dma-buf (or host memory buffer) with a
page-pool. Remove may take time to flush references pushed to hardware
so this would be asynchronous.

3. Create a queue or use an existing queue id and associate a page-pool
with it.

> way, the configuration will take place immediately, and not rely on an
> entire driver reset to actuate the change.

yes

> 
> 2. The uAPI will be changed such that if the netlink socket is closed,
> or the process dies, the rx queue will be unbound from the dma-buf or
> the rx queue will be freed entirely (again, not sure which is better

I think those are separate actions. But, if the queue was created by and
referenced by a process, then closing an fd means it should be freed.

> at the moment). The configuration will take place immediately without
> relying on a driver reset.

yes on the reset.

> 
> 3. I will add 4 new net_device_ops that Jakub specified:
> queue_mem_alloc/free(), and queue_start/stop().
> 
> 4. The uAPI mentioned in #1 will use the new net_device_ops to
> allocate or reconfigure a queue attached to the provided dma-buf.
> 
> Does this sound roughly reasonable here?
> 
> AFAICT the only technical difficulty is that I'm not sure it's
> feasible for a driver to start or stop 1 rx-queue without triggering a
> full driver reset. The (2) drivers I looked at both do a full reset to
> change any queue configuration. I'll investigate.