Re: [PATCH vhost v4 00/10] virtio: drivers maintain dma info for premapped vq

["Michael S. Tsirkin" <mst@redhat.com>]

On Tue, Mar 12, 2024 at 11:35:47AM +0800, Xuan Zhuo wrote:
> As discussed:
> 
> http://lore.kernel.org/all/CACGkMEvq0No8QGC46U4mGsMtuD44fD_cfLcPaVmJ3rHYqRZxYg@mail.gmail.com
> 
> If the virtio is premapped mode, the driver should manage the dma info by self.
> So the virtio core should not store the dma info. We can release the memory used
> to store the dma info.
> 
> For virtio-net xmit queue, if the virtio-net maintains the dma info,
> the virtio-net must allocate too much memory(19 * queue_size for per-queue), so
> we do not plan to make the virtio-net to maintain the dma info by default. The
> virtio-net xmit queue only maintain the dma info when premapped mode is enable
> (such as AF_XDP is enable).

This landed when merge window was open already so I'm deferring this
to the next merge window, just to be safe. Jason can you review please?

> So this patch set try to do:
> 
> 1. make the virtio core to do not store the dma info
>     - But if the desc_extra has not dma info, we face a new question,
>       it is hard to get the dma info of the desc with indirect flag.
>       For split mode, that is easy from desc, but for the packed mode,
>       it is hard to get the dma info from the desc. And hardening
>       the dma unmap is safe, we should store the dma info of indirect
>       descs when the virtio core does not store the bufer dma info.
> 
>       So I introduce the "structure the indirect desc table" to
>       allocate space to store dma info of the desc table.
> 
>         +struct vring_split_desc_indir {
>         +       dma_addr_t addr;                /* Descriptor Array DMA addr. */
>         +       u32 len;                        /* Descriptor Array length. */
>         +       u32 num;
>         +       struct vring_desc desc[];
>         +};
> 
>       The follow patches to this:
>          * virtio_ring: packed: structure the indirect desc table
>          * virtio_ring: split: structure the indirect desc table
> 
>     - On the other side, in the umap handle, we mix the indirect descs with
>       other descs. That make things too complex. I found if we we distinguish
>       the descs with VRING_DESC_F_INDIRECT before unmap, thing will be clearer.
> 
>       The follow patches do this.
>          * virtio_ring: packed: remove double check of the unmap ops
>          * virtio_ring: split: structure the indirect desc table
> 
> 2. make the virtio core to enable premapped mode by find_vqs() params
>     - Because the find_vqs() will try to allocate memory for the dma info.
>       If we set the premapped mode after find_vqs() and release the
>       dma info, that is odd.
> 
> 
> Please review.
> 
> Thanks
> 
> v4:
>     1. virtio-net xmit queue does not enable premapped mode by default
> 
> v3:
>     1. fix the conflict with the vp_modern_create_avq().
> 
> v2:
>     1. change the dma item of virtio-net, every item have MAX_SKB_FRAGS + 2 addr + len pairs.
>     2. introduce virtnet_sq_free_stats for __free_old_xmit
> 
> v1:
>     1. rename transport_vq_config to vq_transport_config
>     2. virtio-net set dma meta number to (ring-size + 1)(MAX_SKB_FRGAS +2)
>     3. introduce virtqueue_dma_map_sg_attrs
>     4. separate vring_create_virtqueue to an independent commit
> 
> Xuan Zhuo (10):
>   virtio_ring: introduce vring_need_unmap_buffer
>   virtio_ring: packed: remove double check of the unmap ops
>   virtio_ring: packed: structure the indirect desc table
>   virtio_ring: split: remove double check of the unmap ops
>   virtio_ring: split: structure the indirect desc table
>   virtio_ring: no store dma info when unmap is not needed
>   virtio: find_vqs: add new parameter premapped
>   virtio_ring: export premapped to driver by struct virtqueue
>   virtio_net: set premapped mode by find_vqs()
>   virtio_ring: virtqueue_set_dma_premapped support disable
> 
>  drivers/net/virtio_net.c      |  57 +++--
>  drivers/virtio/virtio_ring.c  | 436 +++++++++++++++++++++-------------
>  include/linux/virtio.h        |   3 +-
>  include/linux/virtio_config.h |  17 +-
>  4 files changed, 307 insertions(+), 206 deletions(-)
> 
> --
> 2.32.0.3.g01195cf9f