Re: [LSF/MM/BPF BoF]: extend UBLK to cover real storage hardware

[Ming Lei <ming.lei@redhat.com>]

On Mon, Feb 06, 2023 at 10:26:55AM -0800, Bart Van Assche wrote:
> On 2/6/23 07:00, Ming Lei wrote:
> > 4) DMA
> > - DMA requires physical memory address, UBLK driver actually has
> > block request pages, so can we export request SG list(each segment
> > physical address, offset, len) into userspace? If the max_segments
> > limit is not too big(<=64), the needed buffer for holding SG list
> > can be small enough.
> > 
> > - small amount of physical memory for using as DMA descriptor can be
> > pre-allocated from userspace, and ask kernel to pin pages, then still
> > return physical address to userspace for programming DMA
> > 
> > - this way is still zero copy
> 
> Would it be possible to use vfio in such a way that zero-copy
> functionality is achieved? I'm concerned about the code duplication that
> would result if a new interface similar to vfio is introduced.

Here I meant we can export physical address of request sg from
/dev/ublkb* to userspace, which can program the DMA controller
using exported physical address. With this way, the userspace driver
can submit IO without entering kernel, then with high performance.

This should how SPDK/nvme-pci[1] is implemented, but SPDK allocates
hugepage for getting its physical address.

[1] https://spdk.io/doc/memory.html

Thanks,
Ming