Re: [PATCH RFC 00/11] nvmet: Add NVMe target mdev/vfio driver

[Christoph Hellwig <hch@lst.de>]

On Thu, Mar 13, 2025 at 12:18:01AM -0500, Mike Christie wrote:
> 
> If we agree on a new virtual NVMe driver being ok, why mdev vs vhost?
> =====================================================================
> The problem with a vhost nvme is:
> 
> 2.1. If we do a fully vhost nvmet solution, it will require new guest
> drivers that present NVMe interfaces to userspace then perform the
> vhost spec on the backend like how vhost-scsi does.
>
> I don't want to implement a windows or even a linux nvme vhost
> driver. I don't think anyone wants the extra headache.

As in a nvme-virtio spec?  Note that I suspect you could use the
vhost infrastructure for something that isn't virtio, but it would
be a fair amount of work.

> 2.2. We can do a hybrid approach where in the guest it looks like we
> are a normal old local NVMe drive and use the guest's native NVMe driver.
> However in QEMU we would have a vhost nvme module that instead of using
> vhost virtqueues handles virtual PCI memory accesses as well as a vhost
> nvme kernel or user driver to process IO.
> 
> So not as much extra code as option 1 since we don't have to worry about
> the guest but still extra QEMU code.

And it does sound rather inefficient to me.

> Why not a new blk driver or why not vdpa blk?
> =============================================
> Applications want standardized interfaces for things like persistent
> reservations. They have to support them with SCSI and NVMe already
> and don't want to have to support a new virtio block interface.
> 
> Also the nvmet-mdev-pci driver in this patchset can perform was well
> as SPDK vhost blk so that doesn't have the perf advantage like it
> used to.

Maybe I'm too old school, but I find vdpa a complete pain in the neck
to deal with in any way..

> 1. Should the driver integrate with pci-epf (the drivers work very
> differently but could share some code)?

If we can easily share code we should in a library.  But we should
not force sharing code where it just make things more complicated.

> 2. Should it try to fit into the existing configfs interface or implement
> it's own like how pci-epf did? I did an attempt for this but it feels
> wrong.

pci-epf needs to integrate with the pci endpoint configfs interface
exposed by that subsystem.  So the way it works wasn't really a choice
but a requirement to interact with the underlying abstraction.