Re: [PATCH RFC] nvme-ioctl: propagate PRP1 from ioctl to admin cmd

[David Epping <david.epping@missinglinkelectronics.com>]

On Mon, Jun 22, 2026 at 09:15:40AM -0600, Keith Busch wrote:
> On Mon, Jun 22, 2026 at 04:56:22PM +0200, David Epping wrote:
> > On Mon, Jun 22, 2026 at 08:35:42AM -0600, Keith Busch wrote:
> > > On Mon, Jun 22, 2026 at 04:15:42PM +0200, David Epping wrote:
> > > > @@ -306,6 +306,7 @@ static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
> > > >  	c.common.nsid = cpu_to_le32(cmd.nsid);
> > > >  	c.common.cdw2[0] = cpu_to_le32(cmd.cdw2);
> > > >  	c.common.cdw2[1] = cpu_to_le32(cmd.cdw3);
> > > > +	c.common.dptr.prp1 = cpu_to_le64(cmd.addr);
> > > 
> > > This is not correct: the user space virtual address isn't the device
> > > DMA'able address. The driver already handles mapping the user address to
> > > kernel space, then to dma, then sets the PRP accordingly.
> > 
> > To clarify, the ioctl struct addr field is not filled with a memory buffer
> > address by the userspace, but a PCIe mapped BAR address plus offset.
> > It is obtained by the userspace application operating the FPGA vfio device
> > by reading from PCI config space via VFIO_PCI_CONFIG_REGION_INDEX.
> > So it is the address Linux assigned to that BAR (plus offset).
> 
> The driver and block layer should already handle PCIe addresses. You're
> supposed to mmap it to user space first though, and pass that address in
> instead. And you'd also need to set cmd.data_len to a non-zero value so
> the driver doesn't skip setting up the data pointers.
> 
> Note, creating IO queues from user space, while not explicitly prevented
> today, is not supported. The driver doesn't know you've done this so the
> queue isn't properly handled on a controller reset.
> 

Hi Keith, I understand that creating IO queues from user space is not
supported by the current driver. That's why we created patches for that a
couple of years ago and ported them to new Kernels since.

My question is, and maybe I should have put this in my initial email
explicitely, is there interest in having such functionality in the upstream
Linux in-Kernel NVMe driver? An interface and mechanism to request and
manage IO queues that are not used by the Linux NVMe driver to perform IO,
but handed to a separate entity for this purpose.

Of course an upstream implementation would have to take many more things
into account, like the reset you mentioned, and IOMMU setup, and much more.
But that's only worth looking at if there is upstream interest in it.

Thanks for your feedback, David