Re: [PATCH RFC] nvme-ioctl: propagate PRP1 from ioctl to admin cmd

Re: [PATCH RFC] nvme-ioctl: propagate PRP1 from ioctl to admin cmd

[Keith Busch]

On Mon, Jun 22, 2026 at 04:15:42PM +0200, David Epping wrote:
> @@ -306,6 +306,7 @@ static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
>  	c.common.nsid = cpu_to_le32(cmd.nsid);
>  	c.common.cdw2[0] = cpu_to_le32(cmd.cdw2);
>  	c.common.cdw2[1] = cpu_to_le32(cmd.cdw3);
> +	c.common.dptr.prp1 = cpu_to_le64(cmd.addr);

This is not correct: the user space virtual address isn't the device
DMA'able address. The driver already handles mapping the user address to
kernel space, then to dma, then sets the PRP accordingly.

Re: [PATCH RFC] nvme-ioctl: propagate PRP1 from ioctl to admin cmd

[David Epping]

On Mon, Jun 22, 2026 at 08:35:42AM -0600, Keith Busch wrote:
> On Mon, Jun 22, 2026 at 04:15:42PM +0200, David Epping wrote:
> > @@ -306,6 +306,7 @@ static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
> >  	c.common.nsid = cpu_to_le32(cmd.nsid);
> >  	c.common.cdw2[0] = cpu_to_le32(cmd.cdw2);
> >  	c.common.cdw2[1] = cpu_to_le32(cmd.cdw3);
> > +	c.common.dptr.prp1 = cpu_to_le64(cmd.addr);
> 
> This is not correct: the user space virtual address isn't the device
> DMA'able address. The driver already handles mapping the user address to
> kernel space, then to dma, then sets the PRP accordingly.

To clarify, the ioctl struct addr field is not filled with a memory buffer
address by the userspace, but a PCIe mapped BAR address plus offset.
It is obtained by the userspace application operating the FPGA vfio device
by reading from PCI config space via VFIO_PCI_CONFIG_REGION_INDEX.
So it is the address Linux assigned to that BAR (plus offset).

Re: [PATCH RFC] nvme-ioctl: propagate PRP1 from ioctl to admin cmd

[Keith Busch]

On Mon, Jun 22, 2026 at 04:56:22PM +0200, David Epping wrote:
> On Mon, Jun 22, 2026 at 08:35:42AM -0600, Keith Busch wrote:
> > On Mon, Jun 22, 2026 at 04:15:42PM +0200, David Epping wrote:
> > > @@ -306,6 +306,7 @@ static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
> > >  	c.common.nsid = cpu_to_le32(cmd.nsid);
> > >  	c.common.cdw2[0] = cpu_to_le32(cmd.cdw2);
> > >  	c.common.cdw2[1] = cpu_to_le32(cmd.cdw3);
> > > +	c.common.dptr.prp1 = cpu_to_le64(cmd.addr);
> > 
> > This is not correct: the user space virtual address isn't the device
> > DMA'able address. The driver already handles mapping the user address to
> > kernel space, then to dma, then sets the PRP accordingly.
> 
> To clarify, the ioctl struct addr field is not filled with a memory buffer
> address by the userspace, but a PCIe mapped BAR address plus offset.
> It is obtained by the userspace application operating the FPGA vfio device
> by reading from PCI config space via VFIO_PCI_CONFIG_REGION_INDEX.
> So it is the address Linux assigned to that BAR (plus offset).

The driver and block layer should already handle PCIe addresses. You're
supposed to mmap it to user space first though, and pass that address in
instead. And you'd also need to set cmd.data_len to a non-zero value so
the driver doesn't skip setting up the data pointers.

Note, creating IO queues from user space, while not explicitly prevented
today, is not supported. The driver doesn't know you've done this so the
queue isn't properly handled on a controller reset.