Re: [PATCH 0/2] Unprivileged sgl-only passthrough

[Keith Busch <kbusch@kernel.org>]

On Wed, Oct 18, 2023 at 12:40:35PM -0600, Jens Axboe wrote:
> On 10/18/23 12:30 PM, Kanchan Joshi wrote:
> > Patch 1: Prep. Adds the meta-transfer ability in nvme-pci
> > Patch 2: Enables fine-granular passthrough with the change that i/o
> > commands can transfer the data only via SGL.
> > 
> > Requirement:
> > - Prepared against block 6.6 tree.
> > - The patch in uring-passthrough failure handling is required to see the
> >   submission failure (if any)
> >   https://lore.kernel.org/linux-nvme/20231018135718.28820-1-joshi.k@samsung.com/
> 
> I didn't have time to follow the previous discussion, but what's the
> reasoning behind allowing it for SGL only? IIRC, we do have an inline
> vec for a small number of vecs, so presumably this would not hit
> alloc+free for each IO? But even so, I would imagine that SGL is slower
> than PRP? Do we know how much?

SGL for metadata is definitely slower, but it's the only nvme protocol
way to directly specify how much memory is actually available for the
command's transfer. PRP/MPTR vs SGL is like strcpy() vs strncpy().

Similiar to Kanchan's earlier experience though, I haven't found real
nvme devices that support the SGL mode for metadata. The scenarios this
enables might be pretty limited. :(

The other hardware "solution" is turn on your IOMMU (eww).