Re: [PATCH 0/2] Unprivileged sgl-only passthrough

[Jens Axboe <axboe@kernel.dk>]

On 10/18/23 1:35 PM, Keith Busch wrote:
> On Wed, Oct 18, 2023 at 12:40:35PM -0600, Jens Axboe wrote:
>> On 10/18/23 12:30 PM, Kanchan Joshi wrote:
>>> Patch 1: Prep. Adds the meta-transfer ability in nvme-pci
>>> Patch 2: Enables fine-granular passthrough with the change that i/o
>>> commands can transfer the data only via SGL.
>>>
>>> Requirement:
>>> - Prepared against block 6.6 tree.
>>> - The patch in uring-passthrough failure handling is required to see the
>>>   submission failure (if any)
>>>   https://lore.kernel.org/linux-nvme/20231018135718.28820-1-joshi.k@samsung.com/
>>
>> I didn't have time to follow the previous discussion, but what's the
>> reasoning behind allowing it for SGL only? IIRC, we do have an inline
>> vec for a small number of vecs, so presumably this would not hit
>> alloc+free for each IO? But even so, I would imagine that SGL is slower
>> than PRP? Do we know how much?
> 
> SGL for metadata is definitely slower, but it's the only nvme protocol
> way to directly specify how much memory is actually available for the
> command's transfer. PRP/MPTR vs SGL is like strcpy() vs strncpy().

So... is this a metadata only issue? Or does it apply to any read/write,
metadata or not?

> Similiar to Kanchan's earlier experience though, I haven't found real
> nvme devices that support the SGL mode for metadata. The scenarios this
> enables might be pretty limited. :(

Well that certainly makes it way less useful.

> The other hardware "solution" is turn on your IOMMU (eww).

Yep, certainly also a way to pay IO taxes.

-- 
Jens Axboe