Re: [PATCH v2] Support for write stream IDs

[Jens Axboe <axboe@fb.com>]

On 05/05/2015 03:39 PM, Martin K. Petersen wrote:
>>>>>> "Jens" == Jens Axboe <axboe@fb.com> writes:
>
> Jens> The kernel patches deal only with ensuring that the stream
> Jens> information gets passed down. If the device requires explicit
> Jens> stream open/close actions, then that needs to be handled on the
> Jens> side.
>
> Well, I'm deeply concerned about that "on the side" thing. I know you're
> trying to make a shortcut by only caring about the transport mechanism
> and deferring the whole programming model piece of the equation.

I'm not trying to make a shortcut. I deliberately do not want to make ID 
generation/assignment part of the kernel. There's no reason that can't 
exist outside of the kernel, in a libstreamid or similar. In fact I 
already wrote that for the NVMe bits, it could be extended if other 
types of devices require open/close type actions for streams.

> I think the latter is hugely important, though. And if we defer the
> programming model question then the chances are that we'll be stuck with
> something lame in the standards.
>
> The storage vendors appear to think that a handful of stream ids ought
> to be enough for anybody. And that IDs are a scarce resource that
> they'll hand out for a limited time if you ask nicely.

I agree, that's an issue. And I just don't understand why that is. It's 
not like containerized or virtualized setups are esoteric these days. A 
handful of IDs might be enough for a single application type setup, but 
not for anything more elaborate.

> I think that model is completely broken and also of limited use for
> non-NVM types of storage. Sadly it's the same people who are driving the
> NVMe and SCSI proposals so they are similar.

It's completely broken for NVM as well. Most people need to farm out the 
use of such devices, as no single app can drive it to its limit anyways.

> My concern is that by adding just enough plumbing to the kernel to allow
> the current standards proposals to work then we'll be stuck with them
> forever. And I think that would be very unfortunate.

The current API doesn't have any real limits (it'll work from 
1..MAX_UINT), and the transport part handles 255 streams at the moment. 
The latter can be easily extended, we can just steal a few more bits. 
Making it 1023 would be a one liner.

-- 
Jens Axboe