Re: [PATCH v2] Support for write stream IDs

[Jens Axboe <axboe@fb.com>]

On 05/05/2015 02:51 PM, Jeff Moyer wrote:
> Jens Axboe <axboe@fb.com> writes:
>
>> Hi,
>>
>> Changes since the last posting:
>>
>> - Added a specific per-file fadvise setting. POSIX_FADV_STREAMID sets
>>    the inode and file stream ID, POSIX_FADV_STREAMID_FILE sets just the
>>    file stream ID.
>>
>> - Addressed review comments.
>>
>> I've since run some testing with write streams. Test case was a RocksDB
>> overwrite benchmark, using 3 billion keys of 400B in size (numbers set
>> use the full size of the device). WAL/LOG was assigned to stream 1, and
>> each RocksDB compaction level used a separate stream. With streams
>> enabled, user write to device writes (write amplification) was at 2.33.
>> Without streams, the write amplification was 3.05. That is roughly 20%
>> less written NAND, and the streams test subsequently also had 20%
>> higher throughput.
>>
>> Unless there are any grave concerns here, I'd like to merge this for
>> 4.2.
>
> I have a few concerns.  You've added POSIX_FADV_* definitions that do
> not exist in the SUS/POSIX spec.  Do we care?  We (poor reviewers) still
> have no idea what the driver side of this will look like.  Do streams
> need to be opened and closed?  Is that going to be handled transparently
> by the kernel, or exposed to userspace?  If in the kernel, where in the
> kernel?  You've also added a user-visible api without cc-ing linux-api.

Whether this should be fadvise, fcntl, or something else, that's the 
primary review concern.

The driver side depends on the driver! The kernel patches deal only with 
ensuring that the stream information gets passed down. If the device 
requires explicit stream open/close actions, then that needs to be 
handled on the side. There's no reason to include the kernel in that, 
the kernel doesn't care.

> My preference would be to wait for the spec to finalize before pushing
> in changes that depend on it.

I think you are mixing up the write streams with the NVMe proposal. The 
two aren't necessarily connected, and the kernel parts don't really care 
what the NVMe proposal ends up looking like. It's just an interface to 
assign an ID, and a transport for passing that ID down to a driver.

-- 
Jens Axboe