[LSF/MM TOPIC] status update on stream IDs

[LSF/MM TOPIC] status update on stream IDs

[Andreas Dilger]

[-- Attachment #1: Type: text/plain, Size: 470 bytes --]

At LSF/MM'16 and Linux FAST (https://lwn.net/Articles/685499/) there were
discussions about adding stream IDs to the block/device layer to allow higher
layers (filesystems, applications) to identify IO streams so lower layers
(SSDs, hybrid storage, etc.) can make better allocation/placement decisions.

It would be useful to get an update on the state of this work, and discuss
any obstacles that need to be resolved for getting this code landed.

Cheers, Andreas






[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [LSF/MM TOPIC] status update on stream IDs

[Andreas Dilger]

[-- Attachment #1.1: Type: text/plain, Size: 619 bytes --]

[resend to include other relevant lists]

On Jan 6, 2017, at 4:54 PM, Andreas Dilger <adilger@dilger.ca> wrote:
> 
> At LSF/MM'16 and Linux FAST (https://lwn.net/Articles/685499/) there were
> discussions about adding stream IDs to the block/device layer to allow higher
> layers (filesystems, applications) to identify IO streams so lower layers
> (SSDs, hybrid storage, etc.) can make better allocation/placement decisions.
> 
> It would be useful to get an update on the state of this work, and discuss
> any obstacles that need to be resolved for getting this code landed.


Cheers, Andreas






[-- Attachment #1.2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 158 bytes --]

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: [LSF/MM TOPIC] status update on stream IDs

[Andreas Dilger]

[-- Attachment #1: Type: text/plain, Size: 619 bytes --]

[resend to include other relevant lists]

On Jan 6, 2017, at 4:54 PM, Andreas Dilger <adilger@dilger.ca> wrote:
> 
> At LSF/MM'16 and Linux FAST (https://lwn.net/Articles/685499/) there were
> discussions about adding stream IDs to the block/device layer to allow higher
> layers (filesystems, applications) to identify IO streams so lower layers
> (SSDs, hybrid storage, etc.) can make better allocation/placement decisions.
> 
> It would be useful to get an update on the state of this work, and discuss
> any obstacles that need to be resolved for getting this code landed.


Cheers, Andreas






[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[LSF/MM TOPIC] status update on stream IDs

[Andreas Dilger]

[resend to include other relevant lists]

On Jan 6, 2017,@4:54 PM, Andreas Dilger <adilger@dilger.ca> wrote:
> 
> At LSF/MM'16 and Linux FAST (https://lwn.net/Articles/685499/) there were
> discussions about adding stream IDs to the block/device layer to allow higher
> layers (filesystems, applications) to identify IO streams so lower layers
> (SSDs, hybrid storage, etc.) can make better allocation/placement decisions.
> 
> It would be useful to get an update on the state of this work, and discuss
> any obstacles that need to be resolved for getting this code landed.


Cheers, Andreas





-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20170106/2f7127e5/attachment.sig>

Re: [Lsf-pc] [LSF/MM TOPIC] status update on stream IDs

[James Bottomley]

[-- Attachment #1: Type: text/plain, Size: 796 bytes --]

On Fri, 2017-01-06 at 16:54 -0700, Andreas Dilger wrote:
> At LSF/MM'16 and Linux FAST (https://lwn.net/Articles/685499/) there 
> were discussions about adding stream IDs to the block/device layer to
> allow higher layers (filesystems, applications) to identify IO 
> streams so lower layers (SSDs, hybrid storage, etc.) can make better 
> allocation/placement decisions.
> 
> It would be useful to get an update on the state of this work, and 
> discuss any obstacles that need to be resolved for getting this code
> landed.

OK, so I'm a bit at a loss for how to add this one to the spreadsheet. 
 Usually people propose topics *they* would like to talk about.  Since
this is a topic you want to hear about, did you have anyone in mind to
be the speaker on this topic?

James

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [Lsf-pc] [LSF/MM TOPIC] status update on stream IDs

[Andreas Dilger]

[-- Attachment #1: Type: text/plain, Size: 1145 bytes --]

On Jan 6, 2017, at 5:05 PM, James Bottomley <James.Bottomley@HansenPartnership.com> wrote:
> 
> On Fri, 2017-01-06 at 16:54 -0700, Andreas Dilger wrote:
>> At LSF/MM'16 and Linux FAST (https://lwn.net/Articles/685499/) there
>> were discussions about adding stream IDs to the block/device layer to
>> allow higher layers (filesystems, applications) to identify IO
>> streams so lower layers (SSDs, hybrid storage, etc.) can make better
>> allocation/placement decisions.
>> 
>> It would be useful to get an update on the state of this work, and
>> discuss any obstacles that need to be resolved for getting this code
>> landed.
> 
> OK, so I'm a bit at a loss for how to add this one to the spreadsheet.
> Usually people propose topics *they* would like to talk about.  Since
> this is a topic you want to hear about, did you have anyone in mind to
> be the speaker on this topic?

Martin presented on this last year at LSF/MM, and maybe Jens did at FAST
in Feb'16 (Ted or Ric might remember).  Hopefully whomever was working on
patches at that time will step forward once they see this thread.

Cheers, Andreas






[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

RE: [LSF/MM TOPIC] status update on stream IDs

[Kwan (Hingkwan) Huen]

It's been a while since Jens posted the write stream ID patches.

https://lwn.net/Articles/679136/

As recall, these patches provide the framework enabling applications to write data along with the stream ID hint. Writes with the same stream ID indicate that data being written related to each other. This simple ID hint allows SSD to organize data more efficiently internally, resulting better performance of the device in the long run and longer device lifespan. However, without a mature spec describing how the device works, and a device that can be easily acquired for testing, these patches were too early back then.

With the new NVMe v1.3 spec draft available now, which includes the latest streams directive feature, and we already have the device available on hand, would like to bring this up again discuss this with the community for opinions. Here are some ideas we have in mind based on what's been done in Jens' patch from the link above. Any comments and suggestions are welcome and appreciated.

Jens' patches already provides the frame work that passes the stream ID from Application to kernel with new system call streamid().  What's needed is that the block device driver will need to pick up the stream ID from the bio and attached to the write command to the device. This completes the whole data path.

The stream management (enable/allocate/open/close, etc.) can go through functions mapped via backing device info (bdi) to allow same function all from application to manage streams in either a SCSI or NVMe device. The actual functions will be implemented in the block device driver and stream status and statistics can be stored, accessed and updated in gendisk struct. 

If anyone wants kernel to assign the ID, the write requests will need to be intercepted to collect statistics of the workload pattern, run some algorithm that determines and applies the write requests to appropriate stream. Or Kernel maintainers would have already good idea for the stream ID assignment. In this case they can just implement it without adding extra stream detection modules. These can be done in the block device with stream mapping stored in gendisk struct.

Regards, 
kwan
________________________________________
From: Linux-nvme [linux-nvme-bounces@lists.infradead.org] on behalf of Andreas Dilger [adilger@dilger.ca]
Sent: Friday, January 06, 2017 3:58 PM
To: lsf-pc@lists.linux-foundation.org
Cc: linux-fsdevel; linux-block@vger.kernel.org; linux-nvme@lists.infradead.org; linux-scsi@vger.kernel.org; linux-ide@vger.kernel.org
Subject: Re: [LSF/MM TOPIC] status update on stream IDs

[resend to include other relevant lists]

On Jan 6, 2017, at 4:54 PM, Andreas Dilger <adilger@dilger.ca> wrote:
>
> At LSF/MM'16 and Linux FAST (https://lwn.net/Articles/685499/) there were
> discussions about adding stream IDs to the block/device layer to allow higher
> layers (filesystems, applications) to identify IO streams so lower layers
> (SSDs, hybrid storage, etc.) can make better allocation/placement decisions.
>
> It would be useful to get an update on the state of this work, and discuss
> any obstacles that need to be resolved for getting this code landed.


Cheers, Andreas






_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

RE: [LSF/MM TOPIC] status update on stream IDs

[Kwan (Hingkwan) Huen]

It's been a while since Jens posted the write stream ID patches.

https://lwn.net/Articles/679136/

As recall, these patches provide the framework enabling applications to write data along with the stream ID hint. Writes with the same stream ID indicate that data being written related to each other. This simple ID hint allows SSD to organize data more efficiently internally, resulting better performance of the device in the long run and longer device lifespan. However, without a mature spec describing how the device works, and a device that can be easily acquired for testing, these patches were too early back then.

With the new NVMe v1.3 spec draft available now, which includes the latest streams directive feature, and we already have the device available on hand, would like to bring this up again discuss this with the community for opinions. Here are some ideas we have in mind based on what's been done in Jens' patch from the link above. Any comments and suggestions are welcome and appreciated.

Jens' patches already provides the frame work that passes the stream ID from Application to kernel with new system call streamid().  What's needed is that the block device driver will need to pick up the stream ID from the bio and attached to the write command to the device. This completes the whole data path.

The stream management (enable/allocate/open/close, etc.) can go through functions mapped via backing device info (bdi) to allow same function all from application to manage streams in either a SCSI or NVMe device. The actual functions will be implemented in the block device driver and stream status and statistics can be stored, accessed and updated in gendisk struct. 

If anyone wants kernel to assign the ID, the write requests will need to be intercepted to collect statistics of the workload pattern, run some algorithm that determines and applies the write requests to appropriate stream. Or Kernel maintainers would have already good idea for the stream ID assignment. In this case they can just implement it without adding extra stream detection modules. These can be done in the block device with stream mapping stored in gendisk struct.

Regards, 
kwan
________________________________________
From: Linux-nvme [linux-nvme-bounces@lists.infradead.org] on behalf of Andreas Dilger [adilger@dilger.ca]
Sent: Friday, January 06, 2017 3:58 PM
To: lsf-pc@lists.linux-foundation.org
Cc: linux-fsdevel; linux-block@vger.kernel.org; linux-nvme@lists.infradead.org; linux-scsi@vger.kernel.org; linux-ide@vger.kernel.org
Subject: Re: [LSF/MM TOPIC] status update on stream IDs

[resend to include other relevant lists]

On Jan 6, 2017, at 4:54 PM, Andreas Dilger <adilger@dilger.ca> wrote:
>
> At LSF/MM'16 and Linux FAST (https://lwn.net/Articles/685499/) there were
> discussions about adding stream IDs to the block/device layer to allow higher
> layers (filesystems, applications) to identify IO streams so lower layers
> (SSDs, hybrid storage, etc.) can make better allocation/placement decisions.
>
> It would be useful to get an update on the state of this work, and discuss
> any obstacles that need to be resolved for getting this code landed.


Cheers, Andreas

[LSF/MM TOPIC] status update on stream IDs

[Kwan (Hingkwan) Huen]

It's been a while since Jens posted the write stream ID patches.

https://lwn.net/Articles/679136/

As recall, these patches provide the framework enabling applications to write data along with the stream ID hint. Writes with the same stream ID indicate that data being written related to each other. This simple ID hint allows SSD to organize data more efficiently internally, resulting better performance of the device in the long run and longer device lifespan. However, without a mature spec describing how the device works, and a device that can be easily acquired for testing, these patches were too early back then.

With the new NVMe v1.3 spec draft available now, which includes the latest streams directive feature, and we already have the device available on hand, would like to bring this up again discuss this with the community for opinions. Here are some ideas we have in mind based on what's been done in Jens' patch from the link above. Any comments and suggestions are welcome and appreciated.

Jens' patches already provides the frame work that passes the stream ID from Application to kernel with new system call streamid().  What's needed is that the block device driver will need to pick up the stream ID from the bio and attached to the write command to the device. This completes the whole data path.

The stream management (enable/allocate/open/close, etc.) can go through functions mapped via backing device info (bdi) to allow same function all from application to manage streams in either a SCSI or NVMe device. The actual functions will be implemented in the block device driver and stream status and statistics can be stored, accessed and updated in gendisk struct. 

If anyone wants kernel to assign the ID, the write requests will need to be intercepted to collect statistics of the workload pattern, run some algorithm that determines and applies the write requests to appropriate stream. Or Kernel maintainers would have already good idea for the stream ID assignment. In this case they can just implement it without adding extra stream detection modules. These can be done in the block device with stream mapping stored in gendisk struct.

Regards, 
kwan
________________________________________
From: Linux-nvme [linux-nvme-bounces@lists.infradead.org] on behalf of Andreas Dilger [adilger@dilger.ca]
Sent: Friday, January 06, 2017 3:58 PM
To: lsf-pc at lists.linux-foundation.org
Cc: linux-fsdevel; linux-block at vger.kernel.org; linux-nvme at lists.infradead.org; linux-scsi at vger.kernel.org; linux-ide at vger.kernel.org
Subject: Re: [LSF/MM TOPIC] status update on stream IDs

[resend to include other relevant lists]

On Jan 6, 2017,@4:54 PM, Andreas Dilger <adilger@dilger.ca> wrote:
>
> At LSF/MM'16 and Linux FAST (https://lwn.net/Articles/685499/) there were
> discussions about adding stream IDs to the block/device layer to allow higher
> layers (filesystems, applications) to identify IO streams so lower layers
> (SSDs, hybrid storage, etc.) can make better allocation/placement decisions.
>
> It would be useful to get an update on the state of this work, and discuss
> any obstacles that need to be resolved for getting this code landed.


Cheers, Andreas

RE: [LSF/MM TOPIC] status update on stream IDs

[Changho Choi]

Hi Andreas,

Thanks for the heads up!

There has been big progress on it. It was ratified as a NVMe 1.3 standard in Oct 31st, 2016 in NVMe org(As you may already understand it was ratified as SCSI standard in T10 in May, 2015).
The standard compliant product is available for NVMe and SAS in the market.
Also Jens Axboe implemented the features in Kernel, etc. 

We are more than happy to update the status in the meeting. 
Also it would be a great time to revive the Linux Kernel implementation discussion to support both application-assigned stream ID and kernel-assigned stream ID at the same time. 
I think people will start circulating the possible approaches sooner or later unless it is already started.

Thanks,
Changho


[resend to include other relevant lists]

On Jan 6, 2017, at 4:54 PM, Andreas Dilger <adilger@dilger.ca> wrote:
>
> At LSF/MM'16 and Linux FAST (https://lwn.net/Articles/685499/) there 
> were discussions about adding stream IDs to the block/device layer to 
> allow higher layers (filesystems, applications) to identify IO streams 
> so lower layers (SSDs, hybrid storage, etc.) can make better allocation/placement decisions.
>
> It would be useful to get an update on the state of this work, and 
> discuss any obstacles that need to be resolved for getting this code landed.


Cheers, Andreas






_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

RE: [LSF/MM TOPIC] status update on stream IDs

[Changho Choi]

Hi Andreas,

Thanks for the heads up!

There has been big progress on it. It was ratified as a NVMe 1.3 standard in Oct 31st, 2016 in NVMe org(As you may already understand it was ratified as SCSI standard in T10 in May, 2015).
The standard compliant product is available for NVMe and SAS in the market.
Also Jens Axboe implemented the features in Kernel, etc. 

We are more than happy to update the status in the meeting. 
Also it would be a great time to revive the Linux Kernel implementation discussion to support both application-assigned stream ID and kernel-assigned stream ID at the same time. 
I think people will start circulating the possible approaches sooner or later unless it is already started.

Thanks,
Changho


[resend to include other relevant lists]

On Jan 6, 2017, at 4:54 PM, Andreas Dilger <adilger@dilger.ca> wrote:
>
> At LSF/MM'16 and Linux FAST (https://lwn.net/Articles/685499/) there 
> were discussions about adding stream IDs to the block/device layer to 
> allow higher layers (filesystems, applications) to identify IO streams 
> so lower layers (SSDs, hybrid storage, etc.) can make better allocation/placement decisions.
>
> It would be useful to get an update on the state of this work, and 
> discuss any obstacles that need to be resolved for getting this code landed.


Cheers, Andreas

[LSF/MM TOPIC] status update on stream IDs

[Changho Choi]

Hi Andreas,

Thanks for the heads up!

There has been big progress on it. It was ratified as a NVMe 1.3 standard in Oct 31st, 2016 in NVMe org(As you may already understand it was ratified as SCSI standard in T10 in May, 2015).
The standard compliant product is available for NVMe and SAS in the market.
Also Jens Axboe implemented the features in Kernel, etc. 

We are more than happy to update the status in the meeting. 
Also it would be a great time to revive the Linux Kernel implementation discussion to support both application-assigned stream ID and kernel-assigned stream ID at the same time. 
I think people will start circulating the possible approaches sooner or later unless it is already started.

Thanks,
Changho


[resend to include other relevant lists]

On Jan 6, 2017,@4:54 PM, Andreas Dilger <adilger@dilger.ca> wrote:
>
> At LSF/MM'16 and Linux FAST (https://lwn.net/Articles/685499/) there 
> were discussions about adding stream IDs to the block/device layer to 
> allow higher layers (filesystems, applications) to identify IO streams 
> so lower layers (SSDs, hybrid storage, etc.) can make better allocation/placement decisions.
>
> It would be useful to get an update on the state of this work, and 
> discuss any obstacles that need to be resolved for getting this code landed.


Cheers, Andreas