Re: [SPDK] NVMe command ordering

[Jonas Pfefferle <pepperjo at japf.ch>]

[-- Attachment #1: Type: text/plain, Size: 4485 bytes --]

Thanks for the notes/clarifications. Interesting paper.

Jonas

  On Fri, 23 Mar 2018 17:08:47 -0700
  Paul Von-Stamwitz <PVonStamwitz(a)us.fujitsu.com> wrote:
> This is purely FYI:
> 
> Interesting paper from FAST '18. 
> 
> Barrier-Enabled IO Stack for Flash Storage
> https://www.usenix.org/conference/fast18/presentation/won
> 
> -Paul
> 
> -----Original Message-----
>From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Marushak, 
>Nathan
> Sent: Friday, March 23, 2018 9:29 AM
> To: Storage Performance Development Kit
> Subject: Re: [SPDK] NVMe command ordering
> 
> Hey Jonas,
> 
> Looks like you got the answer here hopefully, but I wanted to 
>provide 1 additional point of clarity. For HDD or SSD (or any other 
>type of storage media), it's always been incumbent upon the 
>application (most notably the file system) to provide the checks and 
>balances to ensure ordering.
> 
> Thanks,
> Nate
> 
>> -----Original Message-----
>> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Luse, 
>>Paul 
>> E
>> Sent: Friday, March 23, 2018 9:23 AM
>> To: Storage Performance Development Kit <spdk(a)lists.01.org>; Andrey 
>> Kuzmin <andrey.v.kuzmin(a)gmail.com>
>> Subject: Re: [SPDK] NVMe command ordering
>> 
>> Yup
>> 
>> -----Original Message-----
>> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Jonas 
>> Pfefferle
>> Sent: Friday, March 23, 2018 9:21 AM
>> To: Storage Performance Development Kit <spdk(a)lists.01.org>; Andrey 
>> Kuzmin <andrey.v.kuzmin(a)gmail.com>
>> Subject: Re: [SPDK] NVMe command ordering
>> 
>> One other thing I stumbled upon:
>> 7.1 "There are no ordering restrictions for completions to the 
>>host."
>> 
>> Does that mean if, for example, command A and B were executed in 
>>order 
>> A,B they do not have to complete in this order on the CQ?
>> 
>> Thanks,
>> Jonas
>> 
>>   On Fri, 23 Mar 2018 12:59:54 +0100
>>   "Jonas Pfefferle" <pepperjo(a)japf.ch> wrote:
>> > Thanks!
>> >
>> >  On Fri, 23 Mar 2018 11:25:26 +0000
>> >  Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com> wrote:
>> >> On Fri, Mar 23, 2018, 12:04 Jonas Pfefferle <pepperjo(a)japf.ch>
>> >>wrote:
>> >>
>> >>> Hi @all,
>> >>>
>> >>> Can anyone clarify this for me:
>> >>> NVMe spec 6.3 Command Ordering Requirements  "For all commands 
>> >>>which are not part of a fused operation (refer to section  4.10), 
>> >>>or for which the write size is greater than AWUN, each command is 
>> 
>> >>>processed as an independent entity without reference to other 
>> >>>commands submitted to the same I/O Submission Queue or to 
>>commands 
>> >>>submitted to  other  I/O Submission Queues"
>> >>>
>> >>> So writes <= AWUN size are atomic but there is no ordering 
>>between 
>> >>>the  commands, e.g. write command A issued before write command B 
>> >>>to the same  LBA  can be executed in order AB or BA, the only 
>>thing 
>> >>>guaranteed is that they  are executed atomically? And there is no 
>> >>>way to enforce ordering except on  the host side?
>> >>>
>> >>
>> >> Right, that's exactly what the spec says in that same paragraph 
>>you 
>> >>quote
>> >> above: "If there are ordering requirements between these 
>>commands, 
>> >>host  software or the associated application is required to 
>>enforce 
>> >>that ordering  above the level of the controller. " (1.3a, P. 
>>179).
>> >>
>> >> Regards,
>> >> Andrey
>> >>
>> >>
>> >>> Regards,
>> >>> Jonas
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> SPDK mailing list
>> >>> SPDK(a)lists.01.org
>> >>> https://lists.01.org/mailman/listinfo/spdk
>> >>>
>> >> --
>> >>
>> >> Regards,
>> >> Andrey
>> >
>> >
>> > _______________________________________________
>> > SPDK mailing list
>> > SPDK(a)lists.01.org
>> > https://lists.01.org/mailman/listinfo/spdk
>> 
>> 
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk