Thanks for the notes/clarifications. Interesting paper. Jonas On Fri, 23 Mar 2018 17:08:47 -0700 Paul Von-Stamwitz wrote: > This is purely FYI: > > Interesting paper from FAST '18. > > Barrier-Enabled IO Stack for Flash Storage > https://www.usenix.org/conference/fast18/presentation/won > > -Paul > > -----Original Message----- >From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Marushak, >Nathan > Sent: Friday, March 23, 2018 9:29 AM > To: Storage Performance Development Kit > Subject: Re: [SPDK] NVMe command ordering > > Hey Jonas, > > Looks like you got the answer here hopefully, but I wanted to >provide 1 additional point of clarity. For HDD or SSD (or any other >type of storage media), it's always been incumbent upon the >application (most notably the file system) to provide the checks and >balances to ensure ordering. > > Thanks, > Nate > >> -----Original Message----- >> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Luse, >>Paul >> E >> Sent: Friday, March 23, 2018 9:23 AM >> To: Storage Performance Development Kit ; Andrey >> Kuzmin >> Subject: Re: [SPDK] NVMe command ordering >> >> Yup >> >> -----Original Message----- >> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Jonas >> Pfefferle >> Sent: Friday, March 23, 2018 9:21 AM >> To: Storage Performance Development Kit ; Andrey >> Kuzmin >> Subject: Re: [SPDK] NVMe command ordering >> >> One other thing I stumbled upon: >> 7.1 "There are no ordering restrictions for completions to the >>host." >> >> Does that mean if, for example, command A and B were executed in >>order >> A,B they do not have to complete in this order on the CQ? >> >> Thanks, >> Jonas >> >> On Fri, 23 Mar 2018 12:59:54 +0100 >> "Jonas Pfefferle" wrote: >> > Thanks! >> > >> > On Fri, 23 Mar 2018 11:25:26 +0000 >> > Andrey Kuzmin wrote: >> >> On Fri, Mar 23, 2018, 12:04 Jonas Pfefferle >> >>wrote: >> >> >> >>> Hi @all, >> >>> >> >>> Can anyone clarify this for me: >> >>> NVMe spec 6.3 Command Ordering Requirements "For all commands >> >>>which are not part of a fused operation (refer to section 4.10), >> >>>or for which the write size is greater than AWUN, each command is >> >> >>>processed as an independent entity without reference to other >> >>>commands submitted to the same I/O Submission Queue or to >>commands >> >>>submitted to other I/O Submission Queues" >> >>> >> >>> So writes <= AWUN size are atomic but there is no ordering >>between >> >>>the commands, e.g. write command A issued before write command B >> >>>to the same LBA can be executed in order AB or BA, the only >>thing >> >>>guaranteed is that they are executed atomically? And there is no >> >>>way to enforce ordering except on the host side? >> >>> >> >> >> >> Right, that's exactly what the spec says in that same paragraph >>you >> >>quote >> >> above: "If there are ordering requirements between these >>commands, >> >>host software or the associated application is required to >>enforce >> >>that ordering above the level of the controller. " (1.3a, P. >>179). >> >> >> >> Regards, >> >> Andrey >> >> >> >> >> >>> Regards, >> >>> Jonas >> >>> >> >>> >> >>> _______________________________________________ >> >>> SPDK mailing list >> >>> SPDK(a)lists.01.org >> >>> https://lists.01.org/mailman/listinfo/spdk >> >>> >> >> -- >> >> >> >> Regards, >> >> Andrey >> > >> > >> > _______________________________________________ >> > SPDK mailing list >> > SPDK(a)lists.01.org >> > https://lists.01.org/mailman/listinfo/spdk >> >> >> _______________________________________________ >> SPDK mailing list >> SPDK(a)lists.01.org >> https://lists.01.org/mailman/listinfo/spdk >> _______________________________________________ >> SPDK mailing list >> SPDK(a)lists.01.org >> https://lists.01.org/mailman/listinfo/spdk > _______________________________________________ > SPDK mailing list > SPDK(a)lists.01.org > https://lists.01.org/mailman/listinfo/spdk > _______________________________________________ > SPDK mailing list > SPDK(a)lists.01.org > https://lists.01.org/mailman/listinfo/spdk