From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57076) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1anRsN-0007wv-0s for qemu-devel@nongnu.org; Tue, 05 Apr 2016 10:22:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1anRsI-0001iO-JT for qemu-devel@nongnu.org; Tue, 05 Apr 2016 10:22:38 -0400 Received: from mx1.redhat.com ([209.132.183.28]:35621) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1anRsI-0001iK-CF for qemu-devel@nongnu.org; Tue, 05 Apr 2016 10:22:34 -0400 References: <1459858065-13348-1-git-send-email-alex@alex.org.uk> From: Eric Blake Message-ID: <5703CA27.2010402@redhat.com> Date: Tue, 5 Apr 2016 08:22:31 -0600 MIME-Version: 1.0 In-Reply-To: <1459858065-13348-1-git-send-email-alex@alex.org.uk> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="GHoWSHJjSP3RPC60NotWApMvBkRpSebFp" Subject: Re: [Qemu-devel] [PATCHv3] Improve the documentation of NBD_CMD_FLUSH and NBD_CMD_FLAG_FUA. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alex Bligh , Wouter Verhelst Cc: "nbd-general@lists.sourceforge.net" , "qemu-devel@nongnu.org" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --GHoWSHJjSP3RPC60NotWApMvBkRpSebFp Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 04/05/2016 06:07 AM, Alex Bligh wrote: > Improve the documentation of NBD_CMD_FLUSH and NBD_CMD_FLAG_FUA. Specif= ically > the latter may be set on any command, and its semantics on commands oth= er > than NBD_CMD_WRITE need explaining. Further, explain how these relate t= o > reordering of commands. >=20 > Signed-off-by: Alex Bligh > --- > doc/proto.md | 51 +++++++++++++++++++++++++++++++++++++++++---------- > 1 file changed, 41 insertions(+), 10 deletions(-) >=20 close, but needs a v4 > +++ b/doc/proto.md > @@ -217,6 +217,33 @@ handle as was sent by the client in the correspond= ing request. In > this way, the client can correlate which request is receiving a > response. > =20 > +#### Ordering of messages and writes > + > +The server MAY process commands out of order, and MAY reply out of > +order, save that: maybe s/save/except/ > + > +* All write commands (that includes both `NBD_CMD_WRITE` and > + `NBD_CMD_TRIM`) that the server completes (i.e. replies to) should we also list NBD_CMD_WRITE_ZEROES? > + prior to processing to a `NBD_CMD_FLUSH` MUST be written to non-vola= tile > + storage prior to replying to that `NBD_CMD_FLUSH`. This > + paragraph only applies if `NBD_FLAG_SEND_FLUSH` is set within > + the transmission flags, as otherwise `NBD_CMD_FLUSH` will never > + be sent by the client to the server. > + > +* A server MUST NOT reply to a command that has `NBD_CMD_FLAG_FUA` set= > + in its command flags until the data (if any) written by that command= > + is persisted to non-volatile storage. This only applies if > + `NBD_FLAG_SEND_FLUSH` is set within the transmission flags, as other= wise s/FLUSH/FUA/ > + `NBD_CMD_FLAG_FUA` will not be set on any commands sent to the serve= r > + by the client. > + > +`NBD_CMD_FLUSH` is modelled on the Linux kernel empty bio with > +`REQ_FLUSH` set. `NBD_CMD_FLAG_FUA` is modelled on the Linux > +kernel bio with `REQ_FUA` set. In case of ambiguity in this > +specification, the > +[kernel documentation](https://www.kernel.org/doc/Documentation/block/= writeback_cache_control.txt) > +may be useful. > + > #### Request message > =20 > The request message, sent by the client, looks as follows: > @@ -483,10 +510,20 @@ affects a particular command. Clients MUST NOT s= et a command flag bit > that is not documented for the particular command; and whether a flag = is > valid may depend on negotiation during the handshake phase. > =20 > -- bit 0, `NBD_CMD_FLAG_FUA`; valid during `NBD_CMD_WRITE` and > - `NBD_CMD_WRITE_ZEROES` commands. SHOULD be set to 1 if the client r= equires > - "Force Unit Access" mode of operation. MUST NOT be set unless trans= mission > - flags included `NBD_FLAG_SEND_FUA`. > +- bit 0, `NBD_CMD_FLAG_FUA`; This flag is valid for all commands provi= ded s/commands/commands,/ > + `NBD_FLAG_SEND_FUA` has been negotiated, in which case the server MU= ST > + accept all commands with this bit set (even by ignoring the bit). Th= e > + client SHOULD NOT set this bit unless the command has the potential = of > + writing data (current commands are `NBD_CMD_WRITE`, `NBD_CMD_WRITE_Z= EROES` > + and `NBD_CMD_TRIM`); existing clients are known to set this bit on maybe: s/existing/but existing/ > + other commands; subject to that, provided `NBD_FLAG_SEND_FUA` is > + negotiated, the client MAY set this bit as it wishes. If the server Sounds redundant. I'd strike 'subject to that, ... as it wishes'. > + receives a command with `NBD_CMD_FLAG_FUA` set it MUST NOT send its > + reply to that command until all write operations (if any) associated= with > + that command command have been completed and persisted to non-volati= le > + storage. If the command does not in fact write data (for instance on= an > + `NBD_CMD_TRIM` which does is ignored), the server MAY ignore this bi= t s/does // > + being set on such a command. > - bit 1, `NBD_CMD_NO_HOLE`; defined by the experimental `WRITE_ZEROES`= > extension; see below. > - bit 2, `NBD_CMD_FLAG_DF`; defined by the experimental `STRUCTURED_RE= PLY` > @@ -535,12 +572,6 @@ The following request types exist: > message. The server MAY send the reply message before the data has= > reached permanent storage. > =20 > - If the `NBD_FLAG_SEND_FUA` flag ("Force Unit Access") was set in t= he > - transmission flags field, the client MAY set the flag `NBD_CMD_FLA= G_FUA` in > - the command flags field. If this flag was set, the server MUST NOT= send > - the reply until it has ensured that the newly-written data has rea= ched > - permanent storage. > - You should drop this paragraph from both NBD_CMD_WRITE and NBD_CMD_WRITE_ZEROES, now that the flag is globally described (here, you only dropped it from one of the two). --=20 Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --GHoWSHJjSP3RPC60NotWApMvBkRpSebFp Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJXA8ooAAoJEKeha0olJ0NqLJkH/3xWHYeIYuvFua0++dzD2Vbe Kd4T3A3BhYDnv0ve/e9acvfzQRbOzxhHRVQYMSiVzArLI1f1ZI5nwcRxavP6Chii 0j3Bfqy6pvwsT8QVvPJK1E45lM7rCDfUNckDOqqRX8JEz5M6ggZrUKVE0QOPYCsA gvofnF6tT80oRaOvhgQKK2Z2QA/X3ZjJU5lbkt9KB/41kc+Khk5wMHhL0ItRY3HE ODuck4OqIgLUZMRRrCRgnSUvNcKpV6f7Zx5laGIsY3XRu+qF2ZIjUpYQvx6OcKdk +EgxtrfpzhLSMlUFNk0xCZajtqMnJEK84EoiHkOoVJV3FLROj6pEtZ3FOHg+t3E= =KNBS -----END PGP SIGNATURE----- --GHoWSHJjSP3RPC60NotWApMvBkRpSebFp--