From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32916) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1alpg6-0002NV-Uv for qemu-devel@nongnu.org; Thu, 31 Mar 2016 23:23:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1alpg2-0005bP-IB for qemu-devel@nongnu.org; Thu, 31 Mar 2016 23:23:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45560) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1alpg2-0005bL-AE for qemu-devel@nongnu.org; Thu, 31 Mar 2016 23:23:14 -0400 References: <1459465399-56203-1-git-send-email-alex@alex.org.uk> From: Eric Blake Message-ID: <56FDE99F.90707@redhat.com> Date: Thu, 31 Mar 2016 21:23:11 -0600 MIME-Version: 1.0 In-Reply-To: <1459465399-56203-1-git-send-email-alex@alex.org.uk> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="nkC9dpq0R2Vim6gcNpj9sftRaRlPULFLJ" Subject: Re: [Qemu-devel] [PATCH] Improve documentation of FUA and FLUSH List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alex Bligh , Wouter Verhelst Cc: "nbd-general@lists.sourceforge.net" , "qemu-devel@nongnu.org" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --nkC9dpq0R2Vim6gcNpj9sftRaRlPULFLJ Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 03/31/2016 05:03 PM, Alex Bligh wrote: > Improve the documentation of NBD_CMD_FLUSH and NBD_CMD_FLAG_FUA. Specif= ically > the latter may be set on any command, and its semantics on commands oth= er > than NBD_CMD_WRITE need explaining. Further, explain how these relate t= o > reordering of commands. >=20 > Signed-off-by: Alex Bligh > --- > doc/proto.md | 52 ++++++++++++++++++++++++++++++++++++++++++----------= > 1 file changed, 42 insertions(+), 10 deletions(-) >=20 > diff --git a/doc/proto.md b/doc/proto.md > index c1e05c5..bc4483d 100644 > --- a/doc/proto.md > +++ b/doc/proto.md > @@ -197,6 +197,37 @@ handle as was sent by the client in the correspond= ing request. In > this way, the client can correlate which request is receiving a > response. > =20 > +#### Ordering of messages and writes > + > +The server MAY process commands out of order, and MAY reply out of > +order, save that: > + > +* All write commands (that includes both `NBD_CMD_WRITE` and > + `NBD_CMD_TRIM`) that the server completes (i.e. replies to) > + prior to processing to a `NBD_CMD_FLUSH` MUST be written to non-vola= tile > + storage prior to replying to that `NBD_CMD_FLUSH`. The server SHOULD= ensure > + that all write command received prior to processing the `NBD_CMD_FLU= SH` > + (whether they are replied to or not) are written to non-volatile > + storage prior to processing an `NBD_CMD_FLUSH`; note this is a > + stronger condition than the previous 'MUST' condition. This > + paragram only applies if `NBD_FLAG_SEND_FLUSH` is set within s/paragram/paragraph/ > + the transmission flags, as otherwise `NBD_CMD_FLUSH` will never > + be sent by the client to the server. > + > +* A server MUST NOT reply to a command that has `NBD_CMD_FLAG_FUA` set= > + in its command flags until the data area referred to by that comma= nd why multiple spaces? > + is persisted to non-volatile storage. This only applies if > + `NBD_FLAG_SEND_FLUSH` is set within the transmission flags, as other= wise s/FLUSH/FUA/ > + `NBD_CMD_FLAG_FUA` will not be set on any commands sent to the serve= r > + by the client. > + > +`NBD_CMD_FLUSH` is modelled on the Linux kernel empty bio with > +`REQ_FLUSH` set. `NBD_CMD_FLAG_FUA` is modelled on the Linux > +kernel bio with `REQ_FUA` set. In case of ambiguity in this > +specification, the > +[kernel documentation](https://www.kernel.org/doc/Documentation/block/= writeback_cache_control.txt) > +may be useful. > + > #### Request message > =20 > The request message, sent by the client, looks as follows: > @@ -444,10 +475,17 @@ affects a particular command. Clients MUST NOT s= et a command flag bit > that is not documented for the particular command; and whether a flag = is > valid may depend on negotiation during the handshake phase. > =20 > -- bit 0, `NBD_CMD_FLAG_FUA`; valid during `NBD_CMD_WRITE`. SHOULD be > - set to 1 if the client requires "Force Unit Access" mode of > - operation. MUST NOT be set unless transmission flags included > - `NBD_FLAG_SEND_FUA`. > +- bit 0, `NBD_CMD_FLAG_FUA`. This bit > + MUST be set to 0 unless the `NBD_FLAG_SEND_FUA` flag ("Force Unit Ac= cess") > + was set in the transmission flags field. If the `NBD_FLAG_SEND_FUA` > + is set in the transmission flags field, the client MAY set > + `NBD_CMD_FLAG_FUA` in any request. If this bit is set, the server > + MUST NOT send a reply until it has ensured that any data referred to= > + by this request (i.e. written data on a write or trim, read data on > + a read) has reached permanent storage. There will be certain command= s > + (e.g. `NBD_CMD_DISC`) for which this flag will thus not alter behavi= our > + (as the command does not refer to any data), in which case the serve= r > + MUST ignore this bit. Makes sense, but we now need to fix the reference implementation to match (the recent commit ab22e082 rejects NBD_CMD_FLAG_FUA on all but writes). --=20 Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --nkC9dpq0R2Vim6gcNpj9sftRaRlPULFLJ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJW/emfAAoJEKeha0olJ0NqW4kH/j6t77aNw0xTmD86CV5aRNca mnZaMl3AQr0Txv4xRIJL+z0S+f5UvLK1vl38v4t+U2XZ5/N2oFvQ1YnGttSAk6w1 tIvMFmKgK1iEjqVbOAZdwtCu7b48nAzW4Vdj23eSPIIVthGuHINNC7nC/k475jSQ uXsFpUQukCnIZRP5hiAG5kA96joj5ApSCLjSaKGC2zXiqNq+VwzAapoFmVnTVVZR MwUIhON3ai9Z/TVl+AqElkZAsyYKaVsqG9aA2l1oqSXDlKyefK1Gtn6fSbo7IuH9 /Dtr6+WcXO5l+NdRX+YAs/Tzt+Iv3bR6+XgxcsWPXgglMkKxNnVtv7+mVQzTLZY= =DQu1 -----END PGP SIGNATURE----- --nkC9dpq0R2Vim6gcNpj9sftRaRlPULFLJ--