Re: [Qemu-devel] [Nbd] [PATCH v2] doc: Allow NBD_CMD_FLAG_NO_HOLE during NBD_CMD_WRITE

[Eric Blake <eblake@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 3599 bytes --]

On 04/05/2016 03:38 AM, Markus Pargmann wrote:
> Hi,
> 
> On Monday 04 April 2016 16:15:43 Eric Blake wrote:
>> qemu already has an existing server implementation option that will
>> explicitly search the payload of NBD_CMD_WRITE for large blocks of
>> zeroes, and punch holes in the underlying file.  For old clients
>> that don't know how to use the new NBD_CMD_WRITE_ZEROES, this is a
>> workaround to keep the server's destination file approximately as
>> sparse as the client's source.  However, for new clients that know
>> how to explicitly request holes, it is unnecessary overhead; and
>> can lead to the server punching a hole and risking fragmentation or
>> future ENOSPC even when the client explicitly wanted to write
>> zeroes rather than a hole.  So it makes sense to let the new
>> NBD_CMD_FLAG_NO_HOLE work for WRITE as well as WRITE_ZEROES.
> 
> From the commit message it sounds like this is only for new clients
> supporting WRITE_ZEROES because for those we don't want to search
> through all the data of normal WRITEs. If you don't need to set this for
> each WRITE individually perhaps we could move it to the negotiation
> part?

Interesting idea.  So we'd add a new NBD_OPT_XXX that lets the server
know that "I plan on using WRITE_ZEROS and TRIM as the only places where
I want you to trim, so you can avoid scanning for zeroes in WRITE"; the
server replies with NBD_REP_ACK if it understands the client (in which
case the server _should_ be advertising NBD_FLAG_SEND_WRITE_ZEROES
and/or NBD_FLAG_SEND_TRIM), and with NBD_REP_ERR_UNSUP if it is too old
(the server may still advertise TRIM, but probably should not advertise
WRITE_ZEROES - we are still early enough that we could mandate that any
server that supports WRITE_ZEROES also supports the new NBD_OPT_XXX).
The client then knows that either the server will be efficient with
WRITE and the client uses WRITE_ZEROES and TRIM as desired, or that the
server is old and the client is stuck with WRITE anyways (and whether
the server trims or not is beyond the client's control).

Meanwhile, the new server can unconditionally advertise SEND_TRIM and
SEND_WRITE_ZEROES, whether or not a client uses NBD_OPT_XXX.  If it is
an old client connecting and no NBD_OPT_XXX is sent, chances are the
client is also too old to ever use WRITE_ZEROES, so the server should
feel free to apply its policy on whether to scan for zeroes in WRITE (in
qemu's case, the server policy is set via command line options,
precisely to cater to the scenario where we WANT the server to scan
zeroes to make up for the client being unable to pass sparse regions
efficiently vs. cases where the scanning is deemed too expensive); but
if the client DID send NBD_OPT_XXX, the server SHOULD NOT punch holes
during WRITE, and should not waste time scanning, no matter what the
command line policy permitted.

This also helps the case of clients divided between userspace and
kernel: the way I wrote the proposal, the kernel has to pay attention to
NBD_FLAG_SEND_WRITE_ZEROES, and if present, add NBD_CMD_FLAG_NO_HOLE to
every write.  But with your proposal of option negotiation (done in
userspace), the default of WRITE is now the most efficient on new
servers, and unchanged for old servers, so the kernel doesn't have to do
anything different.

Does the idea of a new NBD_OPT_ make enough sense to write that up
rather than mandating the use of NBD_CMD_FLAG_NO_HOLE with WRITE?

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]