[Qemu-devel] qemu-nbd very slow, patch available

[Qemu-devel] qemu-nbd very slow, patch available

[Tristan Wibberley]

[-- Attachment #1: Type: text/plain, Size: 1458 bytes --]

Hello qemu devs,

A couple of years ago Stephane Chazelas
(https://launchpad.net/~stephane-chazelas) reported a performance
issue with qemu-nbd due to the lack of an option to switch on
writeback caching (ie a mode like any typical program that just
modifies files of data where resilience is not provided for each write
operation). I can't find any information about why that wasn't
applied.

Perhaps it just got lost in the noise, but I could see some
improvements to be made so I've attached a variant to provide extra
speed via a commandline option for users for whom fast, less robust
operation is wanted (ie, where robustness is available through other
means).

The attached diff adds a commandline option "--cache=" with four modes
and makes no change to the default behaviour of qemu-nbd:

  --cache=writethrough [default, O_DSYNC, very slow, very resilient]
  --cache=off [same as --nocache, O_DIRECT, slow, resilient]
  --cache=none [O_DIRECT, O_DSYNC, very very slow, very very resilient]
  --cache=writeback [fast, not resilient]


Here is Stephane's original report:
https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/595117

Versus Stephane's patch I've altered the meaning of --cache=off to
accurately match --nocache.
I have not included Stephane's extra fsync operation on close because
that would make --cache=writeback very slow on disconnect. Users can
sync after disconnect via alternatives available on their platform.

--
Tristan

[-- Attachment #2: fast-nbd.diff --]
[-- Type: application/octet-stream, Size: 1778 bytes --]

--- qemu-nbd.c	2011-12-04 10:38:06.000000000 +0000
+++ ../qemu-nbd.c	2012-09-12 09:02:21.555208119 +0100
@@ -57,7 +57,8 @@ static void usage(const char *name)
 "  -r, --read-only      export read-only\n"
 "  -P, --partition=NUM  only expose partition NUM\n"
 "  -s, --snapshot       use snapshot file\n"
-"  -n, --nocache        disable host cache\n"
+"  -C, --cache=MODE     set cache mode (off [no OS filesystem cache], none [synci data directly to disk - very slow], writethrough [default] or writeback [usually fastest])\n"
+"  -n, --nocache        disable host cache (same as --cache=off)\n"
 "  -c, --connect=DEV    connect FILE to the local NBD device DEV\n"
 "  -d, --disconnect     disconnect the specified device\n"
 "  -e, --shared=NUM     device can be shared by NUM clients (default '1')\n"
@@ -270,6 +271,7 @@ int main(int argc, char **argv)
         { "disconnect", 0, NULL, 'd' },
         { "snapshot", 0, NULL, 's' },
         { "nocache", 0, NULL, 'n' },
+        { "cache", 1, NULL, 'C' },
         { "shared", 1, NULL, 'e' },
         { "persistent", 0, NULL, 't' },
         { "verbose", 0, NULL, 'v' },
@@ -312,6 +314,19 @@ int main(int argc, char **argv)
         case 's':
             flags |= BDRV_O_SNAPSHOT;
             break;
+        case 'C':
+	    if (strcmp(optarg, "off") == 0)
+		flags |= BDRV_O_NOCACHE | BDRV_O_CACHE_WB;
+	    else if (strcmp(optarg, "writethrough") == 0)
+		;/* default */
+	    else if (strcmp(optarg, "writeback") == 0)
+		flags |= BDRV_O_CACHE_WB;
+	    else if (strcmp(optarg, "none"))
+	        flags |= BDRV_O_NOCACHE;
+	    else {
+		errx(EXIT_FAILURE, "Invalid cache option `%s'", optarg);
+	    }
+            break;
         case 'n':
             flags |= BDRV_O_NOCACHE | BDRV_O_CACHE_WB;
             break;

Re: [Qemu-devel] qemu-nbd very slow, patch available

[Paolo Bonzini]

Il 12/09/2012 10:25, Tristan Wibberley ha scritto:
> The attached diff adds a commandline option "--cache=" with four modes
> and makes no change to the default behaviour of qemu-nbd:
> 
>   --cache=writethrough [default, O_DSYNC, very slow, very resilient]
>   --cache=off [same as --nocache, O_DIRECT, slow, resilient]
>   --cache=none [O_DIRECT, O_DSYNC, very very slow, very very resilient]
>   --cache=writeback [fast, not resilient]

A very similar patch is in QEMU 1.2.0, just released last week. :)

  --cache=writethrough [default, O_DSYNC]
  --cache=none [same as --nocache, O_DIRECT]
  --cache=directsync [O_DIRECT, O_DSYNC]
  --cache=writeback

> I have not included Stephane's extra fsync operation on close because
> that would make --cache=writeback very slow on disconnect. Users can
> sync after disconnect via alternatives available on their platform.

An fsync is already done by bdrv_close; however, this only syncs just
before qemu-nbd exits, not after each and every disconnect.  I think
it's a good compromise.  The best solution would be to add support for
fsync in the NBD kernel driver, so that --cache=writeback would be just
as good.

Paolo