From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=46806 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OPEd8-0005u9-AP for qemu-devel@nongnu.org; Thu, 17 Jun 2010 08:55:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OPEd6-0005Qx-Ut for qemu-devel@nongnu.org; Thu, 17 Jun 2010 08:55:38 -0400 Received: from adelie.canonical.com ([91.189.90.139]:42870) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OPEd6-0005Qi-Lz for qemu-devel@nongnu.org; Thu, 17 Jun 2010 08:55:36 -0400 Received: from loganberry.canonical.com ([91.189.90.37]) by adelie.canonical.com with esmtp (Exim 4.69 #1 (Debian)) id 1OPEd4-0006fh-EL for ; Thu, 17 Jun 2010 13:55:34 +0100 Received: from loganberry.canonical.com (localhost [127.0.0.1]) by loganberry.canonical.com (Postfix) with ESMTP id 86B062E8223 for ; Thu, 17 Jun 2010 13:55:33 +0100 (BST) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Date: Thu, 17 Jun 2010 12:48:34 -0000 From: Stephane Chazelas Sender: bounces@canonical.com References: <20100616143321.24259.15137.malonedeb@palladium.canonical.com> <20100616203600.19865.61385.malone@potassium.ubuntu.com> Message-Id: Errors-To: bounces@canonical.com Subject: [Qemu-devel] Re: [Bug 595117] Re: qemu-nbd slow and missing "writeback" cache option Reply-To: Bug 595117 <595117@bugs.launchpad.net> List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org 2010-06-16 20:36:00 -0000, Dustin Kirkland: [...] > Could you please send that patch to the qemu-devel@ mailing list? > Thanks! [...] Hi Dustin, it looks like qemu-devel is subscribed to bugs in there, so the bug report is on the list already. Note that I still consider it as a bug because: - slow performance for no good reason - --nocache option is misleading - no fsync on "-d" which to my mind is a bug. Cheers, Stephane -- = qemu-nbd slow and missing "writeback" cache option https://bugs.launchpad.net/bugs/595117 You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. Status in QEMU: Invalid Status in =E2=80=9Cqemu-kvm=E2=80=9D package in Ubuntu: Incomplete Bug description: Binary package hint: qemu-kvm dpkg -l | grep qemu ii kvm 1:84+dfsg-0ubuntu16+0.12.3+noroms+= 0ubuntu9 dummy transitional pacakge from kvm to qemu- ii qemu 0.12.3+noroms-0ubuntu9 = dummy transitional pacakge from qemu to qemu ii qemu-common 0.12.3+noroms-0ubuntu9 = qemu common functionality (bios, documentati ii qemu-kvm 0.12.3+noroms-0ubuntu9 = Full virtualization on i386 and amd64 hardwa ii qemu-kvm-extras 0.12.3+noroms-0ubuntu9 = fast processor emulator binaries for non-x86 ii qemu-launcher 1.7.4-1ubuntu2 = GTK+ front-end to QEMU computer emulator ii qemuctl 0.2-2 = controlling GUI for qemu lucid amd64. qemu-nbd is a lot slower when writing to disk than say nbd-server. It appears it is because by default the disk image it serves is open with O= _SYNC. The --nocache option, unintuitively, makes matters a bit better beca= use it causes the image to be open with O_DIRECT instead of O_SYNC. The qemu code allows an image to be open without any of those flags, but un= fortunately qemu-nbd doesn't have the option to do that (qemu doesn't allow= the image to be open with both O_SYNC and O_DIRECT though). The default of qemu-img (of using O_SYNC) is not very sensible because anyw= ay, the client (the kernel) uses caches (write-back), (and "qemu-nbd -d" do= esn't flush those by the way). So if for instance qemu-nbd is killed, regar= dless of whether qemu-nbd uses O_SYNC, O_DIRECT or not, the data in the ima= ge will not be consistent anyway, unless "syncs" are done by the client (li= ke fsync on the nbd device or sync mount option), and with qemu-nbd's O_SYN= C mode, those "sync"s will be extremely slow. Attached is a patch that adds a --cache=3D{off,none,writethrough,writeback}= option to qemu-nbd. --cache=3Doff is the same as --nocache (that is use O_DIRECT), writethrough= is using O_SYNC and is still the default so this patch doesn't change the = functionality. writeback is none of those flags, so is the addition of this= patch. The patch also does an fsync upon "qemu-nbd -d" to make sure data i= s flushed to the image before removing the nbd. Consider this test scenario: dd bs=3D1M count=3D100 of=3Da < /dev/null qemu-nbd --cache=3D -c /dev/nbd0 a cp /dev/zero /dev/nbd0 time perl -MIO::Handle -e 'STDOUT->sync or die$!' 1<> /dev/nbd0 With cache=3Dwritethrough (the default), it takes over 10 minutes to write = those 100MB worth of zeroes. Running a strace, we see the recvfrom and sent= os delayed by each 1kb write(2)s to disk (10 to 30 ms per write). With cache=3Doff, it takes about 30 seconds. With cache=3Dwriteback, it takes about 3 seconds, which is similar to the p= erformance you get with nbd-server Note that the cp command runs instantly as the data is buffered by the clie= nt (the kernel), and not sent to qemu-nbd until the fsync(2) is called.