From mboxrd@z Thu Jan 1 00:00:00 1970 From: Richard Weinberger Subject: Re: Very poor write performance Date: Sat, 19 Apr 2014 16:51:18 +0200 Message-ID: <53528D66.6010800@nod.at> References: <53527F3A.4010705@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit To: Paolo Bonzini , Richard Weinberger , kvm , Stefan Hajnoczi Return-path: Received: from b.ns.miles-group.at ([95.130.255.144]:1660 "EHLO radon.swed.at" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751132AbaDSOv1 (ORCPT ); Sat, 19 Apr 2014 10:51:27 -0400 In-Reply-To: <53527F3A.4010705@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: Am 19.04.2014 15:50, schrieb Paolo Bonzini: > Il 19/04/2014 08:04, Richard Weinberger ha scritto: >> Hi! >> >> I hope this is the right place to ask. :) >> >> On a rather recent x86_64 server I'm facing very bad write performance. >> The Server is a 8 Core Xeon E5 with 64GiB ram. >> >> Storage is a ext4 filesystem on top of LVM which is backed by DRBD. >> On the host side dd can easily write with 100MiB/s to the ext4. >> OS is Centos6 with kernel 3.12.x. >> >> Within a KVM Linux guest the seq write throughput is always only >> between 20 and 30MiB/s. >> The guest OS is Centos6, it uses virtio-blk, cache=none, io=natvie and >> the deadline IO scheduler. >> >> The worst thing is that the total IO bandwidth of KVM seems to 30MiB/s. >> If I run the same write benchmark within 5 guests each one achieves >> only 6 or 7 MiB/s. >> I see the same values also if the guest writes directly to a disk like vdb. >> Having the guest disk directly on LVM instead of a ext4 file also didn't help. >> It really looks like 30MiB/s is the upper bound for KVM disk IO. > > As a first guess, can you try XFS or direct DRBD? There seems to be a bug in ext4 that limits queue depth to a very low value. Problem solved. I was hunting a non-issue. The guests had swap enabled. m( Therefore each guest was busy with reading/writing pages do disk. Now I see write rates between 90 and 100 MiB/s within my guests. :-) Thanks, //richard