From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45323) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YgyvB-0005Q1-2B for qemu-devel@nongnu.org; Sat, 11 Apr 2015 13:10:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Ygyv6-0007X9-2M for qemu-devel@nongnu.org; Sat, 11 Apr 2015 13:10:17 -0400 Received: from mail-la0-x236.google.com ([2a00:1450:4010:c03::236]:35820) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Ygyv5-0007Tb-Ls for qemu-devel@nongnu.org; Sat, 11 Apr 2015 13:10:11 -0400 Received: by labbd9 with SMTP id bd9so32236185lab.2 for ; Sat, 11 Apr 2015 10:10:10 -0700 (PDT) Message-ID: <55295570.6010900@gmail.com> Date: Sat, 11 Apr 2015 19:10:08 +0200 From: ein MIME-Version: 1.0 References: <552834C1.9070105@gmail.com> <55291CF2.4000905@redhat.com> In-Reply-To: <55291CF2.4000905@redhat.com> Content-Type: multipart/mixed; boundary="------------040405000302020806020002" Subject: Re: [Qemu-devel] Very poor IO performance which looks like some design problem. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini , qemu-devel This is a multi-part message in MIME format. --------------040405000302020806020002 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 04/11/2015 03:09 PM, Paolo Bonzini wrote: > On 10/04/2015 22:38, ein wrote: >> Qemu creates more than 70 threads and everyone of them tries to write = to >> disk, which results in: >> 1. High I/O time. >> 2. Large latency. >> 2. Poor sequential read/write speeds. >> >> When I limited number of cores, I guess I limited number of threads as= >> well. That's why I got better numbers. >> >> I've tried to combine AIO native and thread setting with deadline >> scheduler. Native AIO was much more worse. >> >> The final question, is there any way to prevent Qemu for making so lar= ge >> number of processes when VM does only one sequential R/W operation? > Use "aio=3Dnative,cache=3Dnone". If that's not enough, you'll need to = use > XFS or a block device; ext4 suffers from spinlock contention on O_DIREC= T > I/O. Hello Paolo and thank you for reply. Firstly, I do use ext2 now, which gave me more MiB/s than XFS in the past. I've tried combination with XFS and block_device with NTFS (4KB) on it. I did tests with AIO=3Dnative,cache=3Dnone. Results in this worklo= ad was significantly worse. I don't have numbers on me right now but if somebody is interested, I'll redo the tests. From my experience I can say that disabling every software caches gives significant boost in sequential RW ops. I mean: Qemu cache, linux kernel dirty pages or even caching on VM itself. It makes somehow speed of data flow softer and more stable. Using cache creates hiccups. Firstly there's enormous speed for couple of seconds, more than hardware is capable of, then flush and no data flow at all (or very little) in few / over a dozen / seconds. --------------040405000302020806020002 Content-Type: application/pgp-keys; name="0xF2C6EA10.asc" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="0xF2C6EA10.asc" -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v2.0.19 (GNU/Linux) mQINBFIvGgYBEADZ96zOyz3HuWYwaM4NzFnxJEl+ANCi2stqCdTJFZqRlJYHNVKC b5KlTI9ctkhfGEwNMsGEU4Ox68U8+JIp6jR4e+yyCs8E5t1djlj+WFqJcb7nenQ4 wqr1PW2dangJ3i6ORwuJlsx1rUlVY5rkYwFGwo30EezJ8KXSjMFt27JI8CT9PX16 jEmp2qxj/Z4LfkLDusQ6PbiRwb6TflAENSTH6bVzHK0cNH4YRZa1DjK9JX+4Bfkh 5rECDmSUxLjRV3xnMrpooQmSY5FB3zPl2zUMmrliC0TdFqVZ4BU5JBULq9WvggBY 5OvYy1BkQOu0ZiRuIjygNSmKs3kCSzUhgHg33R8QsGrP68Bev4m9uqYc2odFjr3W 26WF4xIJ5upQbbnnkLaZWoYTNmtuzVfQEcO38ELrE9yfKR8Xk15WeknoOX2cWuvh rRAPyqaUHnrkMWxAggvzfFuwXNIv5ctabWbo25P2lzIoAM3oOyajbSJW/4eiZyDB cR01LrKpur8jMyCR3AViJH3j5E5Bg/uYV2LlTMLb6t220uYq9U2IAK5t33bLyNhz 4rW7x8bZVvtak5MOldK5R3nAXuRdAgjV8ymWQABXQMZqmdm0Z1Q1l1GVm+i5wMRQ XND5k2+O655PXIAAwasmLQsfIqwRDzCh90JzFnfIY9dwjq6LgnpBMYxUowARAQAB tDlTb2tvxYJvd3NraSBNaWNoYcWCIChubyBwYXNzd29yZCkgPG1zb2tvbG93c2tp QGluYm94LmNvbT6JAj4EEwECACgFAlIvGgYCGyMFCQ0oaIAGCwkIBwMCBhUIAgkK CwQWAgMBAh4BAheAAAoJEHr/aeHyxuoQruwQAJh+zY6y9icu/heUrApLeKYr3am5 O71rI+TKnVnCOOS0n4XCkRDEW0UrXy3IOZIhIUZX0Fc5YHJPOfnfot6DvtgDvJnN K54Ik4k9rQNw2J+Iei4egDyTIDlYMOXwViHB9gFU3keuo/Pf2Jrn1bwVE4o66tQM 7E6V+hh20boQ1u95f3nb+81NX/38mb5a8QtCd1/B69tZc8HWwhViv9Of97fbtZ2R z2adnQT6mdYerMLxtag1TLq2ppOoVNbv6RR3X0qbeCm1VP6g3lAkGepwwPDADzd3 IIGnq+crvOPXsa6UkDJrgoyfAxlidneCampqdkepq1kfRPjRYkm5qZ1lrcm4NArf 1kSRI2VPQptTMeiqGQm58aIJOwoQpXe6lg7hZpOcF23JyCxRunpN3ov3tAjc+oVy 6umO5JZRr/LIqdjD7ogGqjZ4EGyAnlxD0wNSjkXCLb1ehqi+h0en19T3n5Xsoj54 nZJSjz1utwxLb8OLammrBBlNjdrbnjPhMT9pkiSJ7UMQjOtOe9rcSP3/dLu+durd Y8UZNheTcbqIb0DgTfn/HeZC93cf1+CJE8HwUrHCO712pWBuCTu+03JtmPKYFE9l BX6+LVfYhbfrjt64QXEJ/zN+OGtZoYI96yJeBbTQjaKk6DZXHroiRm0qnM68HUVK vMqpiG8wS/uSKUnEtCVlaW4ubmV0QGdtYWlsLmNvbSA8ZWluLm5ldEBnbWFpbC5j b20+iQI+BBMBAgAoBQJSLxseAhsjBQkNKGiABgsJCAcDAgYVCAIJCgsEFgIDAQIe AQIXgAAKCRB6/2nh8sbqEE6ED/93jZSp7d1aFa0UTDxsCmlveN1ti7+hQFpRVqRO duC3XEIClPZxXMI4mTNuHVQTUkIK2KHy1CLNKvyGOVZkaHELTV9+u5oYnZamqZ7E UtHbUFuIZThE2XLhz8/z5Q1575+rLUdus9J9GVYj2qI1decA2NJSWfrR9LWVygFj ICP70b3hEwLiQWUDYBUNXkA+BwM0BFz0mwy+ci8kMuo7pM0D9M2wQcgRj5VPFVKI Q3/tEthN3rZdwDebgvd1f3+FZ97HKR1GJt8n0EC0O5cIehso09rLnLdDfggxdYg3 oGhK50ZIy0oqUJiTUVTdyh2PrOG2pybR1XnHqIWObRsEWfnVJiIWoPKAkGegqfU+ S+M1MrJXlMiFaU16FIDdjK8Pai8bcm4S3GAasttOegG5MirBRTTgI7lqFkSMi4aj TvQvgwT4VgzoWcHrMXpQkyPyzOVESTW5nbn3jLuU9HIl55UKzi0HTmUd0z/pwWAO Djsq8JQO1IbU5/iNTad4F2HfPWp21zvvhzYH6Ihy3Fzcjv2cjfPfZFxJmfmEqsIK Vvvj+YbHEHBi0dkWF3Dkf56e/eA96DcAUyMvAwvp38cl1K6mEHRLfn3YmoW1dqwL p7GrvrnhS+Jb+6px/atvw+fVjk7OfQY1wkcsu5oxFUY7NxkXwHpNi7aIM8tNGiwu LMcSILkCDQRSLxoGARAAr0cXmo0WBLsQj1JAW3sMd+bzCIfv2tOHbK2Rh/NvwwF2 LaLHmmLaEuFzMihdBLwZdgXTVi7EOdLcHK7qlTxYaNAOJYh2e3oBhSi3CBbvVg/S vrIvWKjAuMr7zuJWwFMWZsXp9iuaGf3OKHPkF4R2kxWC3JWcE/DTiejxu0QpLc3y Un7J3zAAep/LAmYbKg+YXH72TaVm6pXcktW1CUGcu32T1XtupJNFZRvT2OBHXyhw Ay+N7tfWGF+wJuOAYLfhdM/MWsrWirHk8mmRYSZtJmCcpmQe5kBHgYhDyqj5nS3q mg1HIJCnupPMe6tdD0836eGVrimTj4aOJE2UAPdgwBvW5j7Wx/vrS1L4qYdiCwK6 KK95qNv5GwfSj5+3unq4FDLWtAeuAdQeCNGhs4RWyOxwHFxaFkJAspGYsRu5ue3w +ftELeVY/e/IaQFykKIYzGBKrv0ASyQOsd0U3I8BNWmhkKlFz8l+Sdw3Co+mvGM6 M/SHQTAefe6/aaxbyEbEjA6s//i1DeBPFtLSxzR9s/Vkv+GJZzhIyYdQShZ9TPgv tSeVS1KVlLWDRK+qrixFIr/3oeZGv8sz3FU9xa7eODCRcFcUQO+mG2fBDztF5i4C HyQIa/M65c8GQxpe/IhMFVyFWDrLLFozLyT1NgYWShcag14+bpReZkRBdnknF8UA EQEAAYkCJQQYAQIADwUCUi8aBgIbDAUJDShogAAKCRB6/2nh8sbqEHHCEACvwgON ZsG7jywBhYFKk90AoZgDjQ47utRREvKshi09R2pMUGrfGwhMhJvFsr65aEY2Q6Sx /SKBb06uBlAyGoedBNwSy8uN+0Iv5sDMnz78KpS9MG/eF4e/5KT8zRn9JTdGQiTx VMAK3rMKbcYrp6VpzYyAbZpR93MvFXauFJFc6fy8hFmhCDDtxtwkVwGqpl+pCk7x pAERp6T0/TXJpGKuFNgZ9oONFbBWOaDnqEP+NHttNPAz40nDb4I85g65zDe26MQv GVmmYHHmNzesOqwTAHxT6KfEwo15+cv3S2GY52WG9PpYXsbapOmN4R41i0/hu/NE RUNVk1YOR7vvpNMgRQ54oXv04aCDNwdOPeCbhSwqfddbYpGUz4yGe6lL677YMJoB JnMsse5Em6jV2av73QUFel+s4DlWeko2b3ZERLEWm04EclC8pAWmkewdPNhOJ0Va W5pfG6a0/NKIQCXxK3wkkXQpmKMdemyE7j43Hy0C3/40IkvQKbwlygad6NiCz862 a53Z/oGuvYcQbt5rye54xhEGgoc0KmPmvretcoKm2udFrDkSjoBfS5FM40Pc6/4L abxySE+B/h2JjTOJUjsxmsMwWexFd9heUrgF30HGW7B5GFMR0edMsTfyn1HN1MYZ 3cTLfZCrJDxCJ022/Kajr+QOjjaNI7s4uxpizg=3D=3D =3Dw/Gc -----END PGP PUBLIC KEY BLOCK----- --------------040405000302020806020002--