From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jamie Lokier Subject: Re: [kvm-devel] Re: [PATCH 1/3] Refactor AIO interface to allow other AIO implementations Date: Tue, 22 Apr 2008 16:23:22 +0100 Message-ID: <20080422152321.GB10229@shareable.org> References: <4808CD10.8010609@qumranet.com> <20080420154943.GB14268@shareable.org> <480B8EDC.6060507@qumranet.com> <20080420233913.GA23292@shareable.org> <480C36A3.6010900@qumranet.com> <20080421121028.GD4193@shareable.org> <480D9D74.5070801@qumranet.com> <20080422142847.GC4849@shareable.org> <480DFBD9.4030802@codemonkey.ws> <480DFEBC.1020504@qumranet.com> Reply-To: qemu-devel@nongnu.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: kvm-devel@lists.sourceforge.net, Marcelo Tosatti To: qemu-devel@nongnu.org Return-path: Content-Disposition: inline In-Reply-To: <480DFEBC.1020504@qumranet.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org Errors-To: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org List-Id: kvm.vger.kernel.org Avi Kivity wrote: > Anthony Liguori wrote: > >>If I submit sequential O_DIRECT reads with aio_read(), will they enter > >>the device read queue in the same order, and reach the disk in that > >>order (allowing for reordering when worthwhile by the elevator)? > >> > >There's no guarantee that any sort of order will be preserved by AIO > >requests. The same is true with writes. This is what fdsync is for, > >to guarantee ordering. > > I believe he'd like a hint to get good scheduling, not a guarantee. > With a thread pool if the threads are scheduled out of order, so are > your requests. > If the elevator doesn't plug the queue, the first few requests may > not be optimally sorted. That's right. Then they tend to settle to a good order. But any delay in scheduling one of the threads, or a signal received by one of them, can make it lose order briefly, making the streaming stutter as the disk performes a few local seeks until it settles to good order again. You can mitigate the disruption in various ways. 1. If all threads share an "offset" variable, and reads and increments that atomically just prior to calling pread(), that helps especially at the start. (If threaded I/O is used for QEMU disk emulation, I would suggest doing that, in the more general form of popping a request from QEMU's internal shared queue at the last moment.) 2. Using more threads helps keep it sustained, at the cost of more wasted I/O when there's a cancellation (changed mind), and more memory. However, AIO, in principle (if not implementations...) could be better at keeping the suggested I/O order than thread, without special tricks. -- Jamie