From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1L61E0-0001Qu-BB for qemu-devel@nongnu.org; Fri, 28 Nov 2008 06:09:28 -0500 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1L61Dz-0001Q9-FV for qemu-devel@nongnu.org; Fri, 28 Nov 2008 06:09:27 -0500 Received: from [199.232.76.173] (port=49895 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1L61Dz-0001Pv-5Z for qemu-devel@nongnu.org; Fri, 28 Nov 2008 06:09:27 -0500 Received: from mail2.shareable.org ([80.68.89.115]:44578) by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1L61Dy-0001Wp-Dp for qemu-devel@nongnu.org; Fri, 28 Nov 2008 06:09:26 -0500 Received: from jamie by mail2.shareable.org with local (Exim 4.63) (envelope-from ) id 1L61Dx-0001rs-6c for qemu-devel@nongnu.org; Fri, 28 Nov 2008 11:09:25 +0000 Date: Fri, 28 Nov 2008 11:09:25 +0000 From: Jamie Lokier Subject: Re: [Qemu-devel] [RFC 2/2] bdrv_aio_readv/writev_em Message-ID: <20081128110924.GE6138@shareable.org> References: <20081127123538.GC10348@random.random> <20081127124351.GD10348@random.random> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20081127124351.GD10348@random.random> Reply-To: qemu-devel@nongnu.org List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Andrea Arcangeli wrote: > pthread_create() and do aio with pthreads by calling writev by hand. > > Use kernel based linux aio (I think it's much better as it won't > screwup with contiguous I/O, and it handles o_direct random writes and > random reads by keeping the lowlevel I/O pipeline full without threads > but by just queuing _in_order_ [in order only from the point of view > of the I/O scheduler of course] and asynchronously the commands of > every different direct-io aio_readv/writev in the lowlevel storage > queue without needing any scheduler and thread synchronization > involvement). The good things about linux-aio are as you described. The bad thing is that I'm told linux-aio can block sometimes. Apart from memory pressure (which can block the calling process anyway), an aio implementation which can block is not so good. Doing it in a pthread is less good for a lot of reasons. On the other hand, I/O in a pthread should never block the main thread. We discussed the "in order" part a while ago, and several people pointed out that once there are a few I/Os in flight, the order should stabilise automatically, but perhaps that's not so good in transient situations. (Ideally we'd use scheduler activations (syslets!) to avoid blocking in any kernel I/O path no matter how complex, while keeping aio ordering and avoiding unnecessary task switching. I haven't heard much about progress on this since earlier this year...) -- Jamie