From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1MXx6y-0006Iv-Cb for qemu-devel@nongnu.org; Mon, 03 Aug 2009 08:57:56 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1MXx6t-0006GZ-En for qemu-devel@nongnu.org; Mon, 03 Aug 2009 08:57:56 -0400 Received: from [199.232.76.173] (port=50116 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1MXx6t-0006GR-8j for qemu-devel@nongnu.org; Mon, 03 Aug 2009 08:57:51 -0400 Received: from mx2.redhat.com ([66.187.237.31]:55309) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1MXx6s-0002Th-Eq for qemu-devel@nongnu.org; Mon, 03 Aug 2009 08:57:50 -0400 Message-ID: <4A76E002.2060301@redhat.com> Date: Mon, 03 Aug 2009 16:02:58 +0300 From: Avi Kivity MIME-Version: 1.0 Subject: Re: [Qemu-devel] [PATCH] RFC: Add new block driver for the VDI format References: <4A4E5AFC.4020206@mail.berlios.de> <4A51FE07.2040307@codemonkey.ws> <4A75A242.9020005@redhat.com> <4A764AAA.1030509@codemonkey.ws> In-Reply-To: <4A764AAA.1030509@codemonkey.ws> Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Anthony Liguori Cc: Kevin Wolf , QEMU Developers , Christoph Hellwig On 08/03/2009 05:25 AM, Anthony Liguori wrote: > Avi Kivity wrote: >> On 07/06/2009 04:37 PM, Anthony Liguori wrote: >>> >>> I'd really like to get rid of synchronous IO functions in the block >>> layer. One way to do this is to insist that all new block drivers >>> only implement the AIO functions. >>> >>> I think we should make this decree but I'd like to know if other >>> people think this is unreasonable first. One potential model of >>> block drivers would involve synchronous IO and threads. I'm not a >>> big fan of that model and I don't think it's an easy conversion from >>> today's synchronous IO drivers to that model because the locking and >>> re-entrance needs careful consideration. >>> >> >> I agree that sync+threads is not easy, but well performing async is >> much, much harder. Consider that qcow2 still has synchronous >> operations, and that eliminating the RMW when writing a partial >> cluster concurrently (a very common operation with 64K clusters) is >> very hard to do ayncly and much easier syncly. > > Supporting parallel RMW operations is certainly difficult, but you're > confusing parallel RMW ops with asynchronous RMW ops. You just have > to queue requests and handle them in order. It's only mildly more > difficult to deal with asynchronous I/O and it avoids all the > nastiness associated with threads and locking. I'm talking about a guest sequential write emitted as multiple adjacent requests in parallel. Currently we'll write the first request and the second request in different locations, then do a rmw to merge the two blocks (I think...). > Fundamentally, threads don't help the RMW problem because you probably > would just hold a look for the entire RMW operation so you're > effectively queuing any RMW op. You do get some queuing but layout is improved, and it's not a W-RMW; instead it's a W-WW. Theoretically anything you can do with threads you can do with async operations but experience has proven that async is much more difficult. Consider the last qcow2 bug. > >> Given in addition the large numbers of format drivers, I think we >> should prefer sync+threads over trying to convert all format drivers >> to full async. > > It's just shifting the problem from one place to another. Instead of > figuring out the state machine, you have to figure out how to do the > locking. The danger of the later is that it gives you the illusion > that it's an easy problem and is therefore prone to error. Locking _is_ an easier problem than figuring out the state machine. I can't prove this but there's numerous anecdotal evidence on the subject. -- error compiling committee.c: too many arguments to function