From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1NwLnJ-0005u5-L7 for qemu-devel@nongnu.org; Mon, 29 Mar 2010 16:42:45 -0400 Received: from [140.186.70.92] (port=58434 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1NwLnH-0005qg-Tu for qemu-devel@nongnu.org; Mon, 29 Mar 2010 16:42:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1NwLnG-0006y6-62 for qemu-devel@nongnu.org; Mon, 29 Mar 2010 16:42:43 -0400 Received: from e35.co.us.ibm.com ([32.97.110.153]:49607) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1NwLnG-0006xo-15 for qemu-devel@nongnu.org; Mon, 29 Mar 2010 16:42:42 -0400 Received: from d03relay01.boulder.ibm.com (d03relay01.boulder.ibm.com [9.17.195.226]) by e35.co.us.ibm.com (8.14.3/8.13.1) with ESMTP id o2TKbrrZ028306 for ; Mon, 29 Mar 2010 14:37:53 -0600 Received: from d03av05.boulder.ibm.com (d03av05.boulder.ibm.com [9.17.195.85]) by d03relay01.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id o2TKgTuJ133430 for ; Mon, 29 Mar 2010 14:42:29 -0600 Received: from d03av05.boulder.ibm.com (loopback [127.0.0.1]) by d03av05.boulder.ibm.com (8.14.3/8.13.1/NCO v10.0 AVout) with ESMTP id o2TKgSNG017006 for ; Mon, 29 Mar 2010 14:42:29 -0600 Message-ID: <4BB110B3.8040300@linux.vnet.ibm.com> Date: Mon, 29 Mar 2010 15:42:27 -0500 From: Anthony Liguori MIME-Version: 1.0 Subject: Re: [Qemu-devel] [PATCH -V3 09/32] virtio-9p: Implement P9_TWRITE/ Thread model in QEMU References: <1269535420-31206-1-git-send-email-aneesh.kumar@linux.vnet.ibm.com> <1269535420-31206-10-git-send-email-aneesh.kumar@linux.vnet.ibm.com> <4BB04A59.1040204@linux.vnet.ibm.com> <4BB0C09D.7090204@linux.vnet.ibm.com> <4BB10E30.70908@redhat.com> In-Reply-To: <4BB10E30.70908@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Avi Kivity Cc: ericvh@gmail.com, jvrao , "Aneesh Kumar K.V" , qemu-devel@nongnu.org On 03/29/2010 03:31 PM, Avi Kivity wrote: > On 03/29/2010 06:00 PM, Anthony Liguori wrote: >> >> In qemu, we tend to prefer state-machine based code. >> >> There are a few different models that we have discussed at different >> points: >> >> 1) state machines with a thread pool to make blocking functions >> asynchronous (what we have today) >> >> 2) co-operative threading >> >> 3) pre-emptive threading >> >> All three models are independent of making device models re-entrant. >> In order to allow VCPU threads to run in qemu simultaneously, we need >> each device to carry a lock and for that lock to be acquired upon any >> of the public functions for the device model. >> >> For individual device models or host services, I think (3) is >> probably the worst model overall. I personally think that (1) is >> better in the long run but ultimately would need an existence proof >> to compare against (2). (2) looks appealing until you actually try >> to have the device handle multiple requests at a time. > > Sooner or later nature and the ever more complicated code will force > us towards (3). As an example, we've observed live migration to > throttle vcpus when sending a large guest's zeroed memory over; the > bandwidth control doesn't kick in since zero pages are compressed, so > the iothread spends large amounts of time reading memory. Making things re-entrant is different than (3) in my mind. There's no reason that VCPU threads should run in lock-step with live migration during the live phase. Making device models re-entrant and making live migration depend not depend on the big global lock is a good thing to do. What I'm skeptical of, is whether converting virtio-9p or qcow2 to handle each request in a separate thread is really going to improve things. The VNC server is another area that I think multithreading would be a bad idea. > We could fix this by yielding every so often (and a patch will be > posted soon), but it remains an issue. We have too much work in the > I/O thread and that defers I/O completion and timers. > >> For virtio-9p, I don't think (1) is much of a burden to be honest. >> In particular, when the 9p2000.L dialect is used, there should be a >> 1-1 correlation between protocol operations and system calls which >> means that for the most part, there's really no advantage to >> threading from a complexity point of view. > > But if those system calls are blocking, you need a thread? You can dispatch just the system call to a thread pool. The advantage of doing that is that you don't need to worry about locking since the system calls are not (usually) handling shared state. > On a philosophical note, threads may be easier to model complex > hardware that includes a processor, for example our scsi card (and how > about using tcg as a jit to boost it :) Yeah, it's hard to argue that script evaluation shouldn't be done in a thread. But that doesn't prevent me from being very cautious about how and where we use threading :-) Regards, Anthony Liguori