From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=47739 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1P6JiI-0002AR-1T for qemu-devel@nongnu.org; Thu, 14 Oct 2010 05:03:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1P6JiF-000690-Tq for qemu-devel@nongnu.org; Thu, 14 Oct 2010 05:03:01 -0400 Received: from mail-iw0-f173.google.com ([209.85.214.173]:48141) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1P6JiF-00068v-Mn for qemu-devel@nongnu.org; Thu, 14 Oct 2010 05:02:59 -0400 Received: by iwn34 with SMTP id 34so6435083iwn.4 for ; Thu, 14 Oct 2010 02:02:58 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20101013153110.21735.16669.stgit@localhost6.localdomain6> References: <20101013152921.21735.87339.stgit@localhost6.localdomain6> <20101013153110.21735.16669.stgit@localhost6.localdomain6> Date: Thu, 14 Oct 2010 10:02:57 +0100 Message-ID: Subject: Re: [Qemu-devel] [PATCH 1/3] Introduce threadlets From: Stefan Hajnoczi Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Arun R Bharadwaj Cc: qemu-devel@nongnu.org On Wed, Oct 13, 2010 at 4:31 PM, Arun R Bharadwaj wrote: > From: Aneesh Kumar K.V > > This patch creates a generic asynchronous-task-offloading infrastructure = named > threadlets. The core idea has been borrowed from the threading framework = that > is being used by paio. > > The reason for creating this generic infrastructure is so that other subs= ystems, > such as virtio-9p could make use of it for offloading tasks that could bl= ock. > > The patch creates a global queue on-to which subsystems can queue their t= asks to > be executed asynchronously. > > The patch also provides API's that allow a subsystem to create a private = queue > with an associated pool of threads. > > [ego@in.ibm.com: Facelift of the code, Documentation, cancel_threadlet > and other helpers] > > Signed-off-by: Aneesh Kumar K.V > Signed-off-by: Gautham R Shenoy > Signed-off-by: Sripathi Kodi > Signed-off-by: Arun R Bharadwaj > --- > =A0Makefile.objs =A0 =A0 =A0 =A0 =A0| =A0 =A03 + > =A0docs/async-support.txt | =A0141 ++++++++++++++++++++++++++++++++++++++= ++ > =A0qemu-threadlets.c =A0 =A0 =A0| =A0169 ++++++++++++++++++++++++++++++++= ++++++++++++++++ > =A0qemu-threadlets.h =A0 =A0 =A0| =A0 48 ++++++++++++++ > =A04 files changed, 360 insertions(+), 1 deletions(-) > =A0create mode 100644 docs/async-support.txt > =A0create mode 100644 qemu-threadlets.c > =A0create mode 100644 qemu-threadlets.h > > diff --git a/Makefile.objs b/Makefile.objs > index cd5a24b..2cf8aba 100644 > --- a/Makefile.objs > +++ b/Makefile.objs > @@ -9,6 +9,8 @@ qobject-obj-y +=3D qerror.o > > =A0block-obj-y =3D cutils.o cache-utils.o qemu-malloc.o qemu-option.o mod= ule.o > =A0block-obj-y +=3D nbd.o block.o aio.o aes.o osdep.o qemu-config.o > +block-obj-$(CONFIG_POSIX) +=3D qemu-thread.o > +block-obj-$(CONFIG_POSIX) +=3D qemu-threadlets.o > =A0block-obj-$(CONFIG_POSIX) +=3D posix-aio-compat.o > =A0block-obj-$(CONFIG_LINUX_AIO) +=3D linux-aio.o > > @@ -124,7 +126,6 @@ endif > =A0common-obj-y +=3D $(addprefix ui/, $(ui-obj-y)) > > =A0common-obj-y +=3D iov.o acl.o > -common-obj-$(CONFIG_THREAD) +=3D qemu-thread.o > =A0common-obj-y +=3D notify.o event_notifier.o > =A0common-obj-y +=3D qemu-timer.o > > diff --git a/docs/async-support.txt b/docs/async-support.txt > new file mode 100644 > index 0000000..2e8adc9 > --- /dev/null > +++ b/docs/async-support.txt Why not call it docs/threadlets.txt? > @@ -0,0 +1,141 @@ > +=3D=3D How to use the various asynchronous models supported in Qemu =3D= =3D This only describes threadlets. > + > +=3D=3D Threadlets =3D=3D > + > +Q.1: What are threadlets ? > +A.1: Threadlets is an infrastructure within QEMU that allows other subsy= stems > + =A0 =A0 to offload possibly blocking work to a queue to be processed by= a pool > + =A0 =A0 of threads asynchrnonously. asynchronously typo > + > +Q.2: When would one want to use threadlets ? > +A.2: Threadlets are useful when there are operations that can be perform= ed > + =A0 =A0 outside the context of the VCPU/IO threads inorder to free thes= e latter > + =A0 =A0 to service any other guest requests. > + > +Q.3: I have some work that can be executed in an asynchronous context. H= ow > + =A0 =A0 should I go about it ? > +A.3: One could follow the steps listed below: > + > + =A0 =A0 - Define a function which would do the asynchrnous work. asynchronous typo > + =A0 =A0 =A0 void my_threadlet_func(ThreadletWork *work) Usually these functions will be static. > + =A0 =A0 =A0 { > + =A0 =A0 =A0 } > + > + =A0 =A0 - Declare an object of type ThreadletWork; > + =A0 =A0 =A0 ThreadletWork work; > + > + > + =A0 =A0 - Assign a value to the "func" member of ThreadletWork object. > + =A0 =A0 =A0 work.func =3D my_threadlet_func; > + > + =A0 =A0 - Submit the threadlet to the global queue. > + =A0 =A0 =A0 submit_threadletwork(&work); > + > + =A0 =A0 - Continue servicing some other guest operations. > + > +Q.4: I want to my_threadlet_func to access some non-global data. How do = I do > + =A0 =A0 that ? > +A.4: Suppose you want my_threadlet_func to access some non-global data-o= bject > + =A0 =A0 of type myPrivateData. In that case one could follow the follow= ing steps. > + > + =A0 =A0 - Define a member of the type ThreadletWork within myPrivateDat= a. > + =A0 =A0 =A0 typdef myPrivateData { typedef typo, struct missing Also, the QEMU coding style usually requires CapsNames for struct types... > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ...; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ...; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ...; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ThreadletWork work; > + =A0 =A0 =A0 } myPrivateData; > + > + =A0 =A0 =A0 myPrivateData myData; ...and myData would be my_data. > + > + =A0 =A0 - Initialize myData.work as described in A.3 > + =A0 =A0 =A0 myData.work.func =3D my_threadlet_func; > + =A0 =A0 =A0 submit_threadletwork(&myData.work); > + > + =A0 =A0 - Access the myData object inside my_threadlet_func() using con= tainer_of > + =A0 =A0 =A0 primitive > + =A0 =A0 =A0 void my_threadlet_func(ThreadletWork *work) > + =A0 =A0 =A0 { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 myPrivateData *mydata_ptr; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 mydata_ptr =3D container_of(work, myPrivate= Data, work); > + > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 /* mydata_ptr now points to myData object *= / > + =A0 =A0 =A0 } > + > +Q.5: Are there any precautions one must take while sharing data with the > + =A0 =A0 Asynchrnous thread-pool ? asynchronous typo > +A.5: Yes, make sure that the helper function of the type my_threadlet_fu= nc() > + =A0 =A0 does not access/modify data when it can be accessed or modified= in the > + =A0 =A0 context of VCPU thread or IO thread. This is because the asynch= ronous > + =A0 =A0 threads in the pool can run in parallel with the VCPU/IOThreads= as shown > + =A0 =A0 in the figure. > + > + =A0 =A0 A typical workflow is as follows: > + > + =A0 =A0 =A0 =A0 =A0 =A0 =A0VCPU/IOThread > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | (1) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 V > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Offload work =A0 =A0 =A0 =A0 =A0 =A0 =A0= (2) > + =A0 =A0 =A0|-------> to threadlets -----------------------------> Helpe= r thread > + =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | > + =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | > + =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0| (3) =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | (4) > + =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | > + =A0 =A0 =A0| =A0 =A0 =A0 =A0 Handle other Guest requests =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0| > + =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | > + =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 V > + =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0| (3) =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Signal the I/O Thread > + =A0 =A0 =A0|(6) =A0 =A0 =A0 =A0 | =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | > + =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/ > + =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 / > + =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0V =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/ > + =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0Do the post <--------------------------= -------/ > + =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0processing =A0 =A0 =A0 =A0 =A0 =A0 =A0 = (5) > + =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0| > + =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0| (6) > + =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0V > + =A0 =A0 =A0|-Yes------ More async work? > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | (7) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0No > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 . > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 . > + > + =A0 =A0Hence one needs to make sure that in the steps (3) and (4) which= run in > + =A0 =A0parallel, any global data is accessed within only one context. > + > +Q.6: I have queued a threadlet which I want to cancel. How do I do that = ? > +A.6: Threadlets framework provides the API cancel_threadlet: > + =A0 =A0 =A0 - int cancel_threadletwork(ThreadletWork *work) > + > + =A0 =A0 The API scans the ThreadletQueue to see if (work) is present. I= f it finds > + =A0 =A0 work, it'll dequeue work and return 0. > + > + =A0 =A0 On the other hand, if it does not find the (work) in the Thread= letQueue, > + =A0 =A0 then it'll return 1. This can imply two things. Either the work= is being > + =A0 =A0 processed by one of the helper threads or it has been processed= . The > + =A0 =A0 threadlet infrastructure currently _does_not_ distinguish betwe= en these > + =A0 =A0 two and the onus is on the caller to do that. > + > +Q.7: Apart from the global pool of threads, can I have my own private Qu= eue ? > +A.7: Yes, the threadlets framework allows subsystems to create their own= private > + =A0 =A0 queues with associated pools of threads. > + > + =A0 =A0 - Define a PrivateQueue > + =A0 =A0 =A0 ThreadletQueue myQueue; > + > + =A0 =A0 - Initialize it: > + =A0 =A0 =A0 threadlet_queue_init(&myQueue, my_max_threads, my_min_threa= ds); > + =A0 =A0 =A0 where my_max_threads is the maximum number of threads that = can be in the > + =A0 =A0 =A0 thread pool and my_min_threads is the minimum number of act= ive threads > + =A0 =A0 =A0 that can be in the thread-pool. > + > + =A0 =A0 - Submit work: > + =A0 =A0 =A0 submit_threadletwork_to_queue(&myQueue, &my_work); > + > + =A0 =A0 - Cancel work: > + =A0 =A0 =A0 cancel_threadletwork_on_queue(&myQueue, &my_work); > diff --git a/qemu-threadlets.c b/qemu-threadlets.c > new file mode 100644 > index 0000000..1442122 > --- /dev/null > +++ b/qemu-threadlets.c > @@ -0,0 +1,169 @@ > +/* > + * Threadlet support for offloading tasks to be executed asynchronously > + * > + * Copyright IBM, Corp. 2008 > + * Copyright IBM, Corp. 2010 > + * > + * Authors: > + * =A0Anthony Liguori =A0 =A0 > + * =A0Aneesh Kumar K.V =A0 =A0 > + * =A0Gautham R Shenoy =A0 =A0 > + * > + * This work is licensed under the terms of the GNU GPL, version 2. =A0S= ee > + * the COPYING file in the top-level directory. > + */ > + > +#include "qemu-threadlets.h" > +#include "osdep.h" > + > +#define MAX_GLOBAL_THREADS =A064 > +#define MIN_GLOBAL_THREADS =A064 > +ThreadletQueue globalqueue; This should be static. > +static int globalqueue_init; > + > +static void *threadlet_worker(void *data) > +{ > + =A0 =A0ThreadletQueue *queue =3D data; > + > + =A0 =A0qemu_mutex_lock(&(queue->lock)); > + =A0 =A0while (1) { > + =A0 =A0 =A0 =A0ThreadletWork *work; > + =A0 =A0 =A0 =A0int ret =3D 0; > + > + =A0 =A0 =A0 =A0while (QTAILQ_EMPTY(&(queue->request_list)) && > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 (ret !=3D ETIMEDOUT)) { > + =A0 =A0 =A0 =A0 =A0 =A0ret =3D qemu_cond_timedwait(&(queue->cond), > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0&(queue->lock), 10*100000); > + =A0 =A0 =A0 =A0} > + > + =A0 =A0 =A0 =A0assert(queue->idle_threads !=3D 0); > + =A0 =A0 =A0 =A0if (QTAILQ_EMPTY(&(queue->request_list))) { > + =A0 =A0 =A0 =A0 =A0 =A0if (queue->cur_threads > queue->min_threads) { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* We retain the minimum number of threa= ds */ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0break; > + =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0} else { > + =A0 =A0 =A0 =A0 =A0 =A0work =3D QTAILQ_FIRST(&(queue->request_list)); > + =A0 =A0 =A0 =A0 =A0 =A0QTAILQ_REMOVE(&(queue->request_list), work, node= ); > + > + =A0 =A0 =A0 =A0 =A0 =A0queue->idle_threads--; > + =A0 =A0 =A0 =A0 =A0 =A0qemu_mutex_unlock(&(queue->lock)); > + > + =A0 =A0 =A0 =A0 =A0 =A0/* execute the work function */ > + =A0 =A0 =A0 =A0 =A0 =A0work->func(work); > + > + =A0 =A0 =A0 =A0 =A0 =A0qemu_mutex_lock(&(queue->lock)); > + =A0 =A0 =A0 =A0 =A0 =A0queue->idle_threads++; > + =A0 =A0 =A0 =A0} > + =A0 =A0} > + > + =A0 =A0queue->idle_threads--; > + =A0 =A0queue->cur_threads--; > + =A0 =A0qemu_mutex_unlock(&(queue->lock)); > + > + =A0 =A0return NULL; > +} > + > +static void spawn_threadlet(ThreadletQueue *queue) > +{ > + =A0 =A0QemuThread thread; > + > + =A0 =A0queue->cur_threads++; > + =A0 =A0queue->idle_threads++; > + > + =A0 =A0qemu_thread_create(&thread, threadlet_worker, queue); > +} > + > +/** > + * submit_threadletwork_to_queue: Submit a new task to a private queue t= o be > + * =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0executed async= hronously. > + * @queue: Per-subsystem private queue to which the new task needs > + * =A0 =A0 =A0 =A0 to be submitted. > + * @work: Contains information about the task that needs to be submitted= . > + */ > +void submit_threadletwork_to_queue(ThreadletQueue *queue, ThreadletWork = *work) > +{ > + =A0 =A0qemu_mutex_lock(&(queue->lock)); > + =A0 =A0if (queue->idle_threads =3D=3D 0 && queue->cur_threads < queue->= max_threads) { > + =A0 =A0 =A0 =A0spawn_threadlet(queue); > + =A0 =A0} > + =A0 =A0QTAILQ_INSERT_TAIL(&(queue->request_list), work, node); > + =A0 =A0qemu_mutex_unlock(&(queue->lock)); > + =A0 =A0qemu_cond_signal(&(queue->cond)); Worker thread signalling and spawning has race conditions. See my previous email: "There are race conditions here: 1. When a new threadlet is started because there are no idle threads, qemu_cond_signal() may fire a blank because the threadlet isn't inside qemu_cond_timedwait() yet. The result, the work item is deadlocked until another thread grabs more work off the queue. If I'm reading the code correctly this bug is currently present! 2. Moving qemu_cond_signal() outside queue->lock is dangerous for the same reason: you need to be careful not to qemu_cond_signal() when the thread isn't inside qemu_cond_timedwait()." > +} > + > +/** > + * submit_threadletwork: Submit to the global queue a new task to be exe= cuted > + * =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 asynchronously. > + * @work: Contains information about the task that needs to be submitted= . > + */ > +void submit_threadletwork(ThreadletWork *work) > +{ > + =A0 =A0if (unlikely(!globalqueue_init)) { > + =A0 =A0 =A0 =A0threadlet_queue_init(&globalqueue, MAX_GLOBAL_THREADS, > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0MIN_GLOB= AL_THREADS); > + =A0 =A0 =A0 =A0globalqueue_init =3D 1; > + =A0 =A0} > + > + =A0 =A0submit_threadletwork_to_queue(&globalqueue, work); > +} > + > +/** > + * cancel_threadletwork_on_queue: Cancel a task queued on a Queue. > + * @queue: The queue containing the task to be cancelled. > + * @work: Contains the information of the task that needs to be cancelle= d. > + * > + * Returns: 0 if the task is successfully cancelled. > + * =A0 =A0 =A0 =A0 =A01 otherwise. > + */ > +int cancel_threadletwork_on_queue(ThreadletQueue *queue, ThreadletWork *= work) > +{ > + =A0 =A0ThreadletWork *ret_work; > + =A0 =A0int found =3D 0; > + > + =A0 =A0qemu_mutex_lock(&(queue->lock)); > + =A0 =A0QTAILQ_FOREACH(ret_work, &(queue->request_list), node) { > + =A0 =A0 =A0 =A0if (ret_work =3D=3D work) { > + =A0 =A0 =A0 =A0 =A0 =A0QTAILQ_REMOVE(&(queue->request_list), ret_work, = node); > + =A0 =A0 =A0 =A0 =A0 =A0found =3D 1; > + =A0 =A0 =A0 =A0 =A0 =A0break; > + =A0 =A0 =A0 =A0} > + =A0 =A0} > + =A0 =A0qemu_mutex_unlock(&(queue->lock)); > + > + =A0 =A0if (found) { > + =A0 =A0 =A0 =A0return 0; > + =A0 =A0} > + > + =A0 =A0return 1; Perhaps invert the logic of "found" and just call it "ret", then you don't need this if statement that flips its value. > +} > + > +/** > + * cancel_threadletwork: Cancel a task queued on the global queue. > + * @work: Contains the information of the task that needs to be cancelle= d. > + * > + * Returns: 0 if the task is successfully cancelled. > + * =A0 =A0 =A0 =A0 =A01 otherwise. > + */ > +int cancel_threadletwork(ThreadletWork *work) > +{ > + =A0 =A0return cancel_threadletwork_on_queue(&globalqueue, work); > +} > + > +/** > + * threadlet_queue_init: Initialize a threadlet queue. > + * @queue: The threadlet queue to be initialized. > + * @max_threads: Maximum number of threads processing the queue. > + * @min_threads: Minimum number of threads processing the queue. > + */ > +void threadlet_queue_init(ThreadletQueue *queue, > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 int= max_threads, int min_threads) > +{ > + =A0 =A0queue->cur_threads =A0=3D 0; > + =A0 =A0queue->idle_threads =3D 0; > + =A0 =A0queue->max_threads =A0=3D max_threads; > + =A0 =A0queue->min_threads =A0=3D min_threads; > + =A0 =A0QTAILQ_INIT(&(queue->request_list)); > + =A0 =A0qemu_mutex_init(&(queue->lock)); > + =A0 =A0qemu_cond_init(&(queue->cond)); > +} > diff --git a/qemu-threadlets.h b/qemu-threadlets.h > new file mode 100644 > index 0000000..3df9b10 > --- /dev/null > +++ b/qemu-threadlets.h > @@ -0,0 +1,48 @@ > +/* > + * Threadlet support for offloading tasks to be executed asynchronously > + * > + * Copyright IBM, Corp. 2008 > + * Copyright IBM, Corp. 2010 > + * > + * Authors: > + * =A0Anthony Liguori =A0 =A0 > + * =A0Aneesh Kumar K.V =A0 =A0 > + * =A0Gautham R Shenoy =A0 =A0 > + * > + * This work is licensed under the terms of the GNU GPL, version 2. =A0S= ee > + * the COPYING file in the top-level directory. > + */ > + > +#ifndef QEMU_ASYNC_WORK_H > +#define QEMU_ASYNC_WORK_H QEMU_THREADLETS_H? > + > +#include "qemu-queue.h" > +#include "qemu-common.h" > +#include "qemu-thread.h" > + > +typedef struct ThreadletQueue > +{ > + =A0 =A0QemuMutex lock; > + =A0 =A0QemuCond cond; > + =A0 =A0int max_threads; > + =A0 =A0int min_threads; > + =A0 =A0int cur_threads; > + =A0 =A0int idle_threads; > + =A0 =A0QTAILQ_HEAD(, threadlet_work) request_list; > +} ThreadletQueue; > + > +typedef struct threadlet_work struct ThreadletWork follows coding style more closely. Stefan