* [Qemu-devel] [PATCH v8 1/5] coroutine: add gthread dependency
2011-07-26 9:21 [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming Stefan Hajnoczi
@ 2011-07-26 9:21 ` Stefan Hajnoczi
2011-07-26 9:21 ` [Qemu-devel] [PATCH v8 2/5] coroutine: introduce coroutines API Stefan Hajnoczi
` (6 subsequent siblings)
7 siblings, 0 replies; 14+ messages in thread
From: Stefan Hajnoczi @ 2011-07-26 9:21 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, Blue Swirl, Anthony Liguori, Venkateswararao Jujjuri,
Stefan Hajnoczi
Commit 1fc7bd4a86a2bfeafcec29445871eb97469a2699 removed the gthread and
gio dependency since qemu-ga did not require it. Coroutines require
gthread, so add it back in.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
---
configure | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/configure b/configure
index 600da9b..5150372 100755
--- a/configure
+++ b/configure
@@ -1811,9 +1811,9 @@ fi
##########################################
# glib support probe
-if $pkg_config --modversion glib-2.0 > /dev/null 2>&1 ; then
- glib_cflags=`$pkg_config --cflags glib-2.0 2>/dev/null`
- glib_libs=`$pkg_config --libs glib-2.0 2>/dev/null`
+if $pkg_config --modversion gthread-2.0 > /dev/null 2>&1 ; then
+ glib_cflags=`$pkg_config --cflags gthread-2.0 2>/dev/null`
+ glib_libs=`$pkg_config --libs gthread-2.0 2>/dev/null`
libs_softmmu="$glib_libs $libs_softmmu"
libs_tools="$glib_libs $libs_tools"
else
--
1.7.5.4
^ permalink raw reply related [flat|nested] 14+ messages in thread* [Qemu-devel] [PATCH v8 2/5] coroutine: introduce coroutines API
2011-07-26 9:21 [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming Stefan Hajnoczi
2011-07-26 9:21 ` [Qemu-devel] [PATCH v8 1/5] coroutine: add gthread dependency Stefan Hajnoczi
@ 2011-07-26 9:21 ` Stefan Hajnoczi
2011-07-26 9:21 ` [Qemu-devel] [PATCH v8 3/5] coroutine: add ucontext and win32 implementations Stefan Hajnoczi
` (5 subsequent siblings)
7 siblings, 0 replies; 14+ messages in thread
From: Stefan Hajnoczi @ 2011-07-26 9:21 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, Anthony Liguori, Stefan Hajnoczi, Blue Swirl,
Aneesh Kumar K.V, Venkateswararao Jujjuri
Asynchronous code is becoming very complex. At the same time
synchronous code is growing because it is convenient to write.
Sometimes duplicate code paths are even added, one synchronous and the
other asynchronous. This patch introduces coroutines which allow code
that looks synchronous but is asynchronous under the covers.
A coroutine has its own stack and is therefore able to preserve state
across blocking operations, which traditionally require callback
functions and manual marshalling of parameters.
Creating and starting a coroutine is easy:
coroutine = qemu_coroutine_create(my_coroutine);
qemu_coroutine_enter(coroutine, my_data);
The coroutine then executes until it returns or yields:
void coroutine_fn my_coroutine(void *opaque) {
MyData *my_data = opaque;
/* do some work */
qemu_coroutine_yield();
/* do some more work */
}
Yielding switches control back to the caller of qemu_coroutine_enter().
This is typically used to switch back to the main thread's event loop
after issuing an asynchronous I/O request. The request callback will
then invoke qemu_coroutine_enter() once more to switch back to the
coroutine.
Note that if coroutines are used only from threads which hold the global
mutex they will never execute concurrently. This makes programming with
coroutines easier than with threads. Race conditions cannot occur since
only one coroutine may be active at any time. Other coroutines can only
run across yield.
This patch includes a gthread-based coroutines implementation which
should work on all platforms.
[Original gthread patch by Aneesh, made consistent with
coroutine-ucontext.c and switched to GStaticPrivate by Stefan. Tested
on Linux and OpenBSD.]
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
---
Makefile.objs | 5 ++
coroutine-gthread.c | 131 ++++++++++++++++++++++++++++++++++++++++++++++++++
qemu-coroutine-int.h | 48 ++++++++++++++++++
qemu-coroutine.c | 75 ++++++++++++++++++++++++++++
qemu-coroutine.h | 95 ++++++++++++++++++++++++++++++++++++
trace-events | 5 ++
6 files changed, 359 insertions(+), 0 deletions(-)
create mode 100644 coroutine-gthread.c
create mode 100644 qemu-coroutine-int.h
create mode 100644 qemu-coroutine.c
create mode 100644 qemu-coroutine.h
diff --git a/Makefile.objs b/Makefile.objs
index 6991a9f..db907c4 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -11,6 +11,10 @@ oslib-obj-$(CONFIG_WIN32) += oslib-win32.o qemu-thread-win32.o
oslib-obj-$(CONFIG_POSIX) += oslib-posix.o qemu-thread-posix.o
#######################################################################
+# coroutines
+coroutine-obj-y = qemu-coroutine.o coroutine-gthread.o
+
+#######################################################################
# block-obj-y is code used by both qemu system emulation and qemu-img
block-obj-y = cutils.o cache-utils.o qemu-malloc.o qemu-option.o module.o async.o
@@ -69,6 +73,7 @@ common-obj-y += readline.o console.o cursor.o qemu-error.o
common-obj-y += $(oslib-obj-y)
common-obj-$(CONFIG_WIN32) += os-win32.o
common-obj-$(CONFIG_POSIX) += os-posix.o
+common-obj-y += $(coroutine-obj-y)
common-obj-y += tcg-runtime.o host-utils.o
common-obj-y += irq.o ioport.o input.o
diff --git a/coroutine-gthread.c b/coroutine-gthread.c
new file mode 100644
index 0000000..f09877e
--- /dev/null
+++ b/coroutine-gthread.c
@@ -0,0 +1,131 @@
+/*
+ * GThread coroutine initialization code
+ *
+ * Copyright (C) 2006 Anthony Liguori <anthony@codemonkey.ws>
+ * Copyright (C) 2011 Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.0 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <glib.h>
+#include "qemu-common.h"
+#include "qemu-coroutine-int.h"
+
+typedef struct {
+ Coroutine base;
+ GThread *thread;
+ bool runnable;
+ CoroutineAction action;
+} CoroutineGThread;
+
+static GCond *coroutine_cond;
+static GStaticMutex coroutine_lock = G_STATIC_MUTEX_INIT;
+static GStaticPrivate coroutine_key = G_STATIC_PRIVATE_INIT;
+
+static void __attribute__((constructor)) coroutine_init(void)
+{
+ if (!g_thread_supported()) {
+ g_thread_init(NULL);
+ }
+
+ coroutine_cond = g_cond_new();
+}
+
+static void coroutine_wait_runnable_locked(CoroutineGThread *co)
+{
+ while (!co->runnable) {
+ g_cond_wait(coroutine_cond, g_static_mutex_get_mutex(&coroutine_lock));
+ }
+}
+
+static void coroutine_wait_runnable(CoroutineGThread *co)
+{
+ g_static_mutex_lock(&coroutine_lock);
+ coroutine_wait_runnable_locked(co);
+ g_static_mutex_unlock(&coroutine_lock);
+}
+
+static gpointer coroutine_thread(gpointer opaque)
+{
+ CoroutineGThread *co = opaque;
+
+ g_static_private_set(&coroutine_key, co, NULL);
+ coroutine_wait_runnable(co);
+ co->base.entry(co->base.entry_arg);
+ qemu_coroutine_switch(&co->base, co->base.caller, COROUTINE_TERMINATE);
+ return NULL;
+}
+
+Coroutine *qemu_coroutine_new(void)
+{
+ CoroutineGThread *co;
+
+ co = qemu_mallocz(sizeof(*co));
+ co->thread = g_thread_create_full(coroutine_thread, co, 0, TRUE, TRUE,
+ G_THREAD_PRIORITY_NORMAL, NULL);
+ if (!co->thread) {
+ qemu_free(co);
+ return NULL;
+ }
+ return &co->base;
+}
+
+void qemu_coroutine_delete(Coroutine *co_)
+{
+ CoroutineGThread *co = DO_UPCAST(CoroutineGThread, base, co_);
+
+ g_thread_join(co->thread);
+ qemu_free(co);
+}
+
+CoroutineAction qemu_coroutine_switch(Coroutine *from_,
+ Coroutine *to_,
+ CoroutineAction action)
+{
+ CoroutineGThread *from = DO_UPCAST(CoroutineGThread, base, from_);
+ CoroutineGThread *to = DO_UPCAST(CoroutineGThread, base, to_);
+
+ g_static_mutex_lock(&coroutine_lock);
+ from->runnable = false;
+ from->action = action;
+ to->runnable = true;
+ to->action = action;
+ g_cond_broadcast(coroutine_cond);
+
+ if (action != COROUTINE_TERMINATE) {
+ coroutine_wait_runnable_locked(from);
+ }
+ g_static_mutex_unlock(&coroutine_lock);
+ return from->action;
+}
+
+Coroutine *qemu_coroutine_self(void)
+{
+ CoroutineGThread *co = g_static_private_get(&coroutine_key);
+
+ if (!co) {
+ co = qemu_mallocz(sizeof(*co));
+ co->runnable = true;
+ g_static_private_set(&coroutine_key, co, (GDestroyNotify)qemu_free);
+ }
+
+ return &co->base;
+}
+
+bool qemu_in_coroutine(void)
+{
+ CoroutineGThread *co = g_static_private_get(&coroutine_key);
+
+ return co && co->base.caller;
+}
diff --git a/qemu-coroutine-int.h b/qemu-coroutine-int.h
new file mode 100644
index 0000000..64915c2
--- /dev/null
+++ b/qemu-coroutine-int.h
@@ -0,0 +1,48 @@
+/*
+ * Coroutine internals
+ *
+ * Copyright (c) 2011 Kevin Wolf <kwolf@redhat.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#ifndef QEMU_COROUTINE_INT_H
+#define QEMU_COROUTINE_INT_H
+
+#include "qemu-queue.h"
+#include "qemu-coroutine.h"
+
+typedef enum {
+ COROUTINE_YIELD = 1,
+ COROUTINE_TERMINATE = 2,
+} CoroutineAction;
+
+struct Coroutine {
+ CoroutineEntry *entry;
+ void *entry_arg;
+ Coroutine *caller;
+ QLIST_ENTRY(Coroutine) pool_next;
+};
+
+Coroutine *qemu_coroutine_new(void);
+void qemu_coroutine_delete(Coroutine *co);
+CoroutineAction qemu_coroutine_switch(Coroutine *from, Coroutine *to,
+ CoroutineAction action);
+
+#endif
diff --git a/qemu-coroutine.c b/qemu-coroutine.c
new file mode 100644
index 0000000..600be26
--- /dev/null
+++ b/qemu-coroutine.c
@@ -0,0 +1,75 @@
+/*
+ * QEMU coroutines
+ *
+ * Copyright IBM, Corp. 2011
+ *
+ * Authors:
+ * Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
+ * Kevin Wolf <kwolf@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#include "trace.h"
+#include "qemu-common.h"
+#include "qemu-coroutine.h"
+#include "qemu-coroutine-int.h"
+
+Coroutine *qemu_coroutine_create(CoroutineEntry *entry)
+{
+ Coroutine *co = qemu_coroutine_new();
+ co->entry = entry;
+ return co;
+}
+
+static void coroutine_swap(Coroutine *from, Coroutine *to)
+{
+ CoroutineAction ret;
+
+ ret = qemu_coroutine_switch(from, to, COROUTINE_YIELD);
+
+ switch (ret) {
+ case COROUTINE_YIELD:
+ return;
+ case COROUTINE_TERMINATE:
+ trace_qemu_coroutine_terminate(to);
+ qemu_coroutine_delete(to);
+ return;
+ default:
+ abort();
+ }
+}
+
+void qemu_coroutine_enter(Coroutine *co, void *opaque)
+{
+ Coroutine *self = qemu_coroutine_self();
+
+ trace_qemu_coroutine_enter(self, co, opaque);
+
+ if (co->caller) {
+ fprintf(stderr, "Co-routine re-entered recursively\n");
+ abort();
+ }
+
+ co->caller = self;
+ co->entry_arg = opaque;
+ coroutine_swap(self, co);
+}
+
+void coroutine_fn qemu_coroutine_yield(void)
+{
+ Coroutine *self = qemu_coroutine_self();
+ Coroutine *to = self->caller;
+
+ trace_qemu_coroutine_yield(self, to);
+
+ if (!to) {
+ fprintf(stderr, "Co-routine is yielding to no one\n");
+ abort();
+ }
+
+ self->caller = NULL;
+ coroutine_swap(self, to);
+}
diff --git a/qemu-coroutine.h b/qemu-coroutine.h
new file mode 100644
index 0000000..08255c7
--- /dev/null
+++ b/qemu-coroutine.h
@@ -0,0 +1,95 @@
+/*
+ * QEMU coroutine implementation
+ *
+ * Copyright IBM, Corp. 2011
+ *
+ * Authors:
+ * Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#ifndef QEMU_COROUTINE_H
+#define QEMU_COROUTINE_H
+
+#include <stdbool.h>
+
+/**
+ * Coroutines are a mechanism for stack switching and can be used for
+ * cooperative userspace threading. These functions provide a simple but
+ * useful flavor of coroutines that is suitable for writing sequential code,
+ * rather than callbacks, for operations that need to give up control while
+ * waiting for events to complete.
+ *
+ * These functions are re-entrant and may be used outside the global mutex.
+ */
+
+/**
+ * Mark a function that executes in coroutine context
+ *
+ * Functions that execute in coroutine context cannot be called directly from
+ * normal functions. In the future it would be nice to enable compiler or
+ * static checker support for catching such errors. This annotation might make
+ * it possible and in the meantime it serves as documentation.
+ *
+ * For example:
+ *
+ * static void coroutine_fn foo(void) {
+ * ....
+ * }
+ */
+#define coroutine_fn
+
+typedef struct Coroutine Coroutine;
+
+/**
+ * Coroutine entry point
+ *
+ * When the coroutine is entered for the first time, opaque is passed in as an
+ * argument.
+ *
+ * When this function returns, the coroutine is destroyed automatically and
+ * execution continues in the caller who last entered the coroutine.
+ */
+typedef void coroutine_fn CoroutineEntry(void *opaque);
+
+/**
+ * Create a new coroutine
+ *
+ * Use qemu_coroutine_enter() to actually transfer control to the coroutine.
+ */
+Coroutine *qemu_coroutine_create(CoroutineEntry *entry);
+
+/**
+ * Transfer control to a coroutine
+ *
+ * The opaque argument is passed as the argument to the entry point when
+ * entering the coroutine for the first time. It is subsequently ignored.
+ */
+void qemu_coroutine_enter(Coroutine *coroutine, void *opaque);
+
+/**
+ * Transfer control back to a coroutine's caller
+ *
+ * This function does not return until the coroutine is re-entered using
+ * qemu_coroutine_enter().
+ */
+void coroutine_fn qemu_coroutine_yield(void);
+
+/**
+ * Get the currently executing coroutine
+ */
+Coroutine *coroutine_fn qemu_coroutine_self(void);
+
+/**
+ * Return whether or not currently inside a coroutine
+ *
+ * This can be used to write functions that work both when in coroutine context
+ * and when not in coroutine context. Note that such functions cannot use the
+ * coroutine_fn annotation since they work outside coroutine context.
+ */
+bool qemu_in_coroutine(void);
+
+#endif /* QEMU_COROUTINE_H */
diff --git a/trace-events b/trace-events
index 713f042..136f775 100644
--- a/trace-events
+++ b/trace-events
@@ -425,3 +425,8 @@ disable qemu_put_ram_ptr(void* addr) "%p"
# hw/xen_platform.c
disable xen_platform_log(char *s) "xen platform: %s"
+
+# qemu-coroutine.c
+disable qemu_coroutine_enter(void *from, void *to, void *opaque) "from %p to %p opaque %p"
+disable qemu_coroutine_yield(void *from, void *to) "from %p to %p"
+disable qemu_coroutine_terminate(void *co) "self %p"
--
1.7.5.4
^ permalink raw reply related [flat|nested] 14+ messages in thread* [Qemu-devel] [PATCH v8 3/5] coroutine: add ucontext and win32 implementations
2011-07-26 9:21 [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming Stefan Hajnoczi
2011-07-26 9:21 ` [Qemu-devel] [PATCH v8 1/5] coroutine: add gthread dependency Stefan Hajnoczi
2011-07-26 9:21 ` [Qemu-devel] [PATCH v8 2/5] coroutine: introduce coroutines API Stefan Hajnoczi
@ 2011-07-26 9:21 ` Stefan Hajnoczi
2011-07-26 9:21 ` [Qemu-devel] [PATCH v8 4/5] coroutine: add test-coroutine automated tests Stefan Hajnoczi
` (4 subsequent siblings)
7 siblings, 0 replies; 14+ messages in thread
From: Stefan Hajnoczi @ 2011-07-26 9:21 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, Blue Swirl, Anthony Liguori, Venkateswararao Jujjuri,
Stefan Hajnoczi
From: Kevin Wolf <kwolf@redhat.com>
This coroutines implementation is based on the gtk-vnc implementation written
by Anthony Liguori <anthony@codemonkey.ws> but it has been significantly
rewritten by Kevin Wolf <kwolf@redhat.com> to use setjmp()/longjmp() instead of
the more expensive swapcontext() and by Paolo Bonzini <pbonzini@redhat.com> for
Windows Fibers support.
Darwin has makecontext(3) but getcontext(3) is stubbed out to return
ENOTSUP. Andreas Färber <andreas.faerber@web.de> debugged this and
contributed the ./configure test which solves the issue for Darwin/ppc64
(and ppc) v10.5.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
---
Makefile.objs | 8 ++-
configure | 18 ++++
coroutine-ucontext.c | 230 ++++++++++++++++++++++++++++++++++++++++++++++++++
coroutine-win32.c | 91 ++++++++++++++++++++
4 files changed, 346 insertions(+), 1 deletions(-)
create mode 100644 coroutine-ucontext.c
create mode 100644 coroutine-win32.c
diff --git a/Makefile.objs b/Makefile.objs
index db907c4..5679e1f 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -12,7 +12,13 @@ oslib-obj-$(CONFIG_POSIX) += oslib-posix.o qemu-thread-posix.o
#######################################################################
# coroutines
-coroutine-obj-y = qemu-coroutine.o coroutine-gthread.o
+coroutine-obj-y = qemu-coroutine.o
+ifeq ($(CONFIG_UCONTEXT_COROUTINE),y)
+coroutine-obj-$(CONFIG_POSIX) += coroutine-ucontext.o
+else
+coroutine-obj-$(CONFIG_POSIX) += coroutine-gthread.o
+endif
+coroutine-obj-$(CONFIG_WIN32) += coroutine-win32.o
#######################################################################
# block-obj-y is code used by both qemu system emulation and qemu-img
diff --git a/configure b/configure
index 5150372..a4d7921 100755
--- a/configure
+++ b/configure
@@ -2499,6 +2499,20 @@ if test "$trace_backend" = "dtrace"; then
fi
##########################################
+# check if we have makecontext
+
+ucontext_coroutine=no
+if test "$darwin" != "yes"; then
+ cat > $TMPC << EOF
+#include <ucontext.h>
+int main(void) { makecontext(0, 0, 0); }
+EOF
+ if compile_prog "" "" ; then
+ ucontext_coroutine=yes
+ fi
+fi
+
+##########################################
# End of CC checks
# After here, no more $cc or $ld runs
@@ -2970,6 +2984,10 @@ if test "$rbd" = "yes" ; then
echo "CONFIG_RBD=y" >> $config_host_mak
fi
+if test "$ucontext_coroutine" = "yes" ; then
+ echo "CONFIG_UCONTEXT_COROUTINE=y" >> $config_host_mak
+fi
+
# USB host support
case "$usb" in
linux)
diff --git a/coroutine-ucontext.c b/coroutine-ucontext.c
new file mode 100644
index 0000000..41c2379
--- /dev/null
+++ b/coroutine-ucontext.c
@@ -0,0 +1,230 @@
+/*
+ * ucontext coroutine initialization code
+ *
+ * Copyright (C) 2006 Anthony Liguori <anthony@codemonkey.ws>
+ * Copyright (C) 2011 Kevin Wolf <kwolf@redhat.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.0 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+/* XXX Is there a nicer way to disable glibc's stack check for longjmp? */
+#ifdef _FORTIFY_SOURCE
+#undef _FORTIFY_SOURCE
+#endif
+#include <stdlib.h>
+#include <setjmp.h>
+#include <stdint.h>
+#include <pthread.h>
+#include <ucontext.h>
+#include "qemu-common.h"
+#include "qemu-coroutine-int.h"
+
+enum {
+ /* Maximum free pool size prevents holding too many freed coroutines */
+ POOL_MAX_SIZE = 64,
+};
+
+typedef struct {
+ Coroutine base;
+ void *stack;
+ jmp_buf env;
+} CoroutineUContext;
+
+/**
+ * Per-thread coroutine bookkeeping
+ */
+typedef struct {
+ /** Currently executing coroutine */
+ Coroutine *current;
+
+ /** Free list to speed up creation */
+ QLIST_HEAD(, Coroutine) pool;
+ unsigned int pool_size;
+
+ /** The default coroutine */
+ CoroutineUContext leader;
+} CoroutineThreadState;
+
+static pthread_key_t thread_state_key;
+
+/*
+ * va_args to makecontext() must be type 'int', so passing
+ * the pointer we need may require several int args. This
+ * union is a quick hack to let us do that
+ */
+union cc_arg {
+ void *p;
+ int i[2];
+};
+
+static CoroutineThreadState *coroutine_get_thread_state(void)
+{
+ CoroutineThreadState *s = pthread_getspecific(thread_state_key);
+
+ if (!s) {
+ s = qemu_mallocz(sizeof(*s));
+ s->current = &s->leader.base;
+ QLIST_INIT(&s->pool);
+ pthread_setspecific(thread_state_key, s);
+ }
+ return s;
+}
+
+static void qemu_coroutine_thread_cleanup(void *opaque)
+{
+ CoroutineThreadState *s = opaque;
+ Coroutine *co;
+ Coroutine *tmp;
+
+ QLIST_FOREACH_SAFE(co, &s->pool, pool_next, tmp) {
+ qemu_free(DO_UPCAST(CoroutineUContext, base, co)->stack);
+ qemu_free(co);
+ }
+ qemu_free(s);
+}
+
+static void __attribute__((constructor)) coroutine_init(void)
+{
+ int ret;
+
+ ret = pthread_key_create(&thread_state_key, qemu_coroutine_thread_cleanup);
+ if (ret != 0) {
+ fprintf(stderr, "unable to create leader key: %s\n", strerror(errno));
+ abort();
+ }
+}
+
+static void coroutine_trampoline(int i0, int i1)
+{
+ union cc_arg arg;
+ CoroutineUContext *self;
+ Coroutine *co;
+
+ arg.i[0] = i0;
+ arg.i[1] = i1;
+ self = arg.p;
+ co = &self->base;
+
+ /* Initialize longjmp environment and switch back the caller */
+ if (!setjmp(self->env)) {
+ longjmp(*(jmp_buf *)co->entry_arg, 1);
+ }
+
+ while (true) {
+ co->entry(co->entry_arg);
+ qemu_coroutine_switch(co, co->caller, COROUTINE_TERMINATE);
+ }
+}
+
+static Coroutine *coroutine_new(void)
+{
+ const size_t stack_size = 1 << 20;
+ CoroutineUContext *co;
+ ucontext_t old_uc, uc;
+ jmp_buf old_env;
+ union cc_arg arg;
+
+ /* The ucontext functions preserve signal masks which incurs a system call
+ * overhead. setjmp()/longjmp() does not preserve signal masks but only
+ * works on the current stack. Since we need a way to create and switch to
+ * a new stack, use the ucontext functions for that but setjmp()/longjmp()
+ * for everything else.
+ */
+
+ if (getcontext(&uc) == -1) {
+ abort();
+ }
+
+ co = qemu_mallocz(sizeof(*co));
+ co->stack = qemu_malloc(stack_size);
+ co->base.entry_arg = &old_env; /* stash away our jmp_buf */
+
+ uc.uc_link = &old_uc;
+ uc.uc_stack.ss_sp = co->stack;
+ uc.uc_stack.ss_size = stack_size;
+ uc.uc_stack.ss_flags = 0;
+
+ arg.p = co;
+
+ makecontext(&uc, (void (*)(void))coroutine_trampoline,
+ 2, arg.i[0], arg.i[1]);
+
+ /* swapcontext() in, longjmp() back out */
+ if (!setjmp(old_env)) {
+ swapcontext(&old_uc, &uc);
+ }
+ return &co->base;
+}
+
+Coroutine *qemu_coroutine_new(void)
+{
+ CoroutineThreadState *s = coroutine_get_thread_state();
+ Coroutine *co;
+
+ co = QLIST_FIRST(&s->pool);
+ if (co) {
+ QLIST_REMOVE(co, pool_next);
+ s->pool_size--;
+ } else {
+ co = coroutine_new();
+ }
+ return co;
+}
+
+void qemu_coroutine_delete(Coroutine *co_)
+{
+ CoroutineThreadState *s = coroutine_get_thread_state();
+ CoroutineUContext *co = DO_UPCAST(CoroutineUContext, base, co_);
+
+ if (s->pool_size < POOL_MAX_SIZE) {
+ QLIST_INSERT_HEAD(&s->pool, &co->base, pool_next);
+ co->base.caller = NULL;
+ s->pool_size++;
+ return;
+ }
+
+ qemu_free(co->stack);
+ qemu_free(co);
+}
+
+CoroutineAction qemu_coroutine_switch(Coroutine *from_, Coroutine *to_,
+ CoroutineAction action)
+{
+ CoroutineUContext *from = DO_UPCAST(CoroutineUContext, base, from_);
+ CoroutineUContext *to = DO_UPCAST(CoroutineUContext, base, to_);
+ CoroutineThreadState *s = coroutine_get_thread_state();
+ int ret;
+
+ s->current = to_;
+
+ ret = setjmp(from->env);
+ if (ret == 0) {
+ longjmp(to->env, action);
+ }
+ return ret;
+}
+
+Coroutine *qemu_coroutine_self(void)
+{
+ CoroutineThreadState *s = coroutine_get_thread_state();
+
+ return s->current;
+}
+
+bool qemu_in_coroutine(void)
+{
+ CoroutineThreadState *s = pthread_getspecific(thread_state_key);
+
+ return s && s->current->caller;
+}
diff --git a/coroutine-win32.c b/coroutine-win32.c
new file mode 100644
index 0000000..a13d1d1
--- /dev/null
+++ b/coroutine-win32.c
@@ -0,0 +1,91 @@
+/*
+ * Win32 coroutine initialization code
+ *
+ * Copyright (c) 2011 Kevin Wolf <kwolf@redhat.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu-common.h"
+#include "qemu-coroutine-int.h"
+
+typedef struct {
+ Coroutine base;
+
+ LPVOID fiber;
+ CoroutineAction action;
+} CoroutineWin32;
+
+static __thread CoroutineWin32 leader;
+static __thread Coroutine *current;
+
+CoroutineAction qemu_coroutine_switch(Coroutine *from_, Coroutine *to_,
+ CoroutineAction action)
+{
+ CoroutineWin32 *from = DO_UPCAST(CoroutineWin32, base, from_);
+ CoroutineWin32 *to = DO_UPCAST(CoroutineWin32, base, to_);
+
+ current = to_;
+
+ to->action = action;
+ SwitchToFiber(to->fiber);
+ return from->action;
+}
+
+static void CALLBACK coroutine_trampoline(void *co_)
+{
+ Coroutine *co = co_;
+
+ while (true) {
+ co->entry(co->entry_arg);
+ qemu_coroutine_switch(co, co->caller, COROUTINE_TERMINATE);
+ }
+}
+
+Coroutine *qemu_coroutine_new(void)
+{
+ const size_t stack_size = 1 << 20;
+ CoroutineWin32 *co;
+
+ co = qemu_mallocz(sizeof(*co));
+ co->fiber = CreateFiber(stack_size, coroutine_trampoline, &co->base);
+ return &co->base;
+}
+
+void qemu_coroutine_delete(Coroutine *co_)
+{
+ CoroutineWin32 *co = DO_UPCAST(CoroutineWin32, base, co_);
+
+ DeleteFiber(co->fiber);
+ qemu_free(co);
+}
+
+Coroutine *qemu_coroutine_self(void)
+{
+ if (!current) {
+ current = &leader.base;
+ leader.fiber = ConvertThreadToFiber(NULL);
+ }
+ return current;
+}
+
+bool qemu_in_coroutine(void)
+{
+ return current && current->caller;
+}
--
1.7.5.4
^ permalink raw reply related [flat|nested] 14+ messages in thread* [Qemu-devel] [PATCH v8 4/5] coroutine: add test-coroutine automated tests
2011-07-26 9:21 [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming Stefan Hajnoczi
` (2 preceding siblings ...)
2011-07-26 9:21 ` [Qemu-devel] [PATCH v8 3/5] coroutine: add ucontext and win32 implementations Stefan Hajnoczi
@ 2011-07-26 9:21 ` Stefan Hajnoczi
2011-08-02 13:43 ` Kevin Wolf
2011-07-26 9:21 ` [Qemu-devel] [PATCH v8 5/5] coroutine: add test-coroutine --benchmark-lifecycle Stefan Hajnoczi
` (3 subsequent siblings)
7 siblings, 1 reply; 14+ messages in thread
From: Stefan Hajnoczi @ 2011-07-26 9:21 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, Blue Swirl, Anthony Liguori, Venkateswararao Jujjuri,
Stefan Hajnoczi
To run automated tests for coroutines:
make test-coroutine
./test-coroutine
On success the program terminates with exit status 0. On failure an
error message is written to stderr and the program exits with exit
status 1.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
---
.gitignore | 1 +
Makefile | 3 +-
test-coroutine.c | 162 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 165 insertions(+), 1 deletions(-)
create mode 100644 test-coroutine.c
diff --git a/.gitignore b/.gitignore
index 54835bc..59c343c 100644
--- a/.gitignore
+++ b/.gitignore
@@ -36,6 +36,7 @@ qemu-io
qemu-ga
qemu-monitor.texi
QMP/qmp-commands.txt
+test-coroutine
.gdbinit
*.a
*.aux
diff --git a/Makefile b/Makefile
index f3a03ad..ea8c0ac 100644
--- a/Makefile
+++ b/Makefile
@@ -151,7 +151,7 @@ qemu-io$(EXESUF): qemu-io.o cmd.o qemu-tool.o qemu-error.o $(oslib-obj-y) $(trac
qemu-img-cmds.h: $(SRC_PATH)/qemu-img-cmds.hx
$(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -h < $< > $@," GEN $@")
-check-qint.o check-qstring.o check-qdict.o check-qlist.o check-qfloat.o check-qjson.o: $(GENERATED_HEADERS)
+check-qint.o check-qstring.o check-qdict.o check-qlist.o check-qfloat.o check-qjson.o test-coroutine.o: $(GENERATED_HEADERS)
CHECK_PROG_DEPS = qemu-malloc.o $(oslib-obj-y) $(trace-obj-y) qemu-tool.o
@@ -161,6 +161,7 @@ check-qdict: check-qdict.o qdict.o qfloat.o qint.o qstring.o qbool.o qlist.o $(C
check-qlist: check-qlist.o qlist.o qint.o $(CHECK_PROG_DEPS)
check-qfloat: check-qfloat.o qfloat.o $(CHECK_PROG_DEPS)
check-qjson: check-qjson.o qfloat.o qint.o qdict.o qstring.o qlist.o qbool.o qjson.o json-streamer.o json-lexer.o json-parser.o error.o qerror.o qemu-error.o $(CHECK_PROG_DEPS)
+test-coroutine: test-coroutine.o qemu-timer-common.o $(coroutine-obj-y) $(CHECK_PROG_DEPS)
$(qapi-obj-y): $(GENERATED_HEADERS)
qapi-dir := qapi-generated
diff --git a/test-coroutine.c b/test-coroutine.c
new file mode 100644
index 0000000..9e9d3c9
--- /dev/null
+++ b/test-coroutine.c
@@ -0,0 +1,162 @@
+/*
+ * Coroutine tests
+ *
+ * Copyright IBM, Corp. 2011
+ *
+ * Authors:
+ * Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#include <glib.h>
+#include "qemu-coroutine.h"
+
+/*
+ * Check that qemu_in_coroutine() works
+ */
+
+static void coroutine_fn verify_in_coroutine(void *opaque)
+{
+ g_assert(qemu_in_coroutine());
+}
+
+static void test_in_coroutine(void)
+{
+ Coroutine *coroutine;
+
+ g_assert(!qemu_in_coroutine());
+
+ coroutine = qemu_coroutine_create(verify_in_coroutine);
+ qemu_coroutine_enter(coroutine, NULL);
+}
+
+/*
+ * Check that qemu_coroutine_self() works
+ */
+
+static void coroutine_fn verify_self(void *opaque)
+{
+ g_assert(qemu_coroutine_self() == opaque);
+}
+
+static void test_self(void)
+{
+ Coroutine *coroutine;
+
+ coroutine = qemu_coroutine_create(verify_self);
+ qemu_coroutine_enter(coroutine, coroutine);
+}
+
+/*
+ * Check that coroutines may nest multiple levels
+ */
+
+typedef struct {
+ unsigned int n_enter; /* num coroutines entered */
+ unsigned int n_return; /* num coroutines returned */
+ unsigned int max; /* maximum level of nesting */
+} NestData;
+
+static void coroutine_fn nest(void *opaque)
+{
+ NestData *nd = opaque;
+
+ nd->n_enter++;
+
+ if (nd->n_enter < nd->max) {
+ Coroutine *child;
+
+ child = qemu_coroutine_create(nest);
+ qemu_coroutine_enter(child, nd);
+ }
+
+ nd->n_return++;
+}
+
+static void test_nesting(void)
+{
+ Coroutine *root;
+ NestData nd = {
+ .n_enter = 0,
+ .n_return = 0,
+ .max = 128,
+ };
+
+ root = qemu_coroutine_create(nest);
+ qemu_coroutine_enter(root, &nd);
+
+ /* Must enter and return from max nesting level */
+ g_assert_cmpint(nd.n_enter, ==, nd.max);
+ g_assert_cmpint(nd.n_return, ==, nd.max);
+}
+
+/*
+ * Check that yield/enter transfer control correctly
+ */
+
+static void coroutine_fn yield_5_times(void *opaque)
+{
+ bool *done = opaque;
+ int i;
+
+ for (i = 0; i < 5; i++) {
+ qemu_coroutine_yield();
+ }
+ *done = true;
+}
+
+static void test_yield(void)
+{
+ Coroutine *coroutine;
+ bool done = false;
+ int i = -1; /* one extra time to return from coroutine */
+
+ coroutine = qemu_coroutine_create(yield_5_times);
+ while (!done) {
+ qemu_coroutine_enter(coroutine, &done);
+ i++;
+ }
+ g_assert_cmpint(i, ==, 5); /* coroutine must yield 5 times */
+}
+
+/*
+ * Check that creation, enter, and return work
+ */
+
+static void coroutine_fn set_and_exit(void *opaque)
+{
+ bool *done = opaque;
+
+ *done = true;
+}
+
+static void test_lifecycle(void)
+{
+ Coroutine *coroutine;
+ bool done = false;
+
+ /* Create, enter, and return from coroutine */
+ coroutine = qemu_coroutine_create(set_and_exit);
+ qemu_coroutine_enter(coroutine, &done);
+ g_assert(done); /* expect done to be true (first time) */
+
+ /* Repeat to check that no state affects this test */
+ done = false;
+ coroutine = qemu_coroutine_create(set_and_exit);
+ qemu_coroutine_enter(coroutine, &done);
+ g_assert(done); /* expect done to be true (second time) */
+}
+
+int main(int argc, char **argv)
+{
+ g_test_init(&argc, &argv, NULL);
+ g_test_add_func("/basic/lifecycle", test_lifecycle);
+ g_test_add_func("/basic/yield", test_yield);
+ g_test_add_func("/basic/nesting", test_nesting);
+ g_test_add_func("/basic/self", test_self);
+ g_test_add_func("/basic/in_coroutine", test_in_coroutine);
+ return g_test_run();
+}
--
1.7.5.4
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [Qemu-devel] [PATCH v8 4/5] coroutine: add test-coroutine automated tests
2011-07-26 9:21 ` [Qemu-devel] [PATCH v8 4/5] coroutine: add test-coroutine automated tests Stefan Hajnoczi
@ 2011-08-02 13:43 ` Kevin Wolf
0 siblings, 0 replies; 14+ messages in thread
From: Kevin Wolf @ 2011-08-02 13:43 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: Blue Swirl, Anthony Liguori, Venkateswararao Jujjuri, qemu-devel
Am 26.07.2011 11:21, schrieb Stefan Hajnoczi:
> To run automated tests for coroutines:
>
> make test-coroutine
> ./test-coroutine
>
> On success the program terminates with exit status 0. On failure an
> error message is written to stderr and the program exits with exit
> status 1.
>
> Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Adding this build fix in the block branch:
diff --git a/Makefile b/Makefile
index fdd620c..2becedc 100644
--- a/Makefile
+++ b/Makefile
@@ -161,7 +161,7 @@ check-qdict: check-qdict.o qdict.o qfloat.o qint.o
qstring.o qbool.o qlist.o $(C
check-qlist: check-qlist.o qlist.o qint.o $(CHECK_PROG_DEPS)
check-qfloat: check-qfloat.o qfloat.o $(CHECK_PROG_DEPS)
check-qjson: check-qjson.o qfloat.o qint.o qdict.o qstring.o qlist.o
qbool.o qjson.o json-streamer.o json-lexer.o json-parser.o error.o
qerror.o qemu-error.o $(CHECK_PROG_DEPS)
-test-coroutine: test-coroutine.o qemu-timer-common.o $(coroutine-obj-y)
$(CHECK_PROG_DEPS)
+test-coroutine: test-coroutine.o qemu-timer-common.o async.o
$(coroutine-obj-y) $(CHECK_PROG_DEPS)
$(qapi-obj-y): $(GENERATED_HEADERS)
qapi-dir := qapi-generated
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [Qemu-devel] [PATCH v8 5/5] coroutine: add test-coroutine --benchmark-lifecycle
2011-07-26 9:21 [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming Stefan Hajnoczi
` (3 preceding siblings ...)
2011-07-26 9:21 ` [Qemu-devel] [PATCH v8 4/5] coroutine: add test-coroutine automated tests Stefan Hajnoczi
@ 2011-07-26 9:21 ` Stefan Hajnoczi
2011-07-26 9:23 ` [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming Stefan Hajnoczi
` (2 subsequent siblings)
7 siblings, 0 replies; 14+ messages in thread
From: Stefan Hajnoczi @ 2011-07-26 9:21 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, Blue Swirl, Anthony Liguori, Venkateswararao Jujjuri,
Stefan Hajnoczi
Add a microbenchmark for coroutine create, enter, and return (aka
lifecycle). This is a useful benchmark because users are expected to
create many coroutines, one per I/O request for example, and we
therefore need to provide good performance in that scenario.
To run:
make test-coroutine
./test-coroutine --benchmark-lifecycle 20000000
This will do 20,000,000 coroutine create, enter, return iterations and
print the resulting time.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
---
test-coroutine.c | 30 ++++++++++++++++++++++++++++++
1 files changed, 30 insertions(+), 0 deletions(-)
diff --git a/test-coroutine.c b/test-coroutine.c
index 9e9d3c9..bf9f3e9 100644
--- a/test-coroutine.c
+++ b/test-coroutine.c
@@ -150,6 +150,33 @@ static void test_lifecycle(void)
g_assert(done); /* expect done to be true (second time) */
}
+/*
+ * Lifecycle benchmark
+ */
+
+static void coroutine_fn empty_coroutine(void *opaque)
+{
+ /* Do nothing */
+}
+
+static void perf_lifecycle(void)
+{
+ Coroutine *coroutine;
+ unsigned int i, max;
+ double duration;
+
+ max = 1000000;
+
+ g_test_timer_start();
+ for (i = 0; i < max; i++) {
+ coroutine = qemu_coroutine_create(empty_coroutine);
+ qemu_coroutine_enter(coroutine, NULL);
+ }
+ duration = g_test_timer_elapsed();
+
+ g_test_message("Lifecycle %u iterations: %f s\n", max, duration);
+}
+
int main(int argc, char **argv)
{
g_test_init(&argc, &argv, NULL);
@@ -158,5 +185,8 @@ int main(int argc, char **argv)
g_test_add_func("/basic/nesting", test_nesting);
g_test_add_func("/basic/self", test_self);
g_test_add_func("/basic/in_coroutine", test_in_coroutine);
+ if (g_test_perf()) {
+ g_test_add_func("/perf/lifecycle", perf_lifecycle);
+ }
return g_test_run();
}
--
1.7.5.4
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming
2011-07-26 9:21 [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming Stefan Hajnoczi
` (4 preceding siblings ...)
2011-07-26 9:21 ` [Qemu-devel] [PATCH v8 5/5] coroutine: add test-coroutine --benchmark-lifecycle Stefan Hajnoczi
@ 2011-07-26 9:23 ` Stefan Hajnoczi
2011-07-27 9:45 ` Aneesh Kumar K.V
2011-07-29 12:54 ` Stefan Hajnoczi
7 siblings, 0 replies; 14+ messages in thread
From: Stefan Hajnoczi @ 2011-07-26 9:23 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: Kevin Wolf, Blue Swirl, Anthony Liguori, Venkateswararao Jujjuri,
qemu-devel
On Tue, Jul 26, 2011 at 10:21 AM, Stefan Hajnoczi
<stefanha@linux.vnet.ibm.com> wrote:
> QEMU is event-driven and suffers when blocking operations are performed because
> VM execution may be stopped until the operation completes. Therefore many
> operations that could block are performed asynchronously and a callback is
> invoked when the operation has completed. This allows QEMU to continue
> executing while the operation is pending.
>
> The downside to callbacks is that they split up code into many smaller
> functions, each of which is a single step in a state machine that quickly
> becomes complex and hard to understand. Callback functions also result in lots
> of noise as variables are packed and unpacked into temporary structs that pass
> state to the callback function.
>
> This patch series introduces coroutines as a solution for writing asynchronous
> code while still having a nice sequential control flow. The semantics are
> explained in the second patch. The fourth patch adds automated tests.
>
> A nice feature of coroutines is that it is relatively easy to take synchronous
> code and lift it into a coroutine to make it asynchronous. Work has been done
> to move qcow2 request processing into coroutines and thereby make it
> asynchronous (today qcow2 will perform synchronous metadata accesses). This
> qcow2 work is still ongoing and not quite ready for mainline yet.
>
> v8:
> * Bisectability: introduce gthread implementation before ucontext/fibers
Forgot to mention:
* Depend on gthread, not just glib, for multithreaded programming
Stefan
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming
2011-07-26 9:21 [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming Stefan Hajnoczi
` (5 preceding siblings ...)
2011-07-26 9:23 ` [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming Stefan Hajnoczi
@ 2011-07-27 9:45 ` Aneesh Kumar K.V
2011-07-27 10:03 ` Kevin Wolf
2011-07-27 11:34 ` Aneesh Kumar K.V
2011-07-29 12:54 ` Stefan Hajnoczi
7 siblings, 2 replies; 14+ messages in thread
From: Aneesh Kumar K.V @ 2011-07-27 9:45 UTC (permalink / raw)
To: Stefan Hajnoczi, qemu-devel
Cc: Kevin Wolf, Blue Swirl, Anthony Liguori, Venkateswararao Jujjuri
On Tue, 26 Jul 2011 10:21:12 +0100, Stefan Hajnoczi <stefanha@linux.vnet.ibm.com> wrote:
> QEMU is event-driven and suffers when blocking operations are performed because
> VM execution may be stopped until the operation completes. Therefore many
> operations that could block are performed asynchronously and a callback is
> invoked when the operation has completed. This allows QEMU to continue
> executing while the operation is pending.
>
> The downside to callbacks is that they split up code into many smaller
> functions, each of which is a single step in a state machine that quickly
> becomes complex and hard to understand. Callback functions also result in lots
> of noise as variables are packed and unpacked into temporary structs that pass
> state to the callback function.
>
> This patch series introduces coroutines as a solution for writing asynchronous
> code while still having a nice sequential control flow. The semantics are
> explained in the second patch. The fourth patch adds automated tests.
>
> A nice feature of coroutines is that it is relatively easy to take synchronous
> code and lift it into a coroutine to make it asynchronous. Work has been done
> to move qcow2 request processing into coroutines and thereby make it
> asynchronous (today qcow2 will perform synchronous metadata accesses). This
> qcow2 work is still ongoing and not quite ready for mainline yet.
>
> v8:
> * Bisectability: introduce gthread implementation before ucontext/fibers
Can we also get CoMutex and CoQueue patches also merged. I use them in
the VirtFS series.
http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/coroutine-devel
http://repo.or.cz/w/qemu/v9fs.git/commit/318ef0b9b01cd296f8c30d8288139b9bed859892
-aneesh
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming
2011-07-27 9:45 ` Aneesh Kumar K.V
@ 2011-07-27 10:03 ` Kevin Wolf
2011-07-27 11:39 ` Aneesh Kumar K.V
2011-07-27 11:34 ` Aneesh Kumar K.V
1 sibling, 1 reply; 14+ messages in thread
From: Kevin Wolf @ 2011-07-27 10:03 UTC (permalink / raw)
To: Aneesh Kumar K.V
Cc: Blue Swirl, Anthony Liguori, Venkateswararao Jujjuri,
Stefan Hajnoczi, qemu-devel
Am 27.07.2011 11:45, schrieb Aneesh Kumar K.V:
> On Tue, 26 Jul 2011 10:21:12 +0100, Stefan Hajnoczi <stefanha@linux.vnet.ibm.com> wrote:
>> QEMU is event-driven and suffers when blocking operations are performed because
>> VM execution may be stopped until the operation completes. Therefore many
>> operations that could block are performed asynchronously and a callback is
>> invoked when the operation has completed. This allows QEMU to continue
>> executing while the operation is pending.
>>
>> The downside to callbacks is that they split up code into many smaller
>> functions, each of which is a single step in a state machine that quickly
>> becomes complex and hard to understand. Callback functions also result in lots
>> of noise as variables are packed and unpacked into temporary structs that pass
>> state to the callback function.
>>
>> This patch series introduces coroutines as a solution for writing asynchronous
>> code while still having a nice sequential control flow. The semantics are
>> explained in the second patch. The fourth patch adds automated tests.
>>
>> A nice feature of coroutines is that it is relatively easy to take synchronous
>> code and lift it into a coroutine to make it asynchronous. Work has been done
>> to move qcow2 request processing into coroutines and thereby make it
>> asynchronous (today qcow2 will perform synchronous metadata accesses). This
>> qcow2 work is still ongoing and not quite ready for mainline yet.
>>
>> v8:
>> * Bisectability: introduce gthread implementation before ucontext/fibers
>
> Can we also get CoMutex and CoQueue patches also merged. I use them in
> the VirtFS series.
>
> http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/coroutine-devel
> http://repo.or.cz/w/qemu/v9fs.git/commit/318ef0b9b01cd296f8c30d8288139b9bed859892
I introduce these in my block coroutine patches. I posted a RFC last
week and the first "real" patch series yesterday, so I hope they get
review and can be merged into master soon.
Kevin
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming
2011-07-27 10:03 ` Kevin Wolf
@ 2011-07-27 11:39 ` Aneesh Kumar K.V
2011-07-27 12:14 ` Kevin Wolf
0 siblings, 1 reply; 14+ messages in thread
From: Aneesh Kumar K.V @ 2011-07-27 11:39 UTC (permalink / raw)
To: Kevin Wolf
Cc: Blue Swirl, Anthony Liguori, Venkateswararao Jujjuri,
Stefan Hajnoczi, qemu-devel
On Wed, 27 Jul 2011 12:03:57 +0200, Kevin Wolf <kwolf@redhat.com> wrote:
> Am 27.07.2011 11:45, schrieb Aneesh Kumar K.V:
> > On Tue, 26 Jul 2011 10:21:12 +0100, Stefan Hajnoczi <stefanha@linux.vnet.ibm.com> wrote:
> >> QEMU is event-driven and suffers when blocking operations are performed because
> >> VM execution may be stopped until the operation completes. Therefore many
> >> operations that could block are performed asynchronously and a callback is
> >> invoked when the operation has completed. This allows QEMU to continue
> >> executing while the operation is pending.
> >>
> >> The downside to callbacks is that they split up code into many smaller
> >> functions, each of which is a single step in a state machine that quickly
> >> becomes complex and hard to understand. Callback functions also result in lots
> >> of noise as variables are packed and unpacked into temporary structs that pass
> >> state to the callback function.
> >>
> >> This patch series introduces coroutines as a solution for writing asynchronous
> >> code while still having a nice sequential control flow. The semantics are
> >> explained in the second patch. The fourth patch adds automated tests.
> >>
> >> A nice feature of coroutines is that it is relatively easy to take synchronous
> >> code and lift it into a coroutine to make it asynchronous. Work has been done
> >> to move qcow2 request processing into coroutines and thereby make it
> >> asynchronous (today qcow2 will perform synchronous metadata accesses). This
> >> qcow2 work is still ongoing and not quite ready for mainline yet.
> >>
> >> v8:
> >> * Bisectability: introduce gthread implementation before ucontext/fibers
> >
> > Can we also get CoMutex and CoQueue patches also merged. I use them in
> > the VirtFS series.
> >
> > http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/coroutine-devel
> > http://repo.or.cz/w/qemu/v9fs.git/commit/318ef0b9b01cd296f8c30d8288139b9bed859892
>
> I introduce these in my block coroutine patches. I posted a RFC last
> week and the first "real" patch series yesterday, so I hope they get
> review and can be merged into master soon.
>
Can you review the patch that add CoRWlock ?
http://article.gmane.org/gmane.comp.emulators.qemu/105402
Message-id:1307382497-3737-2-git-send-email-aneesh.kumar@linux.vnet.ibm.com
commit 8c787d8b81aca1f4f7be45adb67b9e1a6dde7f1f
Author: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Date: Tue May 24 22:14:04 2011 +0530
coroutine: Add CoRwlock support
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
diff --git a/qemu-coroutine-lock.c b/qemu-coroutine-lock.c
index 5071fb8..5ecaa94 100644
--- a/qemu-coroutine-lock.c
+++ b/qemu-coroutine-lock.c
@@ -115,3 +115,47 @@ void qemu_co_mutex_unlock(CoMutex *mutex)
trace_qemu_co_mutex_unlock_return(mutex, self);
}
+
+void qemu_co_rwlock_init(CoRwlock *lock)
+{
+ memset(lock, 0, sizeof(*lock));
+ qemu_co_queue_init(&lock->queue);
+}
+
+void qemu_co_rwlock_rdlock(CoRwlock *lock)
+{
+ while (lock->writer) {
+ qemu_co_queue_wait(&lock->queue);
+ }
+ lock->reader++;
+}
+
+void qemu_co_rwlock_unlock(CoRwlock *lock)
+{
+ assert(qemu_in_coroutine());
+ if (lock->writer) {
+ lock->writer--;;
+ assert(lock->writer == 0);
+ while (!qemu_co_queue_empty(&lock->queue)) {
+ /*
+ * Wakeup every body. This will include some
+ * writers too.
+ */
+ qemu_co_queue_next(&lock->queue);
+ }
+ } else {
+ lock->reader--;
+ assert(lock->reader >= 0);
+ /* Wakeup only one waiting writer */
+ qemu_co_queue_next(&lock->queue);
+ }
+}
+
+void qemu_co_rwlock_wrlock(CoRwlock *lock)
+{
+ while (lock->writer || lock->reader) {
+ qemu_co_queue_wait(&lock->queue);
+ }
+ lock->writer++;
+ assert(lock->writer == 1);
+}
diff --git a/qemu-coroutine.h b/qemu-coroutine.h
index 71cfa5a..a9735fb 100644
--- a/qemu-coroutine.h
+++ b/qemu-coroutine.h
@@ -17,6 +17,7 @@
#include <stdbool.h>
#include "qemu-queue.h"
+#include "qemu-thread.h"
/**
* Coroutines are a mechanism for stack switching and can be used for
@@ -114,4 +115,15 @@ void qemu_co_mutex_init(CoMutex *mutex);
void qemu_co_mutex_lock(CoMutex *mutex);
void qemu_co_mutex_unlock(CoMutex *mutex);
+typedef struct CoRwlock {
+ bool writer;
+ int reader;
+ CoQueue queue;
+} CoRwlock;
+
+void qemu_co_rwlock_init(CoRwlock *lock);
+void qemu_co_rwlock_rdlock(CoRwlock *lock);
+void qemu_co_rwlock_wrlock(CoRwlock *lock);
+void qemu_co_rwlock_unlock(CoRwlock *lock);
+
#endif /* QEMU_COROUTINE_H */
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming
2011-07-27 11:39 ` Aneesh Kumar K.V
@ 2011-07-27 12:14 ` Kevin Wolf
0 siblings, 0 replies; 14+ messages in thread
From: Kevin Wolf @ 2011-07-27 12:14 UTC (permalink / raw)
To: Aneesh Kumar K.V
Cc: Blue Swirl, Anthony Liguori, Venkateswararao Jujjuri,
Stefan Hajnoczi, qemu-devel
Am 27.07.2011 13:39, schrieb Aneesh Kumar K.V:
> Can you review the patch that add CoRWlock ?
>
> http://article.gmane.org/gmane.comp.emulators.qemu/105402
> Message-id:1307382497-3737-2-git-send-email-aneesh.kumar@linux.vnet.ibm.com
>
> commit 8c787d8b81aca1f4f7be45adb67b9e1a6dde7f1f
> Author: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> Date: Tue May 24 22:14:04 2011 +0530
>
> coroutine: Add CoRwlock support
>
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Nice! I think I'll need this, too.
> diff --git a/qemu-coroutine-lock.c b/qemu-coroutine-lock.c
> index 5071fb8..5ecaa94 100644
> --- a/qemu-coroutine-lock.c
> +++ b/qemu-coroutine-lock.c
> @@ -115,3 +115,47 @@ void qemu_co_mutex_unlock(CoMutex *mutex)
>
> trace_qemu_co_mutex_unlock_return(mutex, self);
> }
> +
> +void qemu_co_rwlock_init(CoRwlock *lock)
> +{
> + memset(lock, 0, sizeof(*lock));
> + qemu_co_queue_init(&lock->queue);
> +}
> +
> +void qemu_co_rwlock_rdlock(CoRwlock *lock)
> +{
> + while (lock->writer) {
> + qemu_co_queue_wait(&lock->queue);
> + }
> + lock->reader++;
> +}
> +
> +void qemu_co_rwlock_unlock(CoRwlock *lock)
> +{
> + assert(qemu_in_coroutine());
> + if (lock->writer) {
> + lock->writer--;;
Please don't do arithmetics on bools, just say lock->write = false;
Also there's a double semicolon.
> + assert(lock->writer == 0);
> + while (!qemu_co_queue_empty(&lock->queue)) {
> + /*
> + * Wakeup every body. This will include some
> + * writers too.
> + */
> + qemu_co_queue_next(&lock->queue);
> + }
> + } else {
> + lock->reader--;
> + assert(lock->reader >= 0);
> + /* Wakeup only one waiting writer */
> + qemu_co_queue_next(&lock->queue);
This is only useful if lock->reader == 0.
> + }
> +}
> +
> +void qemu_co_rwlock_wrlock(CoRwlock *lock)
> +{
> + while (lock->writer || lock->reader) {
> + qemu_co_queue_wait(&lock->queue);
> + }
> + lock->writer++;
> + assert(lock->writer == 1);
> +}
I wonder if we should have a mechanism that stops new readers from
taking the lock while a writer is waiting in order to avoid starvation.
Anyway, the locking itself looks correct.
Kevin
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming
2011-07-27 9:45 ` Aneesh Kumar K.V
2011-07-27 10:03 ` Kevin Wolf
@ 2011-07-27 11:34 ` Aneesh Kumar K.V
1 sibling, 0 replies; 14+ messages in thread
From: Aneesh Kumar K.V @ 2011-07-27 11:34 UTC (permalink / raw)
To: Stefan Hajnoczi, qemu-devel
Cc: Kevin Wolf, Blue Swirl, Anthony Liguori, Venkateswararao Jujjuri
On Wed, 27 Jul 2011 15:15:20 +0530, "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com> wrote:
> On Tue, 26 Jul 2011 10:21:12 +0100, Stefan Hajnoczi <stefanha@linux.vnet.ibm.com> wrote:
> > QEMU is event-driven and suffers when blocking operations are performed because
> > VM execution may be stopped until the operation completes. Therefore many
> > operations that could block are performed asynchronously and a callback is
> > invoked when the operation has completed. This allows QEMU to continue
> > executing while the operation is pending.
> >
> > The downside to callbacks is that they split up code into many smaller
> > functions, each of which is a single step in a state machine that quickly
> > becomes complex and hard to understand. Callback functions also result in lots
> > of noise as variables are packed and unpacked into temporary structs that pass
> > state to the callback function.
> >
> > This patch series introduces coroutines as a solution for writing asynchronous
> > code while still having a nice sequential control flow. The semantics are
> > explained in the second patch. The fourth patch adds automated tests.
> >
> > A nice feature of coroutines is that it is relatively easy to take synchronous
> > code and lift it into a coroutine to make it asynchronous. Work has been done
> > to move qcow2 request processing into coroutines and thereby make it
> > asynchronous (today qcow2 will perform synchronous metadata accesses). This
> > qcow2 work is still ongoing and not quite ready for mainline yet.
> >
> > v8:
> > * Bisectability: introduce gthread implementation before ucontext/fibers
>
> Can we also get CoMutex and CoQueue patches also merged. I use them in
> the VirtFS series.
>
> http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/coroutine-devel
> http://repo.or.cz/w/qemu/v9fs.git/commit/318ef0b9b01cd296f8c30d8288139b9bed859892
>
I have pushed VirtFS related changes to
git://repo.or.cz/qemu/v9fs.git for-upstream-1
Once the co-routine patches are merged I can send a proper pull request.
There are more changes pending for VirtFS. We will push them in batches.
The following changes since commit c729e893aba79680bf410a2b9c671fb9e2c2c1a4:
coroutine: add test-coroutine --benchmark-lifecycle (2011-07-27 16:30:26 +0530)
are available in the git repository at:
git://repo.or.cz/qemu/v9fs.git for-upstream-1
Aneesh Kumar K.V (32):
hw/9pfs: Add yield support for readdir related coroutines
hw/9pfs: Update v9fs_readdir to use coroutines
hw/9pfs: Add yield support to statfs coroutine
hw/9pfs: Update v9fs_statfs to use coroutines
hw/9pfs: Add yield support to lstat coroutine
hw/9pfs: Update v9fs_getattr to use coroutines
hw/9pfs: Add yield support to setattr related coroutines
hw/9pfs: Update v9fs_setattr to use coroutines
hw/9pfs: Add yield support to xattr related coroutine
hw/9pfs: Update v9fs_xattrwalk to coroutines
hw/9pfs: Update v9fs_xattrcreate to use coroutines
hw/9pfs: Add yield support to mknod coroutine
hw/9pfs: Update v9fs_mknod to use coroutines
hw/9pfs: Add yeild support to rename coroutine
hw/9pfs: Update vfs_rename to use coroutines
hw/9pfs: Add yeild support for fstat coroutine
hw/9pfs: Update v9fs_lock to use coroutines
hw/9pfs: Update v9fs_getlock to use coroutines
hw/9pfs: Add yield support for open and opendir coroutine
hw/9pfs: Update v9fs_open to use coroutines
hw/9pfs: Update v9fs_stat to use coroutines
hw/9pfs: Update v9fs_walk to use coroutines
hw/9pfs: Add yeild support for clunk related coroutine
hw/9pfs: Update v9fs_clunk to use coroutines
hw/9pfs: Add yield support for fsync coroutine
hw/9pfs: Update v9fs_fsync to use coroutines
hw/9pfs: Add yield support for pwritev coroutine
hw/9pfs: Update v9fs_write to use coroutines
hw/9pfs: Update v9fs_wstat to use coroutines
hw/9pfs: Update v9fs_attach to use coroutines
hw/9pfs: Add yield support for preadv coroutine
hw/9pfs: Update v9fs_read to use coroutines
Harsh Prateek Bora (1):
use readdir_r instead of readdir for reentrancy
Venkateswararao Jujjuri (JV) (20):
[virtio-9p] Add infrastructure to support glib threads and coroutines.
[virtio-9p] Change all pdu handlers to coroutines.
[virtio-9p] Remove post functions for v9fs_readlink.
[virtio-9p] clean up v9fs_readlink.
[virtio-9p] coroutines for readlink
[virtio-9p] Remove post functions for v9fs_mkdir.
[virtio-9p] clean up v9fs_mkdir.
[virtio-9p] coroutine and threading for mkdir
[virtio-9p] Remove post functions for v9fs_remove
[virtio-9p] clean up v9fs_remove.
[virtio-9p] coroutine and threading for remove/unlink
[virtio-9p] Remove post functions for v9fs_lcreate
[virtio-9p] clean up v9fs_lcreate
[virtio-9p] coroutine and threading for open2
[virtio-9p] Remove post functions for v9fs_create
[virtio-9p] clean up v9fs_create Rearrange the code
[virtio-9p] Remove post functions for v9fs_symlink
[virtio-9p] clean up v9fs_symlink
[virtio-9p] coroutine and threading for v9fs_do_symlink
[virtio-9p] coroutine and threading for v9fs_do_link
Makefile.objs | 3 +
fsdev/file-op-9p.h | 2 +-
hw/9pfs/codir.c | 117 ++
hw/9pfs/cofile.c | 163 +++
hw/9pfs/cofs.c | 191 +++
hw/9pfs/coxattr.c | 84 ++
hw/9pfs/virtio-9p-coth.c | 102 ++
hw/9pfs/virtio-9p-coth.h | 96 ++
hw/9pfs/virtio-9p-device.c | 7 +-
hw/9pfs/virtio-9p-local.c | 7 +-
hw/9pfs/virtio-9p.c | 3052 ++++++++++++++++----------------------------
hw/9pfs/virtio-9p.h | 155 +---
12 files changed, 1840 insertions(+), 2139 deletions(-)
create mode 100644 hw/9pfs/codir.c
create mode 100644 hw/9pfs/cofile.c
create mode 100644 hw/9pfs/cofs.c
create mode 100644 hw/9pfs/coxattr.c
create mode 100644 hw/9pfs/virtio-9p-coth.c
create mode 100644 hw/9pfs/virtio-9p-coth.h
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming
2011-07-26 9:21 [Qemu-devel] [PATCH v8 0/5] Coroutines for better asynchronous programming Stefan Hajnoczi
` (6 preceding siblings ...)
2011-07-27 9:45 ` Aneesh Kumar K.V
@ 2011-07-29 12:54 ` Stefan Hajnoczi
7 siblings, 0 replies; 14+ messages in thread
From: Stefan Hajnoczi @ 2011-07-29 12:54 UTC (permalink / raw)
To: Anthony Liguori
Cc: Kevin Wolf, Blue Swirl, Venkateswararao Jujjuri, qemu-devel,
Stefan Hajnoczi
On Tue, Jul 26, 2011 at 10:21 AM, Stefan Hajnoczi
<stefanha@linux.vnet.ibm.com> wrote:
> QEMU is event-driven and suffers when blocking operations are performed because
> VM execution may be stopped until the operation completes. Therefore many
> operations that could block are performed asynchronously and a callback is
> invoked when the operation has completed. This allows QEMU to continue
> executing while the operation is pending.
>
> The downside to callbacks is that they split up code into many smaller
> functions, each of which is a single step in a state machine that quickly
> becomes complex and hard to understand. Callback functions also result in lots
> of noise as variables are packed and unpacked into temporary structs that pass
> state to the callback function.
>
> This patch series introduces coroutines as a solution for writing asynchronous
> code while still having a nice sequential control flow. The semantics are
> explained in the second patch. The fourth patch adds automated tests.
>
> A nice feature of coroutines is that it is relatively easy to take synchronous
> code and lift it into a coroutine to make it asynchronous. Work has been done
> to move qcow2 request processing into coroutines and thereby make it
> asynchronous (today qcow2 will perform synchronous metadata accesses). This
> qcow2 work is still ongoing and not quite ready for mainline yet.
>
> v8:
> * Bisectability: introduce gthread implementation before ucontext/fibers
Aneesh's code on top of this will go in separately.
We're good to go.
Stefan
^ permalink raw reply [flat|nested] 14+ messages in thread