Re: [Qemu-devel] [PATCH 1/2] coroutine: introduce coroutines

[Jamie Lokier <jamie@shareable.org>]

Daniel P. Berrange wrote:
> On Wed, May 11, 2011 at 03:45:39PM +0200, Paolo Bonzini wrote:
> > On 05/11/2011 03:05 PM, Anthony Liguori wrote:
> > >>
> > >>A very slow way, too (on Windows at least if you use qemu_cond...).
> > >
> > >That doesn't mean you can't do a fiber implementation for Windows... but
> > >having a highly portable fallback is a good thing.
> > 
> > I agree but where would you place it, since QEMU is only portable to
> > POSIX and Windows?
> > 
> > osdep-$(CONFIG_POSIX) += coroutine-posix.c
> > osdep-$(CONFIG_WIN32) += coroutine-win32.c
> > osdep-??? += coroutine-fallback.c
> 
> NetBSD forbids the use of 'makecontext' in any application
> which also links to libpthread.so[1]. We used makecontext in
> GTK-VNC's coroutines and got random crashes in threaded
> apps running on NetBSD. So for NetBSD we tell people to use
> the thread based coroutines instead.

You have to use swapcontext(), no wait, you have to use setjmp(), no wait,
_setjmp(), no wait, threads.... Read on.

>From Glibc's FAQ, setjmp/longjmp are not portable choices:

    - UNIX provides no other (portable) way of effecting a synchronous
      context switch (also known as co-routine switch).  Some versions
      support this via setjmp()/longjmp() but this does not work
      universally.

So in principle you should use swapcontext() in portable code.

(By the way, Glibc goes on about how it won't support swapcontext()
from async signal handlers, i.e. preemption, on some architectures
(IA-64/S-390), and I know it has been very subtly broken from a signal
handler on ARM.  Fair enough, somehow disappointing, but doesn't
matter for QEMU coroutines.)

But swapcontext() etc. have been withdrawn from POSIX 2008:

    - Functions to be deleted

      Legacy: Delete all legacy functions except utimes (which should not be legacy).
      OB: Default position is to delete all OB functions.

      XSI Functions to change state

      ....
      _setjmp and _longjmp. Should become obsolete.
      ....
      getcontext, setcontext, makecontext and swapcontext are already
      marked OB and should be withdrawn. And header file <ucontext.h>. 

OB means obsolescent.  They were marked obsolescent a few versions
prior, with the rationale that you can use threads instead...

It's not surprising that NetBSD forbids makecontext() with
libpthread.so.  I suspect old versions of FreeBSD, OpenBSD, DragonFly
BSD, (and Mac OS X?), have the same restriction, because they have a
similar pthreads evolutionary history to LinuxThreads.  LinuxThreads
also breaks when using coroutines that switch stacks, because it uses
the stack pointer to know the current thread.

(LinuxThreads is old now, but that particular quirk still affects me
because some uCLinux platforms, on which I wish to use coroutines, still
don't have working NPTL - but they aren't likely to be running QEMU :-)

Finally, if you are using setjmp/longjmp, consider (from FreeBSD man page):

    The setjmp()/longjmp() pairs save and restore the signal mask
    while _setjmp()/_longjmp() pairs save and restore only the
    register set and the stack.  (See sigprocmask(2).)

As setjmp/longjmp were chosen for performance, you may wish to use
_setjmp/_longjmp instead (when available), as swizzling the signal
mask on each switch may involve a system call and be rather slow.

-- Jamie