Re: [Qemu-devel] [PATCH 1/2] coroutine: introduce coroutines

[Anthony Liguori <anthony@codemonkey.ws>]

On 05/24/2011 02:58 PM, Stefan Hajnoczi wrote:
> On Tue, May 24, 2011 at 08:37:50PM +0100, Jamie Lokier wrote:
>> Daniel P. Berrange wrote:
>>> On Wed, May 11, 2011 at 03:45:39PM +0200, Paolo Bonzini wrote:
>>>> On 05/11/2011 03:05 PM, Anthony Liguori wrote:
>>>>>>
>>>>>> A very slow way, too (on Windows at least if you use qemu_cond...).
>>>>>
>>>>> That doesn't mean you can't do a fiber implementation for Windows... but
>>>>> having a highly portable fallback is a good thing.
>>>>
>>>> I agree but where would you place it, since QEMU is only portable to
>>>> POSIX and Windows?
>>>>
>>>> osdep-$(CONFIG_POSIX) += coroutine-posix.c
>>>> osdep-$(CONFIG_WIN32) += coroutine-win32.c
>>>> osdep-??? += coroutine-fallback.c
>>>
>>> NetBSD forbids the use of 'makecontext' in any application
>>> which also links to libpthread.so[1]. We used makecontext in
>>> GTK-VNC's coroutines and got random crashes in threaded
>>> apps running on NetBSD. So for NetBSD we tell people to use
>>> the thread based coroutines instead.
>>
>> You have to use swapcontext(), no wait, you have to use setjmp(), no wait,
>> _setjmp(), no wait, threads.... Read on.
>>
>>  From Glibc's FAQ, setjmp/longjmp are not portable choices:
>>
>>      - UNIX provides no other (portable) way of effecting a synchronous
>>        context switch (also known as co-routine switch).  Some versions
>>        support this via setjmp()/longjmp() but this does not work
>>        universally.
>>
>> So in principle you should use swapcontext() in portable code.
>>
>> (By the way, Glibc goes on about how it won't support swapcontext()
>> from async signal handlers, i.e. preemption, on some architectures
>> (IA-64/S-390), and I know it has been very subtly broken from a signal
>> handler on ARM.  Fair enough, somehow disappointing, but doesn't
>> matter for QEMU coroutines.)
>>
>> But swapcontext() etc. have been withdrawn from POSIX 2008:
>>
>>      - Functions to be deleted
>>
>>        Legacy: Delete all legacy functions except utimes (which should not be legacy).
>>        OB: Default position is to delete all OB functions.
>>
>>        XSI Functions to change state
>>
>>        ....
>>        _setjmp and _longjmp. Should become obsolete.
>>        ....
>>        getcontext, setcontext, makecontext and swapcontext are already
>>        marked OB and should be withdrawn. And header file<ucontext.h>.
>>
>> OB means obsolescent.  They were marked obsolescent a few versions
>> prior, with the rationale that you can use threads instead...
>
> Yep, aware of this but at the end of the day these functions are
> commonly available.
>
>> It's not surprising that NetBSD forbids makecontext() with
>> libpthread.so.  I suspect old versions of FreeBSD, OpenBSD, DragonFly
>> BSD, (and Mac OS X?), have the same restriction, because they have a
>> similar pthreads evolutionary history to LinuxThreads.  LinuxThreads
>> also breaks when using coroutines that switch stacks, because it uses
>> the stack pointer to know the current thread.
>>
>> (LinuxThreads is old now, but that particular quirk still affects me
>> because some uCLinux platforms, on which I wish to use coroutines, still
>> don't have working NPTL - but they aren't likely to be running QEMU :-)
>
> That is nasty.
>
>> Finally, if you are using setjmp/longjmp, consider (from FreeBSD man page):
>>
>>      The setjmp()/longjmp() pairs save and restore the signal mask
>>      while _setjmp()/_longjmp() pairs save and restore only the
>>      register set and the stack.  (See sigprocmask(2).)
>>
>> As setjmp/longjmp were chosen for performance, you may wish to use
>> _setjmp/_longjmp instead (when available), as swizzling the signal
>> mask on each switch may involve a system call and be rather slow.
>
> Thanks, I read about that but didn't try to implement special cases
> because I don't have relevant OSes here to test against.
>
> My current plan is to try using sigaltstack(2) instead of
> makecontext()/swapcontext() as a hack since OpenBSD doesn't have
> makecontext()/swapcontext().
>
> TBH I'm almost at the stage where I think we should just use threads
> and/or async callbacks, as appropriate.  Hopefully I'll be able to cook
> up a reasonably portable implementation of coroutines though, because
> the prospect of having to go fully threaded or do async callbacks isn't
> attractive in many cases.

I'm meant to say threads as a coroutine fallback.

Regards,

Anthony Liguori

>
> Stefan
>