[Qemu-devel] Coroutines and ucontext

[Qemu-devel] Coroutines and ucontext

[Alex Barcelo]

I have read that one of the reasons for using makecontext is that it
saves the signal state. But there also exist functions like
"sigsetjmp" and "siglongjmp" which can be used to jump around the
coroutines while preserving signal masks.

I have a patch that uses sigsetjmp and siglongjmp instead of
makecontext and getcontext (and all the ucontext stuff), and it
*seems* to work... but I'm not sure if it works "by accident" (not
sure what I'm doing to the stack, not sure what I should be doing to
the stack). I will test more, but first I wanted to ask a little bit
for advice and comments. (Well, I have to admit it: the only benchmark
I have done is "qemu-img create -f qcow2 imgfile.qcow2 5G"... an
extremely poor test, but enough to see if something works at all).

On a related side note, this is not very well-written:
    /* The ucontext functions preserve signal masks which incurs a system call
     * overhead.  setjmp()/longjmp() does not preserve signal masks but only
     * works on the current stack.  Since we need a way to create and switch to
     * a new stack, use the ucontext functions for that but setjmp()/longjmp()
     * for everything else.
     */
[coroutine-ucontext.c, static Coroutine *coroutine_new(void)]
Because it is not clear (IMHO) why are the exact reasons for not using
setjmp and longjmp. Is it because the signal masks? Or is it (also?)
because the "only works on the current stack"? But which system call
are we talking about? Related to stack or related to signals? The
"but" is very ill-placed. And my knowledge is far below the needed for
correcting (or maybe simply understanding) this... sorry.

Re: [Qemu-devel] Coroutines and ucontext

[Paolo Bonzini]

On 01/27/2012 01:39 PM, Alex Barcelo wrote:
> I have read that one of the reasons for using makecontext is that it
> saves the signal state. But there also exist functions like
> "sigsetjmp" and "siglongjmp" which can be used to jump around the
> coroutines while preserving signal masks.
>
> I have a patch that uses sigsetjmp and siglongjmp instead of
> makecontext and getcontext (and all the ucontext stuff), and it
> *seems* to work... but I'm not sure if it works "by accident" (not
> sure what I'm doing to the stack, not sure what I should be doing to
> the stack).

You can post it, don't worry.  I'm curious how you are switching stacks 
when creating the coroutine.

> I will test more, but first I wanted to ask a little bit
> for advice and comments. (Well, I have to admit it: the only benchmark
> I have done is "qemu-img create -f qcow2 imgfile.qcow2 5G"... an
> extremely poor test, but enough to see if something works at all).

Booting a guest (even a raw image will do) is a decent smoke test.

> On a related side note, this is not very well-written:
>      /* The ucontext functions preserve signal masks which incurs a system call
>       * overhead.  setjmp()/longjmp() does not preserve signal masks but only
>       * works on the current stack.  Since we need a way to create and switch to
>       * a new stack, use the ucontext functions for that but setjmp()/longjmp()
>       * for everything else.
>       */
> [coroutine-ucontext.c, static Coroutine *coroutine_new(void)]
> Because it is not clear (IMHO) why are the exact reasons for not using
> setjmp and longjmp. Is it because the signal masks? Or is it (also?)
> because the "only works on the current stack"?

It's because you have to create a new stack for the new coroutine. 
makecontext does it for you; you can later use it with setcontext. 
Anything else (getcontext+setcontext, setjmp+longjmp, 
sigsetjmp+siglongjmp) will only work on an existing stack.

> But which system call are we talking about?

sigprocmask, which is invoked by sigsetjmp/siglongjmp and also 
getcontext/setcontext.  That's what we want to avoid.

Paolo

Re: [Qemu-devel] Coroutines and ucontext

[Daniel P. Berrange]

On Fri, Jan 27, 2012 at 03:39:22PM +0100, Paolo Bonzini wrote:
> On 01/27/2012 01:39 PM, Alex Barcelo wrote:
> >I have read that one of the reasons for using makecontext is that it
> >saves the signal state. But there also exist functions like
> >"sigsetjmp" and "siglongjmp" which can be used to jump around the
> >coroutines while preserving signal masks.
> >
> >I have a patch that uses sigsetjmp and siglongjmp instead of
> >makecontext and getcontext (and all the ucontext stuff), and it
> >*seems* to work... but I'm not sure if it works "by accident" (not
> >sure what I'm doing to the stack, not sure what I should be doing to
> >the stack).
> 
> You can post it, don't worry.  I'm curious how you are switching
> stacks when creating the coroutine.

If you're curious about this kind of thing you might also want to
take a look at the GNU Pth code. In particular its pth_mctx.c
file which has several different implementations of userspace thread
switching, one using makecontext, another with sigstck/altstack
and some others doing something fugly I don't understand :-)

Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] Coroutines and ucontext

[Alex Barcelo]

On Fri, Jan 27, 2012 at 15:39, Paolo Bonzini wrote:
>> I have a patch that uses sigsetjmp and siglongjmp instead of
>> makecontext and getcontext (and all the ucontext stuff), and it
>> *seems* to work... but I'm not sure if it works "by accident" (not
>> sure what I'm doing to the stack, not sure what I should be doing to
>> the stack).
>
> You can post it, don't worry.  I'm curious how you are switching stacks when
> creating the coroutine.
>

I think that I'm not switching, so at first I didn't understand why
could it run at all. Now I think that qemu-img is too simple and a new
stack is not really needed (probably only one coroutine and
straightforward, I'll check it). So it doesn't crash.

Thank you very much for the advice and comments. I'll do some more
tests, I expect to achieve something useful ^^


On Fri, Jan 27, 2012 at 15:48, Daniel P. Berrange wrote:
>(...)
> If you're curious about this kind of thing you might also want to
> take a look at the GNU Pth code. In particular its pth_mctx.c
> file which has several different implementations of userspace thread
> switching, one using makecontext, another with sigstck/altstack
> and some others doing something fugly I don't understand :-)

Thanks! I will definetely look at it/them.

Re: [Qemu-devel] Coroutines and ucontext

[Stefan Hajnoczi]

On Sat, Jan 28, 2012 at 9:31 AM, Alex Barcelo <abarcelo@ac.upc.edu> wrote:
> On Fri, Jan 27, 2012 at 15:39, Paolo Bonzini wrote:
>>> I have a patch that uses sigsetjmp and siglongjmp instead of
>>> makecontext and getcontext (and all the ucontext stuff), and it
>>> *seems* to work... but I'm not sure if it works "by accident" (not
>>> sure what I'm doing to the stack, not sure what I should be doing to
>>> the stack).
>>
>> You can post it, don't worry.  I'm curious how you are switching stacks when
>> creating the coroutine.
>>
>
> I think that I'm not switching, so at first I didn't understand why
> could it run at all. Now I think that qemu-img is too simple and a new
> stack is not really needed (probably only one coroutine and
> straightforward, I'll check it). So it doesn't crash.
>
> Thank you very much for the advice and comments. I'll do some more
> tests, I expect to achieve something useful ^^

The way that you can use the setjmp/longjmp family of functions for
creating stacks is by manipulating the fields inside jmpbuf.  However,
it seemed slightly more portable and less hacky to use ucontext
functions instead of doing that.

Stefan

Re: [Qemu-devel] Coroutines and ucontext

[Alex Barcelo]

On Tue, Feb 7, 2012 at 12:18, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> On Sat, Jan 28, 2012 at 9:31 AM, Alex Barcelo <abarcelo@ac.upc.edu> wrote:
>> On Fri, Jan 27, 2012 at 15:39, Paolo Bonzini wrote:
>>>> I have a patch that uses sigsetjmp and siglongjmp instead of
>>>> makecontext and getcontext (and all the ucontext stuff), and it
>>>> *seems* to work... but I'm not sure if it works "by accident" (not
>>>> sure what I'm doing to the stack, not sure what I should be doing to
>>>> the stack).
>>>
>>> You can post it, don't worry.  I'm curious how you are switching stacks when
>>> creating the coroutine.
>>>
>>
>> I think that I'm not switching, so at first I didn't understand why
>> could it run at all. Now I think that qemu-img is too simple and a new
>> stack is not really needed (probably only one coroutine and
>> straightforward, I'll check it). So it doesn't crash.
>>
>> Thank you very much for the advice and comments. I'll do some more
>> tests, I expect to achieve something useful ^^
>
> The way that you can use the setjmp/longjmp family of functions for
> creating stacks is by manipulating the fields inside jmpbuf.  However,
> it seemed slightly more portable and less hacky to use ucontext
> functions instead of doing that.
>
> Stefan

Yes, I agree. But makecontext and swapcontext are not OK for my
project (mainly because makecontext and swapcontext are not supported
by qemu under PPC) so I was looking for alternatives. My best shot at
the moment is using sigaltstack, if it works I will post a patch for
coroutines using sigaltstack, which is also quite portable (GNU
Portable Thread can use both, so I was trying to copy its
functionality).

Alex

Re: [Qemu-devel] Coroutines and ucontext

[Paolo Bonzini]

On 02/07/2012 05:06 PM, Alex Barcelo wrote:
> Yes, I agree. But makecontext and swapcontext are not OK for my
> project (mainly because makecontext and swapcontext are not supported
> by qemu under PPC) so I was looking for alternatives. My best shot at
> the moment is using sigaltstack, if it works I will post a patch for
> coroutines using sigaltstack, which is also quite portable (GNU
> Portable Thread can use both, so I was trying to copy its
> functionality).

If you get stuck you can use the coroutine-fix branch at 
git://github.com/bonzini/qemu.git

Paolo