Re: [Qemu-devel] [0/25] Async threading for VirtFS using glib threads & coroutines.

[Stefan Hajnoczi <stefanha@gmail.com>]

On Tue, May 24, 2011 at 10:00 PM, Jamie Lokier <jamie@shareable.org> wrote:
> Venkateswararao Jujjuri wrote:
>> This model makes the code simple and also in one shot we can convert
>> all v9fs_do_syscalls into asynchronous threads. But as Aneesh raised
>> will there be any additional overhead for the additional jumps?  We
>> can quickly test it out too.
>
> I'm not sure if this is exactly the right place (I haven't followed
> the whole discussion), but there is a useful trick for getting rid of
> one of the thread context switches:
>
> Swizzle *which* thread is your "main" coroutine thread.
>
> Instead of queuing up an item on the work queue, waking the worker
> thread pool, and having a worker thread pick up the coroutine, you:
>
> Declare the current thread to *be* a worker through from this point,
> and queue the calling context for a worker thread to pick up.  When it
> picks it up, *that* thread declares itself to be the main thread
> coroutine thread.
>
> So the coroutine entry step is just queuing a context for another
> thread to pick up, and then diving into the blocking system call
> (optimising out the enqueue/dequeue and thread switch).
>
> In a sense, you make the "main" thread a long-lived work queue entry,
> and have a symmetric pool, except that the main thread tends to behave
> differently than the other work items.
>
> This only works if the main thread's state is able to follow the
> swizzling.  I don't know if KVM VCPUs will do that, for example, or if
> there's other per-thread state that won't work.
>
> If the main thread can't be swizzled, you can still use this trick
> when doing the coroutine->syscall step starting form an existing
> worker thread.

The point of this trick is to reduce the latency of issuing the
blocking operation?  You still have to tell another thread that it has
now become the main thread, so this seems like an optimization biased
towards I/O latency at the expense of main loop performance.

If the "main" thread is a vcpu thread, I'm not sure we want to
schedule another thread to become the vcpu thread since this adds an
unbounded time until the new vcpu thread is scheduled.  The vcpu
thread is supposed to be non-blocking so that we don't steal time from
the guest and pause execution.

Stefan