[Qemu-devel] QEMU/CPC project: an experience report

[Qemu-devel] QEMU/CPC project: an experience report

[Gabriel Kerneis]

Dear all,

We were so excited about our QEMU/CPC GSoC project that we have decided
to write a 12-page article to share our experience:

           QEMU/CPC: static analysis and CPS conversion
           for safe, portable, and efficient coroutines
       http://gabriel.kerneis.info/research/files/qemu-cpc.pdf

If you want to learn more about QEMU internals, how we fixed hundreds of
lines of code with a small checking tool, or discover a nice compilation
technique for coroutines, you will probably be interested in this
article.

We also hope that other mentors will find it useful as a resource for
prospective students looking for an example of past GSoC projects.

Thanks to Google, and everybody involved, for making this great project
possible.

-- 
Gabriel Kerneis
http://gabriel.kerneis.info/

Re: [Qemu-devel] QEMU/CPC project: an experience report

[Stefan Hajnoczi]

On Thu, Oct 10, 2013 at 11:14:22PM +0100, Gabriel Kerneis wrote:
> We were so excited about our QEMU/CPC GSoC project that we have decided
> to write a 12-page article to share our experience:
> 
>            QEMU/CPC: static analysis and CPS conversion
>            for safe, portable, and efficient coroutines
>        http://gabriel.kerneis.info/research/files/qemu-cpc.pdf
> 
> If you want to learn more about QEMU internals, how we fixed hundreds of
> lines of code with a small checking tool, or discover a nice compilation
> technique for coroutines, you will probably be interested in this
> article.

Summary & status:

Charlie showed it's possible to use the CPC tool to implement QEMU
coroutines.  CPC is a source-to-source translator that outputs C code.
That means no stack switching or threads are necessary to implement
coroutines.  In other words, high performance coroutines that are also
portable.

There are still some challenges so QEMU cannot switch to CPC yet.  Not
all existing QEMU coroutine code is amenable to CPC translation, work is
needed to refactor the code to remove functions that change behavior
depending on if (qemu_in_coroutine()).  Also, CPC is not yet widely
packaged and available on all QEMU host platforms so we cannot add it as
a build dependency at this time.

The paper offers details on how CPC works, tools that Gabriel and
Charlie developed and refined during the Google Summer of Code project,
and performance results.

The most immediate result is that a tool will soon be available to check
coroutine_fn annotations in the QEMU source tree.  The tool can detect
code paths where a yielding function is invoked outside coroutine
context (which would result in an abort at runtime!).

Thanks to Charlie Shepherd and Gabriel Kerneis, it was a pleasure
working together!

Stefan

Re: [Qemu-devel] [GSoC Mentors] Re: QEMU/CPC project: an experience report

[Pierre Tardy]

[-- Attachment #1: Type: text/plain, Size: 1317 bytes --]

On Fri, Oct 11, 2013 at 11:35 AM, Stefan Hajnoczi <stefanha@gmail.com>wrote:

> On Thu, Oct 10, 2013 at 11:14:22PM +0100, Gabriel Kerneis wrote:
> > We were so excited about our QEMU/CPC GSoC project that we have decided
> > to write a 12-page article to share our experience:
> >
> >            QEMU/CPC: static analysis and CPS conversion
> >            for safe, portable, and efficient coroutines
> >        http://gabriel.kerneis.info/research/files/qemu-cpc.pdf
> >
> > If you want to learn more about QEMU internals, how we fixed hundreds of
> > lines of code with a small checking tool, or discover a nice compilation
> > technique for coroutines, you will probably be interested in this
> > article.
>
> Summary & status:
>
> Charlie showed it's possible to use the CPC tool to implement QEMU
> coroutines.  CPC is a source-to-source translator that outputs C code.
> That means no stack switching or threads are necessary to implement
> coroutines.  In other words, high performance coroutines that are also
> portable.
>
> Do I summarized too much if I say that this is about automatically
translating thread based code to event driven code?

This indeed deserve some congrats, and looks interesting. Do you have an
epub version, so that one can study this confortably in an ebook reader?

Regards,
Pierre

[-- Attachment #2: Type: text/html, Size: 1890 bytes --]

Re: [Qemu-devel] QEMU/CPC project: an experience report

[Gabriel Kerneis]

Hi Pierre,

On Fri, Oct 11, 2013 at 01:34:02PM +0200, Pierre Tardy wrote:
> Do I summarized too much if I say that this is about automatically
> translating thread based code to event driven code?

This is exactly what CPC is about.  If you want more details on the
translation itself, you might be interested in my PhD thesis [1], or the
journal article Juliusz Chroboczek and I wrote about it [2].  Note that
those have been written before this GSoC project.

[1] http://gabriel.kerneis.info/research/files/kerneis-phd-thesis.pdf
[2] http://gabriel.kerneis.info/research/files/cpc-2012.pdf

> This indeed deserve some congrats, and looks interesting. Do you have an
> epub version, so that one can study this confortably in an ebook reader?

Unfortunately not, but I could easily recompile it if anyone knows a
simple way to turn a LaTeX file into an epub.

Best,
-- 
Gabriel

Re: [Qemu-devel] QEMU/CPC project: an experience report

[Pierre Tardy]

[-- Attachment #1: Type: text/plain, Size: 655 bytes --]

> [1] http://gabriel.kerneis.info/research/files/kerneis-phd-thesis.pdf
> [2] http://gabriel.kerneis.info/research/files/cpc-2012.pdf
>
> Looks like you ruined my evening!


> > This indeed deserve some congrats, and looks interesting. Do you have an
> > epub version, so that one can study this confortably in an ebook reader?
>
> Unfortunately not, but I could easily recompile it if anyone knows a
> simple way to turn a LaTeX file into an epub.
>

I would indeed though this obvious issue would have a simpler answer.
http://tex.stackexchange.com/questions/1551/use-latex-to-produce-epub

I'll probably try what calibre can do with the pdfs..

Pierre

[-- Attachment #2: Type: text/html, Size: 1606 bytes --]

Re: [Qemu-devel] [GSoC Mentors] Re: QEMU/CPC project: an experience report

[Gabriel Kerneis]

On Fri, Oct 11, 2013 at 01:34:02PM +0200, Pierre Tardy wrote:
> This indeed deserve some congrats, and looks interesting. Do you have an
> epub version, so that one can study this confortably in an ebook reader?

HTML: http://gabriel.kerneis.info/research/files/qemu-cpc/
      (generated from latex source with htlatex)
EPUB: http://gabriel.kerneis.info/research/files/qemu-cpc.epub
      (generated from the above html with calibre)

Thanks to John Nash for his help with this conversion.
-- 
Gabriel

Re: [Qemu-devel] QEMU/CPC project: an experience report

[Gabriel Kerneis]

On Thu, Oct 10, 2013 at 11:14:22PM +0100, Gabriel Kerneis wrote:
> We were so excited about our QEMU/CPC GSoC project that we have decided
> to write a 12-page article to share our experience

Our "QEMU/CPC" paper has been accepted for publication at PEPM'14:
http://www.program-transformation.org/PEPM14/

We received a number of very useful comments from the reviewers that should help
improving the final paper.  The version on my webpage will be updated once this
is done.

Best regards,
-- 
Gabriel Kerneis   ---   http://gabriel.kerneis.info/