[Qemu-devel] [RfC] using pixman in qemu for raster ops

[Qemu-devel] [RfC] using pixman in qemu for raster ops

[Gerd Hoffmann]

  Hi,

While working with the pixman library in the fbdev branch and discussing
patches with Stefano Stabellini I came to the conclusion that it would
be a good idea to:

  (a) use the pixman library not just as vehicle in fbdev,
      but as core service for all raster ops in qemu.
  (b) do a major cleanup / revamp of the console system in qemu.

That implies pixman will join zlib and glib as hard dependency, i.e.
qemu will stop building without pixman.

What is pixman?

As the name implies it is a library for pixel rendering ops.  It is
portable.  It's used by Xorg, cairo and spice.  On any recent linux
distro you'll find it already installed.  When you compile qemu with
spice you'll have a indirect dependency on pixman.  Merging anthonys
gtk/cairo patches will add another indirect dependency.

Even though pixman is used by core linux packages it is still a young
project. On older distros (RHEL-5 for example) it isn't included and
must be compiled manually.

What is the console revamp plan?

 (1) Drop DisplayAllocator.  Make DisplayChangeListeners support all
     important formats directly instead (using pixman), then make gfx
     card emulation use qemu_create_displaysurface_from() to give
     DisplayChangeListeners direct access to the guests framebuffer.

 (2) Replace DisplaySurface with pixman_image_t.  Gives us reference
     counting for free, which simplifies thread usage in ui/ rendering
     code.  Also makes it easier to use pixman in ui/ code.

 (3) Assign a pixman_image_t to each qemu console.  No more redrawing
     needed on console switching.  vga_hw_invalidate() can be dropped,
     screendump code can be simplified.

 (4) API fixes.  DisplayChangeListeners don't get a opaque pointer (or
     pointer to struct DisplayChangeListener for container_of usage)
     for example.

 (5) Investigate optimizations.  Such as passing pixman regions to
     DisplayChangeListensers, so they get all updates in one go instead
     of being called for each rectangle individually.

Comments?

cheers,
  Gerd

Re: [Qemu-devel] [RfC] using pixman in qemu for raster ops

[Peter Maydell]

On 25 September 2012 09:47, Gerd Hoffmann <kraxel@redhat.com> wrote:
> Even though pixman is used by core linux packages it is still a young
> project. On older distros (RHEL-5 for example) it isn't included and
> must be compiled manually.

For me "not a standard library package on RHEL5" is a strong argument
against adding a hard dependency. (For instance, most of the compute
cluster machines here are RHEL5 and it would be pretty awkward to
deal with manually building a dependent library.)

So they're a new library -- how good are they at API/ABI stability?

I also note that http://pixman.org/ says "There is currently no
documentation besides the source code itself", which makes me
really really wary of using it.

-- PMM

Re: [Qemu-devel] [RfC] using pixman in qemu for raster ops

[Gerd Hoffmann]

On 09/25/12 11:31, Peter Maydell wrote:
> On 25 September 2012 09:47, Gerd Hoffmann <kraxel@redhat.com> wrote:
>> Even though pixman is used by core linux packages it is still a young
>> project. On older distros (RHEL-5 for example) it isn't included and
>> must be compiled manually.
> 
> For me "not a standard library package on RHEL5" is a strong argument
> against adding a hard dependency. (For instance, most of the compute
> cluster machines here are RHEL5 and it would be pretty awkward to
> deal with manually building a dependent library.)

Why it is that a big deal?  Whatever is used to distribute qemu to the
cluster machines (local yum repo?) can be used to distribute pixman too, no?

I rarely found a piece of software which builds as easy as pixman.  No
unusual dependencies.  No tricks stuff in the specfile.  You can grab as
source rpm from rhel6 or fedora, rebuild it for rhel5 and be done with
it.  Backports on other distros should likewise be simple.

> So they're a new library -- how good are they at API/ABI stability?

I didn't notice any issues until now (didn't search though).  No soname
bump yet.  And given Xorg + cairo are using pixman they hardly can get
it wrong unnoticed, things would break left+right all over the place if
they did.

> I also note that http://pixman.org/ says "There is currently no
> documentation besides the source code itself", which makes me
> really really wary of using it.

That made me worry too, but it was surprisingly easy to get along with
just the header file and some sample code.  At the end of the day it is
just standard raster ops and transformations built into a library.

cheers,
  Gerd

Re: [Qemu-devel] [RfC] using pixman in qemu for raster ops

[Peter Maydell]

On 25 September 2012 11:37, Gerd Hoffmann <kraxel@redhat.com> wrote:
> On 09/25/12 11:31, Peter Maydell wrote:
>> For me "not a standard library package on RHEL5" is a strong argument
>> against adding a hard dependency. (For instance, most of the compute
>> cluster machines here are RHEL5 and it would be pretty awkward to
>> deal with manually building a dependent library.)
>
> Why it is that a big deal?  Whatever is used to distribute qemu to the
> cluster machines (local yum repo?) can be used to distribute pixman too, no?

There's a big leap between "run configure, put the qemu executable
in some generally available directory" and "you have to first get
and build some third party library, install it somewhere, tell
configure where you put it, then build qemu, then make sure that
not just the qemu executable but also this third party library
are on all the machines". At the moment you can build qemu without
any non-system dependencies, and we should have a really good
reason for breaking that.

-- PMM

Re: [Qemu-devel] [RfC] using pixman in qemu for raster ops

[Gerd Hoffmann]

On 09/25/12 12:48, Peter Maydell wrote:
> On 25 September 2012 11:37, Gerd Hoffmann <kraxel@redhat.com> wrote:
>> On 09/25/12 11:31, Peter Maydell wrote:
>>> For me "not a standard library package on RHEL5" is a strong argument
>>> against adding a hard dependency. (For instance, most of the compute
>>> cluster machines here are RHEL5 and it would be pretty awkward to
>>> deal with manually building a dependent library.)
>>
>> Why it is that a big deal?  Whatever is used to distribute qemu to the
>> cluster machines (local yum repo?) can be used to distribute pixman too, no?
> 
> There's a big leap between "run configure, put the qemu executable
> in some generally available directory" and "you have to first get
> and build some third party library, install it somewhere, tell
> configure where you put it, then build qemu, then make sure that
> not just the qemu executable but also this third party library
> are on all the machines". At the moment you can build qemu without
> any non-system dependencies, and we should have a really good
> reason for breaking that.

Hmm, we could import a pixman copy into the qemu tree and use that as
fallback if we don't find pixman installed on the system ...

cheers,
  Gerd

Re: [Qemu-devel] [RfC] using pixman in qemu for raster ops

[Anthony Liguori]

Gerd Hoffmann <kraxel@redhat.com> writes:

> On 09/25/12 12:48, Peter Maydell wrote:
>> On 25 September 2012 11:37, Gerd Hoffmann <kraxel@redhat.com> wrote:
>>> On 09/25/12 11:31, Peter Maydell wrote:
>>>> For me "not a standard library package on RHEL5" is a strong argument
>>>> against adding a hard dependency. (For instance, most of the compute
>>>> cluster machines here are RHEL5 and it would be pretty awkward to
>>>> deal with manually building a dependent library.)
>>>
>>> Why it is that a big deal?  Whatever is used to distribute qemu to the
>>> cluster machines (local yum repo?) can be used to distribute pixman too, no?
>> 
>> There's a big leap between "run configure, put the qemu executable
>> in some generally available directory" and "you have to first get
>> and build some third party library, install it somewhere, tell
>> configure where you put it, then build qemu, then make sure that
>> not just the qemu executable but also this third party library
>> are on all the machines". At the moment you can build qemu without
>> any non-system dependencies, and we should have a really good
>> reason for breaking that.
>
> Hmm, we could import a pixman copy into the qemu tree and use that as
> fallback if we don't find pixman installed on the system ...

We can include a pixman submodule in qemu.git and have configure try to
find a local version, if it's not available, build the embedded version.

After RHEL5 becomes ancient, we can simply remove the submodule.  Should
make everyone happy.

Regards,

Anthony Liguori

>
> cheers,
>   Gerd

Re: [Qemu-devel] [RfC] using pixman in qemu for raster ops

[Paolo Bonzini]

Il 25/09/2012 12:48, Peter Maydell ha scritto:
> On 25 September 2012 11:37, Gerd Hoffmann <kraxel@redhat.com> wrote:
>> On 09/25/12 11:31, Peter Maydell wrote:
>>> For me "not a standard library package on RHEL5" is a strong argument
>>> against adding a hard dependency. (For instance, most of the compute
>>> cluster machines here are RHEL5 and it would be pretty awkward to
>>> deal with manually building a dependent library.)
>>
>> Why it is that a big deal?  Whatever is used to distribute qemu to the
>> cluster machines (local yum repo?) can be used to distribute pixman too, no?
> 
> There's a big leap between "run configure, put the qemu executable
> in some generally available directory" and "you have to first get
> and build some third party library, install it somewhere, tell
> configure where you put it, then build qemu, then make sure that
> not just the qemu executable but also this third party library
> are on all the machines". At the moment you can build qemu without
> any non-system dependencies, and we should have a really good
> reason for breaking that.

Would EPEL satisfy your requirements?  We can add a pixman package to
EPEL-5 in a day or two.

Paolo

Re: [Qemu-devel] [RfC] using pixman in qemu for raster ops

[Peter Maydell]

On 25 September 2012 15:32, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 25/09/2012 12:48, Peter Maydell ha scritto:
>> There's a big leap between "run configure, put the qemu executable
>> in some generally available directory" and "you have to first get
>> and build some third party library, install it somewhere, tell
>> configure where you put it, then build qemu, then make sure that
>> not just the qemu executable but also this third party library
>> are on all the machines". At the moment you can build qemu without
>> any non-system dependencies, and we should have a really good
>> reason for breaking that.
>
> Would EPEL satisfy your requirements?  We can add a pixman package to
> EPEL-5 in a day or two.

I doubt it -- sysadmins are always conservative :-) In any case I
mention RHEL5 not just as a local issue but as a nice baseline for
being conservative about what we allow in as a QEMU dependency
(and for things like "what version of glib are we OK to require").

-- PMM

Re: [Qemu-devel] [RfC] using pixman in qemu for raster ops

[Paolo Bonzini]

Il 25/09/2012 11:31, Peter Maydell ha scritto:
>> > Even though pixman is used by core linux packages it is still a young
>> > project. On older distros (RHEL-5 for example) it isn't included and
>> > must be compiled manually.
> For me "not a standard library package on RHEL5" is a strong argument
> against adding a hard dependency. (For instance, most of the compute
> cluster machines here are RHEL5 and it would be pretty awkward to
> deal with manually building a dependent library.)
> 
> So they're a new library -- how good are they at API/ABI stability?

Very.

pixman is not that young actually (2003), but it was built into both
cairo and Xorg until ~2007.  Unfortunately, the pixman code that is in
cairo prefixes all names with "_cairo_" (so they look like
"_cairo_pixman_image_create") and there is no public header file.  But I
don't see any reason why pixman couldn't be included in EPEL.

Paolo

Re: [Qemu-devel] [RfC] using pixman in qemu for raster ops

[Søren Sandmann]

Peter Maydell <peter.maydell@linaro.org> writes:

> On 25 September 2012 09:47, Gerd Hoffmann <kraxel@redhat.com> wrote:
>> Even though pixman is used by core linux packages it is still a young
>> project. On older distros (RHEL-5 for example) it isn't included and
>> must be compiled manually.
>
> For me "not a standard library package on RHEL5" is a strong argument
> against adding a hard dependency. (For instance, most of the compute
> cluster machines here are RHEL5 and it would be pretty awkward to
> deal with manually building a dependent library.)

Pixman was actually included in RHEL 5.8 which was released in February
this year.


Soren

Re: [Qemu-devel] [RfC] using pixman in qemu for raster ops

[Gerd Hoffmann]

  Hi,

> Pixman was actually included in RHEL 5.8 which was released in February
> this year.

Where is it?  'yum list pixman' returns nothing ...

thanks,
  Gerd

Re: [Qemu-devel] [RfC] using pixman in qemu for raster ops

[Søren Sandmann]

Gerd Hoffmann <kraxel@redhat.com> writes:

>> Pixman was actually included in RHEL 5.8 which was released in February
>> this year.
>
> Where is it?  'yum list pixman' returns nothing ...

I don't have an installation of 5.8 so I don't know why yum list doesn't
return anything, but pixman-0.22.0 certainly was included as a
dependency of spice-client:

    https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/5/html/5.8_Release_Notes/virtualization.html#spice

    https://bugzilla.redhat.com/show_bug.cgi?id=718810


Søren

Re: [Qemu-devel] [RfC] using pixman in qemu for raster ops

[Markus Armbruster]

sandmann@cs.au.dk (Søren Sandmann) writes:

> Gerd Hoffmann <kraxel@redhat.com> writes:
>
>>> Pixman was actually included in RHEL 5.8 which was released in February
>>> this year.
>>
>> Where is it?  'yum list pixman' returns nothing ...
>
> I don't have an installation of 5.8 so I don't know why yum list doesn't
> return anything, but pixman-0.22.0 certainly was included as a
> dependency of spice-client:
>
>     https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/5/html/5.8_Release_Notes/virtualization.html#spice
>
>     https://bugzilla.redhat.com/show_bug.cgi?id=718810

$ cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 5.8 (Tikanga)
$ yum -C search pixman
Loaded plugins: downloadonly, fastestmirror
=============================== Matched: pixman ================================
qpixman.i386 : Modified version of pixman for spice
qpixman.x86_64 : Modified version of pixman for spice
qpixman-devel.i386 : Pixel manipulation library development package
qpixman-devel.x86_64 : Pixel manipulation library development package

Re: [Qemu-devel] [RfC] using pixman in qemu for raster ops

[Søren Sandmann]

Markus Armbruster <armbru@redhat.com> writes:

> sandmann@cs.au.dk (Søren Sandmann) writes:
>
>> Gerd Hoffmann <kraxel@redhat.com> writes:
>>
>>>> Pixman was actually included in RHEL 5.8 which was released in February
>>>> this year.
>>>
>>> Where is it?  'yum list pixman' returns nothing ...
>>
>> I don't have an installation of 5.8 so I don't know why yum list doesn't
>> return anything, but pixman-0.22.0 certainly was included as a
>> dependency of spice-client:
>>
>>     https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/5/html/5.8_Release_Notes/virtualization.html#spice
>>
>>     https://bugzilla.redhat.com/show_bug.cgi?idq8810
>
> $ cat /etc/redhat-release 
> Red Hat Enterprise Linux Server release 5.8 (Tikanga)
> $ yum -C search pixman
> Loaded plugins: downloadonly, fastestmirror
> =============================== Matched: pixman ===============================qpixman.i386 : Modified version of pixman for spice
> qpixman.x86_64 : Modified version of pixman for spice
> qpixman-devel.i386 : Pixel manipulation library development package
> qpixman-devel.x86_64 : Pixel manipulation library development package

That is not the real pixman though. qpixman is an ancient version that
has been modified to include some rasterops and to install the header
files in /usr/include/qpixman-1/.


Søren

Re: [Qemu-devel] [RfC] using pixman in qemu for raster ops

[Gerd Hoffmann]

  Hi,

> $ yum -C search pixman
> Loaded plugins: downloadonly, fastestmirror
> =============================== Matched: pixman ================================
> qpixman.i386 : Modified version of pixman for spice
> qpixman.x86_64 : Modified version of pixman for spice
> qpixman-devel.i386 : Pixel manipulation library development package
> qpixman-devel.x86_64 : Pixel manipulation library development package

Found it, it is in the "RHEL Optional Productivity Apps" channel.

cheers,
  Gerd

Re: [Qemu-devel] [RfC] using pixman in qemu for raster ops

[Stefano Stabellini]

On Tue, 25 Sep 2012, Gerd Hoffmann wrote:
>   Hi,
> 
> While working with the pixman library in the fbdev branch and discussing
> patches with Stefano Stabellini I came to the conclusion that it would
> be a good idea to:
> 
>   (a) use the pixman library not just as vehicle in fbdev,
>       but as core service for all raster ops in qemu.
>   (b) do a major cleanup / revamp of the console system in qemu.
> 
> That implies pixman will join zlib and glib as hard dependency, i.e.
> qemu will stop building without pixman.
> 
> What is pixman?
> 
> As the name implies it is a library for pixel rendering ops.  It is
> portable.  It's used by Xorg, cairo and spice.  On any recent linux
> distro you'll find it already installed.  When you compile qemu with
> spice you'll have a indirect dependency on pixman.  Merging anthonys
> gtk/cairo patches will add another indirect dependency.
> 
> Even though pixman is used by core linux packages it is still a young
> project. On older distros (RHEL-5 for example) it isn't included and
> must be compiled manually.

I think that not being in RHEL5 could create lots of problems for many
people. For example most Xen virtualization products I know today are
based on something from the same era as RHEL5.


> What is the console revamp plan?
> 
>  (1) Drop DisplayAllocator.  Make DisplayChangeListeners support all
>      important formats directly instead (using pixman), then make gfx
>      card emulation use qemu_create_displaysurface_from() to give
>      DisplayChangeListeners direct access to the guests framebuffer.

"important formats" should include 24 bpp (repeating here for people
that didn't follow the other discussion)


>  (2) Replace DisplaySurface with pixman_image_t.  Gives us reference
>      counting for free, which simplifies thread usage in ui/ rendering
>      code.  Also makes it easier to use pixman in ui/ code.
> 
>  (3) Assign a pixman_image_t to each qemu console.  No more redrawing
>      needed on console switching.  vga_hw_invalidate() can be dropped,
>      screendump code can be simplified.
> 
>  (4) API fixes.  DisplayChangeListeners don't get a opaque pointer (or
>      pointer to struct DisplayChangeListener for container_of usage)
>      for example.

opaque pointers are not always necessary or a must have clean up


>  (5) Investigate optimizations.  Such as passing pixman regions to
>      DisplayChangeListensers, so they get all updates in one go instead
>      of being called for each rectangle individually.
> 
> Comments?