[Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Benjamin Herrenschmidt]

So I got cirrus working on ppc with cirrusdrmfb...

The fun part is that it works :-)

Basically, the issue is that normally, for it to work, one would have to
access the framebuffer using the appropriate aperture for byteswapping
based on the bpp.

However, qemu doesn't emulate those apertures ... and cirrusdrmfb
either.

In fact, qemu cirrus model is just dumb and assumes guest native
byteorder for the framebuffer.

The good thing is that this makes it work... the bad thing is that it's
a completely incorrect HW model and if the linux driver wasn't also
buggy it wouldn't work.

However it's also pretty much unfixable without making it also unusable
in terms of performance so I want to check with you guys if it's ok to
just leave it as-is.

Basically, if the fb was LE as it's supposed to be, one would have to
use the byteswapped apertures. But those can only be emulated by
trapping on every access to turn it into MMIO emulation, which means
unusable performances.

So we end up with what is effectively a BE framebuffer thanks to qemu
hard coding what it thinks the guest endian is (btw, this is quite
busted in theory as well since PPC can be bi-endian for example).

Anyways, it works today, it's just that the HW model is wrong... and I
don't want to fix it. Any objection ?

As for the work I'm doing to brush up pci-vga a bit, I'm tempted to add
an MMIO reg or a VBE config reg bit to allow configuring the endianness
of the underlying fb with a default to what qemu does today.

Cheers,
Ben.

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Avi Kivity]

On 07/30/2012 09:24 AM, Benjamin Herrenschmidt wrote:
> So I got cirrus working on ppc with cirrusdrmfb...
> 
> The fun part is that it works :-)
> 
> Basically, the issue is that normally, for it to work, one would have to
> access the framebuffer using the appropriate aperture for byteswapping
> based on the bpp.
> 
> However, qemu doesn't emulate those apertures ... and cirrusdrmfb
> either.
> 
> In fact, qemu cirrus model is just dumb and assumes guest native
> byteorder for the framebuffer.
> 
> The good thing is that this makes it work... the bad thing is that it's
> a completely incorrect HW model and if the linux driver wasn't also
> buggy it wouldn't work.
> 
> However it's also pretty much unfixable without making it also unusable
> in terms of performance so I want to check with you guys if it's ok to
> just leave it as-is.
> 
> Basically, if the fb was LE as it's supposed to be, one would have to
> use the byteswapped apertures. But those can only be emulated by
> trapping on every access to turn it into MMIO emulation, which means
> unusable performances.
> 
> So we end up with what is effectively a BE framebuffer thanks to qemu
> hard coding what it thinks the guest endian is (btw, this is quite
> busted in theory as well since PPC can be bi-endian for example).
> 
> Anyways, it works today, it's just that the HW model is wrong... and I
> don't want to fix it. Any objection ?
> 

Yes.  If a correct guest comes along and tries to use cirrus, it will break.

> As for the work I'm doing to brush up pci-vga a bit, I'm tempted to add
> an MMIO reg or a VBE config reg bit to allow configuring the endianness
> of the underlying fb with a default to what qemu does today.

What are those byteswapped apertures?  Some chipset thing that does the
byteswap?

IIRC ppc has a bit in the TLB entry that tells it to byteswap.  Can't we
use it directly map the framebuffer with byteswapping?



-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Benjamin Herrenschmidt]

On Mon, 2012-07-30 at 13:08 +0300, Avi Kivity wrote:
>  
> > So we end up with what is effectively a BE framebuffer thanks to qemu
> > hard coding what it thinks the guest endian is (btw, this is quite
> > busted in theory as well since PPC can be bi-endian for example).
> > 
> > Anyways, it works today, it's just that the HW model is wrong... and I
> > don't want to fix it. Any objection ?
> > 
> 
> Yes.  If a correct guest comes along and tries to use cirrus, it will break.

Right. Cirrus on ppc was used on PReP and Amiga for example though not
many people really care about those platforms anymore. I'm not too
worried at this point with that possibility but we shall know about it.

> > As for the work I'm doing to brush up pci-vga a bit, I'm tempted to add
> > an MMIO reg or a VBE config reg bit to allow configuring the endianness
> > of the underlying fb with a default to what qemu does today.
> 
> What are those byteswapped apertures?  Some chipset thing that does the
> byteswap?

The card itself. The 16M BAR is divided in 4 "apertures" (at least some
Cirrus models do that including the one we emulate by default). One is
no byteswap, two are 16-bit and 32-bit byteswap and the last one uses a
specific byteswap for video overlay which I haven't looked at in detail.

> IIRC ppc has a bit in the TLB entry that tells it to byteswap.  Can't we
> use it directly map the framebuffer with byteswapping?

Unfortunately only embedded ppc's have that :-(

We can also make the fbdev/fbcon driver do the swapping in SW, but it's
a relatively unusual code path and I don't think it works properly with
X, I don't think it can be made to work properly with the generic X KMS
at this point.

Now, cirrusdrmfb is already specific to the qemu cirrus variant in
several ways, I wouldn't mind keeping it that way and if we "fix" the
endianness model, maybe having a "hidden" register to flip it back to
it's current mode of operation that cirrusdrmfb would use...

Note that in the long run, I don't think cirrus should have much future
for us (ppc) anyway. I'm working on some simple improvements to qemu-vga
to give it basic 2D accel and the corresponding kernel drivers which
should give us overall better functionality than cirrus with simpler &
more maintainable code, along possibly with a virtio tunnel if we want
to pipe spice or similar through without using the virtio-serial
crackpot :-)

Cheers,
Ben.

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Avi Kivity]

On 07/30/2012 02:20 PM, Benjamin Herrenschmidt wrote:
> On Mon, 2012-07-30 at 13:08 +0300, Avi Kivity wrote:
>>  
>> > So we end up with what is effectively a BE framebuffer thanks to qemu
>> > hard coding what it thinks the guest endian is (btw, this is quite
>> > busted in theory as well since PPC can be bi-endian for example).
>> > 
>> > Anyways, it works today, it's just that the HW model is wrong... and I
>> > don't want to fix it. Any objection ?
>> > 
>> 
>> Yes.  If a correct guest comes along and tries to use cirrus, it will break.
> 
> Right. Cirrus on ppc was used on PReP and Amiga for example though not
> many people really care about those platforms anymore. I'm not too
> worried at this point with that possibility but we shall know about it.

Emulating something incorrectly on purpose is wrong.  Use qxl or stdvga
(you can make either the default for ppc), but don't break cirrus or
rely on it being broken.

>> > As for the work I'm doing to brush up pci-vga a bit, I'm tempted to add
>> > an MMIO reg or a VBE config reg bit to allow configuring the endianness
>> > of the underlying fb with a default to what qemu does today.
>> 
>> What are those byteswapped apertures?  Some chipset thing that does the
>> byteswap?
> 
> The card itself. The 16M BAR is divided in 4 "apertures" (at least some
> Cirrus models do that including the one we emulate by default). One is
> no byteswap, two are 16-bit and 32-bit byteswap and the last one uses a
> specific byteswap for video overlay which I haven't looked at in detail.
> 
>> IIRC ppc has a bit in the TLB entry that tells it to byteswap.  Can't we
>> use it directly map the framebuffer with byteswapping?
> 
> Unfortunately only embedded ppc's have that :-(

Too bad :(

> 
> We can also make the fbdev/fbcon driver do the swapping in SW, but it's
> a relatively unusual code path and I don't think it works properly with
> X, I don't think it can be made to work properly with the generic X KMS
> at this point.
> 
> Now, cirrusdrmfb is already specific to the qemu cirrus variant in
> several ways, I wouldn't mind keeping it that way and if we "fix" the
> endianness model, maybe having a "hidden" register to flip it back to
> it's current mode of operation that cirrusdrmfb would use...

That's possible, but why not go all the way to qxl?

That will give you better graphics performance with no need to hack.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Benjamin Herrenschmidt]

On Mon, 2012-07-30 at 14:25 +0300, Avi Kivity wrote:

> > Right. Cirrus on ppc was used on PReP and Amiga for example though not
> > many people really care about those platforms anymore. I'm not too
> > worried at this point with that possibility but we shall know about it.
> 
> Emulating something incorrectly on purpose is wrong.  Use qxl or stdvga
> (you can make either the default for ppc), but don't break cirrus or
> rely on it being broken.

Well, that's what we do today it seems :-) But yes, it would be nice to
fix it, the problem is that the fix doesn't seem to be possible while
keeping a usable cirrus with KVM on big endian, at least as far as I
have investigated so far.

> >> > As for the work I'm doing to brush up pci-vga a bit, I'm tempted to add
> >> > an MMIO reg or a VBE config reg bit to allow configuring the endianness
> >> > of the underlying fb with a default to what qemu does today.
> >> 
> >> What are those byteswapped apertures?  Some chipset thing that does the
> >> byteswap?
> > 
> > The card itself. The 16M BAR is divided in 4 "apertures" (at least some
> > Cirrus models do that including the one we emulate by default). One is
> > no byteswap, two are 16-bit and 32-bit byteswap and the last one uses a
> > specific byteswap for video overlay which I haven't looked at in detail.
> > 
> >> IIRC ppc has a bit in the TLB entry that tells it to byteswap.  Can't we
> >> use it directly map the framebuffer with byteswapping?
> > 
> > Unfortunately only embedded ppc's have that :-(
> 
> Too bad :(
> 
> > 
> > We can also make the fbdev/fbcon driver do the swapping in SW, but it's
> > a relatively unusual code path and I don't think it works properly with
> > X, I don't think it can be made to work properly with the generic X KMS
> > at this point.
> > 
> > Now, cirrusdrmfb is already specific to the qemu cirrus variant in
> > several ways, I wouldn't mind keeping it that way and if we "fix" the
> > endianness model, maybe having a "hidden" register to flip it back to
> > it's current mode of operation that cirrusdrmfb would use...
> 
> That's possible, but why not go all the way to qxl?
>
> That will give you better graphics performance with no need to hack.

Well, qxl is pretty awful from what I can see so far. I'm more tempted
to continue improving qemu-vga, adding a virtio transport, and maybe
adding a way to tunnel spice into it if that makes sense but so far,
that's stuff was designed for Windows as far as I can tell and is pretty
horrible whatever way you look at it...

Cheers,
Ben.

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Avi Kivity]

On 07/30/2012 02:54 PM, Benjamin Herrenschmidt wrote:
>> 
>> > 
>> > We can also make the fbdev/fbcon driver do the swapping in SW, but it's
>> > a relatively unusual code path and I don't think it works properly with
>> > X, I don't think it can be made to work properly with the generic X KMS
>> > at this point.
>> > 
>> > Now, cirrusdrmfb is already specific to the qemu cirrus variant in
>> > several ways, I wouldn't mind keeping it that way and if we "fix" the
>> > endianness model, maybe having a "hidden" register to flip it back to
>> > it's current mode of operation that cirrusdrmfb would use...
>> 
>> That's possible, but why not go all the way to qxl?
>>
>> That will give you better graphics performance with no need to hack.
> 
> Well, qxl is pretty awful from what I can see so far. I'm more tempted
> to continue improving qemu-vga, adding a virtio transport, and maybe
> adding a way to tunnel spice into it if that makes sense but so far,
> that's stuff was designed for Windows as far as I can tell and is pretty
> horrible whatever way you look at it...

Let's balkanize some more then?

No, qxl is our paravirt vga, we should improve it instead of spawning
new ones (which will be horrible in the eyes of the next person to look
at them).  You should also be getting the drm driver for free.

http://spice-space.org/page/DRM

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Benjamin Herrenschmidt]

On Mon, 2012-07-30 at 14:58 +0300, Avi Kivity wrote:
> Let's balkanize some more then?
> 
> No, qxl is our paravirt vga, we should improve it instead of spawning
> new ones (which will be horrible in the eyes of the next person to look
> at them).  You should also be getting the drm driver for free.
> 
> http://spice-space.org/page/DRM

"Free" is a big word here, I wouldn't be surprised if it was totally
endian busted :-)

Why so much effort into a bad design on top of a crack transport btw ?
Is it just RH politics of there's a good reason hiding somewhere ? Some
kind of morbid fascination for anything Windows ?

People I've talked to around in the community seem to agree that some
minor improvements on qemu-vga are worthwhile, so I'm doing them,
nothing drastic, mostly having a mirror of the legacy IO registers in an
MMIO BAR so it's more usable for non-x86 platforms, and I'm about to add
some simplistic 2D blit facility so we can have a semi-decent fb console
over vnc. I can trivially do an equivalent of cirrusdrmfb for it so we
get X mode setting / RandR.

That gives us a baseline for mostly unaccelerated 2D.

Something tells me that getting that spice/qxl gunk will be more than a
trivial effort (but I might be wrong) and I'm reluctant to start
committing effort on it since so far I yet have to see it being actually
picked up by people.

Cheers,
Ben.

 

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Avi Kivity]

On 07/30/2012 03:08 PM, Benjamin Herrenschmidt wrote:
> On Mon, 2012-07-30 at 14:58 +0300, Avi Kivity wrote:
>> Let's balkanize some more then?
>> 
>> No, qxl is our paravirt vga, we should improve it instead of spawning
>> new ones (which will be horrible in the eyes of the next person to look
>> at them).  You should also be getting the drm driver for free.
>> 
>> http://spice-space.org/page/DRM
> 
> "Free" is a big word here, I wouldn't be surprised if it was totally
> endian busted :-)

Probably... well fixing endianness is the price of doing business if
you're big endian.

> 
> Why so much effort into a bad design on top of a crack transport btw ?
> Is it just RH politics of there's a good reason hiding somewhere ? Some
> kind of morbid fascination for anything Windows ?

It was indeed designed for Windows, which is the prime target for
desktop virtualization.  As to the rest, I don't know much about it,
except that no politicians were involved in the design.

> 
> People I've talked to around in the community seem to agree that some
> minor improvements on qemu-vga are worthwhile, so I'm doing them,
> nothing drastic, mostly having a mirror of the legacy IO registers in an
> MMIO BAR so it's more usable for non-x86 platforms, and I'm about to add
> some simplistic 2D blit facility so we can have a semi-decent fb console
> over vnc. I can trivially do an equivalent of cirrusdrmfb for it so we
> get X mode setting / RandR.
> 
> That gives us a baseline for mostly unaccelerated 2D.

That's fine.

> Something tells me that getting that spice/qxl gunk will be more than a
> trivial effort (but I might be wrong) and I'm reluctant to start
> committing effort on it since so far I yet have to see it being actually
> picked up by people.

I think it's prudent to wait until it's merged upstream, then we can
take another look to see why ppc is so special that it can't just use it
after fixing any endianness issues.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Benjamin Herrenschmidt]

On Mon, 2012-07-30 at 15:15 +0300, Avi Kivity wrote:
> > Something tells me that getting that spice/qxl gunk will be more than a
> > trivial effort (but I might be wrong) and I'm reluctant to start
> > committing effort on it since so far I yet have to see it being actually
> > picked up by people.
> 
> I think it's prudent to wait until it's merged upstream, then we can
> take another look to see why ppc is so special that it can't just use it
> after fixing any endianness issues. 

If it flies there's no reason why ppc can't use it, I'm just dubious at
this stage but we'll see.

I'd love to spend some time endian-fix it (some of that fixing can be
pretty hard when/if things aren't designed "properly" from the start)
but at this stage I don't have the bandwidth to dive into it, understand
the code in details etc... so that will have to wait unless I can find
somebody else to deal with it within the community or my employer :-)

Cheers,
Ben.

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Alon Levy]

On Mon, Jul 30, 2012 at 10:08:07PM +1000, Benjamin Herrenschmidt wrote:
> On Mon, 2012-07-30 at 14:58 +0300, Avi Kivity wrote:
> > Let's balkanize some more then?
> > 
> > No, qxl is our paravirt vga, we should improve it instead of spawning
> > new ones (which will be horrible in the eyes of the next person to look
> > at them).  You should also be getting the drm driver for free.
> > 
> > http://spice-space.org/page/DRM
> 
> "Free" is a big word here, I wouldn't be surprised if it was totally
> endian busted :-)
> 
> Why so much effort into a bad design on top of a crack transport btw ?
Could you elaborate about the design and transport issues that you see?

> Is it just RH politics of there's a good reason hiding somewhere ? Some
No politics AFAIK. Spice code may suck but it's doing a good job while
sucking, so we prefer to fix it unless a good design that makes it
easier to rebuild from scratch comes along. I'm all ears.

> kind of morbid fascination for anything Windows ?
We do have bad linux performance but the is work going to improve it, by
adding RENDER implementation to our driver and protocol additions, among
others.

> 
> People I've talked to around in the community seem to agree that some
> minor improvements on qemu-vga are worthwhile, so I'm doing them,
> nothing drastic, mostly having a mirror of the legacy IO registers in an
> MMIO BAR so it's more usable for non-x86 platforms, and I'm about to add
> some simplistic 2D blit facility so we can have a semi-decent fb console
> over vnc. I can trivially do an equivalent of cirrusdrmfb for it so we
> get X mode setting / RandR.
> 
> That gives us a baseline for mostly unaccelerated 2D.
> 
> Something tells me that getting that spice/qxl gunk will be more than a
> trivial effort (but I might be wrong) and I'm reluctant to start
> committing effort on it since so far I yet have to see it being actually
> picked up by people.

I'll be happy to elaborate on any part of qxl/spice that I understand
and help you help us improve it.

> 
> Cheers,
> Ben.
> 
>  
> 
> 

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Anthony Liguori]

Alon Levy <alevy@redhat.com> writes:

> On Mon, Jul 30, 2012 at 10:08:07PM +1000, Benjamin Herrenschmidt wrote:
>> On Mon, 2012-07-30 at 14:58 +0300, Avi Kivity wrote:
>> > Let's balkanize some more then?
>> > 
>> > No, qxl is our paravirt vga, we should improve it instead of spawning
>> > new ones (which will be horrible in the eyes of the next person to look
>> > at them).  You should also be getting the drm driver for free.
>> > 
>> > http://spice-space.org/page/DRM
>> 
>> "Free" is a big word here, I wouldn't be surprised if it was totally
>> endian busted :-)
>> 
>> Why so much effort into a bad design on top of a crack transport btw ?
> Could you elaborate about the design and transport issues that you
> see?

"bad design" is too simplistic.  The design is fine for what it
initially targetted.

But we have better ways of doing things now and a much broader scope.
There's no way we can support non-PCI architectures without heavy
lifting.  We also have the opportunity to improve things in such a way
to have a single unified graphics adapter which is a very good thing.

Basically, the core requirements are VESA-compatible + virtio for
command queue.

This gives us reasonable performance/features when a guest driver isn't
available and a path to supporting non-PCI platforms.

Non-PCI graphics is actually very common place.  Many embedded systems
have some form of framebuffer.  Having a graphics platform that supports
non-PCI systems would be quite valuable.

>
>> Is it just RH politics of there's a good reason hiding somewhere ? Some
> No politics AFAIK. Spice code may suck but it's doing a good job while
> sucking, so we prefer to fix it unless a good design that makes it
> easier to rebuild from scratch comes along. I'm all ears.
>
>> kind of morbid fascination for anything Windows ?
> We do have bad linux performance but the is work going to improve it, by
> adding RENDER implementation to our driver and protocol additions, among
> others.

FWIW, when I talk about PV graphics, I'm really only interested in
having a variable sized linear framebuffer (in a sane format--32-bit
8-bit pixels), 2d acceleration like blit and solid fill, and ARGB cursor
offload.

I strongly suspect that even Linux has this today with Spice.  My only
problem with Spice is that it's all or nothing.  If we could negotate
capabilities, I'd be very happy with it.

Offscreen bitmaps, YUV surfaces, etc, are all just icing on the cake.
Requring libspice for that kind of stuff is fine by me.

Regards,

Anthony Liguori

>> 
>> People I've talked to around in the community seem to agree that some
>> minor improvements on qemu-vga are worthwhile, so I'm doing them,
>> nothing drastic, mostly having a mirror of the legacy IO registers in an
>> MMIO BAR so it's more usable for non-x86 platforms, and I'm about to add
>> some simplistic 2D blit facility so we can have a semi-decent fb console
>> over vnc. I can trivially do an equivalent of cirrusdrmfb for it so we
>> get X mode setting / RandR.
>> 
>> That gives us a baseline for mostly unaccelerated 2D.
>> 
>> Something tells me that getting that spice/qxl gunk will be more than a
>> trivial effort (but I might be wrong) and I'm reluctant to start
>> committing effort on it since so far I yet have to see it being actually
>> picked up by people.
>
> I'll be happy to elaborate on any part of qxl/spice that I understand
> and help you help us improve it.
>
>> 
>> Cheers,
>> Ben.
>> 
>>  
>> 
>> 

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Benjamin Herrenschmidt]

On Mon, 2012-07-30 at 18:24 +0200, Alon Levy wrote:
> On Mon, Jul 30, 2012 at 10:08:07PM +1000, Benjamin Herrenschmidt wrote:
> > On Mon, 2012-07-30 at 14:58 +0300, Avi Kivity wrote:
> > > Let's balkanize some more then?
> > > 
> > > No, qxl is our paravirt vga, we should improve it instead of spawning
> > > new ones (which will be horrible in the eyes of the next person to look
> > > at them).  You should also be getting the drm driver for free.
> > > 
> > > http://spice-space.org/page/DRM
> > 
> > "Free" is a big word here, I wouldn't be surprised if it was totally
> > endian busted :-)
> > 
> > Why so much effort into a bad design on top of a crack transport btw ?
> Could you elaborate about the design and transport issues that you see?

So Anthony listed a few, the transport being inconsistent with all our
other paravirt model is also part of the problem, and the spice code
base just hurts my eyes. The fact that it's essentially GDI centric
makes it a non starter for me anyway.

> > Is it just RH politics of there's a good reason hiding somewhere ? Some
> No politics AFAIK. Spice code may suck but it's doing a good job while
> sucking, so we prefer to fix it unless a good design that makes it
> easier to rebuild from scratch comes along. I'm all ears.
> 
> > kind of morbid fascination for anything Windows ?
> We do have bad linux performance but the is work going to improve it, by
> adding RENDER implementation to our driver and protocol additions, among
> others.

Which still makes me feel like we should be doing something better
targeted at Linux exclusively.

> > People I've talked to around in the community seem to agree that some
> > minor improvements on qemu-vga are worthwhile, so I'm doing them,
> > nothing drastic, mostly having a mirror of the legacy IO registers in an
> > MMIO BAR so it's more usable for non-x86 platforms, and I'm about to add
> > some simplistic 2D blit facility so we can have a semi-decent fb console
> > over vnc. I can trivially do an equivalent of cirrusdrmfb for it so we
> > get X mode setting / RandR.
> > 
> > That gives us a baseline for mostly unaccelerated 2D.
> > 
> > Something tells me that getting that spice/qxl gunk will be more than a
> > trivial effort (but I might be wrong) and I'm reluctant to start
> > committing effort on it since so far I yet have to see it being actually
> > picked up by people.
> 
> I'll be happy to elaborate on any part of qxl/spice that I understand
> and help you help us improve it.

Well, the main thing to begin with from my perspective would be to make
it work on powerpc, and thus handle all the endianness issues etc... but
at this stage, it seems pretty far out.

Cheers,
Ben.

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Alon Levy]

On Tue, Jul 31, 2012 at 08:24:23AM +1000, Benjamin Herrenschmidt wrote:
> On Mon, 2012-07-30 at 18:24 +0200, Alon Levy wrote:
> > On Mon, Jul 30, 2012 at 10:08:07PM +1000, Benjamin Herrenschmidt wrote:
> > > On Mon, 2012-07-30 at 14:58 +0300, Avi Kivity wrote:
> > > > Let's balkanize some more then?
> > > > 
> > > > No, qxl is our paravirt vga, we should improve it instead of spawning
> > > > new ones (which will be horrible in the eyes of the next person to look
> > > > at them).  You should also be getting the drm driver for free.
> > > > 
> > > > http://spice-space.org/page/DRM
> > > 
> > > "Free" is a big word here, I wouldn't be surprised if it was totally
> > > endian busted :-)
> > > 
> > > Why so much effort into a bad design on top of a crack transport btw ?
> > Could you elaborate about the design and transport issues that you see?
> 
> So Anthony listed a few, the transport being inconsistent with all our
> other paravirt model is also part of the problem, and the spice code
> base just hurts my eyes. The fact that it's essentially GDI centric
> makes it a non starter for me anyway.
> 
> > > Is it just RH politics of there's a good reason hiding somewhere ? Some
> > No politics AFAIK. Spice code may suck but it's doing a good job while
> > sucking, so we prefer to fix it unless a good design that makes it
> > easier to rebuild from scratch comes along. I'm all ears.
> > 
> > > kind of morbid fascination for anything Windows ?
> > We do have bad linux performance but the is work going to improve it, by
> > adding RENDER implementation to our driver and protocol additions, among
> > others.
> 
> Which still makes me feel like we should be doing something better
> targeted at Linux exclusively.

Using virtio - +1 - I'm not actively working on it because of priorities
issues, but if it comes along I'll be happy to help making it work as
well as the existing device.

There is really not much different between virtio + linear framebuffer
and the qxl pci device, so I really don't see a lot of issues in
porting.

But other then this issue, I'm not sure what you think is windows
specific: The protocol is not the transport. GDI over a wire is
accurate, but adding RENDER makes it X over the wire (the wire here
being both qxl/future-virtio and the tcp spice protocol). And for the
future we will be passing bo's and TGSI command streams probably to the
host, rendering it and pushing video to the client. But that's still a
way off. Nothing window's specific there either (except the driver part
of course).

> 
> > > People I've talked to around in the community seem to agree that some
> > > minor improvements on qemu-vga are worthwhile, so I'm doing them,
> > > nothing drastic, mostly having a mirror of the legacy IO registers in an
> > > MMIO BAR so it's more usable for non-x86 platforms, and I'm about to add
> > > some simplistic 2D blit facility so we can have a semi-decent fb console
> > > over vnc. I can trivially do an equivalent of cirrusdrmfb for it so we
> > > get X mode setting / RandR.
> > > 
> > > That gives us a baseline for mostly unaccelerated 2D.
> > > 
> > > Something tells me that getting that spice/qxl gunk will be more than a
> > > trivial effort (but I might be wrong) and I'm reluctant to start
> > > committing effort on it since so far I yet have to see it being actually
> > > picked up by people.
> > 
> > I'll be happy to elaborate on any part of qxl/spice that I understand
> > and help you help us improve it.
> 
> Well, the main thing to begin with from my perspective would be to make
> it work on powerpc, and thus handle all the endianness issues etc... but
> at this stage, it seems pretty far out.

I'm sure we can make spice endianess clean. If you look at how it works,
server/red_parse.c reads from the device (so this is the single point to
change if the transport moves from QXLPHYSICAL meaning a encoded
slot/offset into a BAR to some other location identifier like a handle
when using virtio) and writes to spice specific structs, which are then,
like Anthony pointed out, stored in a tree for dropping operations that
have become obsolete before being sent to the client, and also put on a
pipe (queue) of messages to be sent to the client.

That would be the place to do any endianess convertions between guest
and host. The network protocol is LE (yes, counter to the norm, I know.
It is possible to add a capability to spice to change this though if we
really want. So only the initial handshake would have to be LE and the
rest would be BE).

> 
> Cheers,
> Ben.
> 
> 

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Avi Kivity]

On 07/31/2012 01:24 AM, Benjamin Herrenschmidt wrote:
> 
> So Anthony listed a few, the transport being inconsistent with all our
> other paravirt model is also part of the problem, and the spice code
> base just hurts my eyes. The fact that it's essentially GDI centric
> makes it a non starter for me anyway.

Cirrus too is gdi centric.  That's an "I dislike Windows" objection, not
a technical one.

>> We do have bad linux performance but the is work going to improve it, by
>> adding RENDER implementation to our driver and protocol additions, among
>> others.
> 
> Which still makes me feel like we should be doing something better
> targeted at Linux exclusively.

On x86 we run a wide variety of guests, targeting Linux exclusively
excludes too many guests.  Plus the let's throw everything away and
start from scratch until we move to something else before it's really
completed strategy is incredibly wasteful.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Gerd Hoffmann]

On 07/30/12 14:08, Benjamin Herrenschmidt wrote:
> On Mon, 2012-07-30 at 14:58 +0300, Avi Kivity wrote:
>> Let's balkanize some more then?
>>
>> No, qxl is our paravirt vga, we should improve it instead of spawning
>> new ones (which will be horrible in the eyes of the next person to look
>> at them).  You should also be getting the drm driver for free.
>>
>> http://spice-space.org/page/DRM
> 
> "Free" is a big word here, I wouldn't be surprised if it was totally
> endian busted :-)

Well, not totally.  But it used to be, and although a bunch of cleanup &
fixing happened already I doubt spice will run out-of-the-box on
bigendian machines.

> People I've talked to around in the community seem to agree that some
> minor improvements on qemu-vga are worthwhile, so I'm doing them,
> nothing drastic, mostly having a mirror of the legacy IO registers in an
> MMIO BAR so it's more usable for non-x86 platforms, and I'm about to add
> some simplistic 2D blit facility so we can have a semi-decent fb console
> over vnc. I can trivially do an equivalent of cirrusdrmfb for it so we
> get X mode setting / RandR.
> 
> That gives us a baseline for mostly unaccelerated 2D.

Makes sense.

> Something tells me that getting that spice/qxl gunk will be more than a
> trivial effort (but I might be wrong) and I'm reluctant to start
> committing effort on it since so far I yet have to see it being actually
> picked up by people.

For a decent paravirt gfx card (supporting more than just basic X11 &
fbcon with scroll-bitblit) going qxl makes more sense.  Oh, and most of
the endian issues in the spice code base are *not* in hw/qxl*.[ch], but
in the spice-server library and spice client code.  So plumbing in
spice-over-virtio into stdvga doesn't buy you much ...

cheers,
  Gerd

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Anthony Liguori]

Avi Kivity <avi@redhat.com> writes:

> On 07/30/2012 02:54 PM, Benjamin Herrenschmidt wrote:
>>> 
>>> > 
>>> > We can also make the fbdev/fbcon driver do the swapping in SW, but it's
>>> > a relatively unusual code path and I don't think it works properly with
>>> > X, I don't think it can be made to work properly with the generic X KMS
>>> > at this point.
>>> > 
>>> > Now, cirrusdrmfb is already specific to the qemu cirrus variant in
>>> > several ways, I wouldn't mind keeping it that way and if we "fix" the
>>> > endianness model, maybe having a "hidden" register to flip it back to
>>> > it's current mode of operation that cirrusdrmfb would use...
>>> 
>>> That's possible, but why not go all the way to qxl?
>>>
>>> That will give you better graphics performance with no need to hack.
>> 
>> Well, qxl is pretty awful from what I can see so far. I'm more tempted
>> to continue improving qemu-vga, adding a virtio transport, and maybe
>> adding a way to tunnel spice into it if that makes sense but so far,
>> that's stuff was designed for Windows as far as I can tell and is pretty
>> horrible whatever way you look at it...
>
> Let's balkanize some more then?

Minor improvements to stdvga actual help qxl (presumably).  qxl still
provides a vga interface which is used when guest drivers aren't
available.

It's not clear to me why it doesn't enable VBE but presumably if it did,
then accelerations could be mapped through VBE.

>
> No, qxl is our paravirt vga, we should improve it instead of spawning
> new ones (which will be horrible in the eyes of the next person to look
> at them).  You should also be getting the drm driver for free.

Actually, Gerd et al have expressed interest in moving to a virtio-based
device model for Spice in the past.

I think done correctly, it could help bring graphics to other platforms
like S390 where PCI doesn't exist and will never exist.

Regards,

Anthony Liguori

>
> http://spice-space.org/page/DRM
>
> -- 
> error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Avi Kivity]

On 07/30/2012 04:18 PM, Anthony Liguori wrote:
> Avi Kivity <avi@redhat.com> writes:
> 
>> On 07/30/2012 02:54 PM, Benjamin Herrenschmidt wrote:
>>>> 
>>>> > 
>>>> > We can also make the fbdev/fbcon driver do the swapping in SW, but it's
>>>> > a relatively unusual code path and I don't think it works properly with
>>>> > X, I don't think it can be made to work properly with the generic X KMS
>>>> > at this point.
>>>> > 
>>>> > Now, cirrusdrmfb is already specific to the qemu cirrus variant in
>>>> > several ways, I wouldn't mind keeping it that way and if we "fix" the
>>>> > endianness model, maybe having a "hidden" register to flip it back to
>>>> > it's current mode of operation that cirrusdrmfb would use...
>>>> 
>>>> That's possible, but why not go all the way to qxl?
>>>>
>>>> That will give you better graphics performance with no need to hack.
>>> 
>>> Well, qxl is pretty awful from what I can see so far. I'm more tempted
>>> to continue improving qemu-vga, adding a virtio transport, and maybe
>>> adding a way to tunnel spice into it if that makes sense but so far,
>>> that's stuff was designed for Windows as far as I can tell and is pretty
>>> horrible whatever way you look at it...
>>
>> Let's balkanize some more then?
> 
> Minor improvements to stdvga actual help qxl (presumably).  qxl still
> provides a vga interface which is used when guest drivers aren't
> available.

The premise is that guest drivers will be used, otherwise you may as
well stay with stdvga.

> It's not clear to me why it doesn't enable VBE but presumably if it did,
> then accelerations could be mapped through VBE.

I believe the idea is that you don't want to map the framebuffer into
the guest, this allows one-directional communication so you can defer
rendering to the client and not suffer from the latency.  But I may be
mixing things up.

> 
>>
>> No, qxl is our paravirt vga, we should improve it instead of spawning
>> new ones (which will be horrible in the eyes of the next person to look
>> at them).  You should also be getting the drm driver for free.
> 
> Actually, Gerd et al have expressed interest in moving to a virtio-based
> device model for Spice in the past.
> 
> I think done correctly, it could help bring graphics to other platforms
> like S390 where PCI doesn't exist and will never exist.

I thought the plan was to render into a virtual card punch, then flip
through the cards at 60 fps?

Virtio makes sense for qxl, but for now we have the original pci model
which I don't see a reason why it can't work for ppc.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Anthony Liguori]

Avi Kivity <avi@redhat.com> writes:

> On 07/30/2012 04:18 PM, Anthony Liguori wrote:
>> Avi Kivity <avi@redhat.com> writes:
>> 
>>> On 07/30/2012 02:54 PM, Benjamin Herrenschmidt wrote:
>>>>> 
>>>>> > 
>>>>> > We can also make the fbdev/fbcon driver do the swapping in SW, but it's
>>>>> > a relatively unusual code path and I don't think it works properly with
>>>>> > X, I don't think it can be made to work properly with the generic X KMS
>>>>> > at this point.
>>>>> > 
>>>>> > Now, cirrusdrmfb is already specific to the qemu cirrus variant in
>>>>> > several ways, I wouldn't mind keeping it that way and if we "fix" the
>>>>> > endianness model, maybe having a "hidden" register to flip it back to
>>>>> > it's current mode of operation that cirrusdrmfb would use...
>>>>> 
>>>>> That's possible, but why not go all the way to qxl?
>>>>>
>>>>> That will give you better graphics performance with no need to hack.
>>>> 
>>>> Well, qxl is pretty awful from what I can see so far. I'm more tempted
>>>> to continue improving qemu-vga, adding a virtio transport, and maybe
>>>> adding a way to tunnel spice into it if that makes sense but so far,
>>>> that's stuff was designed for Windows as far as I can tell and is pretty
>>>> horrible whatever way you look at it...
>>>
>>> Let's balkanize some more then?
>> 
>> Minor improvements to stdvga actual help qxl (presumably).  qxl still
>> provides a vga interface which is used when guest drivers aren't
>> available.
>
> The premise is that guest drivers will be used, otherwise you may as
> well stay with stdvga.

The trouble is predicting which guests have drivers and which guests
don't.  Having a VGA model that could be enabled universally with good
VBE support for guests without drivers would be a very nice default
model.

We've never made the switch because WinXP doesn't have VESA support
natively.  But we're slowly getting to the point in time where it's
acceptable to require a special command line option for running WinXP
guests such that we could consider changing the default machine type.

>> It's not clear to me why it doesn't enable VBE but presumably if it did,
>> then accelerations could be mapped through VBE.
>
> I believe the idea is that you don't want to map the framebuffer into
> the guest, this allows one-directional communication so you can defer
> rendering to the client and not suffer from the latency.  But I may be
> mixing things up.

Hrm, that seems like an odd strategy for legacy VGA.  Spice isn't
remoting every pixel update, right?  I would assume it's using the same
logic as the rest of the VGA cards and doing bulk updates based on the
refresh timer.  In that case, exposing the framebuffer shouldn't matter
at all.

>>> No, qxl is our paravirt vga, we should improve it instead of spawning
>>> new ones (which will be horrible in the eyes of the next person to look
>>> at them).  You should also be getting the drm driver for free.
>> 
>> Actually, Gerd et al have expressed interest in moving to a virtio-based
>> device model for Spice in the past.
>> 
>> I think done correctly, it could help bring graphics to other platforms
>> like S390 where PCI doesn't exist and will never exist.
>
> I thought the plan was to render into a virtual card punch, then flip
> through the cards at 60 fps?

48.5 fps actually.  In 1960 when the system was designed, there were two
competing frame rates.  Everything else standardized on 60Hz but S390
still uses the old 48.5 refresh rate (and it's obviously superior).

>
> Virtio makes sense for qxl, but for now we have the original pci model
> which I don't see a reason why it can't work for ppc.

I'm sure it can work for PPC given enough effort.  But I think the
question becomes, why not invest that effort in moving qxl to the
standard transport that the rest of our PV devices use.

Regards,

Anthony Liguori

>
> -- 
> error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Avi Kivity]

On 07/30/2012 04:45 PM, Anthony Liguori wrote:
> Avi Kivity <avi@redhat.com> writes:
> 
>> On 07/30/2012 04:18 PM, Anthony Liguori wrote:
>>> Avi Kivity <avi@redhat.com> writes:
>>> 
>>>> On 07/30/2012 02:54 PM, Benjamin Herrenschmidt wrote:
>>>>>> 
>>>>>> > 
>>>>>> > We can also make the fbdev/fbcon driver do the swapping in SW, but it's
>>>>>> > a relatively unusual code path and I don't think it works properly with
>>>>>> > X, I don't think it can be made to work properly with the generic X KMS
>>>>>> > at this point.
>>>>>> > 
>>>>>> > Now, cirrusdrmfb is already specific to the qemu cirrus variant in
>>>>>> > several ways, I wouldn't mind keeping it that way and if we "fix" the
>>>>>> > endianness model, maybe having a "hidden" register to flip it back to
>>>>>> > it's current mode of operation that cirrusdrmfb would use...
>>>>>> 
>>>>>> That's possible, but why not go all the way to qxl?
>>>>>>
>>>>>> That will give you better graphics performance with no need to hack.
>>>>> 
>>>>> Well, qxl is pretty awful from what I can see so far. I'm more tempted
>>>>> to continue improving qemu-vga, adding a virtio transport, and maybe
>>>>> adding a way to tunnel spice into it if that makes sense but so far,
>>>>> that's stuff was designed for Windows as far as I can tell and is pretty
>>>>> horrible whatever way you look at it...
>>>>
>>>> Let's balkanize some more then?
>>> 
>>> Minor improvements to stdvga actual help qxl (presumably).  qxl still
>>> provides a vga interface which is used when guest drivers aren't
>>> available.
>>
>> The premise is that guest drivers will be used, otherwise you may as
>> well stay with stdvga.
> 
> The trouble is predicting which guests have drivers and which guests
> don't.  Having a VGA model that could be enabled universally with good
> VBE support for guests without drivers would be a very nice default
> model.

I agree.  Hopefully it won't be difficult to get the guest to unmap, or
maybe we can just unregister the direct RAM mapping in qemu.

> We've never made the switch because WinXP doesn't have VESA support
> natively.  But we're slowly getting to the point in time where it's
> acceptable to require a special command line option for running WinXP
> guests such that we could consider changing the default machine type.

Yes.

> 
>>> It's not clear to me why it doesn't enable VBE but presumably if it did,
>>> then accelerations could be mapped through VBE.
>>
>> I believe the idea is that you don't want to map the framebuffer into
>> the guest, this allows one-directional communication so you can defer
>> rendering to the client and not suffer from the latency.  But I may be
>> mixing things up.
> 
> Hrm, that seems like an odd strategy for legacy VGA.  Spice isn't
> remoting every pixel update, right?  I would assume it's using the same
> logic as the rest of the VGA cards and doing bulk updates based on the
> refresh timer.  In that case, exposing the framebuffer shouldn't matter
> at all.

I'd assume so too, but we need to make sure the framebuffer is unmapped
when in accelerated mode, or at least the guest has no expectations of
using it.

> 
>>>> No, qxl is our paravirt vga, we should improve it instead of spawning
>>>> new ones (which will be horrible in the eyes of the next person to look
>>>> at them).  You should also be getting the drm driver for free.
>>> 
>>> Actually, Gerd et al have expressed interest in moving to a virtio-based
>>> device model for Spice in the past.
>>> 
>>> I think done correctly, it could help bring graphics to other platforms
>>> like S390 where PCI doesn't exist and will never exist.
>>
>> I thought the plan was to render into a virtual card punch, then flip
>> through the cards at 60 fps?
> 
> 48.5 fps actually.  In 1960 when the system was designed, there were two
> competing frame rates.  Everything else standardized on 60Hz but S390
> still uses the old 48.5 refresh rate (and it's obviously superior).

s390 can outwierd anyone and anything.

> 
>>
>> Virtio makes sense for qxl, but for now we have the original pci model
>> which I don't see a reason why it can't work for ppc.
> 
> I'm sure it can work for PPC given enough effort.  But I think the
> question becomes, why not invest that effort in moving qxl to the
> standard transport that the rest of our PV devices use.

The drm drivers for the current model are needed anyway; so moving to
virtio is extra effort, not an alternative.

Note virtio doesn't support mapping framebuffers yet, or the entire vga
compatibility stuff, so the pc-oriented card will have to be a mix of
virtio and stdvga multiplexed on one pci card (maybe two functions, but
I'd rather avoid that).

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Anthony Liguori]

Avi Kivity <avi@redhat.com> writes:

>>> Virtio makes sense for qxl, but for now we have the original pci model
>>> which I don't see a reason why it can't work for ppc.
>> 
>> I'm sure it can work for PPC given enough effort.  But I think the
>> question becomes, why not invest that effort in moving qxl to the
>> standard transport that the rest of our PV devices use.
>
> The drm drivers for the current model are needed anyway; so moving to
> virtio is extra effort, not an alternative.

This is just a point in time statement.  If we were serious about using
virtio then we could quickly introduce a virtio transport and only
target the DRM drivers at the virtio transport.

> Note virtio doesn't support mapping framebuffers yet

Yes.  I haven't seen a good proposal yet on how to handle this.  I think
this is the main problem to solve.

> or the entire vga compatibility stuff

This is actually independent of virtio.  A virtio-pci device could
expose it's class code as a VGA adapter and also handle I/O accesses for
the legacy region.  This is strictly a PC-ism.

For non-virtio-pci versions of the device, the legacy I/O area would not exist.

> so the pc-oriented card will have to be a mix of
> virtio and stdvga multiplexed on one pci card (maybe two functions, but
> I'd rather avoid that).

Yes.  We could modify stdvga to expose the VGA ram area as the second
bar and make the first bar a virtio-pci compatible area.  This would
require modifying the VGA bios to understand the change but otherwise,
should be compatible.

It would take modeling VGACommonState as a proper device and then it's a
pretty simple process of embedding a VGACommonState within a virtio-pci
device.  It should work fairly well.

It gets a little complicated in terms of who owns the DisplayState but
that's a solvable problem.

Regards,

Anthony Liguori

>
> -- 
> error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Avi Kivity]

On 07/30/2012 05:29 PM, Anthony Liguori wrote:
> Avi Kivity <avi@redhat.com> writes:
> 
>>>> Virtio makes sense for qxl, but for now we have the original pci model
>>>> which I don't see a reason why it can't work for ppc.
>>> 
>>> I'm sure it can work for PPC given enough effort.  But I think the
>>> question becomes, why not invest that effort in moving qxl to the
>>> standard transport that the rest of our PV devices use.
>>
>> The drm drivers for the current model are needed anyway; so moving to
>> virtio is extra effort, not an alternative.
> 
> This is just a point in time statement.  If we were serious about using
> virtio then we could quickly introduce a virtio transport and only
> target the DRM drivers at the virtio transport.

That doesn't help all the deployed hypervisors out there.  IMO we're
mature enough, and the difference doesn't warrant a new interface.

>> Note virtio doesn't support mapping framebuffers yet
> 
> Yes.  I haven't seen a good proposal yet on how to handle this.  I think
> this is the main problem to solve.

It doesn't seem to be such a huge problem, though it does turn virtio
into a respec'ed PCI.

> 
>> or the entire vga compatibility stuff
> 
> This is actually independent of virtio.  A virtio-pci device could
> expose it's class code as a VGA adapter and also handle I/O accesses for
> the legacy region.  This is strictly a PC-ism.

We have to share the BAR space with VGA; not a huge problem.


-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Anthony Liguori]

Avi Kivity <avi@redhat.com> writes:

> On 07/30/2012 05:29 PM, Anthony Liguori wrote:
>> Avi Kivity <avi@redhat.com> writes:
>> 
>>>>> Virtio makes sense for qxl, but for now we have the original pci model
>>>>> which I don't see a reason why it can't work for ppc.
>>>> 
>>>> I'm sure it can work for PPC given enough effort.  But I think the
>>>> question becomes, why not invest that effort in moving qxl to the
>>>> standard transport that the rest of our PV devices use.
>>>
>>> The drm drivers for the current model are needed anyway; so moving to
>>> virtio is extra effort, not an alternative.
>> 
>> This is just a point in time statement.  If we were serious about using
>> virtio then we could quickly introduce a virtio transport and only
>> target the DRM drivers at the virtio transport.
>
> That doesn't help all the deployed hypervisors out there.  IMO we're
> mature enough, and the difference doesn't warrant a new interface.

QXL has a lot of short comings.  Here's a short list:

- It's 100% PC centric.  It requires PCI and is completely oblivious to
  endianness.

- It's all-or-nothing with respect to Spice support.  libspice is a big
  external dependency.  It cannot be mandated as a QEMU requirement
  because it's not portable enough.  This means we can never make qxl
  the default device because we can't guarantee it's there.

- There's no obvious way to map to non-PCI systems (either S390 or
  embedded platforms).

I'm not saying that we should remove qxl.c from QEMU.  We can continue
to support that ABI forever.

But there's a lot of value in a new graphics interface that uses virtio
and negotiates support for the Spice protocol.  That way, if QEMU
doesn't have Spice support, the feature won't be exposed to the guest
and you can still have a legacy VGA interface.

Then we can change the default.  Basic 2d commands (like blit and solid
fill) can be done without going through libspice.

Then we can stop messing around with having multiple display types.
It would be a huge usability improvement and would allow us to focus on
a single graphics adapter for all architectures.

>
>>> Note virtio doesn't support mapping framebuffers yet
>> 
>> Yes.  I haven't seen a good proposal yet on how to handle this.  I think
>> this is the main problem to solve.
>
> It doesn't seem to be such a huge problem, though it does turn virtio
> into a respec'ed PCI.

Virtio was originally designed to be a DMA API (although not ABI).  From
a virtio-pci perspective, adding a large memory area that can be
referenced is not a big deal at all.  You can take PFNs into the memory
area and just handle it like you would any other address reference.

But for the virtio API within Linux, it presents a challenge.
Originally, there was a desire to support DMA-based interfaces like
Xen's grant tables or PowerVM's TCE transfers.

While there have been proof of concepts, it's never landed in an
upstream kernel.  I'd be perfectly happy forcing the virtio API to
assume the ability to share large areas of memory between the host and
guest and eliminate the possibility to support all types of hypervisors
for some devices.

(While Xen supports shared memory, PowerVM does not--I don't give a damn
about supporting PowerVM FWIW).

Rusty, what do you think about the above?

Regards,

Anthony Liguori

>
>> 
>>> or the entire vga compatibility stuff
>> 
>> This is actually independent of virtio.  A virtio-pci device could
>> expose it's class code as a VGA adapter and also handle I/O accesses for
>> the legacy region.  This is strictly a PC-ism.
>
> We have to share the BAR space with VGA; not a huge problem.
>
>
> -- 
> error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Rusty Russell]

On Mon, 30 Jul 2012 11:01:20 -0500, Anthony Liguori <anthony@codemonkey.ws> wrote:
> Avi Kivity <avi@redhat.com> writes:
> > It doesn't seem to be such a huge problem, though it does turn virtio
> > into a respec'ed PCI.
> 
> Virtio was originally designed to be a DMA API (although not ABI).  From
> a virtio-pci perspective, adding a large memory area that can be
> referenced is not a big deal at all.  You can take PFNs into the memory
> area and just handle it like you would any other address reference.
> 
> But for the virtio API within Linux, it presents a challenge.
> Originally, there was a desire to support DMA-based interfaces like
> Xen's grant tables or PowerVM's TCE transfers.
> 
> While there have been proof of concepts, it's never landed in an
> upstream kernel.  I'd be perfectly happy forcing the virtio API to
> assume the ability to share large areas of memory between the host and
> guest and eliminate the possibility to support all types of hypervisors
> for some devices.
> 
> (While Xen supports shared memory, PowerVM does not--I don't give a damn
> about supporting PowerVM FWIW).
> 
> Rusty, what do you think about the above?

Shared memory is an efficiency thing, not a requirement.  If the virtio
side-channel tells the device about the location of framebuffer
changes, it could still be quite efficient.

Cheers,
Rusty.

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Benjamin Herrenschmidt]

On Tue, 2012-07-31 at 09:17 +0930, Rusty Russell wrote:
> 
> Shared memory is an efficiency thing, not a requirement.  If the
> virtio
> side-channel tells the device about the location of framebuffer
> changes, it could still be quite efficient.

But potentially tricky to get things like BIOSes working etc... you want
to keep basic access simple.

At this point, I'm hesitating between those two models:

1- Keep qemu-vga and hack virtio to have the virtio BAR not be BAR 0
(BAR 3 in my current code) and be an optional feature (you can get basic
qemu-vga without virtio if you disable CONFIG_VIRTIO for example, you
just don't get acceleration). It's still a PCI device and mode setting
is done via either legacy PIO or MMIO registers, so it's not necessarily
s390 friendly, though at least the MMIO registers are BAR mapped.

The advantage is that it's compatible with the existing qemu PCI VGA,
however it makes it harder to "Fix" the vendor ID, among other cons.

2- Create a -new- virtio-vga (ie, leave vga-std alone and eventually
deprecate it). This is basically a virtio-pci base with the virtio BAR
as BAR 0 (so compatible with overall virtio) and we get a proper virtio
vendor/device. However, I do add a second BAR for the linear framebuffer
and it supports optionally legacy IO ports for mode setting.

For non-legacy mode setting, I either keep another MMIO BAR for doing a
BAR-variant of the legacy IO ports or provide a simple virtio packet
format that can be used to populate the VGA/VBE registers & palette.

Finally, I adapt the vbios so that VBE & co still works, including text
mode.

The latter sounds like a better long term approach, however it lacks
backward compat with qemu-vga, but I doubt it's a big deal especially if
we provide a working VBE ROM for x86.

Cheers,
Ben.

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Gerd Hoffmann]

  Hi,

> The latter sounds like a better long term approach, however it lacks
> backward compat with qemu-vga, but I doubt it's a big deal especially if
> we provide a working VBE ROM for x86.

A vbe rom isn't a big deal.  You probably want support the 0x01CE and
0x01CF ports (on x86) so the vgabios running in real mode can easily
reach the bochs interface registers without a protected mode round trip
for mmio access.

I *think* for seavgabios this needs just a rebuild with an approximate
configuration (see roms/Makefile and roms/config.vga.*).  The code
already checks whenever bar 0 is actually memory not mmio because this
is needed for the vmware vga too.

HTH,
  Gerd

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Benjamin Herrenschmidt]

On Mon, 2012-08-06 at 16:02 +0200, Gerd Hoffmann wrote:
> 
> A vbe rom isn't a big deal.  You probably want support the 0x01CE and
> 0x01CF ports (on x86) so the vgabios running in real mode can easily
> reach the bochs interface registers without a protected mode round
> trip
> for mmio access.
> 
> I *think* for seavgabios this needs just a rebuild with an approximate
> configuration (see roms/Makefile and roms/config.vga.*).  The code
> already checks whenever bar 0 is actually memory not mmio because this
> is needed for the vmware vga too. 

My current layout is

 BAR 0 : virtio (including commands to access all the registers)

 BAR 1 : MMIO regs (there just in case, they might come in handy
                    to make the DRM driver more oops-resistant,
                    but they are unused by my WIP code)

 BAR 2 : framebuffer for scanout

I have legacy VGA & Bochs VBE ports wired up as well (it's trivial), did
a quick vgabios change and that works, I haven't looked at seabios yet
but it looks trivial enough.

I have a port of SLOF that uses the virtio channel to configure the
linear mode (using the VBE regs via the virtio tunnel) and that works.

I've started putting together a DRM +fb driver for the guest side, but
that's not done yet.

I've also done a trivial 2D blitter mostly for use by the fb and/or SLOF
but I haven't wired it up yet.

Cheers,
Ben.

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Andreas Färber]

Am 30.07.2012 18:01, schrieb Anthony Liguori:
> I'm not saying that we should remove qxl.c from QEMU.  We can continue
> to support that ABI forever.
> 
> But there's a lot of value in a new graphics interface that uses virtio
> and negotiates support for the Spice protocol.  That way, if QEMU
> doesn't have Spice support, the feature won't be exposed to the guest
> and you can still have a legacy VGA interface.
> 
> Then we can change the default.  Basic 2d commands (like blit and solid
> fill) can be done without going through libspice.
> 
> Then we can stop messing around with having multiple display types.
> It would be a huge usability improvement and would allow us to focus on
> a single graphics adapter for all architectures.

Focus maybe, but legacy and non-x86 systems will still need the existing
cirrus/vga graphics adapters. Also a reminder that Hervé and me had
prepared an IBM8514 PCI card that we will still be interested in merging
for PReP whatever virtio devices come along.

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Gerd Hoffmann]

  Hi,

> QXL has a lot of short comings.  Here's a short list:
> 
> - It's 100% PC centric.  It requires PCI and is completely oblivious to
>   endianness.

No.  The endianess is actually clearly defined.  It's little endian for
both guest/host interface (aka qxl) and the network protocol.

So it is "only" the libspice code which needs fixing.  Should be not as
bad as it sounds.  There is a code generator for writing/parsing the
network protocol, winding up native endian <=> little endian handling
there shouldn't be that hard.  Likewise there is a piece of code
translating qxl device structs into internal spice-server structs
(applying sanity checks along the way).  There we can hook up the
byteswapping for the guest/host interface.

> - It's all-or-nothing with respect to Spice support.  libspice is a big
>   external dependency.  It cannot be mandated as a QEMU requirement
>   because it's not portable enough.  This means we can never make qxl
>   the default device because we can't guarantee it's there.

Indeed.

> But there's a lot of value in a new graphics interface that uses virtio
> and negotiates support for the Spice protocol.  That way, if QEMU
> doesn't have Spice support, the feature won't be exposed to the guest
> and you can still have a legacy VGA interface.

What does it buy us?

Even with -vga std-which-might-have-spice-over-virtio-support you still
have to figure whenever qemu has spice support and pass / don't pass
-spice $opts accordingly.

> Then we can change the default.  Basic 2d commands (like blit and solid
> fill) can be done without going through libspice.

We can create a set of basic 2d accel commands and implement them in
both stdvga and qxl, where qxl would translate them into spice ops of
course.

> Then we can stop messing around with having multiple display types.
> It would be a huge usability improvement and would allow us to focus on
> a single graphics adapter for all architectures.

Improving stdvga to support basic 2d accel isn't that much effort.  I
think it is worth doing it, even when it is obsoleted by a redesigned /
rewritten qxl2 some day.

cheers,
  Gerd

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Anthony Liguori]

Gerd Hoffmann <kraxel@redhat.com> writes:

>   Hi,
>
>> QXL has a lot of short comings.  Here's a short list:
>> 
>> - It's 100% PC centric.  It requires PCI and is completely oblivious to
>>   endianness.
>
> No.  The endianess is actually clearly defined.  It's little endian for
> both guest/host interface (aka qxl) and the network protocol.

Right, because it was designed without regard to endianness and then
declared as little endian since it was written that's what x86 happens
to be :-)

One of the nice properties of virtio is that the ring is defined to be
in guest-endian.  This does make access a bit faster.

> So it is "only" the libspice code which needs fixing.  Should be not as
> bad as it sounds.  There is a code generator for writing/parsing the
> network protocol, winding up native endian <=> little endian handling
> there shouldn't be that hard.  Likewise there is a piece of code
> translating qxl device structs into internal spice-server structs
> (applying sanity checks along the way).  There we can hook up the
> byteswapping for the guest/host interface.

I have no doubt that it's fixable and ought to be fixed.

>
>> - It's all-or-nothing with respect to Spice support.  libspice is a big
>>   external dependency.  It cannot be mandated as a QEMU requirement
>>   because it's not portable enough.  This means we can never make qxl
>>   the default device because we can't guarantee it's there.
>
> Indeed.
>
>> But there's a lot of value in a new graphics interface that uses virtio
>> and negotiates support for the Spice protocol.  That way, if QEMU
>> doesn't have Spice support, the feature won't be exposed to the guest
>> and you can still have a legacy VGA interface.
>
> What does it buy us?

A single guest driver/configuration that works whether or not libspice
is available on the host.

A single graphics driver configuration which means that users can stop
worry about what graphics card type is specified.

> Even with -vga std-which-might-have-spice-over-virtio-support you still
> have to figure whenever qemu has spice support and pass / don't pass
> -spice $opts accordingly.

If you want to configure a Spice server, yes.  If you're just doing
local graphics, it shouldn't matter, right?

>> Then we can change the default.  Basic 2d commands (like blit and solid
>> fill) can be done without going through libspice.
>
> We can create a set of basic 2d accel commands and implement them in
> both stdvga and qxl, where qxl would translate them into spice ops of
> course.
>
>> Then we can stop messing around with having multiple display types.
>> It would be a huge usability improvement and would allow us to focus on
>> a single graphics adapter for all architectures.
>
> Improving stdvga to support basic 2d accel isn't that much effort.  I
> think it is worth doing it, even when it is obsoleted by a redesigned /
> rewritten qxl2 some day.

I think we're pretty much in agreement here.  I think if we improve
stdvga by making it a virtio device, then it becomes pretty easy to
eventually make it qxl2.

Regards,

Anthony Liguori

>
> cheers,
>   Gerd

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Alon Levy]

On Mon, Jul 30, 2012 at 09:29:01AM -0500, Anthony Liguori wrote:
> Avi Kivity <avi@redhat.com> writes:
> 
> >>> Virtio makes sense for qxl, but for now we have the original pci model
> >>> which I don't see a reason why it can't work for ppc.
> >> 
> >> I'm sure it can work for PPC given enough effort.  But I think the
> >> question becomes, why not invest that effort in moving qxl to the
> >> standard transport that the rest of our PV devices use.
> >
> > The drm drivers for the current model are needed anyway; so moving to
> > virtio is extra effort, not an alternative.
> 
> This is just a point in time statement.  If we were serious about using
> virtio then we could quickly introduce a virtio transport and only
> target the DRM drivers at the virtio transport.
> 
> > Note virtio doesn't support mapping framebuffers yet
> 
> Yes.  I haven't seen a good proposal yet on how to handle this.  I think
> this is the main problem to solve.

The only thing I can add here is perhaps we could use scatter-gather
lists of pages instead of large framebuffer. When just passing commands
from guest->host->client this would mean guests constructs list -> host
reads from list to socket -> client unchanged. For rendering locally on
the host (client wants a screenshot, host wants a screenshot) this would
be a lot of work, basically having a non linear framebuffer version of
pixman. The upside is that you don't need to modify virtio and can use
guest ram.

> 
> > or the entire vga compatibility stuff
> 
> This is actually independent of virtio.  A virtio-pci device could
> expose it's class code as a VGA adapter and also handle I/O accesses for
> the legacy region.  This is strictly a PC-ism.
> 
> For non-virtio-pci versions of the device, the legacy I/O area would not exist.
> 
> > so the pc-oriented card will have to be a mix of
> > virtio and stdvga multiplexed on one pci card (maybe two functions, but
> > I'd rather avoid that).
> 
> Yes.  We could modify stdvga to expose the VGA ram area as the second
> bar and make the first bar a virtio-pci compatible area.  This would
> require modifying the VGA bios to understand the change but otherwise,
> should be compatible.
> 
> It would take modeling VGACommonState as a proper device and then it's a
> pretty simple process of embedding a VGACommonState within a virtio-pci
> device.  It should work fairly well.
> 
> It gets a little complicated in terms of who owns the DisplayState but
> that's a solvable problem.
> 
> Regards,
> 
> Anthony Liguori
> 
> >
> > -- 
> > error compiling committee.c: too many arguments to function
> 

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Benjamin Herrenschmidt]

On Mon, 2012-07-30 at 16:55 +0300, Avi Kivity wrote:
> > The trouble is predicting which guests have drivers and which guests
> > don't.  Having a VGA model that could be enabled universally with good
> > VBE support for guests without drivers would be a very nice default
> > model.
> 
> I agree.  Hopefully it won't be difficult to get the guest to unmap, or
> maybe we can just unregister the direct RAM mapping in qemu.

I don't understand that part... why on earth would the fb has to be
unmapped ?

> > We've never made the switch because WinXP doesn't have VESA support
> > natively.  But we're slowly getting to the point in time where it's
> > acceptable to require a special command line option for running WinXP
> > guests such that we could consider changing the default machine type.
> 
> Yes.
> 
> > 
> >>> It's not clear to me why it doesn't enable VBE but presumably if it did,
> >>> then accelerations could be mapped through VBE.
> >>
> >> I believe the idea is that you don't want to map the framebuffer into
> >> the guest, this allows one-directional communication so you can defer
> >> rendering to the client and not suffer from the latency.  But I may be
> >> mixing things up.
> > 
> > Hrm, that seems like an odd strategy for legacy VGA.  Spice isn't
> > remoting every pixel update, right?  I would assume it's using the same
> > logic as the rest of the VGA cards and doing bulk updates based on the
> > refresh timer.  In that case, exposing the framebuffer shouldn't matter
> > at all.
> 
> I'd assume so too, but we need to make sure the framebuffer is unmapped
> when in accelerated mode, or at least the guest has no expectations of
> using it.

Well, unmapping it is easy but why would you want to do or enforce
that ? 

> The drm drivers for the current model are needed anyway; so moving to
> virtio is extra effort, not an alternative.
> 
> Note virtio doesn't support mapping framebuffers yet, or the entire vga
> compatibility stuff, so the pc-oriented card will have to be a mix of
> virtio and stdvga multiplexed on one pci card (maybe two functions, but
> I'd rather avoid that).

Well what I'm hacking as a proof of concept right now is std-vga with a
BAR for MMIO regs to supplement the legacy VGA IO and an optional virtio
BAR (which is not BAR 0, see below).

So to be compatible with the existing std-vga I made the virtio BAR be
BAR 3 or something like that (trivial patch in virtio-pci to allow that)
but of course that means a hack in the guest to find it which is
sub-optimal.

We had a chat with Rusty and it would be ideal if we could have a PCI
capability indicating where to find the virtio config space. However
this is a bit problematic because either we use the vendor cap which
means limiting outselves to RH vendor ID and hijacking the vendor cap
for it for ever, or we get the SIG to allocate a capability for virtual
IO devices....

The later is ideal but I don't have contacts at the SIG. It could be
done in a way that is usable by many vendors, ie, the cap itself could
contain a vendor ID indicating the virtualization interface owner along
with some additional data (in our case virtio version, BAR index, BAR
offset).

It does generally make sense to be able to have a device expose a
more/less standard or existing HW interface (one of the USB HCIs, AHCI,
VGA stuff, etc...) and also have a virtio channel for paravirt.

Finally as for s390 & co, well... std-vga is still very PCI'ish, so we'd
have to do a bit more work if we are to disconnect that.

Cheers,
Ben.

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Anthony Liguori]

Benjamin Herrenschmidt <benh@kernel.crashing.org> writes:

> On Mon, 2012-07-30 at 16:55 +0300, Avi Kivity wrote:
>> > The trouble is predicting which guests have drivers and which guests
>> > don't.  Having a VGA model that could be enabled universally with good
>> > VBE support for guests without drivers would be a very nice default
>> > model.
>> 
>> I agree.  Hopefully it won't be difficult to get the guest to unmap, or
>> maybe we can just unregister the direct RAM mapping in qemu.
>
> I don't understand that part... why on earth would the fb has to be
> unmapped ?

This is a detail of how Spice/QXL works.  It is not a framebuffer
protocol.

Spice sends a series of rendering commands.  It not rendering to a flat
framebuffer but rather to window-like objects.  It maintains a list of
these commands and objects and even maintain a tree to understand
z-ordering.

It remotes these commands over the network and the client does all the
magic that your compositing manager/window manager/X server would
normally do.  This is why it's so complicated, it's doing an awful lot.

Normally, the framebuffer that the guest sees (it must exist, it's VGA
after all) is never updated.  If the guest attempts to read the
framebuffer (it normally doesn't), Spice/QXL will render the entire
queue all at once to produce the framebuffer result.  This is a slow
path because it doesn't happen normally.

Unmapping is probably the wrong word.  Quiescing is probably a better
term.

>> > We've never made the switch because WinXP doesn't have VESA support
>> > natively.  But we're slowly getting to the point in time where it's
>> > acceptable to require a special command line option for running WinXP
>> > guests such that we could consider changing the default machine type.
>> 
>> Yes.
>> 
>> > 
>> >>> It's not clear to me why it doesn't enable VBE but presumably if it did,
>> >>> then accelerations could be mapped through VBE.
>> >>
>> >> I believe the idea is that you don't want to map the framebuffer into
>> >> the guest, this allows one-directional communication so you can defer
>> >> rendering to the client and not suffer from the latency.  But I may be
>> >> mixing things up.
>> > 
>> > Hrm, that seems like an odd strategy for legacy VGA.  Spice isn't
>> > remoting every pixel update, right?  I would assume it's using the same
>> > logic as the rest of the VGA cards and doing bulk updates based on the
>> > refresh timer.  In that case, exposing the framebuffer shouldn't matter
>> > at all.
>> 
>> I'd assume so too, but we need to make sure the framebuffer is unmapped
>> when in accelerated mode, or at least the guest has no expectations of
>> using it.
>
> Well, unmapping it is easy but why would you want to do or enforce
> that ? 

Because Spice depends on keeping the framebuffer inconsistent.

>
>> The drm drivers for the current model are needed anyway; so moving to
>> virtio is extra effort, not an alternative.
>> 
>> Note virtio doesn't support mapping framebuffers yet, or the entire vga
>> compatibility stuff, so the pc-oriented card will have to be a mix of
>> virtio and stdvga multiplexed on one pci card (maybe two functions, but
>> I'd rather avoid that).
>
> Well what I'm hacking as a proof of concept right now is std-vga with a
> BAR for MMIO regs to supplement the legacy VGA IO and an optional virtio
> BAR (which is not BAR 0, see below).
>
> So to be compatible with the existing std-vga I made the virtio BAR be
> BAR 3 or something like that (trivial patch in virtio-pci to allow that)
> but of course that means a hack in the guest to find it which is
> sub-optimal.

Why does it need to be compatible?  I don't think there's anything in
VESA that mandates the framebuffer be in BAR 0.  In fact, I don't think
VESA even mandates PCI.

VBE is strictly a BIOS interface.  So you could move the framebuffer to
BAR 1 by just modifying SeaBIOS.

> We had a chat with Rusty and it would be ideal if we could have a PCI
> capability indicating where to find the virtio config space.

Typically, PCI capabilities point to the PCI config space.  But the
virtio config space is not part of the PCI config space.  It's in BAR0.
So this doesn't make a lot of sense to me.

>From a virtio perspective, if there was an API to "map area of memory
from host", virtio-pci could use a transport feature flag to indicate
that such an area existed.  It would be a fully compatible change to
virtio-pci.

>From a VGA perspective, as long as we set the class code correctly and
handle legacy VGA access, it should all be fine.  We would need to
implement VBE in terms of virtio commands but that shouldn't be a
problem.

I think the best approach is to have some basic commands for managing
things like resolution setting, 2d accelerations, etc.  A feature flag
could be used to say "this device speaks Spice too."

Regards,

Anthony Liguori

> However
> this is a bit problematic because either we use the vendor cap which
> means limiting outselves to RH vendor ID and hijacking the vendor cap
> for it for ever, or we get the SIG to allocate a capability for virtual
> IO devices....
>
> The later is ideal but I don't have contacts at the SIG. It could be
> done in a way that is usable by many vendors, ie, the cap itself could
> contain a vendor ID indicating the virtualization interface owner along
> with some additional data (in our case virtio version, BAR index, BAR
> offset).
>
> It does generally make sense to be able to have a device expose a
> more/less standard or existing HW interface (one of the USB HCIs, AHCI,
> VGA stuff, etc...) and also have a virtio channel for paravirt.
>
> Finally as for s390 & co, well... std-vga is still very PCI'ish, so we'd
> have to do a bit more work if we are to disconnect that.
>
> Cheers,
> Ben.

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Benjamin Herrenschmidt]

On Mon, 2012-07-30 at 19:17 -0500, Anthony Liguori wrote:

> This is a detail of how Spice/QXL works.  It is not a framebuffer
> protocol.
> 
> Spice sends a series of rendering commands.  It not rendering to a flat
> framebuffer but rather to window-like objects.  It maintains a list of
> these commands and objects and even maintain a tree to understand
> z-ordering.

So it's GDI on the wire roughly :-)

> It remotes these commands over the network and the client does all the
> magic that your compositing manager/window manager/X server would
> normally do.  This is why it's so complicated, it's doing an awful lot.

So akin to a rootless X server ? 

> Normally, the framebuffer that the guest sees (it must exist, it's VGA
> after all) is never updated.  If the guest attempts to read the
> framebuffer (it normally doesn't), Spice/QXL will render the entire
> queue all at once to produce the framebuffer result.  This is a slow
> path because it doesn't happen normally.
> 
> Unmapping is probably the wrong word.  Quiescing is probably a better
> term.

I'd say we keep the fb around, whether it's actually actively updated or
not indeed depends on the active driver in the guest. That way we can
switch back to dumb fb easily for oopses or kernel text console.

I'll dig a bit more later see if I can figure out more of the innards of
spice/qxl. Note also that the .br folks are working on a qxl over
virtio, I'll catch up with them this week to make sure we coordinate our
efforts.

> >> > We've never made the switch because WinXP doesn't have VESA support
> >> > natively.  But we're slowly getting to the point in time where it's
> >> > acceptable to require a special command line option for running WinXP
> >> > guests such that we could consider changing the default machine type.
> >> 
> >> Yes.
> >> 
> >> > 
> >> >>> It's not clear to me why it doesn't enable VBE but presumably if it did,
> >> >>> then accelerations could be mapped through VBE.
> >> >>
> >> >> I believe the idea is that you don't want to map the framebuffer into
> >> >> the guest, this allows one-directional communication so you can defer
> >> >> rendering to the client and not suffer from the latency.  But I may be
> >> >> mixing things up.
> >> > 
> >> > Hrm, that seems like an odd strategy for legacy VGA.  Spice isn't
> >> > remoting every pixel update, right?  I would assume it's using the same
> >> > logic as the rest of the VGA cards and doing bulk updates based on the
> >> > refresh timer.  In that case, exposing the framebuffer shouldn't matter
> >> > at all.
> >> 
> >> I'd assume so too, but we need to make sure the framebuffer is unmapped
> >> when in accelerated mode, or at least the guest has no expectations of
> >> using it.
> >
> > Well, unmapping it is easy but why would you want to do or enforce
> > that ? 
> 
> Because Spice depends on keeping the framebuffer inconsistent.

That's ok, we don't need to unmap the object for that. We can "switch"
to "GDI mode" in which case it goes back to not triggering updates, it
can still be written to, this just has no visible effect. That way the
kernel can trivially switch it back on with a single command of some
sort for things like emergency oops display, switch to text consoles,
you name it ....

> >> The drm drivers for the current model are needed anyway; so moving to
> >> virtio is extra effort, not an alternative.
> >> 
> >> Note virtio doesn't support mapping framebuffers yet, or the entire vga
> >> compatibility stuff, so the pc-oriented card will have to be a mix of
> >> virtio and stdvga multiplexed on one pci card (maybe two functions, but
> >> I'd rather avoid that).
> >
> > Well what I'm hacking as a proof of concept right now is std-vga with a
> > BAR for MMIO regs to supplement the legacy VGA IO and an optional virtio
> > BAR (which is not BAR 0, see below).
> >
> > So to be compatible with the existing std-vga I made the virtio BAR be
> > BAR 3 or something like that (trivial patch in virtio-pci to allow that)
> > but of course that means a hack in the guest to find it which is
> > sub-optimal.
> 
> Why does it need to be compatible?  I don't think there's anything in
> VESA that mandates the framebuffer be in BAR 0.  In fact, I don't think
> VESA even mandates PCI.
> 
> VBE is strictly a BIOS interface.  So you could move the framebuffer to
> BAR 1 by just modifying SeaBIOS.

Well, I didn't want to diverge too much from the existing one bcs things
like openbios know about it but I'm starting to think that this is a
stupid idea. See my other email on that subject.

> > We had a chat with Rusty and it would be ideal if we could have a PCI
> > capability indicating where to find the virtio config space.
> 
> Typically, PCI capabilities point to the PCI config space.  But the
> virtio config space is not part of the PCI config space.  It's in BAR0.
> So this doesn't make a lot of sense to me.

No the idea was to have a PCI cap akin to the MSI-X cap that indicates
that the device supports virtio, which version of it, and indicates
which BAR has the virtio config space (just like an MSI-X cap tells you
which BAR has the MSI-X config table).

But I'm not thinking I'll just do a proper virtio with BAR 0, it's
cleaner to do that and fix the various firmwares to look for BAR 2 for
the fb instead and those FW aren't very hard to fix (tho I might ask for
a hand with the x86 asm in vbe bios). Also that way I get a clean break
so I can use a proper vendor ID (btw. BAR 2 rather than BAR 1 so it can
be a BAR pair in 64-bit space in the future).

> >From a virtio perspective, if there was an API to "map area of memory
> from host", virtio-pci could use a transport feature flag to indicate
> that such an area existed.  It would be a fully compatible change to
> virtio-pci.
> 
> >From a VGA perspective, as long as we set the class code correctly and
> handle legacy VGA access, it should all be fine.  We would need to
> implement VBE in terms of virtio commands but that shouldn't be a
> problem.
> 
> I think the best approach is to have some basic commands for managing
> things like resolution setting, 2d accelerations, etc.  A feature flag
> could be used to say "this device speaks Spice too."

Yup, that's where I'm headed. One remaining question is whether I keep
an MMIO programming interface for mode setting, which might be easier
for firmwares/BIOSen so they don't have to deal with DMA. There is the
legacy ports but those can be hard to reach on some platforms.

My current thinking is thus:

- BAR 0 -> virtio
- BAR 1 -> MMIO image of the legacy ports
- BAR 2/3 -> fb (2 initially, 2+3 64-bit BAR in the future)

Additionally, the virtio transport will provide commands to access the
mode setting/VGA registers as well so a guest driver doesn't have to
access BAR 1. It also makes it easier to have a non-PCI variant.

This is all pretty easy so far so I'll give that a shot and come back
when I have actual patches, we can discuss further from that point.

It might be a good idea to implement VBE 3 with blits as well btw
provided Windows can make use of it (can it ?)

Cheers,
Ben.

> Regards,
> 
> Anthony Liguori
> 
> > However
> > this is a bit problematic because either we use the vendor cap which
> > means limiting outselves to RH vendor ID and hijacking the vendor cap
> > for it for ever, or we get the SIG to allocate a capability for virtual
> > IO devices....
> >
> > The later is ideal but I don't have contacts at the SIG. It could be
> > done in a way that is usable by many vendors, ie, the cap itself could
> > contain a vendor ID indicating the virtualization interface owner along
> > with some additional data (in our case virtio version, BAR index, BAR
> > offset).
> >
> > It does generally make sense to be able to have a device expose a
> > more/less standard or existing HW interface (one of the USB HCIs, AHCI,
> > VGA stuff, etc...) and also have a virtio channel for paravirt.
> >
> > Finally as for s390 & co, well... std-vga is still very PCI'ish, so we'd
> > have to do a bit more work if we are to disconnect that.
> >
> > Cheers,
> > Ben.

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Gerd Hoffmann]

  Hi,
  Hi,

> Minor improvements to stdvga actual help qxl (presumably).  qxl still
> provides a vga interface which is used when guest drivers aren't
> available.
> 
> It's not clear to me why it doesn't enable VBE but presumably if it did,
> then accelerations could be mapped through VBE.

QXL actually enables VBE.  And, yes, making qemu translate simple 2D
bitblits into spice rendering ops is possible.

Mixing direct framebuffer writes and bitblit ops will give you bad
performance though, so when going that route I suggest to store the
console font somewhere in vga memory, then use only blit ops to draw the
letters and scroll the console.

> Actually, Gerd et al have expressed interest in moving to a virtio-based
> device model for Spice in the past.

There are discussions about re-doing the guest/host interface (command
rings etc) now and then, by adding a qxl2 device (or maybe even extend
stdvga), dropping a bunch of backward compatibility stuff in qxl.c.

Sending the spice commands over virtio is certainly an option here.  I'm
not sure it is possible to drive virtio rings from userspace (Xorg
driver), but that issue should go away with the qxl kms driver in the works.

> I think done correctly, it could help bring graphics to other platforms
> like S390 where PCI doesn't exist and will never exist.

Yes.

cheers,
  Gerd

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Benjamin Herrenschmidt]

On Mon, 2012-08-06 at 15:20 +0200, Gerd Hoffmann wrote:
> There are discussions about re-doing the guest/host interface (command
> rings etc) now and then, by adding a qxl2 device (or maybe even extend
> stdvga), dropping a bunch of backward compatibility stuff in qxl.c.
> 
> Sending the spice commands over virtio is certainly an option here.
> I'm
> not sure it is possible to drive virtio rings from userspace (Xorg
> driver), but that issue should go away with the qxl kms driver in the
> works.

Anything that gets rid of the qxl memory region API is welcome ;-) Some
of that stuff is really gross.

Ben.

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Gerd Hoffmann]

On 08/06/12 23:16, Benjamin Herrenschmidt wrote:
> On Mon, 2012-08-06 at 15:20 +0200, Gerd Hoffmann wrote:
>> There are discussions about re-doing the guest/host interface (command
>> rings etc) now and then, by adding a qxl2 device (or maybe even extend
>> stdvga), dropping a bunch of backward compatibility stuff in qxl.c.
>>
>> Sending the spice commands over virtio is certainly an option here.
>> I'm
>> not sure it is possible to drive virtio rings from userspace (Xorg
>> driver), but that issue should go away with the qxl kms driver in the
>> works.
> 
> Anything that gets rid of the qxl memory region API is welcome ;-) Some
> of that stuff is really gross.

Well, it is needed for sanity checking memory references (to images etc)
in qxl commands, so we can't just zap it.  But having a kms driver in
kernel space makes things easier again:  You could simply register a
single memory region for the guest ram, then basically use guest
physical addresses for qxl memory references.  qxl commands & data can
live in guest ram and you can stick references to the commands into
virtio rings ...

cheers,
  Gerd

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Benjamin Herrenschmidt]

On Tue, 2012-08-07 at 07:30 +0200, Gerd Hoffmann wrote:
> On 08/06/12 23:16, Benjamin Herrenschmidt wrote:
> > On Mon, 2012-08-06 at 15:20 +0200, Gerd Hoffmann wrote:
> >> There are discussions about re-doing the guest/host interface (command
> >> rings etc) now and then, by adding a qxl2 device (or maybe even extend
> >> stdvga), dropping a bunch of backward compatibility stuff in qxl.c.
> >>
> >> Sending the spice commands over virtio is certainly an option here.
> >> I'm
> >> not sure it is possible to drive virtio rings from userspace (Xorg
> >> driver), but that issue should go away with the qxl kms driver in the
> >> works.
> > 
> > Anything that gets rid of the qxl memory region API is welcome ;-) Some
> > of that stuff is really gross.
> 
> Well, it is needed for sanity checking memory references (to images etc)
> in qxl commands, so we can't just zap it.  But having a kms driver in
> kernel space makes things easier again:  You could simply register a
> single memory region for the guest ram, then basically use guest
> physical addresses for qxl memory references.  qxl commands & data can
> live in guest ram and you can stick references to the commands into
> virtio rings ...

Or you use a sane model in the first place that hides memory references
behind object IDs :-) Yes, going via a KMS driver will go a long way
toward fixing some of that.

I would still like to experiment a bit if time permits with a different
model which is basically GL paravirt via a virtio channel on top of
virtio-vga (possibly limited to GL/ES at least initially).

Guest side, we'd then have a simple DRM and use Glamor to turn all 2D
rendering into GL.

Cheers,
Ben.

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Alon Levy]

On Mon, Jul 30, 2012 at 09:54:27PM +1000, Benjamin Herrenschmidt wrote:
> On Mon, 2012-07-30 at 14:25 +0300, Avi Kivity wrote:
> 
> > > Right. Cirrus on ppc was used on PReP and Amiga for example though not
> > > many people really care about those platforms anymore. I'm not too
> > > worried at this point with that possibility but we shall know about it.
> > 
> > Emulating something incorrectly on purpose is wrong.  Use qxl or stdvga
> > (you can make either the default for ppc), but don't break cirrus or
> > rely on it being broken.
> 
> Well, that's what we do today it seems :-) But yes, it would be nice to
> fix it, the problem is that the fix doesn't seem to be possible while
> keeping a usable cirrus with KVM on big endian, at least as far as I
> have investigated so far.
> 
> > >> > As for the work I'm doing to brush up pci-vga a bit, I'm tempted to add
> > >> > an MMIO reg or a VBE config reg bit to allow configuring the endianness
> > >> > of the underlying fb with a default to what qemu does today.
> > >> 
> > >> What are those byteswapped apertures?  Some chipset thing that does the
> > >> byteswap?
> > > 
> > > The card itself. The 16M BAR is divided in 4 "apertures" (at least some
> > > Cirrus models do that including the one we emulate by default). One is
> > > no byteswap, two are 16-bit and 32-bit byteswap and the last one uses a
> > > specific byteswap for video overlay which I haven't looked at in detail.
> > > 
> > >> IIRC ppc has a bit in the TLB entry that tells it to byteswap.  Can't we
> > >> use it directly map the framebuffer with byteswapping?
> > > 
> > > Unfortunately only embedded ppc's have that :-(
> > 
> > Too bad :(
> > 
> > > 
> > > We can also make the fbdev/fbcon driver do the swapping in SW, but it's
> > > a relatively unusual code path and I don't think it works properly with
> > > X, I don't think it can be made to work properly with the generic X KMS
> > > at this point.
> > > 
> > > Now, cirrusdrmfb is already specific to the qemu cirrus variant in
> > > several ways, I wouldn't mind keeping it that way and if we "fix" the
> > > endianness model, maybe having a "hidden" register to flip it back to
> > > it's current mode of operation that cirrusdrmfb would use...
> > 
> > That's possible, but why not go all the way to qxl?
> >
> > That will give you better graphics performance with no need to hack.
> 
> Well, qxl is pretty awful from what I can see so far. I'm more tempted

I would love to hear something more specific about this. I assume you
are talking about libspice-server and not the device itself, since the
device itself has nothing specifically matching windows.

> to continue improving qemu-vga, adding a virtio transport, and maybe
> adding a way to tunnel spice into it if that makes sense but so far,
> that's stuff was designed for Windows as far as I can tell and is pretty
> horrible whatever way you look at it...
> 
> Cheers,
> Ben.
> 
> 
> 

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Andreas Färber]

Am 30.07.2012 18:19, schrieb Alon Levy:
> On Mon, Jul 30, 2012 at 09:54:27PM +1000, Benjamin Herrenschmidt wrote:
>> On Mon, 2012-07-30 at 14:25 +0300, Avi Kivity wrote:
>>
>>> [...] why not go all the way to qxl?
>>>
>>> That will give you better graphics performance with no need to hack.
>>
>> Well, qxl is pretty awful from what I can see so far. [...]
> 
> I would love to hear something more specific about this. I assume you
> are talking about libspice-server and not the device itself, since the
> device itself has nothing specifically matching windows.

I can't comment on what Ben meant, but from my perspective the really
awful thing about SPICE was its huge tree of dependencies, including a
very specific version of celt that we now need to package and maintain
specifically for SPICE. At least during the big QOM refactorings.

Elsewhere QEMU is built around the principle of opting individual
features in rather than requiring a whole bunch of stuff just to do a
basic qxl compile test for patches.

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Anthony Liguori]

Andreas Färber <afaerber@suse.de> writes:

> Am 30.07.2012 18:19, schrieb Alon Levy:
>> On Mon, Jul 30, 2012 at 09:54:27PM +1000, Benjamin Herrenschmidt wrote:
>>> On Mon, 2012-07-30 at 14:25 +0300, Avi Kivity wrote:
>>>
>>>> [...] why not go all the way to qxl?
>>>>
>>>> That will give you better graphics performance with no need to hack.
>>>
>>> Well, qxl is pretty awful from what I can see so far. [...]
>> 
>> I would love to hear something more specific about this. I assume you
>> are talking about libspice-server and not the device itself, since the
>> device itself has nothing specifically matching windows.
>
> I can't comment on what Ben meant, but from my perspective the really
> awful thing about SPICE was its huge tree of dependencies, including a
> very specific version of celt that we now need to package and maintain
> specifically for SPICE. At least during the big QOM refactorings.

Ack.

This is why I've been advocating for a new PV device model that can
negotiation in full SPICE support.

Then we could keep libspice an optional dependency, but move all guests
to use a single graphics driver.  Likewise, management tools wouldn't
need to worry about multiple types of graphics cards.

Regards,

Anthony Liguori

>
> Elsewhere QEMU is built around the principle of opting individual
> features in rather than requiring a whole bunch of stuff just to do a
> basic qxl compile test for patches.
>
> Andreas
>
> -- 
> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Alon Levy]

On Wed, Aug 01, 2012 at 02:22:37PM -0500, Anthony Liguori wrote:
> Andreas Färber <afaerber@suse.de> writes:
> 
> > Am 30.07.2012 18:19, schrieb Alon Levy:
> >> On Mon, Jul 30, 2012 at 09:54:27PM +1000, Benjamin Herrenschmidt wrote:
> >>> On Mon, 2012-07-30 at 14:25 +0300, Avi Kivity wrote:
> >>>
> >>>> [...] why not go all the way to qxl?
> >>>>
> >>>> That will give you better graphics performance with no need to hack.
> >>>
> >>> Well, qxl is pretty awful from what I can see so far. [...]
> >> 
> >> I would love to hear something more specific about this. I assume you
> >> are talking about libspice-server and not the device itself, since the
> >> device itself has nothing specifically matching windows.
> >
> > I can't comment on what Ben meant, but from my perspective the really
> > awful thing about SPICE was its huge tree of dependencies, including a
> > very specific version of celt that we now need to package and maintain
> > specifically for SPICE. At least during the big QOM refactorings.
> 
> Ack.
> 
> This is why I've been advocating for a new PV device model that can
> negotiation in full SPICE support.
> 
> Then we could keep libspice an optional dependency, but move all guests
> to use a single graphics driver.  Likewise, management tools wouldn't
> need to worry about multiple types of graphics cards.

This sounds great, but how would that negotiation work? Do you intend
for a VGA device (i.e. pci vendor & product id's of cirrus) that is also
a virtio device and a guest driver will recognize this by poking some io
ports or looking at another pci field?

> 
> Regards,
> 
> Anthony Liguori
> 
> >
> > Elsewhere QEMU is built around the principle of opting individual
> > features in rather than requiring a whole bunch of stuff just to do a
> > basic qxl compile test for patches.
> >
> > Andreas
> >
> > -- 
> > SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> > GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
> 

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Anthony Liguori]

Alon Levy <alevy@redhat.com> writes:

> On Wed, Aug 01, 2012 at 02:22:37PM -0500, Anthony Liguori wrote:
>> Andreas Färber <afaerber@suse.de> writes:
>> 
>> > Am 30.07.2012 18:19, schrieb Alon Levy:
>> >> On Mon, Jul 30, 2012 at 09:54:27PM +1000, Benjamin Herrenschmidt wrote:
>> >>> On Mon, 2012-07-30 at 14:25 +0300, Avi Kivity wrote:
>> >>>
>> >>>> [...] why not go all the way to qxl?
>> >>>>
>> >>>> That will give you better graphics performance with no need to hack.
>> >>>
>> >>> Well, qxl is pretty awful from what I can see so far. [...]
>> >> 
>> >> I would love to hear something more specific about this. I assume you
>> >> are talking about libspice-server and not the device itself, since the
>> >> device itself has nothing specifically matching windows.
>> >
>> > I can't comment on what Ben meant, but from my perspective the really
>> > awful thing about SPICE was its huge tree of dependencies, including a
>> > very specific version of celt that we now need to package and maintain
>> > specifically for SPICE. At least during the big QOM refactorings.
>> 
>> Ack.
>> 
>> This is why I've been advocating for a new PV device model that can
>> negotiation in full SPICE support.
>> 
>> Then we could keep libspice an optional dependency, but move all guests
>> to use a single graphics driver.  Likewise, management tools wouldn't
>> need to worry about multiple types of graphics cards.
>
> This sounds great, but how would that negotiation work? Do you intend
> for a VGA device (i.e. pci vendor & product id's of cirrus) that is also
> a virtio device and a guest driver will recognize this by poking some io
> ports or looking at another pci field?

It would be an VGA/SVGA/VESA/VBE compatible virtio-pci device.  If we
take virtio-pci, do vga_init_common, set the class codes correctly, 
move the vram bar from 0->1 and update the VGA BIOS accordingly, it Just
Works.

With no feature bits negotiated, this is all you get--a plain VESA
compatible interface.

We would then add feature bits to allow you to do basic operations like
setting display mode, damage update, and perhaps some 2d acceleration
like blit.  This all happens through messages on a virtqueue.

While this is totally virtio-pci ABI compatible, we'll need to enhance
the virtio API within Linux to allow the notion of "map large memory
region."  It's not entirely clear to me yet how to do this only because
non-PCI transports probably need this memory to be guest allocated.
It's possible that we could add another vring type abstraction layer to
handle this difference.

Anyway, we would then add additional feature bits to things like Spice.
It's not clear to me yet how this would work in detail (I don't know
enough about Spice).  The easiest thing to do is simply introduce a
dedicated virtqueue for Spice and speak exactly the same protocol that
QXL does today.

The trouble with that though is that some of the things in QXL today
probably overlap with features we want if libspice is not available
(like mode setting).

So if it's reasonable, it would be best to negotiate in Spice
feature-by-feature using Spice command format where it makes sense and
something more generic where it makes sense.

QEMU would have to fully decode these commands and hand off the results
to libspice if it was there.

So in QEMU, if libspice is present, QEMU would decode all commands and
hand them to libspice in a form it understands (this may require some
hopefully trivial mapping for things like mode setting).  If libspice
isn't present, QEMU only exposes the features it can handle on its own.

This should give us the best of all worlds.  A legacy VGA compatible
interface that speaks virtio, works on non-PCI architectures, and has
the full capabilities of Spice (with the ability to fallback if libspice
isn't present).

Regards,

Anthony Liguori

>
>> 
>> Regards,
>> 
>> Anthony Liguori
>> 
>> >
>> > Elsewhere QEMU is built around the principle of opting individual
>> > features in rather than requiring a whole bunch of stuff just to do a
>> > basic qxl compile test for patches.
>> >
>> > Andreas
>> >
>> > -- 
>> > SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>> > GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
>> 

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Alon Levy]

On Fri, Aug 03, 2012 at 08:41:36AM -0500, Anthony Liguori wrote:
> Alon Levy <alevy@redhat.com> writes:
> 
> > On Wed, Aug 01, 2012 at 02:22:37PM -0500, Anthony Liguori wrote:
> >> Andreas Färber <afaerber@suse.de> writes:
> >> 
> >> > Am 30.07.2012 18:19, schrieb Alon Levy:
> >> >> On Mon, Jul 30, 2012 at 09:54:27PM +1000, Benjamin Herrenschmidt wrote:
> >> >>> On Mon, 2012-07-30 at 14:25 +0300, Avi Kivity wrote:
> >> >>>
> >> >>>> [...] why not go all the way to qxl?
> >> >>>>
> >> >>>> That will give you better graphics performance with no need to hack.
> >> >>>
> >> >>> Well, qxl is pretty awful from what I can see so far. [...]
> >> >> 
> >> >> I would love to hear something more specific about this. I assume you
> >> >> are talking about libspice-server and not the device itself, since the
> >> >> device itself has nothing specifically matching windows.
> >> >
> >> > I can't comment on what Ben meant, but from my perspective the really
> >> > awful thing about SPICE was its huge tree of dependencies, including a
> >> > very specific version of celt that we now need to package and maintain
> >> > specifically for SPICE. At least during the big QOM refactorings.
> >> 
> >> Ack.
> >> 
> >> This is why I've been advocating for a new PV device model that can
> >> negotiation in full SPICE support.
> >> 
> >> Then we could keep libspice an optional dependency, but move all guests
> >> to use a single graphics driver.  Likewise, management tools wouldn't
> >> need to worry about multiple types of graphics cards.
> >
> > This sounds great, but how would that negotiation work? Do you intend
> > for a VGA device (i.e. pci vendor & product id's of cirrus) that is also
> > a virtio device and a guest driver will recognize this by poking some io
> > ports or looking at another pci field?
> 
> It would be an VGA/SVGA/VESA/VBE compatible virtio-pci device.  If we
> take virtio-pci, do vga_init_common, set the class codes correctly, 
> move the vram bar from 0->1 and update the VGA BIOS accordingly, it Just
> Works.

Sounds good.

> 
> With no feature bits negotiated, this is all you get--a plain VESA
> compatible interface.
> 
> We would then add feature bits to allow you to do basic operations like
> setting display mode, damage update, and perhaps some 2d acceleration
> like blit.  This all happens through messages on a virtqueue.
> 
> While this is totally virtio-pci ABI compatible, we'll need to enhance
> the virtio API within Linux to allow the notion of "map large memory
> region."  It's not entirely clear to me yet how to do this only because
> non-PCI transports probably need this memory to be guest allocated.
> It's possible that we could add another vring type abstraction layer to
> handle this difference.
> 
> Anyway, we would then add additional feature bits to things like Spice.
> It's not clear to me yet how this would work in detail (I don't know
> enough about Spice).  The easiest thing to do is simply introduce a
> dedicated virtqueue for Spice and speak exactly the same protocol that
> QXL does today.
> 

Sounds right.

> The trouble with that though is that some of the things in QXL today
> probably overlap with features we want if libspice is not available
> (like mode setting).
> 
> So if it's reasonable, it would be best to negotiate in Spice
> feature-by-feature using Spice command format where it makes sense and
> something more generic where it makes sense.
> 
> QEMU would have to fully decode these commands and hand off the results
> to libspice if it was there.

Why not make libspice mandatory? fix it to be big endian compatible
(it's already 32 bit LE compatible). I think this can be done either via
git submodules (either just for spice-protocol or for spice) or
continuing to use it as an external dependency.

Any guest without a qxl-virtio driver would work with the VGA/VBE
device, so we can port the driver gradually to all guests.

> 
> So in QEMU, if libspice is present, QEMU would decode all commands and
> hand them to libspice in a form it understands (this may require some
> hopefully trivial mapping for things like mode setting).  If libspice
> isn't present, QEMU only exposes the features it can handle on its own.
> 
> This should give us the best of all worlds.  A legacy VGA compatible
> interface that speaks virtio, works on non-PCI architectures, and has
> the full capabilities of Spice (with the ability to fallback if libspice
> isn't present).
> 
> Regards,
> 
> Anthony Liguori
> 
> >
> >> 
> >> Regards,
> >> 
> >> Anthony Liguori
> >> 
> >> >
> >> > Elsewhere QEMU is built around the principle of opting individual
> >> > features in rather than requiring a whole bunch of stuff just to do a
> >> > basic qxl compile test for patches.
> >> >
> >> > Andreas
> >> >
> >> > -- 
> >> > SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> >> > GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
> >> 

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Gerd Hoffmann]

  Hi,

> Why not make libspice mandatory?

spice needs to build and run on alot of platforms where it doesn't run
today.  So it isn't an option right now.  Which doesn't imply it will
never happen, but spice certainly needs some work before we can
seriously discuss that.

Make spice work on bigendian is probably the biggest part of it.  But
also make spice-server build on windows.  Add opus audio codec support
so we can make celt051 a build time option instead of a hard dependency.
 Probably a bunch of minor portability issues.

cheers,
  Gerd

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Erlon Cruz]

[-- Attachment #1: Type: text/plain, Size: 1102 bytes --]

Em 07/08/2012 05:01, "Gerd Hoffmann" <kraxel@redhat.com> escreveu:
>
>   Hi,
>
> > Why not make libspice mandatory?
>
> spice needs to build and run on alot of platforms where it doesn't run
> today.  So it isn't an option right now.  Which doesn't imply it will
> never happen, but spice certainly needs some work before we can
> seriously discuss that.
>
> Make spice work on bigendian is probably the biggest part of it.

Spice has a good support for endianess issues. The protocol handles
endianess for commands sent from spice server to the client. The only thing
its missing is to fix the endianess for server/client handshaking. We a
patch for that. We can send that later on. We have tested it first running
spice sever tests in a PPC machine and then we run it in an experimental
virtio-qxl driver we are working on.

The device only have support for QXL (nor VGA) and works well in x86 and i
PPC guest with a few issues we still working on. Another limitation is that
in the design we used virtio transport, the device wont work with mixed
guest/host configurations (e.g. Guest ppc host x86)

[-- Attachment #2: Type: text/html, Size: 1297 bytes --]

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Gerd Hoffmann]

On 08/07/12 15:05, Erlon Cruz wrote:
> Em 07/08/2012 05:01, "Gerd Hoffmann" <kraxel@redhat.com> escreveu:
>>
>>   Hi,
>>
>>> Why not make libspice mandatory?
>>
>> spice needs to build and run on alot of platforms where it doesn't run
>> today.  So it isn't an option right now.  Which doesn't imply it will
>> never happen, but spice certainly needs some work before we can
>> seriously discuss that.
>>
>> Make spice work on bigendian is probably the biggest part of it.
> 
> Spice has a good support for endianess issues. The protocol handles
> endianess for commands sent from spice server to the client.

The spice wire protocol is little endian.  Functions for
generating/parsing the wire protocol are generated, maybe the generator
already supports byteswapping as needed, not sure.

> The only thing
> its missing is to fix the endianess for server/client handshaking.

What exactly do you mean here?

> We a
> patch for that. We can send that later on.

Patches welcome.  Please make sure spice-devel is included (additionally
to qemu-devel).

> We have tested it first running
> spice sever tests in a PPC machine and then we run it in an experimental
> virtio-qxl driver we are working on.

Huh?  How does this work?

The QXLCommand passed on to spice-server (via get_command callback) is
supposed to be little endian.  The qxl parser (server/red_parse_qxl.c in
spice repo) doesn't support bigendian hosts yet.  Not that a big deal,
basically just a bunch of le{16,32]_to_cpu() calls when copying
(+checking) fields from struct QXL* (little endian) to struct Spice*
(native endian).  But not done yet and likewise not tested yet ...

So I'm wondering how this works for you on ppc ...

> The device only have support for QXL (nor VGA) and works well in x86 and i
> PPC guest with a few issues we still working on. Another limitation is that
> in the design we used virtio transport, the device wont work with mixed
> guest/host configurations (e.g. Guest ppc host x86)

Which should be fixable.

cheers,
  Gerd

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Erlon Cruz]

>On Tue, Aug 7, 2012 at 11:07 AM, Gerd Hoffmann <kraxel@redhat.com> wrote:
>
> On 08/07/12 15:05, Erlon Cruz wrote:
> > Em 07/08/2012 05:01, "Gerd Hoffmann" <kraxel@redhat.com> escreveu:
> >>
> >>   Hi,
> >>
> >>> Why not make libspice mandatory?
> >>
> >> spice needs to build and run on alot of platforms where it doesn't run
> >> today.  So it isn't an option right now.  Which doesn't imply it will
> >> never happen, but spice certainly needs some work before we can
> >> seriously discuss that.
> >>
> >> Make spice work on bigendian is probably the biggest part of it.
> >
> > Spice has a good support for endianess issues. The protocol handles
> > endianess for commands sent from spice server to the client.
>
> The spice wire protocol is little endian.  Functions for
> generating/parsing the wire protocol are generated, maybe the generator
> already supports byteswapping as needed, not sure.

Yes, it does. There are marshellers that are generated based on SPICE
command definitions.

> > The only thing
> > its missing is to fix the endianess for server/client handshaking.
>
> What exactly do you mean here?

Well that are negotiation messages configuring each channel, its
capabilities, encryption keys , etc. After this negotiation, the
server start to send SPICE data, which are already swapped by the
marshallers.

> > We a
> > patch for that. We can send that later on.
>
> Patches welcome.  Please make sure spice-devel is included (additionally
> to qemu-devel).

I've sent then on spice-devel mailing list.

> > We have tested it first running
> > spice sever tests in a PPC machine and then we run it in an experimental
> > virtio-qxl driver we are working on.
>
> Huh?  How does this work?

Well, our first though was to minimize changes on xf86-video-qxl. The
device allocates the video memory on QEMU and memory on guest (the
same amount). A virtio kernel drivers makes the connection/mapping
between xf86-video-driver and the device. Guest memory then works
mirroring all commands to the device (the X driver tell the kernel
which areas changed and the kernel pushes the pages to the device. we
only need one VQ to to that) and the xf86-driver can work like if does
in the QXL PCI.
'
> The QXLCommand passed on to spice-server (via get_command callback) is
> supposed to be little endian.

Actually the command endianness doesn't matter at this point once
QXLCommand will be marshalled (and then swapped) before get to the
wire.

> The qxl parser (server/red_parse_qxl.c in
> spice repo) doesn't support bigendian hosts yet.  Not that a big deal,
> basically just a bunch of le{16,32]_to_cpu() calls when copying
> (+checking) fields from struct QXL* (little endian) to struct Spice*
> (native endian).  But not done yet and likewise not tested yet ...
>
> So I'm wondering how this works for you on ppc ...

Well, I think that once QEMU and libspice are both BE or LE this code
will run. And we only  server/reds.c, server/red_channel.c, and
server/inputs_channel.c check the patches I send in spice-devel.

> > The device only have support for QXL (nor VGA) and works well in x86 and
> > i
> > PPC guest with a few issues we still working on. Another limitation is
> > that
> > in the design we used virtio transport, the device wont work with mixed
> > guest/host configurations (e.g. Guest ppc host x86)
>
> Which should be fixable.
>
> cheers,
>   Gerd

Cheers,
Erlon

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Gerd Hoffmann]

  Hi,

>>> The only thing
>>> its missing is to fix the endianess for server/client handshaking.
>>
>> What exactly do you mean here?
> 
> Well that are negotiation messages configuring each channel, its
> capabilities, encryption keys , etc. After this negotiation, the
> server start to send SPICE data, which are already swapped by the
> marshallers.

Ah, so the initial handshake isn't covered by the generated marshallers.

>>> We have tested it first running
>>> spice sever tests in a PPC machine and then we run it in an experimental
>>> virtio-qxl driver we are working on.
>>
>> Huh?  How does this work?
> 
> Well, our first though was to minimize changes on xf86-video-qxl.
> [ ... ]

Keep in mind that the linux world is moving to kms & drm.  A virtual
hardware design trying to minimize the changes to todays userland X
driver may turn out to be not be the best pick long-term.  We also must
make sure it works for both linux and windows guests.  I suggest to
discuss the virtual hardware design @ qemu-devel and spice-devel.

>> The QXLCommand passed on to spice-server (via get_command callback) is
>> supposed to be little endian.
> 
> Actually the command endianness doesn't matter at this point once
> QXLCommand will be marshalled (and then swapped) before get to the
> wire.

Well, the flow of a command from the guest to the spice client looks
like this:

  (1) guest places the command into memory (QXLCommand, little endian)
      and queues it up (qxl uses its own ring format, but using virtio
      rings doesn't make a difference).
  (2) qemu takes the QXLCommand and passes it as-is to the spice-server.
  (3) spice-server parses and sanity-checks the command, also
      translates QXLCommand into internal representation (struct
      {Red,Spice}*), see server/red_parse_qxl.c.  It should also
      translate from little endian to native endian but doesn't yet.
  (4) spice-server can process the commands (in native byte order)
      locally in case the rendered screen is needed.  Happens for
      example if you ask for a screenshot via qemu monitor.
  (5) The generated marshaller code translates the command into wire
      format (and swaps again from native to little endian if needed).
  (6) The generated demarshaller code (in the client) translates back
      from the wire format to the internal representation (in native
      endian).
  (7) spice client renders the result.

/me wonders how this works for you with step #3 not byteswapping on
bigendian hosts.

Maybe the guest driver places the qxl commands in native endian instead
of big endian into memory?

cheers,
  Gerd

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Erlon Cruz]

On Wed, Aug 8, 2012 at 3:18 AM, Gerd Hoffmann <kraxel@redhat.com> wrote:
>   Hi,
>
>>>> The only thing
>>>> its missing is to fix the endianess for server/client handshaking.
>>>
>>> What exactly do you mean here?
>>
>> Well that are negotiation messages configuring each channel, its
>> capabilities, encryption keys , etc. After this negotiation, the
>> server start to send SPICE data, which are already swapped by the
>> marshallers.
>
> Ah, so the initial handshake isn't covered by the generated marshallers.
>
>>>> We have tested it first running
>>>> spice sever tests in a PPC machine and then we run it in an experimental
>>>> virtio-qxl driver we are working on.
>>>
>>> Huh?  How does this work?
>>
>> Well, our first though was to minimize changes on xf86-video-qxl.
>> [ ... ]
>
> Keep in mind that the linux world is moving to kms & drm.  A virtual
> hardware design trying to minimize the changes to todays userland X
> driver may turn out to be not be the best pick long-term.  We also must
> make sure it works for both linux and windows guests.  I suggest to
> discuss the virtual hardware design @ qemu-devel and spice-devel.
>
>>> The QXLCommand passed on to spice-server (via get_command callback) is
>>> supposed to be little endian.
>>
>> Actually the command endianness doesn't matter at this point once
>> QXLCommand will be marshalled (and then swapped) before get to the
>> wire.
>
> Well, the flow of a command from the guest to the spice client looks
> like this:
>
>   (1) guest places the command into memory (QXLCommand, little endian)
>       and queues it up (qxl uses its own ring format, but using virtio
>       rings doesn't make a difference).
>   (2) qemu takes the QXLCommand and passes it as-is to the spice-server.
>   (3) spice-server parses and sanity-checks the command, also
>       translates QXLCommand into internal representation (struct
>       {Red,Spice}*), see server/red_parse_qxl.c.  It should also
>       translate from little endian to native endian but doesn't yet.
>   (4) spice-server can process the commands (in native byte order)
>       locally in case the rendered screen is needed.  Happens for
>       example if you ask for a screenshot via qemu monitor.
>   (5) The generated marshaller code translates the command into wire
>       format (and swaps again from native to little endian if needed).
>   (6) The generated demarshaller code (in the client) translates back
>       from the wire format to the internal representation (in native
>       endian).
>   (7) spice client renders the result.
>
> /me wonders how this works for you with step #3 not byteswapping on
> bigendian hosts.
> Maybe the guest driver places the qxl commands in native endian instead
> of big endian into memory?

The guest drivers places the commands in the memory in its native
endianess. That why the device only works when guest/host has the same
endianess. That's sanit-checks doesn't fails.

Regards,
Erlon
>
> cheers,
>   Gerd

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Gerd Hoffmann]

  Hi,

>> Maybe the guest driver places the qxl commands in native endian instead
>> of big endian into memory?
> 
> The guest drivers places the commands in the memory in its native
> endianess. That why the device only works when guest/host has the same
> endianess. That's sanit-checks doesn't fails.

Ah, ok. this explains it.  So we must fix red_parse_qxl.c to byteswap
and also the guest driver to byteswap.  Then it will work even in case
guest and host have different endianess.

cheers,
  Gerd

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Blue Swirl]

On Mon, Jul 30, 2012 at 6:24 AM, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> So I got cirrus working on ppc with cirrusdrmfb...
>
> The fun part is that it works :-)
>
> Basically, the issue is that normally, for it to work, one would have to
> access the framebuffer using the appropriate aperture for byteswapping
> based on the bpp.
>
> However, qemu doesn't emulate those apertures ... and cirrusdrmfb
> either.
>
> In fact, qemu cirrus model is just dumb and assumes guest native
> byteorder for the framebuffer.
>
> The good thing is that this makes it work... the bad thing is that it's
> a completely incorrect HW model and if the linux driver wasn't also
> buggy it wouldn't work.
>
> However it's also pretty much unfixable without making it also unusable
> in terms of performance so I want to check with you guys if it's ok to
> just leave it as-is.
>
> Basically, if the fb was LE as it's supposed to be, one would have to
> use the byteswapped apertures. But those can only be emulated by
> trapping on every access to turn it into MMIO emulation, which means
> unusable performances.
>
> So we end up with what is effectively a BE framebuffer thanks to qemu
> hard coding what it thinks the guest endian is (btw, this is quite
> busted in theory as well since PPC can be bi-endian for example).

I think even this is not fully accurate description of the VGA
insanity, because if the host display happens to use BGR pixel format
instead of RGB, there may be more byte shuffling, but not in all
cases: compare vga_draw_line24() to vga_draw_line32() in
vga_template.h.

For VGA, 24 and 32 bit modes and direct frame buffer access are only
available via VBE which does not match any real HW. The real VGA modes
should be accurately emulated.

>
> Anyways, it works today, it's just that the HW model is wrong... and I
> don't want to fix it. Any objection ?
>
> As for the work I'm doing to brush up pci-vga a bit, I'm tempted to add
> an MMIO reg or a VBE config reg bit to allow configuring the endianness
> of the underlying fb with a default to what qemu does today.
>
> Cheers,
> Ben.
>
>
>

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Peter Maydell]

On 30 July 2012 16:18, Blue Swirl <blauwirbel@gmail.com> wrote:
> I think even this is not fully accurate description of the VGA
> insanity, because if the host display happens to use BGR pixel format
> instead of RGB, there may be more byte shuffling, but not in all
> cases: compare vga_draw_line24() to vga_draw_line32() in
> vga_template.h.

It seems conceptually the wrong layer to have these be done
by the individual device models (similarly there's a bunch of
pixel twiddling done in hw/pl110_template.h). It would be nicer
if this was done by some generic layer instead so the device
model just said "this is a 32 bit BGR framebuffer"...

-- PMM

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Blue Swirl]

On Mon, Jul 30, 2012 at 3:30 PM, Peter Maydell <peter.maydell@linaro.org> wrote:
> On 30 July 2012 16:18, Blue Swirl <blauwirbel@gmail.com> wrote:
>> I think even this is not fully accurate description of the VGA
>> insanity, because if the host display happens to use BGR pixel format
>> instead of RGB, there may be more byte shuffling, but not in all
>> cases: compare vga_draw_line24() to vga_draw_line32() in
>> vga_template.h.
>
> It seems conceptually the wrong layer to have these be done
> by the individual device models (similarly there's a bunch of
> pixel twiddling done in hw/pl110_template.h). It would be nicer
> if this was done by some generic layer instead so the device
> model just said "this is a 32 bit BGR framebuffer"...

Fully agree, but it would be also nice to retain the optimizations
somehow to avoid any byte swapping.

>
> -- PMM

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[ronnie sahlberg]

On Mon, Jul 30, 2012 at 4:24 PM, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> So I got cirrus working on ppc with cirrusdrmfb...
>
> The fun part is that it works :-)
>
> Basically, the issue is that normally, for it to work, one would have to
> access the framebuffer using the appropriate aperture for byteswapping
> based on the bpp.
>
> However, qemu doesn't emulate those apertures ... and cirrusdrmfb
> either.
>
> In fact, qemu cirrus model is just dumb and assumes guest native
> byteorder for the framebuffer.
>
> The good thing is that this makes it work... the bad thing is that it's
> a completely incorrect HW model and if the linux driver wasn't also
> buggy it wouldn't work.
>
> However it's also pretty much unfixable without making it also unusable
> in terms of performance so I want to check with you guys if it's ok to
> just leave it as-is.
>
> Basically, if the fb was LE as it's supposed to be, one would have to
> use the byteswapped apertures. But those can only be emulated by
> trapping on every access to turn it into MMIO emulation, which means
> unusable performances.
>
> So we end up with what is effectively a BE framebuffer thanks to qemu
> hard coding what it thinks the guest endian is (btw, this is quite
> busted in theory as well since PPC can be bi-endian for example).
>
> Anyways, it works today, it's just that the HW model is wrong... and I
> don't want to fix it. Any objection ?
>
> As for the work I'm doing to brush up pci-vga a bit, I'm tempted to add
> an MMIO reg or a VBE config reg bit to allow configuring the endianness
> of the underlying fb with a default to what qemu does today.
>
> Cheers,
> Ben.
>
>
>

I use lots of guests that will never ever get virtio drivers.
So for those guests, any work on making sure bog standard vga keeps
working or even improving it
gets two thumbs up from me!

Re: [Qemu-devel] Cirrus bugs vs endian: how two bugs cancel each other out

[Benjamin Herrenschmidt]

On Tue, 2012-07-31 at 18:44 +1000, ronnie sahlberg wrote:
> 
> I use lots of guests that will never ever get virtio drivers.
> So for those guests, any work on making sure bog standard vga keeps
> working or even improving it
> gets two thumbs up from me!

So I've been essentially restarting my work to make it a virtio-vga...
however, you don't have to use the virtio channel to set modes & do VBE
etc... so it should work like vga std for a dumb x86 guest using
VBE/BIOS.

I'm hoping to have something to show later this week.

I've also done it like the other virtio drivers in that the PCI
interface is split from the core so it can accomodate a non-PCI virtio
base.

The main difference with the other virtio ones is that it has APIs to
return MemoryRegion's for the linear fb and registers, which the
virtio-pci stub hooks up to BARs but that can be hooked up elsewhere
easily.  

The main thing to do right now to have parity of functionality with vga
std is the modified vbe bios, which I'll toy with tomorrow (ugh .. x86
asm :-) From there, I can start doing things with the virtio channel
(right now it exists but I don't have any request implemented).

I'm thinking about a trivial 2D blitter (based on earlier work posted in
2009 by Alex Graf), maybe a HW cursor, see if I can get that used by
SLOF and do a fbdev driver (or rather a drmfb driver).

>From there, I think the right approach is to sync with the folks working
on virtio-qxl so they can use that pipe for spice & we don't re-invent
that wheel.

Cheers,
Ben.