virtualGraphicCards

virtualGraphicCards

[Retzki, Sascha [Xplain]]

Hi Xen-developers,


I would love to see "virtualGraphicCards":

Each domU is shown a virtual graphics cards (besides the console?). The
domUs graphic system (x11r6 on unix, rio on plan9, $whatever on
$what-OS-ever-support-xen) can use those to startup the graphical
system, window manager, Desktop environment.

The dom0 can $somehow "attach" to those displays. Effectivly, this means
dom0 can interact with all OSes via the graphical system, and the
user(s) can use all domUs via their respective UserInterfaces.

I'd love to see this on Xen, I'd love to work on this, but I am
"unskilled" ;-), so it is a bit hard for me at the moment. Anyway, any
help, brainstorming, code, is highly appreciated.


Who else is in?

Re: virtualGraphicCards

[Daniel Hulme]

[-- Attachment #1.1: Type: text/plain, Size: 747 bytes --]

> I'd love to see this on Xen, I'd love to work on this, but I am
> "unskilled" ;-), so it is a bit hard for me at the moment. Anyway, any
> help, brainstorming, code, is highly appreciated.
So would I, but even for "skilled" people it is a bit hard. This is
already on http://wiki.xensource.com/xenwiki/XenTodoList but the list is
quite long and it is not high-priority. Of course, the more people want
it, the sooner it will get done.

-- 
You can't run away  forever,  but there's  nothing wrong with  getting a
good head start.  You want to shut out the night,  you want to shut down
the  sun,  you  want  to  shut  away  the  pieces  of  a  broken  heart.
`Rock and Roll Dreams Come True' (Steinman)    http://surreal.istic.org/

[-- Attachment #1.2: Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: virtualGraphicCards

[Retzki, Sascha [Xplain]]

> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-
> bounces@lists.xensource.com] On Behalf Of Daniel Hulme
> Sent: Wednesday, September 07, 2005 12:07 PM
> To: xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] virtualGraphicCards
> 
> > I'd love to see this on Xen, I'd love to work on this, but I am
> > "unskilled" ;-), so it is a bit hard for me at the moment. Anyway,
any
> > help, brainstorming, code, is highly appreciated.
> So would I, but even for "skilled" people it is a bit hard. This is
> already on http://wiki.xensource.com/xenwiki/XenTodoList but the list
is
> quite long and it is not high-priority. Of course, the more people
want
> it, the sooner it will get done.


You mean
"Graphic console support - post 3.0"
?
I don't quiet get:
1 kernel fb mapping? or X server?

Do you mean how the virtual graphics card is "seen" by the dom0? Or/And
accessed? 

Imho, a glue-layer could abstract several approaches, for example 9P
(the Plan9-folks love it, I did not have the time to test it and thus I
don't have an opinion yet. However, I believe it is good ;-)) and VNC,
and the dom0 decides itsself howto access the "data" - this would be
pretty good because some OSes (which may be dom0-cabable some day) may
find method-N better (are optimized for it) than method-M... and so on
;-)

Re: virtualGraphicCards

[Mark Williamson]

> You mean
> "Graphic console support - post 3.0"
> ?
> I don't quiet get:
> 1 kernel fb mapping? or X server?

The idea is that the graphical console will have "back/front" structure 
similar to the other devices, with a viewer in dom0 reading screen data from 
the guest and either displaying it, or exporting it over the network.

The question here is: do we write a kernel framebuffer driver for this (and 
get the X server to use that) or implement it in the X server directly?  I 
think the former solution is probably going to have the best cost/benefit.

One thing that's important (IMHO) is not to require networking in order to get 
the virtual display - makes it easier to install, configure stuff, etc via a 
graphical interface.

Cheers,
Mark

> Do you mean how the virtual graphics card is "seen" by the dom0? Or/And
> accessed?
>
> Imho, a glue-layer could abstract several approaches, for example 9P
> (the Plan9-folks love it, I did not have the time to test it and thus I
> don't have an opinion yet. However, I believe it is good ;-)) and VNC,
> and the dom0 decides itsself howto access the "data" - this would be
> pretty good because some OSes (which may be dom0-cabable some day) may
> find method-N better (are optimized for it) than method-M... and so on
> ;-)
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

RE: virtualGraphicCards

[Retzki, Sascha [Xplain]]

> > 1 kernel fb mapping? or X server?
> 
> The idea is that the graphical console will have "back/front"
structure
> similar to the other devices, with a viewer in dom0 reading screen
data
> from
> the guest and either displaying it, or exporting it over the network.
> 
> The question here is: do we write a kernel framebuffer driver for this
> (and
> get the X server to use that) or implement it in the X server
directly?  

I still don't get it, I'd like to make a little ASCII-art:


			Xen
			 |
			 |
			dom0 [graphical system]
			 |
			 |
	-------------+----------------
	|					|
	domU_1 [graphical]	domU_2 [text]


So, dom0 sees the real graphics card, and attaches to it.
Xen offers virtual graphics cards for each domU. domU_2 does not fire up
a graphical system, thus works with the emulated text buffer. domU_1
fires up its graphical system. Both domU write some data to some place,
managed by Xen (because Xen offered the vGraphicscard).
1.) Is the data actually saved as the domUs wrote them? Or is some
glue-layer in-between, converting it to .e.g VNC-compatible data, to
make the memory consumption problem go away?
Anyway, if the dom0 wants to "see" any of domU, it just asks Xen for the
data, getting a pointer or something (it just reads, anyway). It is best
if we can interact with the domUs, so we get some virtual HIDs (Human
Interface Devices)? Anyway the data is accessed, either in a "raw" way
or through some "protocol" like VNC. Still, I don't see why any of the
displaying systems, in the bottom (dom0 graphical system and/or Xen) or
the top (domU graphical systems), and their transporting systems, should
be known; In the end, all of those systems transfer data to a graphic
card, in a standard way (VGA for instance), no matter if it is a linux
framebuffer, a Unix x11, a Plan9 Rio, or any other system.

Re: virtualGraphicCards

[Mark Williamson]

> I still don't get it, I'd like to make a little ASCII-art:
>
>
>    Xen
>
>
>    dom0 [graphical system]
>
>
>  -------------+----------------
>
>  domU_1 [graphical] domU_2 [text]
>

> So, dom0 sees the real graphics card, and attaches to it.

Yep.

> Xen offers virtual graphics cards for each domU.

Not exactly: since Xen 2.0 all virtual devices are handled by dom0.  The guest 
accesses its virtual devices by "proxying" requests through dom0.  The 
virtual graphics device would work in a similar way...

> a graphical system, thus works with the emulated text buffer. domU_1
> fires up its graphical system. Both domU write some data to some place,
> managed by Xen (because Xen offered the vGraphicscard).
> 1.) Is the data actually saved as the domUs wrote them? Or is some
> glue-layer in-between, converting it to .e.g VNC-compatible data, to
> make the memory consumption problem go away?

The guests would be writing to some sort of "virtual framebuffer" memory.  
This memory would be read directly by *something* in dom0 to see what was 
being displayed.

> Anyway, if the dom0 wants to "see" any of domU, it just asks Xen for the
> data, getting a pointer or something (it just reads, anyway). It is best
> if we can interact with the domUs, so we get some virtual HIDs (Human
> Interface Devices)?

Yes, we'd need some sort of virtual HID layer.  This would want to basically 
connect the keyboard and mouse actions in the GUI to the guest, somehow.

> Anyway the data is accessed, either in a "raw" way 
> or through some "protocol" like VNC. Still, I don't see why any of the
> displaying systems, in the bottom (dom0 graphical system and/or Xen) or
> the top (domU graphical systems), and their transporting systems, should
> be known; In the end, all of those systems transfer data to a graphic
> card, in a standard way (VGA for instance), no matter if it is a linux
> framebuffer, a Unix x11, a Plan9 Rio, or any other system.

There's two possibilities:
1) emulate a VESA graphics card
2) write a special "Xen virtual" device, as we have for block, net, etc.

1 will possibly require some support from Xen.  The output in this case will 
be handled by a daemon in dom0 which will map the "framebuffer" memory and 
display it on the screen (in dom0).

2 is to just write another Xen-aware driver.  I suspect that explicitly 
writing a xen-aware display driver will prove easier and more scalable in 
future...  The guests "Xen display driver" will handle requests, draw into a 
"virtual framebuffer" and notify the dom0 daemon of changes, etc.

Either way, there's a component in dom0 that needs to read out this data and 
export it to the user.

The X server vs kernel framebuffer question refers to point 2: if we're going 
to implement a Xen-aware virtual framebuffer device, do we implement it as a 
kernel module or as an X server module?  I think the former makes more sense.

Does that answer your questions?

Cheers,
Mark

Re: virtualGraphicCards

[Daniel Hulme]

[-- Attachment #1.1: Type: text/plain, Size: 2279 bytes --]

> So, dom0 sees the real graphics card, and attaches to it.
> Xen offers virtual graphics cards for each domU. domU_2 does not fire
> up a graphical system, thus works with the emulated text buffer.
> domU_1 fires up its graphical system. Both domU write some data to
> some place, managed by Xen (because Xen offered the vGraphicscard).

> 1.) Is the data actually saved as the domUs wrote them? Or is some
> glue-layer in-between, converting it to .e.g VNC-compatible data, to
> make the memory consumption problem go away?
I think it quite likely that each domain gets its own little framebuffer
in memory.

> Anyway, if the dom0 wants to "see" any of domU, it just asks Xen for
> the data, getting a pointer or something (it just reads, anyway). It
> is best if we can interact with the domUs, so we get some virtual HIDs
> (Human Interface Devices)? Anyway the data is accessed, either in a
> "raw" way or through some "protocol" like VNC.
Makes sense.

> Still, I don't see why any of the displaying systems, in the bottom
> (dom0 graphical system and/or Xen) or the top (domU graphical
> systems), and their transporting systems, should be known; In the end,
> all of those systems transfer data to a graphic card, in a standard
> way (VGA for instance), no matter if it is a linux framebuffer, a Unix
> x11, a Plan9 Rio, or any other system.
Exactly, so you support the first option Mark mentioned. The second
option is that we shift the boundary higher up than that: the domU just
has a special X driver that, instead of writing the data to a gfx card,
packages it up in whatever way for the backend driver in dom0 to look at
later. This is probably an easier way to do it -- we only have to
interface to X calls, rather than having to pretend to be a whole
graphics card, with VESA and everything -- but strikes me as being
almost cripplingly platform-specific.

This is still some way off, though, so everything is up in the air.

-- 
  ,^--^.      ,-----.      Never meddle in the affairs of angry cats,   
 ( +  + )-----   ---'      for they are well-armed and quick to bite.   
 /  --          )          http://surreal.istic.org/   keyid 885b170d   
|_,-|_/--,_|-\_|    I'm not complaining; I'm just commenting negatively.

[-- Attachment #1.2: Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: virtualGraphicCards

[Retzki, Sascha [Xplain]]

> Exactly, so you support the first option Mark mentioned. The second
> option is that we shift the boundary higher up than that: the domU
just
> has a special X driver that, instead of writing the data to a gfx
card,
> packages it up in whatever way for the backend driver in dom0 to look
at
> later. This is probably an easier way to do it -- we only have to
> interface to X calls, rather than having to pretend to be a whole
> graphics card, with VESA and everything -- but strikes me as being
> almost cripplingly platform-specific.
> 

X is not the center of this world. If it is, something is wrong.
I think VESA, VGA, other (better, newer) protocols are way better. They
are more generic, and thus *can* be supported by other Operating
Systems. Despite that X is on another "layer" (layer as in OSI, just for
graphics cards ;-))...

Re: virtualGraphicCards

[Mark Williamson]

> X is not the center of this world. If it is, something is wrong.
> I think VESA, VGA, other (better, newer) protocols are way better. They
> are more generic, and thus *can* be supported by other Operating
> Systems. Despite that X is on another "layer" (layer as in OSI, just for
> graphics cards ;-))...

The intention isn't to limit ourselves to using X.  The plan would be to 
define the "Xen video protocol" then write a driver to make X talk it.  
Anyone else could port their system to talk to it too.

I'd be inclined to implement the Xen video device in the guest kernel, then 
get X to use that (as it can for existing framebuffer drivers).

Cheers,
Mark

Re: virtualGraphicCards

[Anthony Liguori]

Mark Williamson wrote:

>The idea is that the graphical console will have "back/front" structure 
>similar to the other devices, with a viewer in dom0 reading screen data from 
>the guest and either displaying it, or exporting it over the network.
>
>The question here is: do we write a kernel framebuffer driver for this (and 
>get the X server to use that) or implement it in the X server directly?  I 
>think the former solution is probably going to have the best cost/benefit.
>  
>
I agreed with Mark up until very recently.  Doing some (unscientific) 
analysis of the cost of framebuffer rendering leads me to belief that 
it's not necessarily worth having a kernel framebuffer for para-virtual 
domains.

IMHO, a better solution would be to use Xvnc and have a terminal => VNC 
server (that could also proxy a VNC session).  Here's how I think it 
would look:

Every domain has a dedicated network interface for VNC.  Domains are 
configured with Xvnc with reverse VNC.  Each domain has a XenVNC 
instance running on dom0, the XenVNC daemon either 1) exposes the 
console via VNC (perhaps with ggiterm) or 2) proxies the domain's Xvnc 
session.

Depending on where the device model ends up, for full virtual domains 
you can just expose the VNC session from the device model directly or 
proxy it in dom0.

The real benefit with this approach is you do the VNC encoding in the X 
server (and can take advantage of all of that information) as opposed to 
rendering to a flat buffer and then trying to expose that buffer 
directly.  In fact, this is the only way to take advantage of the remote 
cursor pseudo-encodings in VNC which IMO has the most dramatic effect on 
VNC usability.

Just a thought.

Regards,

Anthony Liguori

>One thing that's important (IMHO) is not to require networking in order to get 
>the virtual display - makes it easier to install, configure stuff, etc via a 
>graphical interface.
>
>Cheers,
>Mark
>
>  
>
>>Do you mean how the virtual graphics card is "seen" by the dom0? Or/And
>>accessed?
>>
>>Imho, a glue-layer could abstract several approaches, for example 9P
>>(the Plan9-folks love it, I did not have the time to test it and thus I
>>don't have an opinion yet. However, I believe it is good ;-)) and VNC,
>>and the dom0 decides itsself howto access the "data" - this would be
>>pretty good because some OSes (which may be dom0-cabable some day) may
>>find method-N better (are optimized for it) than method-M... and so on
>>;-)
>>
>>
>>
>>_______________________________________________
>>Xen-devel mailing list
>>Xen-devel@lists.xensource.com
>>http://lists.xensource.com/xen-devel
>>    
>>
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel
>
>  
>

Re: virtualGraphicCards

[Anthony Liguori]

Mark Williamson wrote:

>>X is not the center of this world. If it is, something is wrong.
>>I think VESA, VGA, other (better, newer) protocols are way better. They
>>are more generic, and thus *can* be supported by other Operating
>>Systems. Despite that X is on another "layer" (layer as in OSI, just for
>>graphics cards ;-))...
>>    
>>
>
>The intention isn't to limit ourselves to using X.  The plan would be to 
>define the "Xen video protocol" then write a driver to make X talk it.  
>Anyone else could port their system to talk to it too.
>
>I'd be inclined to implement the Xen video device in the guest kernel, then 
>get X to use that (as it can for existing framebuffer drivers).
>  
>
The problem with framebuffers is that you lose higher level primatives 
(like blitting, rect fills, moves).  Modern remote display protocols 
(NX, VNC) are optimized to take advantage of these higher level primatives.

The thing that really gets you is that modern environments draw software 
cursors a lot.  Paying a round trip for every cursor movement makes the 
mouse visibly laggy.  Even qemu has this problem (when you export it's 
display over VNC) as the cirrus hardware cursor is not often used (as 
it's only black and white and fixed size).

I'm becoming fonder of the idea of virtualizing at a much higher level 
(perhaps even at the OpenGL level).  I'm not sure how we bridge this 
effectively to fully virtualized domains either.

Regards,

Anthony Liguori

>Cheers,
>Mark
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel
>
>  
>

Re: virtualGraphicCards

[Mark Williamson]

> >I'd be inclined to implement the Xen video device in the guest kernel,
> > then get X to use that (as it can for existing framebuffer drivers).
>
> The problem with framebuffers is that you lose higher level primatives
> (like blitting, rect fills, moves).  Modern remote display protocols
> (NX, VNC) are optimized to take advantage of these higher level primatives.

Yep, true.  We'd talked about having a "side channel" that could be used to 
relay accelerated operations to speed up blitting, etc.  This could also 
describe what areas of the framebuffer to rescan...

A nice idea might be to have the kernel framebuffer device, which could be 
used successfully by X (and svgalib, framebuffer console, etc) at some 
performance cost.  An optional X11 module would allow X to also use the 
explicit sidechannel in order to get maximum performance.

That said, I think the NX protocol you mentioned is really slick: it uses 
various round trip suppression mechanisms to combat network latency - I 
imagine they'd work to limit the hit from context switching in the Xen 
environment.

Cheers,
Mark

> The thing that really gets you is that modern environments draw software
> cursors a lot.  Paying a round trip for every cursor movement makes the
> mouse visibly laggy.  Even qemu has this problem (when you export it's
> display over VNC) as the cirrus hardware cursor is not often used (as
> it's only black and white and fixed size).
>
> I'm becoming fonder of the idea of virtualizing at a much higher level
> (perhaps even at the OpenGL level).  I'm not sure how we bridge this
> effectively to fully virtualized domains either.
>
> Regards,
>
> Anthony Liguori
>
> >Cheers,
> >Mark
> >
> >_______________________________________________
> >Xen-devel mailing list
> >Xen-devel@lists.xensource.com
> >http://lists.xensource.com/xen-devel