[Qemu-devel] spicevmv chardev, guest agents and paravirtual mouse

[Qemu-devel] spicevmv chardev, guest agents and paravirtual mouse

[Gerd Hoffmann]

   Hi folks,

Looks like the spicevmc patch kicked the guest qagent discussion, so 
lets start with this, although it isn't related much to the agent issue
itself ...


The spicevmc chardev just pipes data from a chardev user within qemu
to libspice and adds a "type" tag to it so libspice knows now to wind
up the other end.  There are several types:

(1) vdagent, the spice guest agent.  Will discuss this in detail
     below.

(2) smartcard, this basically pipes the smartcard protocol over spice.
     Patches for smartcard support are on the list and should be almost
     ready for merge now. If you want connect a remote smart card reader
     to your guest you can use a tcp chardev, which will build a data
     pipeline like this:

        ccid-passthrough <-> tcp chardev <-> tcp protocol <->
          vcsclient <-> libcacard

     Or you can use the spicevmc chardev to use spice as transport:

        ccid-passthrough <-> spicevmc chardev <-> spice protocol <->
        spice client <-> libcacard

     If someone comes up with a vnc extention one could create something
     simliar for vnc tunneling:

        ccid-passthrough <-> vnctunnel chardev <-> vnc protocol <->
        gtk-vnc widget <-> libcacard

(3) usb forwarding.  Hans is busy with this.  No working code yet.
     Will probably work pretty simliar to smartcard.

(4) termial forwarding.  Just an idea right now.  Nowdays that the spice
     client side moves to gtk it would be easy to embed a termial widget,
     therby allowing easy access to the serial console using something
     like this:

        -chardev spicevmc,id=console,type=terminal
        -device isa-serial,index=0,chardev=console

So even if you put the guest agent discussion completely aside the
spicevmc chardev clearly has its uses.


Ok, now for the spice vdagent.  Alon posted the link to the specs in
the spice wiki already, here it is again:
    http://spice-space.org/page/Whiteboard/AgentProtocol

The header file with the protocol specification comes with the
spice-protocol package, here is a direct link:
    http://cgit.freedesktop.org/spice/spice-protocol/tree/spic/vd_agent.h

Linux agent code is here:
    http://cgit.freedesktop.org/~jwrdegoede/vdagent-linux/

The protocol can be multiplexed via VDIChunkHeader->port.  spice
actually does that as the mouse messages (VDAgentMouseState) are
generated by the spice server itself.  Everything else is just piped
from the guest to the spice client (and back).  The protocol also has
capabilities (VDAgentAnnounceCapabilities).

There isn't much spice-specific stuff in there.  The clipboard bits
for example should work unmodified with vnc, one would just have to
hack up a vnc extention to tunnel the agent protocol over vnc (and vnc
client support of course).

I think the only spice-specific bit in the protocol is the display
enumeration in the VDAgentMonitorsConfig and VDAgentMouseState
messages.  The numbers there simply reference the spice display
channel number of the display in question, which just doesn't exist
outside spice.  Of course one can just ignore that for now as there is
no multihead support outside spice anyway ...


Also related: paravirtual mouse.  I'd suggest to go for something new,
based on virtio-serial, doing just the mouse and nothing else.

The VDAgentMouseState messages have one problem: They send the pointer
position as-is, which introduces a dependency on the screen size.
Spice deals with it, and although that is a little bit ugly it isn't a
big issue either as spice knows how big the screens are.  When using
this outside spice context this becomes nasty though.

Also I think it is a bad idea to mix guest agent stuff with the
paravirtual mouse.  The mouse events need to get routed to the
X-Server, whereas the agent stuff is handled by some other daemon.
Also the multiplexing the spice server has to do to inject the mouse
messages supports that view IMHO.

Related note: It is probably a good idea to also separate stuff which
is handled best by a system-wide guest daemon (such as reboot
requests, package installs, whatever, vdagent doesn't do that kind of
management btw) and stuff which needs to be handled by a per-user
daemon which has access to the user's X-Session (clipboard, ...).


Ok, how to go forward now?  Here is what I think we should do:

(1) Merge spicevmc chardev.  Needed anyway for compatibility and
     for the other use cases outlined above.
(2) Design a paravirtual mouse device.  Make sure it fullfills the
     spice requirements, which are basically:
       * support multiple displays.
       * don't eat tons of cpu like usb-tablet does.
(3) spice can use the new pvmouse now and depricate sending mouse
     events via vdagent protocol.
(4) In parallel review the vdagent protocol, then either extend or
     redesign it for other use cases.


Let the Flam^WDiscussions begin,

   Gerd

Re: [Qemu-devel] spicevmv chardev, guest agents and paravirtual mouse

[Anthony Liguori]

On 01/12/2011 10:12 AM, Gerd Hoffmann wrote:
>
>   Hi folks,
>
> Looks like the spicevmc patch kicked the guest qagent discussion, so 
> lets start with this, although it isn't related much to the agent issue
> itself ...
>
>
> The spicevmc chardev just pipes data from a chardev user within qemu
> to libspice and adds a "type" tag to it so libspice knows now to wind
> up the other end.  There are several types:
>
> (1) vdagent, the spice guest agent.  Will discuss this in detail
>     below.
>
> (2) smartcard, this basically pipes the smartcard protocol over spice.
>     Patches for smartcard support are on the list and should be almost
>     ready for merge now. If you want connect a remote smart card reader
>     to your guest you can use a tcp chardev, which will build a data
>     pipeline like this:
>
>        ccid-passthrough <-> tcp chardev <-> tcp protocol <->
>          vcsclient <-> libcacard
>
>     Or you can use the spicevmc chardev to use spice as transport:
>
>        ccid-passthrough <-> spicevmc chardev <-> spice protocol <->
>        spice client <-> libcacard
>
>     If someone comes up with a vnc extention one could create something
>     simliar for vnc tunneling:
>
>        ccid-passthrough <-> vnctunnel chardev <-> vnc protocol <->
>        gtk-vnc widget <-> libcacard
>
> (3) usb forwarding.  Hans is busy with this.  No working code yet.
>     Will probably work pretty simliar to smartcard.
>
> (4) termial forwarding.  Just an idea right now.  Nowdays that the spice
>     client side moves to gtk it would be easy to embed a termial widget,
>     therby allowing easy access to the serial console using something
>     like this:
>
>        -chardev spicevmc,id=console,type=terminal
>        -device isa-serial,index=0,chardev=console
>
> So even if you put the guest agent discussion completely aside the
> spicevmc chardev clearly has its uses.
>
>
> Ok, now for the spice vdagent.  Alon posted the link to the specs in
> the spice wiki already, here it is again:
>    http://spice-space.org/page/Whiteboard/AgentProtocol
>
> The header file with the protocol specification comes with the
> spice-protocol package, here is a direct link:
>    http://cgit.freedesktop.org/spice/spice-protocol/tree/spic/vd_agent.h
>
> Linux agent code is here:
>    http://cgit.freedesktop.org/~jwrdegoede/vdagent-linux/
>
> The protocol can be multiplexed via VDIChunkHeader->port.  spice
> actually does that as the mouse messages (VDAgentMouseState) are
> generated by the spice server itself.  Everything else is just piped
> from the guest to the spice client (and back).  The protocol also has
> capabilities (VDAgentAnnounceCapabilities).
>
> There isn't much spice-specific stuff in there.  The clipboard bits
> for example should work unmodified with vnc, one would just have to
> hack up a vnc extention to tunnel the agent protocol over vnc (and vnc
> client support of course).

VNC already supports copy/paste as part of the protocol so can the agent 
protocol be terminated in QEMU such that the server can make use of the 
standard protocol extensions?

>
> I think the only spice-specific bit in the protocol is the display
> enumeration in the VDAgentMonitorsConfig and VDAgentMouseState
> messages.  The numbers there simply reference the spice display
> channel number of the display in question, which just doesn't exist
> outside spice.  Of course one can just ignore that for now as there is
> no multihead support outside spice anyway ...
>
>
> Also related: paravirtual mouse.  I'd suggest to go for something new,
> based on virtio-serial, doing just the mouse and nothing else.

I'd agree.  I think we want something that actually terminates in the 
kernel for Linux guests since then we can expose it as an evdev device.  
No special X driver would be needed.

Is this something that makes sense for Spice in the future?

>
> The VDAgentMouseState messages have one problem: They send the pointer
> position as-is, which introduces a dependency on the screen size.
> Spice deals with it, and although that is a little bit ugly it isn't a
> big issue either as spice knows how big the screens are.  When using
> this outside spice context this becomes nasty though.
>
> Also I think it is a bad idea to mix guest agent stuff with the
> paravirtual mouse.  The mouse events need to get routed to the
> X-Server, whereas the agent stuff is handled by some other daemon.
> Also the multiplexing the spice server has to do to inject the mouse
> messages supports that view IMHO.
>
> Related note: It is probably a good idea to also separate stuff which
> is handled best by a system-wide guest daemon (such as reboot
> requests, package installs, whatever, vdagent doesn't do that kind of
> management btw) and stuff which needs to be handled by a per-user
> daemon which has access to the user's X-Session (clipboard, ...).

Definitely.  VMware uses two separate daemons--one that runs in a system 
context and one that runs per user X session.

>
> Ok, how to go forward now?  Here is what I think we should do:
>
> (1) Merge spicevmc chardev.  Needed anyway for compatibility and
>     for the other use cases outlined above.
> (2) Design a paravirtual mouse device.  Make sure it fullfills the
>     spice requirements, which are basically:
>       * support multiple displays.
>       * don't eat tons of cpu like usb-tablet does.
> (3) spice can use the new pvmouse now and depricate sending mouse
>     events via vdagent protocol.
> (4) In parallel review the vdagent protocol, then either extend or
>     redesign it for other use cases.

I'm okay with this as long as there is a commitment from the Spice team 
to convert on a common framework.  I'd like to have a little more 
discussion about agent design first to make sure that we're on the same 
page.

For instance, Spice makes use of a 1-off protocol whereas something like 
virt-agent uses an established RPC protocol (XML-RPC).  I'm not tied to 
using any particular protocol, but I think it's very important to use a 
standardized, well specified protocol.

Regards,

Anthony Liguori

>
>
> Let the Flam^WDiscussions begin,
>
>   Gerd
>

Re: [Spice-devel] [Qemu-devel] spicevmv chardev, guest agents and paravirtual mouse

[Hans de Goede]

Hi,

On 01/12/2011 06:40 PM, Anthony Liguori wrote:
> On 01/12/2011 10:12 AM, Gerd Hoffmann wrote:
>>
>> Hi folks,
>>
>> Looks like the spicevmc patch kicked the guest qagent discussion, so lets start with this, although it isn't related much to the agent issue
>> itself ...
>>
>>
>> The spicevmc chardev just pipes data from a chardev user within qemu
>> to libspice and adds a "type" tag to it so libspice knows now to wind
>> up the other end. There are several types:
>>
>> (1) vdagent, the spice guest agent. Will discuss this in detail
>> below.
>>
>> (2) smartcard, this basically pipes the smartcard protocol over spice.
>> Patches for smartcard support are on the list and should be almost
>> ready for merge now. If you want connect a remote smart card reader
>> to your guest you can use a tcp chardev, which will build a data
>> pipeline like this:
>>
>> ccid-passthrough <-> tcp chardev <-> tcp protocol <->
>> vcsclient <-> libcacard
>>
>> Or you can use the spicevmc chardev to use spice as transport:
>>
>> ccid-passthrough <-> spicevmc chardev <-> spice protocol <->
>> spice client <-> libcacard
>>
>> If someone comes up with a vnc extention one could create something
>> simliar for vnc tunneling:
>>
>> ccid-passthrough <-> vnctunnel chardev <-> vnc protocol <->
>> gtk-vnc widget <-> libcacard
>>
>> (3) usb forwarding. Hans is busy with this. No working code yet.
>> Will probably work pretty simliar to smartcard.
>>
>> (4) termial forwarding. Just an idea right now. Nowdays that the spice
>> client side moves to gtk it would be easy to embed a termial widget,
>> therby allowing easy access to the serial console using something
>> like this:
>>
>> -chardev spicevmc,id=console,type=terminal
>> -device isa-serial,index=0,chardev=console
>>
>> So even if you put the guest agent discussion completely aside the
>> spicevmc chardev clearly has its uses.
>>
>>
>> Ok, now for the spice vdagent. Alon posted the link to the specs in
>> the spice wiki already, here it is again:
>> http://spice-space.org/page/Whiteboard/AgentProtocol
>>
>> The header file with the protocol specification comes with the
>> spice-protocol package, here is a direct link:
>> http://cgit.freedesktop.org/spice/spice-protocol/tree/spic/vd_agent.h
>>
>> Linux agent code is here:
>> http://cgit.freedesktop.org/~jwrdegoede/vdagent-linux/
>>
>> The protocol can be multiplexed via VDIChunkHeader->port. spice
>> actually does that as the mouse messages (VDAgentMouseState) are
>> generated by the spice server itself. Everything else is just piped
>> from the guest to the spice client (and back). The protocol also has
>> capabilities (VDAgentAnnounceCapabilities).
>>
>> There isn't much spice-specific stuff in there. The clipboard bits
>> for example should work unmodified with vnc, one would just have to
>> hack up a vnc extention to tunnel the agent protocol over vnc (and vnc
>> client support of course).
>
> VNC already supports copy/paste as part of the protocol so can the agent protocol be terminated in QEMU such that the server can make use of the standard protocol extensions?
>

That depends on if the VNC copy/paste support was done right. With right
I mean that it should consist of the following messages:
1) A clipboard grab (send by guest -> client if ctrl+c pressed inside
    the guest, other direction if ctrl-c is pressed in any app on the
    client machine.) This should include a list of supported types the
    app now owning the clipboard can offer the clipboard in. For example
    text/plain-utf-8, text/html
2) A clipboard release send when the clipboard owning app exits
3) A clipboard request, send by one side when it wants to get the
    clipboard data, prereq: the other side owns the clipboard, the
    request is for a type in the list of types advertised when grabbing
4) clipboard data, response to clipboard request, could also be a nack
    when a release / request message cross each other.

This is basically how both windows and X11 work wrt copy and paste at
the low level, and by far the sanest way to support guest <-> client
os copy paste (event driven, no polling or some such involved).

I agree with the notion of having 2 agents in the guest, 1 system
wide and 1 per session. Assuming we get a standardized protocol for
the per session agent and we can somehow fit vnc's existing copy and
paste support into the above model, it should be able to use that.

While on the 2 agent topic. As I've just completed written the
spice agent for Linux I've some experience on this. Currently the
spice agent actually already consists of 2 processes an always
running daemon and a per X-session agent, there are 2 reasons for this:
1) Thinks like the paravirt mouse are best done at the system level
2) A multiplexer is needed between the client -> session agent channel,
and the actual session agents, as multiple session agents could be
running (think switch user functionality), and the messages should be
forwarded to the session agent in the active (focussed / showing)
session. spice-vdagentd uses console kit to determine the active
session and monitor session changes.

This does not really impact any protocol designs, but still it is
something to keep in mind.

>>
>> I think the only spice-specific bit in the protocol is the display
>> enumeration in the VDAgentMonitorsConfig and VDAgentMouseState
>> messages. The numbers there simply reference the spice display
>> channel number of the display in question, which just doesn't exist
>> outside spice. Of course one can just ignore that for now as there is
>> no multihead support outside spice anyway ...
>>
>>
>> Also related: paravirtual mouse. I'd suggest to go for something new,
>> based on virtio-serial, doing just the mouse and nothing else.
>
> I'd agree. I think we want something that actually terminates in the kernel for Linux guests since then we can expose it as an evdev device. No special X driver would be needed.

A paravirt mouse would need some sort of guest os support, yes. But
not necessarily in the kernel. currently spice-vdagentd uses uinput
to create an evdev device and send events from userspace. And the spice
agent under windows does the same (although there the events are
generated by the per user session agent process).

I'm not saying this should not be in the kernel, just that it does
not have to be in the kernel. It might even make sense to try and
use such a ipc mechanism for this, that in one guest os it could be
a kernel driver and in the other a userspace solution, but I'm not
sure how feasible that it.

> Is this something that makes sense for Spice in the future?

Yes, as long as guest support is kept in mind. Looking at it from
that pov. I think I might favor a userspace solution, as we likely
need an agent anyways, and adding / doing drivers for some guest
os-s is a pain.

>
>>
>> The VDAgentMouseState messages have one problem: They send the pointer
>> position as-is, which introduces a dependency on the screen size.

Yeah, if we could get rid of that, that would be great. We could even
introduce a new mouse message type to the existing spice vdagent protocol
and use capabilities to switch between the 2.

>> Spice deals with it, and although that is a little bit ugly it isn't a
>> big issue either as spice knows how big the screens are.

Well the client knows, but a guest os system level agent does not
necessarily know. With the current linux spice agent, the system level
daemon depends on the per session agent to notify it of resolution changes
(and if there is no per session agent active, the mouse gets dropped
  into server mode). So I'm in favor of fixing this even within the
current spice specific vdagent protocol

 >> When using
>> this outside spice context this becomes nasty though.
>>
>> Also I think it is a bad idea to mix guest agent stuff with the
>> paravirtual mouse. The mouse events need to get routed to the
>> X-Server, whereas the agent stuff is handled by some other daemon.
>> Also the multiplexing the spice server has to do to inject the mouse
>> messages supports that view IMHO.
>>
>> Related note: It is probably a good idea to also separate stuff which
>> is handled best by a system-wide guest daemon (such as reboot
>> requests, package installs, whatever, vdagent doesn't do that kind of
>> management btw) and stuff which needs to be handled by a per-user
>> daemon which has access to the user's X-Session (clipboard, ...).
>
> Definitely. VMware uses two separate daemons--one that runs in a system context and one that runs per user X session.
>

+1

>>
>> Ok, how to go forward now? Here is what I think we should do:
>>
>> (1) Merge spicevmc chardev. Needed anyway for compatibility and
>> for the other use cases outlined above.
>> (2) Design a paravirtual mouse device. Make sure it fullfills the
>> spice requirements, which are basically:
>> * support multiple displays.
>> * don't eat tons of cpu like usb-tablet does.
>> (3) spice can use the new pvmouse now and depricate sending mouse
>> events via vdagent protocol.
>> (4) In parallel review the vdagent protocol, then either extend or
>> redesign it for other use cases.
>
> I'm okay with this as long as there is a commitment from the Spice team to convert on a common framework.

I (*) am in favor of moving to a common framework. As long as
compatibility is maintained with the old agent, but that all lives inside
spice server, so should not be an issue.

*) Speaking for myself not the team (not that I know of anyone in the
team who is against).

> I'd like to have a little more discussion about agent design first to make sure that we're on the same page.
>
> For instance, Spice makes use of a 1-off protocol whereas something like virt-agent uses an established RPC protocol (XML-RPC). I'm not tied to using any particular protocol, but I think it's very important to use a standardized, well specified protocol.

I'm not sure what something like XML-RPC buys us here, other then
dragging in a lot of extra dependencies. I'm not saying that I'm against
using xmlrpc but I'm not sold on it either.

Note while on the subject of design, I think that having some sort of
capabilities negotiation so that we can provide compatibility between
different versions is important.

Regards,

Hans

Re: [Spice-devel] [Qemu-devel] spicevmv chardev, guest agents and paravirtual mouse

[Alon Levy]

On Wed, Jan 12, 2011 at 07:59:40PM +0100, Hans de Goede wrote:
> Hi,
> 
> On 01/12/2011 06:40 PM, Anthony Liguori wrote:
> >On 01/12/2011 10:12 AM, Gerd Hoffmann wrote:
> >>
> >>Hi folks,
> >>
> >>Looks like the spicevmc patch kicked the guest qagent discussion, so lets start with this, although it isn't related much to the agent issue
> >>itself ...
> >>
> >>
> >>The spicevmc chardev just pipes data from a chardev user within qemu
> >>to libspice and adds a "type" tag to it so libspice knows now to wind
> >>up the other end. There are several types:
> >>
> >>(1) vdagent, the spice guest agent. Will discuss this in detail
> >>below.
> >>
> >>(2) smartcard, this basically pipes the smartcard protocol over spice.
> >>Patches for smartcard support are on the list and should be almost
> >>ready for merge now. If you want connect a remote smart card reader
> >>to your guest you can use a tcp chardev, which will build a data
> >>pipeline like this:
> >>
> >>ccid-passthrough <-> tcp chardev <-> tcp protocol <->
> >>vcsclient <-> libcacard
> >>
> >>Or you can use the spicevmc chardev to use spice as transport:
> >>
> >>ccid-passthrough <-> spicevmc chardev <-> spice protocol <->
> >>spice client <-> libcacard
> >>
> >>If someone comes up with a vnc extention one could create something
> >>simliar for vnc tunneling:
> >>
> >>ccid-passthrough <-> vnctunnel chardev <-> vnc protocol <->
> >>gtk-vnc widget <-> libcacard
> >>
> >>(3) usb forwarding. Hans is busy with this. No working code yet.
> >>Will probably work pretty simliar to smartcard.
> >>
> >>(4) termial forwarding. Just an idea right now. Nowdays that the spice
> >>client side moves to gtk it would be easy to embed a termial widget,
> >>therby allowing easy access to the serial console using something
> >>like this:
> >>
> >>-chardev spicevmc,id=console,type=terminal
> >>-device isa-serial,index=0,chardev=console
> >>
> >>So even if you put the guest agent discussion completely aside the
> >>spicevmc chardev clearly has its uses.
> >>
> >>
> >>Ok, now for the spice vdagent. Alon posted the link to the specs in
> >>the spice wiki already, here it is again:
> >>http://spice-space.org/page/Whiteboard/AgentProtocol
> >>
> >>The header file with the protocol specification comes with the
> >>spice-protocol package, here is a direct link:
> >>http://cgit.freedesktop.org/spice/spice-protocol/tree/spic/vd_agent.h
> >>
> >>Linux agent code is here:
> >>http://cgit.freedesktop.org/~jwrdegoede/vdagent-linux/
> >>
> >>The protocol can be multiplexed via VDIChunkHeader->port. spice
> >>actually does that as the mouse messages (VDAgentMouseState) are
> >>generated by the spice server itself. Everything else is just piped
> >>from the guest to the spice client (and back). The protocol also has
> >>capabilities (VDAgentAnnounceCapabilities).
> >>
> >>There isn't much spice-specific stuff in there. The clipboard bits
> >>for example should work unmodified with vnc, one would just have to
> >>hack up a vnc extention to tunnel the agent protocol over vnc (and vnc
> >>client support of course).
> >
> >VNC already supports copy/paste as part of the protocol so can the agent protocol be terminated in QEMU such that the server can make use of the standard protocol extensions?
> >
> 
> That depends on if the VNC copy/paste support was done right. With right
> I mean that it should consist of the following messages:
> 1) A clipboard grab (send by guest -> client if ctrl+c pressed inside
>    the guest, other direction if ctrl-c is pressed in any app on the
>    client machine.) This should include a list of supported types the
>    app now owning the clipboard can offer the clipboard in. For example
>    text/plain-utf-8, text/html
> 2) A clipboard release send when the clipboard owning app exits
> 3) A clipboard request, send by one side when it wants to get the
>    clipboard data, prereq: the other side owns the clipboard, the
>    request is for a type in the list of types advertised when grabbing
> 4) clipboard data, response to clipboard request, could also be a nack
>    when a release / request message cross each other.
> 
> This is basically how both windows and X11 work wrt copy and paste at
> the low level, and by far the sanest way to support guest <-> client
> os copy paste (event driven, no polling or some such involved).
> 

Reading RealVNC's (authorative afaik) protocol 3.8 there are only two
messages, ClientCutText which supports latin (ISO 8859-1) only, and
ServerCutText, which is the same.

There are a few message extensions that I haven't looked at (defined
in the spec), maybe they include more copy paste functionality? Also,
we could of course propose an extension.

So I think we could easily work with this with the spice agent, just
have to generate some messages in qemu.

> I agree with the notion of having 2 agents in the guest, 1 system
> wide and 1 per session. Assuming we get a standardized protocol for
> the per session agent and we can somehow fit vnc's existing copy and
> paste support into the above model, it should be able to use that.
> 
> While on the 2 agent topic. As I've just completed written the
> spice agent for Linux I've some experience on this. Currently the
> spice agent actually already consists of 2 processes an always
> running daemon and a per X-session agent, there are 2 reasons for this:
> 1) Thinks like the paravirt mouse are best done at the system level
> 2) A multiplexer is needed between the client -> session agent channel,
> and the actual session agents, as multiple session agents could be
> running (think switch user functionality), and the messages should be
> forwarded to the session agent in the active (focussed / showing)
> session. spice-vdagentd uses console kit to determine the active
> session and monitor session changes.
> 
> This does not really impact any protocol designs, but still it is
> something to keep in mind.
> 
> >>
> >>I think the only spice-specific bit in the protocol is the display
> >>enumeration in the VDAgentMonitorsConfig and VDAgentMouseState
> >>messages. The numbers there simply reference the spice display
> >>channel number of the display in question, which just doesn't exist
> >>outside spice. Of course one can just ignore that for now as there is
> >>no multihead support outside spice anyway ...
> >>
> >>
> >>Also related: paravirtual mouse. I'd suggest to go for something new,
> >>based on virtio-serial, doing just the mouse and nothing else.
> >
> >I'd agree. I think we want something that actually terminates in the kernel for Linux guests since then we can expose it as an evdev device. No special X driver would be needed.
> 
> A paravirt mouse would need some sort of guest os support, yes. But
> not necessarily in the kernel. currently spice-vdagentd uses uinput
> to create an evdev device and send events from userspace. And the spice
> agent under windows does the same (although there the events are
> generated by the per user session agent process).
> 
> I'm not saying this should not be in the kernel, just that it does
> not have to be in the kernel. It might even make sense to try and
> use such a ipc mechanism for this, that in one guest os it could be
> a kernel driver and in the other a userspace solution, but I'm not
> sure how feasible that it.
> 
> >Is this something that makes sense for Spice in the future?
> 
> Yes, as long as guest support is kept in mind. Looking at it from
> that pov. I think I might favor a userspace solution, as we likely
> need an agent anyways, and adding / doing drivers for some guest
> os-s is a pain.
> 
> >
> >>
> >>The VDAgentMouseState messages have one problem: They send the pointer
> >>position as-is, which introduces a dependency on the screen size.
> 
> Yeah, if we could get rid of that, that would be great. We could even
> introduce a new mouse message type to the existing spice vdagent protocol
> and use capabilities to switch between the 2.
> 
> >>Spice deals with it, and although that is a little bit ugly it isn't a
> >>big issue either as spice knows how big the screens are.
> 
> Well the client knows, but a guest os system level agent does not
> necessarily know. With the current linux spice agent, the system level
> daemon depends on the per session agent to notify it of resolution changes
> (and if there is no per session agent active, the mouse gets dropped
>  into server mode). So I'm in favor of fixing this even within the
> current spice specific vdagent protocol
> 
> >> When using
> >>this outside spice context this becomes nasty though.
> >>
> >>Also I think it is a bad idea to mix guest agent stuff with the
> >>paravirtual mouse. The mouse events need to get routed to the
> >>X-Server, whereas the agent stuff is handled by some other daemon.
> >>Also the multiplexing the spice server has to do to inject the mouse
> >>messages supports that view IMHO.
> >>
> >>Related note: It is probably a good idea to also separate stuff which
> >>is handled best by a system-wide guest daemon (such as reboot
> >>requests, package installs, whatever, vdagent doesn't do that kind of
> >>management btw) and stuff which needs to be handled by a per-user
> >>daemon which has access to the user's X-Session (clipboard, ...).
> >
> >Definitely. VMware uses two separate daemons--one that runs in a system context and one that runs per user X session.
> >
> 
> +1
> 
> >>
> >>Ok, how to go forward now? Here is what I think we should do:
> >>
> >>(1) Merge spicevmc chardev. Needed anyway for compatibility and
> >>for the other use cases outlined above.
> >>(2) Design a paravirtual mouse device. Make sure it fullfills the
> >>spice requirements, which are basically:
> >>* support multiple displays.
> >>* don't eat tons of cpu like usb-tablet does.
> >>(3) spice can use the new pvmouse now and depricate sending mouse
> >>events via vdagent protocol.
> >>(4) In parallel review the vdagent protocol, then either extend or
> >>redesign it for other use cases.
> >
> >I'm okay with this as long as there is a commitment from the Spice team to convert on a common framework.
> 
> I (*) am in favor of moving to a common framework. As long as
> compatibility is maintained with the old agent, but that all lives inside
> spice server, so should not be an issue.
> 
> *) Speaking for myself not the team (not that I know of anyone in the
> team who is against).

+1

> 
> >I'd like to have a little more discussion about agent design first to make sure that we're on the same page.
> >
> >For instance, Spice makes use of a 1-off protocol whereas something like virt-agent uses an established RPC protocol (XML-RPC). I'm not tied to using any particular protocol, but I think it's very important to use a standardized, well specified protocol.
> 
> I'm not sure what something like XML-RPC buys us here, other then
> dragging in a lot of extra dependencies. I'm not saying that I'm against
> using xmlrpc but I'm not sold on it either.

Regarding wire protocol this would of course be very wasteful. If it's used
between the guest and host except for the translation overhead, which is minimal
I think (except for a copy-paste of a large amount of data - I'm not familiar
enough with XML-RPC but I guess it has some way to pass binary buffers unmodified?),
I don't have a particular objection, other then hoping we consider alternatives (there
are a few).

Regarding one-off protocols in general, I agree it creates code duplication
and should be avoided. In that respect I prefer mechanisems that produce
implementations from a declarative description. But I would prefer a binary
protocol, possibly a convertion of the existing spice protocol to declarative
form (like Alex did to the core spice protocols, see spice.proto and spice1.proto,
http://cgit.freedesktop.org/spice/spice/tree/spice.proto
Of course there are many implementations of this outside of spice, like protocol
buffers). I also don't think a one-off is necessarily a bad thing.

> 
> Note while on the subject of design, I think that having some sort of
> capabilities negotiation so that we can provide compatibility between
> different versions is important.
> 
> Regards,
> 
> Hans
> 

Re: [Spice-devel] [Qemu-devel] spicevmv chardev, guest agents and paravirtual mouse

[Anthony Liguori]

On 01/12/2011 12:59 PM, Hans de Goede wrote:
> Hi,
>
> On 01/12/2011 06:40 PM, Anthony Liguori wrote:
>> On 01/12/2011 10:12 AM, Gerd Hoffmann wrote:
>>>
>>> Hi folks,
>>>
>>> Looks like the spicevmc patch kicked the guest qagent discussion, so 
>>> lets start with this, although it isn't related much to the agent issue
>>> itself ...
>>>
>>>
>>> The spicevmc chardev just pipes data from a chardev user within qemu
>>> to libspice and adds a "type" tag to it so libspice knows now to wind
>>> up the other end. There are several types:
>>>
>>> (1) vdagent, the spice guest agent. Will discuss this in detail
>>> below.
>>>
>>> (2) smartcard, this basically pipes the smartcard protocol over spice.
>>> Patches for smartcard support are on the list and should be almost
>>> ready for merge now. If you want connect a remote smart card reader
>>> to your guest you can use a tcp chardev, which will build a data
>>> pipeline like this:
>>>
>>> ccid-passthrough <-> tcp chardev <-> tcp protocol <->
>>> vcsclient <-> libcacard
>>>
>>> Or you can use the spicevmc chardev to use spice as transport:
>>>
>>> ccid-passthrough <-> spicevmc chardev <-> spice protocol <->
>>> spice client <-> libcacard
>>>
>>> If someone comes up with a vnc extention one could create something
>>> simliar for vnc tunneling:
>>>
>>> ccid-passthrough <-> vnctunnel chardev <-> vnc protocol <->
>>> gtk-vnc widget <-> libcacard
>>>
>>> (3) usb forwarding. Hans is busy with this. No working code yet.
>>> Will probably work pretty simliar to smartcard.
>>>
>>> (4) termial forwarding. Just an idea right now. Nowdays that the spice
>>> client side moves to gtk it would be easy to embed a termial widget,
>>> therby allowing easy access to the serial console using something
>>> like this:
>>>
>>> -chardev spicevmc,id=console,type=terminal
>>> -device isa-serial,index=0,chardev=console
>>>
>>> So even if you put the guest agent discussion completely aside the
>>> spicevmc chardev clearly has its uses.
>>>
>>>
>>> Ok, now for the spice vdagent. Alon posted the link to the specs in
>>> the spice wiki already, here it is again:
>>> http://spice-space.org/page/Whiteboard/AgentProtocol
>>>
>>> The header file with the protocol specification comes with the
>>> spice-protocol package, here is a direct link:
>>> http://cgit.freedesktop.org/spice/spice-protocol/tree/spic/vd_agent.h
>>>
>>> Linux agent code is here:
>>> http://cgit.freedesktop.org/~jwrdegoede/vdagent-linux/
>>>
>>> The protocol can be multiplexed via VDIChunkHeader->port. spice
>>> actually does that as the mouse messages (VDAgentMouseState) are
>>> generated by the spice server itself. Everything else is just piped
>>> from the guest to the spice client (and back). The protocol also has
>>> capabilities (VDAgentAnnounceCapabilities).
>>>
>>> There isn't much spice-specific stuff in there. The clipboard bits
>>> for example should work unmodified with vnc, one would just have to
>>> hack up a vnc extention to tunnel the agent protocol over vnc (and vnc
>>> client support of course).
>>
>> VNC already supports copy/paste as part of the protocol so can the 
>> agent protocol be terminated in QEMU such that the server can make 
>> use of the standard protocol extensions?
>>
>
> That depends on if the VNC copy/paste support was done right.

Nothing is done right in VNC.  But it should be possible to expose it 
through VNC nonetheless without impacting the protocol.  Let me explain.

> With right
> I mean that it should consist of the following messages:
> 1) A clipboard grab (send by guest -> client if ctrl+c pressed inside
>    the guest, other direction if ctrl-c is pressed in any app on the
>    client machine.) This should include a list of supported types the
>    app now owning the clipboard can offer the clipboard in. For example
>    text/plain-utf-8, text/html

In the case of VNC, the QEMU VNC server would advertise text/plain.

> 2) A clipboard release send when the clipboard owning app exits
> 3) A clipboard request, send by one side when it wants to get the
>    clipboard data, prereq: the other side owns the clipboard, the
>    request is for a type in the list of types advertised when grabbing
> 4) clipboard data, response to clipboard request, could also be a nack
>    when a release / request message cross each other.

So in the case of VNC, there is a copy and paste message which 
effectively maps to (3) and (4) combined into a single server or client 
message.  There is no ability to fail but that just means that the VNC 
implementation either ignores failure or always succeeds.

>>> I think the only spice-specific bit in the protocol is the display
>>> enumeration in the VDAgentMonitorsConfig and VDAgentMouseState
>>> messages. The numbers there simply reference the spice display
>>> channel number of the display in question, which just doesn't exist
>>> outside spice. Of course one can just ignore that for now as there is
>>> no multihead support outside spice anyway ...
>>>
>>>
>>> Also related: paravirtual mouse. I'd suggest to go for something new,
>>> based on virtio-serial, doing just the mouse and nothing else.
>>
>> I'd agree. I think we want something that actually terminates in the 
>> kernel for Linux guests since then we can expose it as an evdev 
>> device. No special X driver would be needed.
>
>
> A paravirt mouse would need some sort of guest os support, yes. But
> not necessarily in the kernel. currently spice-vdagentd uses uinput
> to create an evdev device and send events from userspace. And the spice
> agent under windows does the same (although there the events are
> generated by the per user session agent process).
>
> I'm not saying this should not be in the kernel, just that it does
> not have to be in the kernel. It might even make sense to try and
> use such a ipc mechanism for this, that in one guest os it could be
> a kernel driver and in the other a userspace solution, but I'm not
> sure how feasible that it.

I'm not at all tied to it being in the kernel.  It's slightly nice not 
to require special support in userspace but I don't consider it a deal 
breaker.


>>
>>>
>>> The VDAgentMouseState messages have one problem: They send the pointer
>>> position as-is, which introduces a dependency on the screen size.
>
> Yeah, if we could get rid of that, that would be great. We could even
> introduce a new mouse message type to the existing spice vdagent protocol
> and use capabilities to switch between the 2.

I think the typical trick is to scale the coordinates to some large 
resolution.  Would there be any issue doing this in vdagent today?

>> I'd like to have a little more discussion about agent design first to 
>> make sure that we're on the same page.
>>
>> For instance, Spice makes use of a 1-off protocol whereas something 
>> like virt-agent uses an established RPC protocol (XML-RPC). I'm not 
>> tied to using any particular protocol, but I think it's very 
>> important to use a standardized, well specified protocol.
>
> I'm not sure what something like XML-RPC buys us here, other then
> dragging in a lot of extra dependencies. I'm not saying that I'm against
> using xmlrpc but I'm not sold on it either.
>
> Note while on the subject of design, I think that having some sort of
> capabilities negotiation so that we can provide compatibility between
> different versions is important.


Yes, xmlrpc provides a standard method of introspection that can be used 
for this.  I'm not really advocating XML-RPC, I just want a structured 
well specified RPC protocol.

Regards,

Anthony Liguori

>
> Regards,
>
> Hans

Re: [Spice-devel] [Qemu-devel] spicevmv chardev, guest agents and paravirtual mouse

[Anthony Liguori]

On 01/12/2011 01:28 PM, Alon Levy wrote:
>
> Regarding wire protocol this would of course be very wasteful. If it's used
> between the guest and host except for the translation overhead, which is minimal
> I think (except for a copy-paste of a large amount of data - I'm not familiar
> enough with XML-RPC but I guess it has some way to pass binary buffers unmodified?),
> I don't have a particular objection, other then hoping we consider alternatives (there
> are a few).
>
> Regarding one-off protocols in general, I agree it creates code duplication
> and should be avoided. In that respect I prefer mechanisems that produce
> implementations from a declarative description. But I would prefer a binary
> protocol, possibly a convertion of the existing spice protocol to declarative
> form (like Alex did to the core spice protocols, see spice.proto and spice1.proto,
> http://cgit.freedesktop.org/spice/spice/tree/spice.proto
> Of course there are many implementations of this outside of spice, like protocol
> buffers). I also don't think a one-off is necessarily a bad thing.
>    

Yeah, it might be worthwhile to look at protocol buffers.

Regards,

Anthony Liguori

>> Note while on the subject of design, I think that having some sort of
>> capabilities negotiation so that we can provide compatibility between
>> different versions is important.
>>
>> Regards,
>>
>> Hans
>>
>>      

Re: [Qemu-devel] spicevmv chardev, guest agents and paravirtual mouse

[Gerd Hoffmann]

   Hi,

>> There isn't much spice-specific stuff in there. The clipboard bits
>> for example should work unmodified with vnc, one would just have to
>> hack up a vnc extention to tunnel the agent protocol over vnc (and vnc
>> client support of course).
>
> VNC already supports copy/paste as part of the protocol so can the agent
> protocol be terminated in QEMU such that the server can make use of the
> standard protocol extensions?

Should be doable too, didn't look at the vnc extension in detail though.

>> Also related: paravirtual mouse. I'd suggest to go for something new,
>> based on virtio-serial, doing just the mouse and nothing else.
>
> I'd agree. I think we want something that actually terminates in the
> kernel for Linux guests since then we can expose it as an evdev device.
> No special X driver would be needed.

Doesn't have to terminate in the kernel, you can feed the linux input 
subsystem via uinput (vdagent-linux actually does that btw.), X will get 
the events via evdev and everything works fine in the singlehead case. 
I think you need a X driver to handle multihead though.

> Is this something that makes sense for Spice in the future?

Yes.  Supporting multihead is a requirement for spice though.  Having 
the initial implementation not support it would be fine, but we need at 
least a plan how to handle this case.

> For instance, Spice makes use of a 1-off protocol whereas something like
> virt-agent uses an established RPC protocol (XML-RPC). I'm not tied to
> using any particular protocol, but I think it's very important to use a
> standardized, well specified protocol.

If that is a hard requirement it means we'll have to design something 
new for the agent, take care that it fulfills the spice requirements, 
then switch over for new deployments.  Of course we'll have to maintain 
current vdagent for backward compatibility.

Depending on the design we might be able to translate between the 
current vdagent and new agent protocol.  Being able to do that would be 
quite helpful for the transition of course.

cheers,
   Gerd

Re: [Spice-devel] [Qemu-devel] spicevmv chardev, guest agents and paravirtual mouse

[Gerd Hoffmann]

   Hi,

>>> The VDAgentMouseState messages have one problem: They send the pointer
>>> position as-is, which introduces a dependency on the screen size.
>
> Yeah, if we could get rid of that, that would be great. We could even
> introduce a new mouse message type to the existing spice vdagent protocol
> and use capabilities to switch between the 2.

I'd rather go straight to the paravirtual mouse.  spice can use both 
vdagent and usual qemu input devices such as usb tablet to route mouse 
events, so the switchover should be relatively painless and also 
completely independent from the agent design.

cheers,
   Gerd

Re: [Spice-devel] [Qemu-devel] spicevmv chardev, guest agents and paravirtual mouse

[Gerd Hoffmann]

   Hi,

>>>> The VDAgentMouseState messages have one problem: They send the pointer
>>>> position as-is, which introduces a dependency on the screen size.
>>
>> Yeah, if we could get rid of that, that would be great. We could even
>> introduce a new mouse message type to the existing spice vdagent protocol
>> and use capabilities to switch between the 2.
>
> I think the typical trick is to scale the coordinates to some large
> resolution. Would there be any issue doing this in vdagent today?

No.  The vdagent actually registers a uinput device with the size 
matching the screen resolution today.  It could also register a tablet 
with a -- say -- 65536 x 65536 resolution and scale.  Even better would 
be to let spice-server or qemu do the scaling and feed vdagent/pvmouse 
with already scaled coordinates, so the vdagent doesn't has to hop 
through the loops Hans described to get the screen resolution coordinates.

qemu already uses a 0 -> 0xffff range internally for absolute input 
coordinates, the pvmouse (or should we better name it pvtablet?) should 
probably pass those values as-is to the guest.

cheers,
   Gerd

[Qemu-devel] Re: [Spice-devel] spicevmv chardev, guest agents and paravirtual mouse

[Tiziano Müller]

Am Mittwoch, den 12.01.2011, 17:12 +0100 schrieb Gerd Hoffmann:
> Hi folks,
> 
> Looks like the spicevmc patch kicked the guest qagent discussion, so 
> lets start with this, although it isn't related much to the agent issue
> itself ...
> 
> 
> The spicevmc chardev just pipes data from a chardev user within qemu
> to libspice and adds a "type" tag to it so libspice knows now to wind
> up the other end.  There are several types:
> 
> (1) vdagent, the spice guest agent.  Will discuss this in detail
>      below.
> 
> (2) smartcard, this basically pipes the smartcard protocol over spice.
>      Patches for smartcard support are on the list and should be almost
>      ready for merge now. If you want connect a remote smart card reader
>      to your guest you can use a tcp chardev, which will build a data
>      pipeline like this:
> 
>         ccid-passthrough <-> tcp chardev <-> tcp protocol <->
>           vcsclient <-> libcacard
> 
>      Or you can use the spicevmc chardev to use spice as transport:
> 
>         ccid-passthrough <-> spicevmc chardev <-> spice protocol <->
>         spice client <-> libcacard
> 
>      If someone comes up with a vnc extention one could create something
>      simliar for vnc tunneling:
> 
>         ccid-passthrough <-> vnctunnel chardev <-> vnc protocol <->
>         gtk-vnc widget <-> libcacard
> 
> (3) usb forwarding.  Hans is busy with this.  No working code yet.
>      Will probably work pretty simliar to smartcard.
> 
> (4) termial forwarding.  Just an idea right now.  Nowdays that the spice
>      client side moves to gtk it would be easy to embed a termial widget,
>      therby allowing easy access to the serial console using something
>      like this:
> 
>         -chardev spicevmc,id=console,type=terminal
>         -device isa-serial,index=0,chardev=console

I guess this could be used to get more info about the state of a VM from
the host system (like "system is ready", "system is rebooting", etc),
although it needs some adjustments in the guest to use the serial
console and string parsing on the host. Would it therefore make sense to
have a another type to get such status messages from the guest and have
a list of states defined somewhere? If yes, is this something which
should be implemented in the spice-part or is this general qemu stuff?

Furthermore it would be nice if the client side could get a list of
display resolutions supported in the guest and having the possibility to
change the resolution when resizing the guest window. The vdagent
protocol already has a message to pass this info but it only sends it
when going into fullscreen mode.

Yet another use case would be: authentication forwarding/injection to
the guest OS. With smartcard-support you have something like that, but
in our case we have the users authenticate against our management
backend first before they get access to a VM and it would be kind of
nice to be able to inject authentication data into the guest OS, either
via the spice client or coming from the host OS. 

Cheers,
Tiziano

[Qemu-devel] Re: [Spice-devel] spicevmv chardev, guest agents and paravirtual mouse

[Gerd Hoffmann]

   Hi,

>> (4) termial forwarding.  Just an idea right now.  Nowdays that the spice
>>       client side moves to gtk it would be easy to embed a termial widget,
>>       therby allowing easy access to the serial console using something
>>       like this:
>>
>>          -chardev spicevmc,id=console,type=terminal
>>          -device isa-serial,index=0,chardev=console
>
> I guess this could be used to get more info about the state of a VM from
> the host system (like "system is ready", "system is rebooting", etc),
> although it needs some adjustments in the guest to use the serial
> console and string parsing on the host.

That is really just for a terminal / serial console.  Running an guest 
agent protocol over a serial line instead of virtio-serial would be 
possible too, but that end would probably better handled by libvirt or 
some other management tool.

> Would it therefore make sense to
> have a another type to get such status messages from the guest and have
> a list of states defined somewhere? If yes, is this something which
> should be implemented in the spice-part or is this general qemu stuff?

It is general qemu stuff and how to design that best is the whole point 
of this agent discussion ;)

> Furthermore it would be nice if the client side could get a list of
> display resolutions supported in the guest and having the possibility to
> change the resolution when resizing the guest window. The vdagent
> protocol already has a message to pass this info but it only sends it
> when going into fullscreen mode.

That is just a client implementation thing.  The new spice-gtk client 
can send messages on window resize too.  Guest is supposed to pick the 
closest resolution it can handle.

> Yet another use case would be: authentication forwarding/injection to
> the guest OS. With smartcard-support you have something like that, but
> in our case we have the users authenticate against our management
> backend first before they get access to a VM and it would be kind of
> nice to be able to inject authentication data into the guest OS, either
> via the spice client or coming from the host OS.

Depends on the auth infrastructure whenever this is doable I guess.  If 
you are using x509 certificates the smartcard support should serve just 
fine.  I can also imagine that the guest can aquire kerberos tickets via 
spice-client or using some other agent for example (don't know kerberos 
good enougth to be sure though).

cheers,
   Gerd