[Qemu-devel] KVM call minutes for Jan 11

[Qemu-devel] KVM call minutes for Jan 11

[Chris Wright]

KVM Forum 2011
- expand the scope? yes, continue up the stack
- how long?  2 days (maybe 2 1/2 - 3 space permitting)
- where?  Vancouver with LinuxCon

Spice guest agent:
- virt agent, matahari, spice agent...what is in spice agent?
- spice char device
  - mouse, copy 'n paste, screen resolution change
- could be generic (at least input and copy/paste)
  - send protocol details of what is being sent
- need to look at how difficult it is to split it out from spice
  (how to split out in qemu vs. libspice)
- goal to converge on common framework
- more discussion on char device vs. protocol
  - eg. mouse_set breaks if mouse channel is part pv and part spice specific
- Alon will send link to protocol and try to propose new interfaces

migration and block devices:
- need to invalidate data after first read on target,
  because it can be stale
- close + reopen is what was done for NFS
- iscsi: can issue ioctl(BLKFLSBUF) to flush, but it's CAP_SYS_ADMIN only
- O_DIRECT to avoid cache (concerns that it's not guaranteed)
- agree change the default (cache=none for 

qemu patch queue is long:
- slow to return from break
- patience and more patch review will help make sure things are applied
  and don't fall through cracks

spice vdagent protocol, was Re: [Qemu-devel] KVM call minutes for Jan 11

[Alon Levy]

On Tue, Jan 11, 2011 at 07:53:40AM -0800, Chris Wright wrote:
> KVM Forum 2011
> - expand the scope? yes, continue up the stack
> - how long?  2 days (maybe 2 1/2 - 3 space permitting)
> - where?  Vancouver with LinuxCon
> 
> Spice guest agent:
> - virt agent, matahari, spice agent...what is in spice agent?
> - spice char device
>   - mouse, copy 'n paste, screen resolution change
> - could be generic (at least input and copy/paste)
>   - send protocol details of what is being sent
> - need to look at how difficult it is to split it out from spice
>   (how to split out in qemu vs. libspice)
> - goal to converge on common framework
> - more discussion on char device vs. protocol
>   - eg. mouse_set breaks if mouse channel is part pv and part spice specific
> - Alon will send link to protocol and try to propose new interfaces

http://spice-space.org/page/Whiteboard/AgentProtocol

That's the corrent documentation. Notably the clipboard is a todo there, I'll
try to get that filled in. I'll continue this discussion on a separate thread later.

Alon

> 
> migration and block devices:
> - need to invalidate data after first read on target,
>   because it can be stale
> - close + reopen is what was done for NFS
> - iscsi: can issue ioctl(BLKFLSBUF) to flush, but it's CAP_SYS_ADMIN only
> - O_DIRECT to avoid cache (concerns that it's not guaranteed)
> - agree change the default (cache=none for 
> 
> qemu patch queue is long:
> - slow to return from break
> - patience and more patch review will help make sure things are applied
>   and don't fall through cracks
> 

Re: spice vdagent protocol, was Re: [Qemu-devel] KVM call minutes for Jan 11

[Adam Litke]

On Tue, 2011-01-11 at 20:33 +0200, Alon Levy wrote:
> > Spice guest agent:
> > - virt agent, matahari, spice agent...what is in spice agent?
> > - spice char device
> >   - mouse, copy 'n paste, screen resolution change
> > - could be generic (at least input and copy/paste)
> >   - send protocol details of what is being sent
> > - need to look at how difficult it is to split it out from spice
> >   (how to split out in qemu vs. libspice)
> > - goal to converge on common framework
> > - more discussion on char device vs. protocol
> >   - eg. mouse_set breaks if mouse channel is part pv and part spice specific
> > - Alon will send link to protocol and try to propose new interfaces
> 
> http://spice-space.org/page/Whiteboard/AgentProtocol
> 
> That's the corrent documentation. Notably the clipboard is a todo there, I'll
> try to get that filled in. I'll continue this discussion on a separate thread later.

Thanks for sending this out Alon.  The use cases you have outlined are
very similar to the ones we have for virtagent.  The main differences I
can see so far are the the data encoding strategy and the spice agent's
method for delivery of events (mouse, etc).

Virtagent implements an RPC interface and uses xmlrpc-c to encode data
in XML.  This approach seems more general-purpose, extensible and easier
to manage over the long term than relying on a custom binary
representation.

As for event delivery, virtagent does not yet have an interface to allow
external programs to subscribe to events.  I am sure this can be done in
a generic way that is backwards-compatible with the current spice
architecture.  Such an interface should allow arbitrary programs to
subscribe to events but have no dependencies on those programs.  I am
not sure if something like D-Bus would be appropriate for Linux guests.
We'd need to consider Windows too.  

I see no reason why the core qemu use cases (shutdown, exec, copyfile)
and the spice use cases (mouse, copy-paste, graphics reconfiguration)
cannot (and should not) be satisfied by a single agent.  Going forward,
I think we'd need to agree on the wire protocol and guest-side event
subscription.

Thoughts?

-- 
Thanks,
Adam

Re: spice vdagent protocol, was Re: [Qemu-devel] KVM call minutes for Jan 11

[Alon Levy]

On Thu, Jan 13, 2011 at 09:01:34AM -0600, Adam Litke wrote:
> On Tue, 2011-01-11 at 20:33 +0200, Alon Levy wrote:
> > > Spice guest agent:
> > > - virt agent, matahari, spice agent...what is in spice agent?
> > > - spice char device
> > >   - mouse, copy 'n paste, screen resolution change
> > > - could be generic (at least input and copy/paste)
> > >   - send protocol details of what is being sent
> > > - need to look at how difficult it is to split it out from spice
> > >   (how to split out in qemu vs. libspice)
> > > - goal to converge on common framework
> > > - more discussion on char device vs. protocol
> > >   - eg. mouse_set breaks if mouse channel is part pv and part spice specific
> > > - Alon will send link to protocol and try to propose new interfaces
> > 
> > http://spice-space.org/page/Whiteboard/AgentProtocol
> > 
> > That's the corrent documentation. Notably the clipboard is a todo there, I'll
> > try to get that filled in. I'll continue this discussion on a separate thread later.
> 
> Thanks for sending this out Alon.  The use cases you have outlined are
> very similar to the ones we have for virtagent.  The main differences I
> can see so far are the the data encoding strategy and the spice agent's
> method for delivery of events (mouse, etc).
> 
Let me start by saying I'm not yet familiar with virtagent, I need to read
up on it (really didn't expect to try to merge with it at this point, not
that that isn't a good discussion).

> Virtagent implements an RPC interface and uses xmlrpc-c to encode data
> in XML.  This approach seems more general-purpose, extensible and easier
> to manage over the long term than relying on a custom binary
> representation.
> 

Agree with the benefits, but there are other alternatives like I outlined
in the thread that Gerd started, like a binary representation generated from
declarative description. Totally agree with using a single definition to
generate marshalling / demarshalling code. Not saying we can't work with
xmlrpc for qemu<->guest but for qemu<->client I think it is too wasteful.

> As for event delivery, virtagent does not yet have an interface to allow
> external programs to subscribe to events.  I am sure this can be done in
> a generic way that is backwards-compatible with the current spice
> architecture.  Such an interface should allow arbitrary programs to
> subscribe to events but have no dependencies on those programs.  I am
> not sure if something like D-Bus would be appropriate for Linux guests.
> We'd need to consider Windows too.  
> 

Really not sure what you mean by events here, maybe I missed something.

> I see no reason why the core qemu use cases (shutdown, exec, copyfile)
> and the spice use cases (mouse, copy-paste, graphics reconfiguration)
> cannot (and should not) be satisfied by a single agent.  Going forward,
> I think we'd need to agree on the wire protocol and guest-side event
> subscription.

At first glance, i'd say:
 * development - how would we develop a shared agent? two forks and a upstream
  that no one cares about? shared repository? seems easier to me to use
  a shared protocol and separate agents. Also better bug wise (bug in our
  code doesn't bring down your agent) to have separate agents. Plugins are
  not any better bug wise (segfault is a segfault). Also languages become a
  problem (our agents are in C++/C atm win/lin). So in summary I think it
  is better to have a shared qemu code (this has to be a single exe) and
  protocol perhaps.
 * security - we don't need our agent to be able to execute or to copy files
  or to shutdown, so we'd need some mechanism to turn those off. There is
  another different agent (going over virtio-serial too) used for RHEV
  which probably does need these kind of features, so maybe need to involve
  them in this discussion too. (RHEV isn't open source yet but aims to be
  AFAIK).

Like I said in the beginning, I should go and read up on virtagent, and come
back..

> 
> Thoughts?
> 

There you go :)

> -- 
> Thanks,
> Adam
> 
Alon

Re: spice vdagent protocol, was Re: [Qemu-devel] KVM call minutes for Jan 11

[Adam Litke]

On Thu, 2011-01-13 at 21:18 +0200, Alon Levy wrote:
> On Thu, Jan 13, 2011 at 09:01:34AM -0600, Adam Litke wrote:
> > On Tue, 2011-01-11 at 20:33 +0200, Alon Levy wrote:
> > > > Spice guest agent:
> > > > - virt agent, matahari, spice agent...what is in spice agent?
> > > > - spice char device
> > > >   - mouse, copy 'n paste, screen resolution change
> > > > - could be generic (at least input and copy/paste)
> > > >   - send protocol details of what is being sent
> > > > - need to look at how difficult it is to split it out from spice
> > > >   (how to split out in qemu vs. libspice)
> > > > - goal to converge on common framework
> > > > - more discussion on char device vs. protocol
> > > >   - eg. mouse_set breaks if mouse channel is part pv and part spice specific
> > > > - Alon will send link to protocol and try to propose new interfaces
> > > 
> > > http://spice-space.org/page/Whiteboard/AgentProtocol
> > > 
> > > That's the corrent documentation. Notably the clipboard is a todo there, I'll
> > > try to get that filled in. I'll continue this discussion on a separate thread later.
> > 
> > Thanks for sending this out Alon.  The use cases you have outlined are
> > very similar to the ones we have for virtagent.  The main differences I
> > can see so far are the the data encoding strategy and the spice agent's
> > method for delivery of events (mouse, etc).
> > 
> Let me start by saying I'm not yet familiar with virtagent, I need to read
> up on it (really didn't expect to try to merge with it at this point, not
> that that isn't a good discussion).
> 
> > Virtagent implements an RPC interface and uses xmlrpc-c to encode data
> > in XML.  This approach seems more general-purpose, extensible and easier
> > to manage over the long term than relying on a custom binary
> > representation.
> > 
> 
> Agree with the benefits, but there are other alternatives like I outlined
> in the thread that Gerd started, like a binary representation generated from
> declarative description. Totally agree with using a single definition to
> generate marshalling / demarshalling code. Not saying we can't work with
> xmlrpc for qemu<->guest but for qemu<->client I think it is too wasteful.
> 
> > As for event delivery, virtagent does not yet have an interface to allow
> > external programs to subscribe to events.  I am sure this can be done in
> > a generic way that is backwards-compatible with the current spice
> > architecture.  Such an interface should allow arbitrary programs to
> > subscribe to events but have no dependencies on those programs.  I am
> > not sure if something like D-Bus would be appropriate for Linux guests.
> > We'd need to consider Windows too.  
> > 
> 
> Really not sure what you mean by events here, maybe I missed something.

I was trying to get at the method by which a generic guest agent could
forward things like mouse coordinates to Spice.  Unfortunately, I sent
this out before I noticed the other thread which covered this topic
well.

> > I see no reason why the core qemu use cases (shutdown, exec, copyfile)
> > and the spice use cases (mouse, copy-paste, graphics reconfiguration)
> > cannot (and should not) be satisfied by a single agent.  Going forward,
> > I think we'd need to agree on the wire protocol and guest-side event
> > subscription.
> 
> At first glance, i'd say:
>  * development - how would we develop a shared agent? two forks and a upstream
>   that no one cares about? shared repository? seems easier to me to use
>   a shared protocol and separate agents. Also better bug wise (bug in our
>   code doesn't bring down your agent) to have separate agents. Plugins are
>   not any better bug wise (segfault is a segfault). Also languages become a
>   problem (our agents are in C++/C atm win/lin). So in summary I think it
>   is better to have a shared qemu code (this has to be a single exe) and
>   protocol perhaps.

I think it is critical that we have a single shared agent.  The agent
must be part of the default install of most future guests for any
host->guest API to be useful.  I will defer to discussions in the main
thread regarding the structure of the daemon(s).

>  * security - we don't need our agent to be able to execute or to copy files
>   or to shutdown, so we'd need some mechanism to turn those off. There is
>   another different agent (going over virtio-serial too) used for RHEV
>   which probably does need these kind of features, so maybe need to involve
>   them in this discussion too. (RHEV isn't open source yet but aims to be
>   AFAIK).

One idea that has been discussed is to restrict agent actions to
programs/scripts that exist in an virtagent.d/ directory (similar to how
udev rules are configured).  Virtagent could install a set of scripts
in /lib/virtagent/actions.d/ which can be overridden (or disabled) by
the user in /etc/virtagent/actions.d/.  If the 'shutdown' action is
missing, then the agent will not be able to perform shutdowns.

-- 
Thanks,
Adam

Re: spice vdagent protocol, was Re: [Qemu-devel] KVM call minutes for Jan 11

[Alon Levy]

On Thu, Jan 13, 2011 at 01:44:50PM -0600, Adam Litke wrote:
> On Thu, 2011-01-13 at 21:18 +0200, Alon Levy wrote:
> > On Thu, Jan 13, 2011 at 09:01:34AM -0600, Adam Litke wrote:
> > > On Tue, 2011-01-11 at 20:33 +0200, Alon Levy wrote:
> > > > > Spice guest agent:
> > > > > - virt agent, matahari, spice agent...what is in spice agent?
> > > > > - spice char device
> > > > >   - mouse, copy 'n paste, screen resolution change
> > > > > - could be generic (at least input and copy/paste)
> > > > >   - send protocol details of what is being sent
> > > > > - need to look at how difficult it is to split it out from spice
> > > > >   (how to split out in qemu vs. libspice)
> > > > > - goal to converge on common framework
> > > > > - more discussion on char device vs. protocol
> > > > >   - eg. mouse_set breaks if mouse channel is part pv and part spice specific
> > > > > - Alon will send link to protocol and try to propose new interfaces
> > > > 
> > > > http://spice-space.org/page/Whiteboard/AgentProtocol
> > > > 
> > > > That's the corrent documentation. Notably the clipboard is a todo there, I'll
> > > > try to get that filled in. I'll continue this discussion on a separate thread later.
> > > 
> > > Thanks for sending this out Alon.  The use cases you have outlined are
> > > very similar to the ones we have for virtagent.  The main differences I
> > > can see so far are the the data encoding strategy and the spice agent's
> > > method for delivery of events (mouse, etc).
> > > 
> > Let me start by saying I'm not yet familiar with virtagent, I need to read
> > up on it (really didn't expect to try to merge with it at this point, not
> > that that isn't a good discussion).
> > 
> > > Virtagent implements an RPC interface and uses xmlrpc-c to encode data
> > > in XML.  This approach seems more general-purpose, extensible and easier
> > > to manage over the long term than relying on a custom binary
> > > representation.
> > > 
> > 
> > Agree with the benefits, but there are other alternatives like I outlined
> > in the thread that Gerd started, like a binary representation generated from
> > declarative description. Totally agree with using a single definition to
> > generate marshalling / demarshalling code. Not saying we can't work with
> > xmlrpc for qemu<->guest but for qemu<->client I think it is too wasteful.
> > 
> > > As for event delivery, virtagent does not yet have an interface to allow
> > > external programs to subscribe to events.  I am sure this can be done in
> > > a generic way that is backwards-compatible with the current spice
> > > architecture.  Such an interface should allow arbitrary programs to
> > > subscribe to events but have no dependencies on those programs.  I am
> > > not sure if something like D-Bus would be appropriate for Linux guests.
> > > We'd need to consider Windows too.  
> > > 
> > 
> > Really not sure what you mean by events here, maybe I missed something.
> 
> I was trying to get at the method by which a generic guest agent could
> forward things like mouse coordinates to Spice.  Unfortunately, I sent
> this out before I noticed the other thread which covered this topic
> well.
> 
> > > I see no reason why the core qemu use cases (shutdown, exec, copyfile)
> > > and the spice use cases (mouse, copy-paste, graphics reconfiguration)
> > > cannot (and should not) be satisfied by a single agent.  Going forward,
> > > I think we'd need to agree on the wire protocol and guest-side event
> > > subscription.
> > 
> > At first glance, i'd say:
> >  * development - how would we develop a shared agent? two forks and a upstream
> >   that no one cares about? shared repository? seems easier to me to use
> >   a shared protocol and separate agents. Also better bug wise (bug in our
> >   code doesn't bring down your agent) to have separate agents. Plugins are
> >   not any better bug wise (segfault is a segfault). Also languages become a
> >   problem (our agents are in C++/C atm win/lin). So in summary I think it
> >   is better to have a shared qemu code (this has to be a single exe) and
> >   protocol perhaps.
> 
> I think it is critical that we have a single shared agent.  The agent
> must be part of the default install of most future guests for any
> host->guest API to be useful.  I will defer to discussions in the main
> thread regarding the structure of the daemon(s).
> 

Having agents install by default in a guest is great, I guess it would be
easier to justify a single agent install then multiple. But it would probably
still have to be multiple packages, like your /lib/virtagent/actions.d scheme,
where there would be a packages for (talking linux distribution specific for
a second) virtagent-spice and virtagent-admin for instance, plus
a virtagent which contains the actual agent. But this is getting into the structure
of the daemon(s), so I guess better moved there?

> >  * security - we don't need our agent to be able to execute or to copy files
> >   or to shutdown, so we'd need some mechanism to turn those off. There is
> >   another different agent (going over virtio-serial too) used for RHEV
> >   which probably does need these kind of features, so maybe need to involve
> >   them in this discussion too. (RHEV isn't open source yet but aims to be
> >   AFAIK).
> 
> One idea that has been discussed is to restrict agent actions to
> programs/scripts that exist in an virtagent.d/ directory (similar to how
> udev rules are configured).  Virtagent could install a set of scripts
> in /lib/virtagent/actions.d/ which can be overridden (or disabled) by
> the user in /etc/virtagent/actions.d/.  If the 'shutdown' action is
> missing, then the agent will not be able to perform shutdowns.
> 
> -- 
> Thanks,
> Adam
>