[Qemu-devel] [PATCH 0/2] [RFC] qemu-ga: add support for guest command execution

[Qemu-devel] [PATCH 0/2] [RFC] qemu-ga: add support for guest command execution

[Michael Roth]

The code is still in rough shape, but while we're on the topic of guest agents
I wanted to put out a working example of how exec functionality can be added
to qemu-ga to provide a mechansim for building arbitrarilly high-level
interfaces.

The hope is that by allowing qemu-ga to execute commands in the guest, paired
with file read/write access, we can instrument a guest "on the fly" to support
any type of hyperviser functionality, and do so without dramatically enlarging
the role qemu-ga plays as a small, QEMU-specific agent that is tightly
integrated with QEMU/QMP/libvirt.

These patches add the following interfaces:

guest-file-open-pipe
guest-exec
guest-exec-status

The guest-file-open-pipe interface is analagous to the existing guest-file-open
interface (it might be best to roll it into it actually): it returns a handle
that can be handled via the existing guest-file-{read,write,flush,close}
interface. Internally it creates a FIFO pair that we can use to associate
handles to the stdin/stdout/stderr of a guest-exec spawned process. We can also
also use them to redirect output into other processes, giving us the basic
tools to build a basic shell (or a full-blown one if we add TTY support) using
a single qemu-ga.

Theoretically we can even deploy other agents, including session-level agents,
and communicate with them via these same handles. Thus, ovirt could deploy and
run an agent via qemu-ga, Spice could deploy vdagent, etc. Since the interface
is somewhat tedious, I'm working on a wrapper script to try out some of
these scenarios, but a basic use case using the raw QMP interface is included
below.

Any thoughts/comments on this approach are appreciated.

EXAMPLE USAGE (execute `top -b -n1`):

{'execute': 'guest-file-open-pipe'}
{'return': 6}

{'execute': 'guest-exec',                    \
 'arguments': {'detach': True,               \
               'handle_stdout': 6,           \
               'params': [{'param': '-b'},   \
                          {'param': '-n1'}], \
               'path': 'top'}}
{'return': {'exit-code': 0,                  \
            'exited': False,                 \
            'handle_stderr': -1,             \
            'handle_stdin': -1,              \
            'handle_stdout': 6,              \
            'pid': 14267}}

{'execute': 'guest-file-read',               \
 'arguments': {'count': 65536,               \
               'handle': 6}}
{'return': {'buf-b64': '',                   \
            'count': 0,                      \
            'eof': False}}

{'execute': 'guest-file-read',               \
 'arguments': {'count': 65536,               \
               'handle': 6}}
{'return': {'buf-b64': 'dG9wIC0gMjI6N...',   \
            'count': 11064,                  \
            'eof': True}}

/*
top - 22:41:49 up 1 day,  4:30,  3 users,  load average: 0.00, 0.00, 0.00
Tasks: 114 total,   1 running, 113 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.2%us,  0.2%sy,  0.0%ni, 99.6%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    504848k total,   445664k used,    59184k free,    49100k buffers
Swap:   323580k total,      224k used,   323356k free,   256392k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
14267 root      20   0 19272 1248  924 R    2  0.2   0:00.02 top
    1 root      20   0 24008 2048 1280 S    0  0.4   0:00.85 init
    2 root      20   0     0    0    0 S    0  0.0   0:00.30 kthreadd
    3 root      20   0     0    0    0 S    0  0.0   0:01.09 ksoftirqd/0
...
*/

{'execute': 'guest-exec-status',             \
 'arguments': {'pid': 14267}}
{'return': {'exit-code': 0,                  \
            'exited': True,                  \
            'handle_stderr': -1,             \
            'handle_stdin': -1,              \
            'handle_stdout': 6,              \
            'pid': 14267}}

{'execute': 'guest-file-close'}              \
 'arguments': {'handle': 6}}
{'return': {}}

Michael Roth (2):
  guest agent: add guest-file-open-pipe
  guest agent: add guest-exec and guest-exec-status interfaces

 qapi-schema-guest.json     |   79 +++++++-
 qga/guest-agent-commands.c |  478 +++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 531 insertions(+), 26 deletions(-)

-- 
1.7.4.1

Re: [Qemu-devel] [PATCH 0/2] [RFC] qemu-ga: add support for guest command execution

[Daniel P. Berrange]

On Tue, Dec 06, 2011 at 08:34:06AM -0600, Michael Roth wrote:
> The code is still in rough shape, but while we're on the topic of guest agents
> I wanted to put out a working example of how exec functionality can be added
> to qemu-ga to provide a mechansim for building arbitrarilly high-level
> interfaces.
> 
> The hope is that by allowing qemu-ga to execute commands in the guest, paired
> with file read/write access, we can instrument a guest "on the fly" to support
> any type of hyperviser functionality, and do so without dramatically enlarging
> the role qemu-ga plays as a small, QEMU-specific agent that is tightly
> integrated with QEMU/QMP/libvirt.
> 
> These patches add the following interfaces:
> 
> guest-file-open-pipe
> guest-exec
> guest-exec-status
> 
> The guest-file-open-pipe interface is analagous to the existing guest-file-open
> interface (it might be best to roll it into it actually): it returns a handle
> that can be handled via the existing guest-file-{read,write,flush,close}
> interface. Internally it creates a FIFO pair that we can use to associate
> handles to the stdin/stdout/stderr of a guest-exec spawned process. We can also
> also use them to redirect output into other processes, giving us the basic
> tools to build a basic shell (or a full-blown one if we add TTY support) using
> a single qemu-ga.
> 
> Theoretically we can even deploy other agents, including session-level agents,
> and communicate with them via these same handles. Thus, ovirt could deploy and
> run an agent via qemu-ga, Spice could deploy vdagent, etc. Since the interface
> is somewhat tedious, I'm working on a wrapper script to try out some of
> these scenarios, but a basic use case using the raw QMP interface is included
> below.
> 
> Any thoughts/comments on this approach are appreciated.
> 
> EXAMPLE USAGE (execute `top -b -n1`):
> 
> {'execute': 'guest-file-open-pipe'}
> {'return': 6}
> 
> {'execute': 'guest-exec',                    \
>  'arguments': {'detach': True,               \
>                'handle_stdout': 6,           \
>                'params': [{'param': '-b'},   \
>                           {'param': '-n1'}], \
>                'path': 'top'}}

This feels like a rather verbose way of specifying
the ARGV. Why not just allow

  {'execute': 'guest-exec',                    \
   'arguments': {'detach': True,               \
                 'handle_stdout': 6,           \
                 'params': ['-b', '-n1'],      \
                 'path': 'top'}}

Or even


  {'execute': 'guest-exec',                    \
   'arguments': {'detach': True,               \
                 'handle_stdout': 6,           \
                 'argv': ['top', '-b', '-n1']}} \

and just use the first element of argv as the binary to
execute. Also you might need to set env variables for
some tools, so we'd want

  {'execute': 'guest-exec',                    \
   'arguments': {'detach': True,               \
                 'handle_stdout': 6,           \
                 'argv': ['top', '-b', '-n1'], \
                 'env' : ['TMPDIR=/wibble']}}

and perhaps also you might want to run as a non-root
user, so allow a username/groupname ?

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [PATCH 0/2] [RFC] qemu-ga: add support for guest command execution

[Michael Roth]

On 12/06/2011 08:44 AM, Daniel P. Berrange wrote:
> On Tue, Dec 06, 2011 at 08:34:06AM -0600, Michael Roth wrote:
>> The code is still in rough shape, but while we're on the topic of guest agents
>> I wanted to put out a working example of how exec functionality can be added
>> to qemu-ga to provide a mechansim for building arbitrarilly high-level
>> interfaces.
>>
>> The hope is that by allowing qemu-ga to execute commands in the guest, paired
>> with file read/write access, we can instrument a guest "on the fly" to support
>> any type of hyperviser functionality, and do so without dramatically enlarging
>> the role qemu-ga plays as a small, QEMU-specific agent that is tightly
>> integrated with QEMU/QMP/libvirt.
>>
>> These patches add the following interfaces:
>>
>> guest-file-open-pipe
>> guest-exec
>> guest-exec-status
>>
>> The guest-file-open-pipe interface is analagous to the existing guest-file-open
>> interface (it might be best to roll it into it actually): it returns a handle
>> that can be handled via the existing guest-file-{read,write,flush,close}
>> interface. Internally it creates a FIFO pair that we can use to associate
>> handles to the stdin/stdout/stderr of a guest-exec spawned process. We can also
>> also use them to redirect output into other processes, giving us the basic
>> tools to build a basic shell (or a full-blown one if we add TTY support) using
>> a single qemu-ga.
>>
>> Theoretically we can even deploy other agents, including session-level agents,
>> and communicate with them via these same handles. Thus, ovirt could deploy and
>> run an agent via qemu-ga, Spice could deploy vdagent, etc. Since the interface
>> is somewhat tedious, I'm working on a wrapper script to try out some of
>> these scenarios, but a basic use case using the raw QMP interface is included
>> below.
>>
>> Any thoughts/comments on this approach are appreciated.
>>
>> EXAMPLE USAGE (execute `top -b -n1`):
>>
>> {'execute': 'guest-file-open-pipe'}
>> {'return': 6}
>>
>> {'execute': 'guest-exec',                    \
>>   'arguments': {'detach': True,               \
>>                 'handle_stdout': 6,           \
>>                 'params': [{'param': '-b'},   \
>>                            {'param': '-n1'}], \
>>                 'path': 'top'}}
>
> This feels like a rather verbose way of specifying
> the ARGV. Why not just allow
>
>    {'execute': 'guest-exec',                    \
>     'arguments': {'detach': True,               \
>                   'handle_stdout': 6,           \
>                   'params': ['-b', '-n1'],      \
>                   'path': 'top'}}
>
> Or even
>
>
>    {'execute': 'guest-exec',                    \
>     'arguments': {'detach': True,               \
>                   'handle_stdout': 6,           \
>                   'argv': ['top', '-b', '-n1']}} \
>

Agreed, this would look way nicer and is what I was shooting for 
initially. Unfortunately, the QAPI-generated QMP marshalling code, and 
the visitor interfaces it uses, expects lists to be of structured types. 
We might be able to special-case it for int and char* arrays though and 
open-code it though...

The problem I ran into when I looked at it though is that the QMP 
request is a QObject at that point that's contained inside a Visitor. So 
to manipulate it we'd have to extract the QList and manipulate it 
outside the visitor, which would be difficult since the Visitor is using 
a stack to manage it's traversal of the QObject based on the visitor 
calls we make, so it's difficult to pull it out of there...

Although, maybe we could just add a qmp_input_vistor_stack_top() that 
gives us the top of the stack, then call it at the appropriate point in 
the marshalling code and key into it to get qobj{'argv'} and whatnot. 
It's kinda hacky, but it's QMP-specific, and useful, so it might be 
worth it.

Alternatively, we could introduce another Visitor interface for handling 
lists of int64_t and char*.

So maybe it's not so bad, I'll look into it more.

> and just use the first element of argv as the binary to
> execute. Also you might need to set env variables for
> some tools, so we'd want
>
>    {'execute': 'guest-exec',                    \
>     'arguments': {'detach': True,               \
>                   'handle_stdout': 6,           \
>                   'argv': ['top', '-b', '-n1'], \
>                   'env' : ['TMPDIR=/wibble']}}
>
> and perhaps also you might want to run as a non-root
> user, so allow a username/groupname ?
>

Good idea, this would open up a lot of potential use cases (X 
session/display management, copy/paste, etc).

> Regards,
> Daniel

Re: [Qemu-devel] [PATCH 0/2] [RFC] qemu-ga: add support for guest command execution

[Alon Levy]

On Tue, Dec 06, 2011 at 10:43:42AM -0600, Michael Roth wrote:
> On 12/06/2011 08:44 AM, Daniel P. Berrange wrote:
> >On Tue, Dec 06, 2011 at 08:34:06AM -0600, Michael Roth wrote:
> >>The code is still in rough shape, but while we're on the topic of guest agents
> >>I wanted to put out a working example of how exec functionality can be added
> >>to qemu-ga to provide a mechansim for building arbitrarilly high-level
> >>interfaces.
> >>
> >>The hope is that by allowing qemu-ga to execute commands in the guest, paired
> >>with file read/write access, we can instrument a guest "on the fly" to support
> >>any type of hyperviser functionality, and do so without dramatically enlarging
> >>the role qemu-ga plays as a small, QEMU-specific agent that is tightly
> >>integrated with QEMU/QMP/libvirt.
> >>
> >>These patches add the following interfaces:
> >>
> >>guest-file-open-pipe
> >>guest-exec
> >>guest-exec-status
> >>
> >>The guest-file-open-pipe interface is analagous to the existing guest-file-open
> >>interface (it might be best to roll it into it actually): it returns a handle
> >>that can be handled via the existing guest-file-{read,write,flush,close}
> >>interface. Internally it creates a FIFO pair that we can use to associate
> >>handles to the stdin/stdout/stderr of a guest-exec spawned process. We can also
> >>also use them to redirect output into other processes, giving us the basic
> >>tools to build a basic shell (or a full-blown one if we add TTY support) using
> >>a single qemu-ga.
> >>
> >>Theoretically we can even deploy other agents, including session-level agents,
> >>and communicate with them via these same handles. Thus, ovirt could deploy and
> >>run an agent via qemu-ga, Spice could deploy vdagent, etc. Since the interface
> >>is somewhat tedious, I'm working on a wrapper script to try out some of
> >>these scenarios, but a basic use case using the raw QMP interface is included
> >>below.
> >>
> >>Any thoughts/comments on this approach are appreciated.

Is there a seperate transport (another virtio channel) for the launched
guest process? I'm thinking as usual on the copy/paste functionallity,
when passing a large buffer, I would not want to block any other future
qemu-ga command. Also I would like to avoid having to uuencode/decode
the traffic.

> >>
> >>EXAMPLE USAGE (execute `top -b -n1`):
> >>
> >>{'execute': 'guest-file-open-pipe'}
> >>{'return': 6}
> >>
> >>{'execute': 'guest-exec',                    \
> >>  'arguments': {'detach': True,               \
> >>                'handle_stdout': 6,           \
> >>                'params': [{'param': '-b'},   \
> >>                           {'param': '-n1'}], \
> >>                'path': 'top'}}
> >
> >This feels like a rather verbose way of specifying
> >the ARGV. Why not just allow
> >
> >   {'execute': 'guest-exec',                    \
> >    'arguments': {'detach': True,               \
> >                  'handle_stdout': 6,           \
> >                  'params': ['-b', '-n1'],      \
> >                  'path': 'top'}}
> >
> >Or even
> >
> >
> >   {'execute': 'guest-exec',                    \
> >    'arguments': {'detach': True,               \
> >                  'handle_stdout': 6,           \
> >                  'argv': ['top', '-b', '-n1']}} \
> >
> 
> Agreed, this would look way nicer and is what I was shooting for
> initially. Unfortunately, the QAPI-generated QMP marshalling code,
> and the visitor interfaces it uses, expects lists to be of
> structured types. We might be able to special-case it for int and
> char* arrays though and open-code it though...
> 
> The problem I ran into when I looked at it though is that the QMP
> request is a QObject at that point that's contained inside a
> Visitor. So to manipulate it we'd have to extract the QList and
> manipulate it outside the visitor, which would be difficult since
> the Visitor is using a stack to manage it's traversal of the QObject
> based on the visitor calls we make, so it's difficult to pull it out
> of there...
> 
> Although, maybe we could just add a qmp_input_vistor_stack_top()
> that gives us the top of the stack, then call it at the appropriate
> point in the marshalling code and key into it to get qobj{'argv'}
> and whatnot. It's kinda hacky, but it's QMP-specific, and useful, so
> it might be worth it.
> 
> Alternatively, we could introduce another Visitor interface for
> handling lists of int64_t and char*.
> 
> So maybe it's not so bad, I'll look into it more.
> 
> >and just use the first element of argv as the binary to
> >execute. Also you might need to set env variables for
> >some tools, so we'd want
> >
> >   {'execute': 'guest-exec',                    \
> >    'arguments': {'detach': True,               \
> >                  'handle_stdout': 6,           \
> >                  'argv': ['top', '-b', '-n1'], \
> >                  'env' : ['TMPDIR=/wibble']}}
> >
> >and perhaps also you might want to run as a non-root
> >user, so allow a username/groupname ?
> >
> 
> Good idea, this would open up a lot of potential use cases (X
> session/display management, copy/paste, etc).
> 
> >Regards,
> >Daniel
> 
> 

[Qemu-devel] [PATCH 0/2] [RFC] qemu-ga: add support for guest command execution

[srinath reddy]

[-- Attachment #1: Type: text/plain, Size: 196 bytes --]

Hi,

Can someone help me in finding the status of this RFC here
http://lists.gnu.org/archive/html/qemu-devel/2011-12/msg00722.html

I need a similar functionality.

Thanks,
Srinath.

-- 
good day

[-- Attachment #2: Type: text/html, Size: 442 bytes --]

Re: [Qemu-devel] [PATCH 0/2] [RFC] qemu-ga: add support for guest command execution

[Michael Roth]

Quoting srinath reddy (2013-10-07 09:06:04)
> Hi,
>
> Can someone help me in finding the status of this RFC here
> http://lists.gnu.org/archive/html/qemu-devel/2011-12/msg00722.html

I picked it up again a few months back and re-worked the interface
to address some of the initial review comments by using glib's
exec() wrappers. I also added support for windows along with the
guest-file-* interfaces, but then proceeded to blow everything away
in an rm -rf ~ incident. And, of course, I was too cool for backups.

I think I can still piece most of it back together from some vim
backup/temp files that managed to survive, I'm scared to confirm.
But either way I won't have time to take a look at it again until
the QEMU 1.7 development cycle.

>
> I need a similar functionality.
> 
> Thanks,
> Srinath.
> 
> --
> good day

Re: [Qemu-devel] [PATCH 0/2] [RFC] qemu-ga: add support for guest command execution

[Michael Roth]

Quoting Michael Roth (2013-10-08 16:12:52)
> Quoting srinath reddy (2013-10-07 09:06:04)
> > Hi,
> >
> > Can someone help me in finding the status of this RFC here
> > http://lists.gnu.org/archive/html/qemu-devel/2011-12/msg00722.html
> 
> I picked it up again a few months back and re-worked the interface
> to address some of the initial review comments by using glib's
> exec() wrappers. I also added support for windows along with the
> guest-file-* interfaces, but then proceeded to blow everything away
> in an rm -rf ~ incident. And, of course, I was too cool for backups.
> 
> I think I can still piece most of it back together from some vim
> backup/temp files that managed to survive, I'm scared to confirm.
> But either way I won't have time to take a look at it again until
> the QEMU 1.7 development cycle.

1.8 I mean, Nov/Dec timeframe

> 
> >
> > I need a similar functionality.
> > 
> > Thanks,
> > Srinath.
> > 
> > --
> > good day