[Qemu-devel] [ANNOUNCE] virt-mem tools version 0.2.8 released

[Qemu-devel] [ANNOUNCE] virt-mem tools version 0.2.8 released

[Richard W.M. Jones]

I'm pleased to announce the latest release of the virt-mem tools,
version 0.2.8.

These are tools for system administrators which let you find things
like kernel messages, process lists and network information of your
guests.

For example:

  virt-uname
    'uname' command, shows OS version, architecture, etc.
  virt-dmesg
    'dmesg' command, shows kernel messages
  virt-ps
    'ps' command, shows process list 

Nothing needs to be installed in the guest for this to work, and the
tools are specifically designed to allow easy scripting and
integration with databases and monitoring systems.

Source is available from the web page here:

  http://et.redhat.com/~rjones/virt-mem/

The latest version (0.2.8) reworks the internals substantially so that
we have direct access to basically any kernel structure, and this will
allow us to quickly add the remaining features that people have asked
for (memory usage information, lists of network interfaces and so on).

As usual, patches, feedback, suggestions etc. are very welcome!

Binaries will be available in Fedora 10 (Rawhide) at some point soon.

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top

RE: [Qemu-devel] [ANNOUNCE] virt-mem tools version 0.2.8 released

[Alexey Eremenko]

[-- Attachment #1: Type: text/plain, Size: 656 bytes --]


This seems to be great !
I think it is similar to OpenVZ concept of controlling VMs from Host, right ?

How it works, if it is not installed in guest ?
Using PV, in connects into guests virtio-tty device ?

In may become more universal, if more command are added, such as: (like OpenVZ does)

vm 101 enter
(this will enter into guest VM TTY, right from host terminal).
-or-
vm 101 exec [guest-command]
(this will execute command on guest, using bash, right from host terminal).

For more commands from OpenVZ, that can be applied to Qemu/KVM, see: (7.8.2008)
http://www.howtoforge.com/installing-and-using-openvz-on-fedora9-p2

-Alexey

[-- Attachment #2: Type: text/html, Size: 1216 bytes --]

Re: [libvirt] RE: [Qemu-devel] [ANNOUNCE] virt-mem tools version 0.2.8 released

[Jun Koi]

On Thu, Aug 7, 2008 at 7:20 PM, Alexey Eremenko
<alexey.eremenko@qumranet.com> wrote:
>
> This seems to be great !
> I think it is similar to OpenVZ concept of controlling VMs from Host, right
> ?
>
> How it works, if it is not installed in guest ?

Basically he does that by inspecting the VM's memory. Something like
the "instrospection" mechanism.

One of the problem is that these tools work via libvirt, so on a VM is
not managed by libvirt, these tools no longer work.

Jun

Re: [libvirt] RE: [Qemu-devel] [ANNOUNCE] virt-mem tools version 0.2.8 released

[Daniel P. Berrange]

On Thu, Aug 07, 2008 at 07:40:58PM +0900, Jun Koi wrote:
> On Thu, Aug 7, 2008 at 7:20 PM, Alexey Eremenko
> <alexey.eremenko@qumranet.com> wrote:
> >
> > This seems to be great !
> > I think it is similar to OpenVZ concept of controlling VMs from Host, right
> > ?
> >
> > How it works, if it is not installed in guest ?
> 
> Basically he does that by inspecting the VM's memory. Something like
> the "instrospection" mechanism.

Yes they peek at the live guest kernel memory image to extract the data.

> One of the problem is that these tools work via libvirt, so on a VM is
> not managed by libvirt, these tools no longer work.

That's not a problem - that's a reason to use libvirt :-) It allows the
same tools to work whether using Xen, QEMU, KVM or any other full machine
virtualization suported by libvirt, rather than being tied to one particular
hypervisor. Not to mention ability to run them remotely, with authentication
and encryption, etc

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [libvirt] RE: [Qemu-devel] [ANNOUNCE] virt-mem tools version 0.2.8 released

[Alexey Eremenko]

The only problem: virt-mem doesn't compiles.

checking for ocamldoc... ocamldoc
checking for ocamlfind... ./configure: line 5121: WARNING:: command not found
no
configure: error: OCaml findlib is required

And I have installed Ocam.

linux-3wx2:~/Linstall/virt-mem-0.2.9 # rpm -qa | grep -i ocam
ocaml-3.10.2-22.1
ocaml-facile-1.1-121.1
ocaml-ocamldoc-3.10.2-22.1

openSUSE 11.0, 64-bit
-- 
-Alexey Eromenko "Technologov"

Re: [libvirt] RE: [Qemu-devel] [ANNOUNCE] virt-mem tools version 0.2.8 released

[Richard W.M. Jones]

On Thu, Aug 07, 2008 at 03:55:49PM +0300, Alexey Eremenko wrote:
> The only problem: virt-mem doesn't compiles.

It has a huge chain of dependencies actually.  Maybe better off
starting with our RPMs, either source or binary, from here:

https://bugzilla.redhat.com/show_bug.cgi?id=450713

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top

Re: [libvirt] RE: [Qemu-devel] [ANNOUNCE] virt-mem tools version 0.2.8 released

[Samuel Thibault]

Alexey Eremenko, le Thu 07 Aug 2008 15:55:49 +0300, a écrit :
> The only problem: virt-mem doesn't compiles.
> 
> checking for ocamldoc... ocamldoc
> checking for ocamlfind... ./configure: line 5121: WARNING:: command not found
> no
> configure: error: OCaml findlib is required
> 
> And I have installed Ocam.

ocaml findlib is not part of the main caml distribution, look for
something like ocaml-findlib.

Samuel

Re: [et-mgmt-tools] Re: [libvirt] RE: [Qemu-devel] [ANNOUNCE] virt-mem tools version 0.2.8 released

[Richard W.M. Jones]

On Thu, Aug 07, 2008 at 11:47:39AM +0100, Daniel P. Berrange wrote:
> On Thu, Aug 07, 2008 at 07:40:58PM +0900, Jun Koi wrote:
> > One of the problem is that these tools work via libvirt, so on a VM is
> > not managed by libvirt, these tools no longer work.
> 
> That's not a problem - that's a reason to use libvirt :-) It allows the
> same tools to work whether using Xen, QEMU, KVM or any other full machine
> virtualization suported by libvirt, rather than being tied to one particular
> hypervisor. Not to mention ability to run them remotely, with authentication
> and encryption, etc

We also support running the tools from memory images which you can
capture using the QEMU "memsave" command (see the '-t' option).  No
libvirt required for that, _but_ to see any interesting stuff you'd
need to capture the entire guest memory which could obviously be quite
large.

You could do 'virt-mem capture' which captures just the bits of memory
that contain interesting data, and that reduces the amount of data you
need to capture substantially.  Unfortunately I broke 'virt-mem
capture' in the latest release by accident, and in any case it
requires libvirt to do the capturing.

I think the message here is, install libvirt & be happy :-)

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top

Re: [et-mgmt-tools] Re: [libvirt] RE: [Qemu-devel] [ANNOUNCE] virt-mem tools version 0.2.8 released

[Javier Guerra]

On Thu, Aug 7, 2008 at 8:06 AM, Richard W.M. Jones <rjones@redhat.com> wrote:
> I think the message here is, install libvirt & be happy :-)

nice as this tool sounds, i would need far more than this to make me
switch from a simple, easily scriptable command-line to a generic,
'lowest common', solution like libvirt.

of course, i hope it keeps getting better.  who knows? maybe in a year
or so it would be comparable to the CLI.


-- 
Javier

Re: [et-mgmt-tools] Re: [libvirt] RE: [Qemu-devel] [ANNOUNCE] virt-mem tools version 0.2.8 released

[Jamie Lokier]

Javier Guerra wrote:
> On Thu, Aug 7, 2008 at 8:06 AM, Richard W.M. Jones <rjones@redhat.com> wrote:
> > I think the message here is, install libvirt & be happy :-)
> 
> nice as this tool sounds, i would need far more than this to make me
> switch from a simple, easily scriptable command-line to a generic,
> 'lowest common', solution like libvirt.
> 
> of course, i hope it keeps getting better.  who knows? maybe in a year
> or so it would be comparable to the CLI.

Regrettably I agree for the moment.

I ended up writing a Perl management script for my KVM VMs because
libvirt was just too muddled and limited for my needs, and because the
config file format confused me, didn't handle everything I needed, and
I didn't find clear documentation on it.

Also, I wanted to import existing guests from another VM, and
libvirt's tools seemed strongly geared around creating new VMs to use
with libvirt.  So I had to write config files for it - see above.

I like the idea of libvirt a lot and wish it well.

My own Perl script was a nightmare to write even though it's not so
long (synchronisation & monitor issues especially), so I respect
what's done.  It's a good goal.

But I just found it too confusing to use in the ways I needed to use
KVM, that I gave up on libvirt for now rather than spend the
considerable time to get to grips with what it's doing, and it's
config format.

What would be nicer is a VM management protocol build in to QEMU, KVM
and XEN, which is a bit like the monitor, but supports multiple client
connections and overlapping operations (where reasonable), and is a
bit more structured, so e.g. you can get the state of anything whose
state you can set, you can wait for events, etc.  The somewhat
object-based config file work that's been discussed not long ago would
be a good thing to structure it around.

-- Jamie

Re: [et-mgmt-tools] Re: [libvirt] RE: [Qemu-devel] [ANNOUNCE] virt-mem tools version 0.2.8 released

[Richard W.M. Jones]

On Sun, Aug 10, 2008 at 02:28:27AM +0100, Jamie Lokier wrote:
> Javier Guerra wrote:
> > On Thu, Aug 7, 2008 at 8:06 AM, Richard W.M. Jones <rjones@redhat.com> wrote:
> > > I think the message here is, install libvirt & be happy :-)
> > 
> > nice as this tool sounds, i would need far more than this to make me
> > switch from a simple, easily scriptable command-line to a generic,
> > 'lowest common', solution like libvirt.
> > 
> > of course, i hope it keeps getting better.  who knows? maybe in a year
> > or so it would be comparable to the CLI.
> 
> Regrettably I agree for the moment.
> 
> I ended up writing a Perl management script for my KVM VMs because
> libvirt was just too muddled and limited for my needs, and because the
> config file format confused me, didn't handle everything I needed, and
> I didn't find clear documentation on it.

The configuration format is documented here:

  http://libvirt.org/formatdomain.html

You can also print out the configuration from any existing guest using
'virsh dumpxml <domain>' if you need examples.

I'm intrigued by what your Perl management script needed that isn't
exposed by libvirt.  Libvirt deliberately doesn't expose the full
feature set of any one hypervisor which it supports, but instead
exposes common features.  The reason for this is so that you can
switch hypervisor technologies later on.

Clearly, we all love KVM, but people have different needs from
hypervisors and KVM won't fit all of them.  For example, lightweight
container-based approaches are better for some virtualization problems
(particularly where you really need to run 100s or 1000s of guests on
a single machine), and people will still be running Xen and VMWare for
many years to come.

[...]
> What would be nicer is a VM management protocol build in to QEMU, KVM
> and XEN, which is a bit like the monitor, but supports multiple client
> connections and overlapping operations (where reasonable), and is a
> bit more structured, so e.g. you can get the state of anything whose
> state you can set, you can wait for events, etc.  The somewhat
> object-based config file work that's been discussed not long ago would
> be a good thing to structure it around.

This is what libvirt gives you (and lots more, eg. secure remote
access to hypervisors, bindings to Perl & many other languages, etc.).
Can you be more specfic about what you couldn't do with libvirt?

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top

Re: [et-mgmt-tools] Re: [libvirt] RE: [Qemu-devel] [ANNOUNCE] virt-mem tools version 0.2.8 released

[Richard W.M. Jones]

On Sun, Aug 10, 2008 at 11:07:32AM +0100, Richard W.M. Jones wrote:
> Libvirt deliberately doesn't expose the full
> feature set of any one hypervisor which it supports, but instead
> exposes common features.

Judging by one private reply I got, I don't want this to be
misinterpreted.  Libvirt DOESN'T expose the minimum subset of all
hypervisors (because that would be very small and useless).  It
exposes general virtualization features, even if those features only
apply to one or two hypervisors.  For example:

  migration - only works with Xen & KVM, but could be applicable
   to other hypervisors in the future when we support them and they
   support migration

  scheduler tuning - only for Xen, but generally applicable

  adding/dropping interfaces from live guests - a general feature
    supported by many but not all of the backends

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
Read my OCaml programming blog: http://camltastic.blogspot.com/
Fedora now supports 60 OCaml packages (the OPEN alternative to F#)
http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora