[Qemu-devel] [RFC] QEMU Guest Tools ISO Follow-up

[Qemu-devel] [RFC] QEMU Guest Tools ISO Follow-up

[Michael Roth]

These patches apply on top of master (8-22), and can also be obtained from:
git://repo.or.cz/qemu/mdroth.git guest-tools-iso-v1

This is a follow-up to the previous discussion regarding deployment QEMU/KVM guest dependencies via a distributable ISO:

http://lists.gnu.org/archive/html/qemu-devel/2011-06/msg02239.html

At the time a cross-platform byte-compiled python install framework was being explored, but since qemu-ga will likely be the only user for a while I wanted to focus on putting together a dead simple framework for linux guests and going from there.

This current framework works as follows:

Adding a tool:

1) A tool intended for guest deployment adds distribution files (init/config scripts, etc) to qemu.git/guest-tools-iso/projects/<tool>
2) A linux executable/script, preferably a simple shell script to ensure the guest can run it, is placed in that same directory to handle all the install logic.
3) The logic to build the required binaries for that tool/project is added to qemu.git/scripts/build-guest-iso.sh

Generating an ISO/tarball:

1) qemu.git/scripts/build-guest-iso.sh is invoked: scripts/build-guest-iso.sh
2) guest-tools-iso/ gets copied to $tmp/qemu-guest-tools-$VERSION
3) The ISO-generation logic for each project (in build-guest-iso.sh) will build/copy binaries and dynamically-generated files to $tmp/projects/<tool>
4) An ISO and tarball of the contents of $tmp/qemu-guest-tools-$VERSION are then created and placed in the current directory.

libvirt/virsh/hmp/qmp users would install the tools by mounting the ISO and running the installer for their platform. Ideally, this would be done via a new interface that does intelligent things: for instance, automatically locating the appropriate ISO in the host, or checking if the guest agent is already installed and kicking off the installer automatically after mounting (and if not, passing back some indication that something like virt-manager can use to guide the user to the next steps).

For users of GUI-based management tools like virt-manager, this would all manifest in a simple, tooltip-guided "Install Guest Additions" button, as with virtualbox/VMWare.

TODO/KNOWN ISSUES:

 - qemu-ga has a single notable dependency, glib, which makes it easy to build a single executable 32-bit and 64-bit executable that runs on multiple distros. Even so, we have issues with glibc that require an older host, like RHEL5, or a cross-build environment, to build. With other projects this mismatches will be more pronounced. To take some of the subtleties out of the equation, it would be ideal to tie this ISO generation command into Makefile and do some logic during configuration to gather up all these variables and determine whether or not we can generate the ISO and what's needed to build it. So far the known requirements are glibc <= 2.5 and glib >= 2.0. A user could then attempt to resolve these dependencies by building on a different host, or putting together a cross-build environment that meets these dependencies. Similar handling would be needed for MinGW-based builds when support for windows guest tools are added.
 - Support for interactive/graphical installs would be nice. For now we're command-line only.
 - Currently every tool's installer script needs to do stuff to figure out what arch/distro/platform it's on. Ideally this would all be done via the installer harness, which would then pass the information to individual installers as environment variables or parameters.

Any comments on the approach/code are welcome.

guest-tools-iso/install.linux                      |    7 ++
.../projects/qemu-ga/debian/qemu-ga.init           |  112 ++++++++++++++++++++
.../projects/qemu-ga/debian/qemu-ga.sysconfig      |   14 +++
guest-tools-iso/projects/qemu-ga/install.linux     |   61 +++++++++++
.../projects/qemu-ga/redhat/qemu-ga.init           |   97 +++++++++++++++++
.../projects/qemu-ga/redhat/qemu-ga.sysconfig      |   14 +++
scripts/build-guest-iso.sh                         |   46 ++++++++
7 files changed, 351 insertions(+), 0 deletions(-)

[Qemu-devel] [PATCH 1/5] guest tools: add iso/tarball generator for guest tools

[Michael Roth]

This script runs a series of tool-specific builds/configuration, each of
which should result in a self-contained installation directory at
$iso_root/projects/<project>, with a script or executable called
"install.linux". The script/executable must be runnable on standard
distros, so a basic shell script to kick off the install logic for the
guest is preferred.

When run in the guest, the install harness will iterate through these
projects, cd'ing into $iso_root/projects/<project> and executing the
"install.linux" script.

Windows support will be added later in a similar manner.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
 scripts/build-guest-iso.sh |   46 ++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 46 insertions(+), 0 deletions(-)
 create mode 100755 scripts/build-guest-iso.sh

diff --git a/scripts/build-guest-iso.sh b/scripts/build-guest-iso.sh
new file mode 100755
index 0000000..c8fbee8
--- /dev/null
+++ b/scripts/build-guest-iso.sh
@@ -0,0 +1,46 @@
+#!/bin/sh
+
+tmp=${tmp:-"/tmp"}
+qemu_dir=${qemu_dir:-$PWD}
+concurrency=${concurrency:-4}
+output_dir=${output_dir:-$PWD}
+
+if [ -e $qemu_dir/VERSION ]; then
+    version=`cat $qemu_dir/VERSION`
+else
+    echo qemu source directory not found
+    exit 1
+fi
+
+package=qemu-guest-tools-${version}${version_extra}
+iso_root=$tmp/$package
+build_dir=$tmp/qemu_iso_build
+projects_dir=$iso_root/projects
+mkdir -p $projects_dir
+mkdir -p $build_dir
+rm -rf $iso_root/* $build_dir/*
+
+cp -r $qemu_dir/guest-tools-iso/* $iso_root
+cd $build_dir
+
+# BUILD PROJECTS
+
+# qemu-ga
+
+# --enable-io-thread is just a workaround on RHEL5 for lack of signalfd()
+# support (which configure complains about). we don't use either
+$qemu_dir/configure --enable-guest-agent --extra-cflags="-m32" --enable-io-thread
+make -j$concurrency qemu-ga
+cp qemu-ga $projects_dir/qemu-ga/qemu-ga.linux.x86
+make distclean
+
+$qemu_dir/configure --enable-guest-agent
+make -j$concurrency qemu-ga
+cp qemu-ga $projects_dir/qemu-ga/qemu-ga.linux.x86_64
+make distclean
+
+# GENERATE GUEST TOOLS ISO/TARBALL
+
+cd $tmp
+tar zpvcf $output_dir/$package.tar.gz $package
+mkisofs -o $output_dir/$package.iso $package
-- 
1.7.0.4

[Qemu-devel] [PATCH 2/5] guest tools: installer harness

[Michael Roth]

This is what the users run in the guest to install the guest tools.
Since initially only the guest agent will be installed in the manner,
the logic is to simply iterate through each projects, execute the
install script, then do the next. It's up the the project-specific
install scripts to decide whether to upgrade/error/etc. In the future
this may change to support more intelligent dependency checking and
allow for interactive installs.

Only linux is supported initially, windows and other platforms will
follow as needed.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
 guest-tools-iso/install.linux |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)
 create mode 100755 guest-tools-iso/install.linux

diff --git a/guest-tools-iso/install.linux b/guest-tools-iso/install.linux
new file mode 100755
index 0000000..2291444
--- /dev/null
+++ b/guest-tools-iso/install.linux
@@ -0,0 +1,7 @@
+#!/bin/sh
+
+for project in projects/*; do
+    cd $project
+    sudo ./install.linux
+    cd -
+done
-- 
1.7.0.4

[Qemu-devel] [PATCH 3/5] guest tools: qemu-ga, debian init files

[Michael Roth]

Init files for debian/ubuntu guests

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
 .../projects/qemu-ga/debian/qemu-ga.init           |  112 ++++++++++++++++++++
 .../projects/qemu-ga/debian/qemu-ga.sysconfig      |   14 +++
 2 files changed, 126 insertions(+), 0 deletions(-)
 create mode 100755 guest-tools-iso/projects/qemu-ga/debian/qemu-ga.init
 create mode 100644 guest-tools-iso/projects/qemu-ga/debian/qemu-ga.sysconfig

diff --git a/guest-tools-iso/projects/qemu-ga/debian/qemu-ga.init b/guest-tools-iso/projects/qemu-ga/debian/qemu-ga.init
new file mode 100755
index 0000000..0a4fd91
--- /dev/null
+++ b/guest-tools-iso/projects/qemu-ga/debian/qemu-ga.init
@@ -0,0 +1,112 @@
+#!/bin/sh
+#
+# qemu-ga      Control the QEMU guest agent
+#
+# Author:      Adam Litke <agl@us.ibm.com>
+#
+# Copyright 2011 IBM, and/or its affiliates.
+# Released under the GPL
+#
+# chkconfig: 2345 50 40
+# description: The QEMU guest agent provides a mechanism for communication \
+#              with a QEMU host and a host<->guest command API.
+# processname: qemu-ga
+# pidfile: /var/run/qemu-ga.pid
+#
+
+### BEGIN INIT INFO
+# Provides: qemu-ga
+# Required-Start: $syslog
+# Required-Stop:
+# Should-Start:
+# Default-Start: 2 3 4 5
+# Default-Stop: 0 6
+# Short-Description: QEMU guest agent
+# Description: The QEMU guest agent provides a mechanism for communication
+#    with a QEMU host and a host<->guest command API.
+### END INIT INFO
+
+. /lib/lsb/init-functions
+. /etc/sysconfig/qemu-ga
+
+prog=qemu-ga
+bin=/usr/sbin/qemu-ga
+pidfile=${PIDFILE-"/var/run/qemu-ga.pid"}
+RETVAL=0
+
+wait_for_port() {
+    for i in `seq 1 5`; do
+        [ -f "$PORT" ] && return 0
+        sleep 1
+    done
+    return 1
+}
+
+start() {
+    log_daemon_msg "Starting QEMU Guest Agent" "$prog"
+
+    # Ensure the virtio_console module is loaded
+    /sbin/modprobe virtio_console
+    wait_for_port
+
+    log_daemon_msg "$pidfile" "$prog"
+    start_daemon -p $pidfile -- $bin $OPTIONS -f $pidfile
+    RETVAL=$?
+    log_end_msg $RETVAL
+    return $RETVAL
+}
+
+stop() {
+    log_daemon_msg "Stopping QEMU Guest Agent" "$prog"
+    killproc -p ${pidfile}
+    RETVAL=$?
+    log_end_msg $RETVAL
+    return $RETVAL
+}
+
+restart() {
+    stop
+    start
+    return $?
+}
+
+condrestart() {
+    pidofproc
+    if [ $? -eq 0 ]; then
+        log_daemon_msg "QEMU Guest Agent already running" "$prog"
+        RETVAL=0
+        log_end_msg $RETVAL
+    else
+        restart
+        return $?
+    fi
+
+}
+
+case "$1" in
+  start)
+	start
+    RETVAL=$?
+	;;
+  stop)
+	stop
+    RETVAL=$?
+	;;
+  status)
+    status_of_pid -p ${pidfile} $bin $prog
+	RETVAL=$?
+	;;
+  restart|force-reload)
+	restart
+    RETVAL=$?
+	;;
+  condrestart|try-restart)
+	condrestart
+    RETVAL=$?
+	;;
+  *)
+	echo $"Usage: $prog {start|stop|restart|force-reload|condrestart|try-restart|status|help}"
+	RETVAL=2
+esac
+
+exit $RETVAL
diff --git a/guest-tools-iso/projects/qemu-ga/debian/qemu-ga.sysconfig b/guest-tools-iso/projects/qemu-ga/debian/qemu-ga.sysconfig
new file mode 100644
index 0000000..c125a82
--- /dev/null
+++ b/guest-tools-iso/projects/qemu-ga/debian/qemu-ga.sysconfig
@@ -0,0 +1,14 @@
+# This file configures the qemu-ga guest agent daemon
+
+# METHOD= Sets the type of channel being used.  The only type currently
+# supported is virtio-serial
+METHOD="virtio-serial"
+
+# PORT= Sets the name of the channel to use.  This should be a device file
+PORT="/dev/virtio-ports/org.qemu.guest_agent.0"
+
+# LOGFILE= Sets the path to the log file.
+LOGFILE="/var/log/qemu-ga.log"
+
+# Set options according to above
+OPTIONS="-m $METHOD -p $PORT -l $LOGFILE -d"
-- 
1.7.0.4

[Qemu-devel] [PATCH 4/5] guest tools: qemu-ga, redhat init scripts

[Michael Roth]

Init files for redhat/fedora guests

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
 .../projects/qemu-ga/redhat/qemu-ga.init           |   97 ++++++++++++++++++++
 .../projects/qemu-ga/redhat/qemu-ga.sysconfig      |   14 +++
 2 files changed, 111 insertions(+), 0 deletions(-)
 create mode 100755 guest-tools-iso/projects/qemu-ga/redhat/qemu-ga.init
 create mode 100644 guest-tools-iso/projects/qemu-ga/redhat/qemu-ga.sysconfig

diff --git a/guest-tools-iso/projects/qemu-ga/redhat/qemu-ga.init b/guest-tools-iso/projects/qemu-ga/redhat/qemu-ga.init
new file mode 100755
index 0000000..fe7bc98
--- /dev/null
+++ b/guest-tools-iso/projects/qemu-ga/redhat/qemu-ga.init
@@ -0,0 +1,97 @@
+#!/bin/sh
+#
+# qemu-ga     Control the QEMU guest agent
+#
+# Author:      Adam Litke <agl@us.ibm.com>
+#
+# Copyright 2011 IBM, and/or its affiliates.
+# Released under the GPL
+#
+# chkconfig: 2345 50 40
+# description: The QEMU guest agent provides a mechanism for communication \
+#              with a QEMU host and a host<->guest command API.
+# processname: qemu-ga
+# pidfile: /var/run/qemu-ga.pid
+#
+### BEGIN INIT INFO
+# Provides: qemu-ga
+# Required-Start: udev-post
+# Required-Stop:
+# Should-Start:
+# Default-Start: 2 3 4 5
+# Default-Stop: 0 6
+# Short-Description: QEMU guest agent
+# Description: The QEMU guest agent provides a mechanism for communication
+#    with a QEMU host and a host<->guest command API.
+### END INIT INFO
+
+. /etc/rc.d/init.d/functions
+. /etc/sysconfig/qemu-ga
+
+prog=qemu-ga
+bin=/usr/sbin/qemu-ga
+pidfile=${PIDFILE-/var/run/qemu-ga.pid}
+RETVAL=0
+
+wait_for_port() {
+    for i in `seq 1 5`; do
+        [ -f "$PORT" ] && return 0
+        sleep 1
+    done
+    return 1
+}
+
+start() {
+    echo -n $"Starting $prog: "
+    # Ensure the virtio_console module is loaded
+    /sbin/modprobe virtio_console
+    wait_for_port
+
+    #daemon --pidfile=${pidfile} $bin $OPTIONS
+    echo $bin -f ${pidfile} $OPTIONS >/tmp/qemu-ga.out
+    $bin -f ${pidfile} $OPTIONS
+    RETVAL=$?
+    echo
+    [ $RETVAL -eq 0 ] && touch /var/lock/subsys/$prog
+}
+
+stop() {
+    echo -n $"Stopping $prog: "
+    killproc -p ${pidfile}
+    RETVAL=$?
+    echo
+    [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/$prog
+}
+
+restart() {
+    stop
+    start
+}
+
+condrestart() {
+    [ -e /var/lock/subsys/$prog ] && restart || :
+}
+
+case "$1" in
+  start)
+	start
+	;;
+  stop)
+	stop
+	;;
+  status)
+        status -p ${pidfile} $prog
+	RETVAL=$?
+	;;
+  restart|force-reload)
+	restart
+	;;
+  condrestart|try-restart)
+	condrestart
+	;;
+  *)
+	echo $"Usage: $prog {start|stop|restart|force-reload|condrestart|try-restart|status|help}"
+	RETVAL=2
+esac
+
+exit $RETVAL
diff --git a/guest-tools-iso/projects/qemu-ga/redhat/qemu-ga.sysconfig b/guest-tools-iso/projects/qemu-ga/redhat/qemu-ga.sysconfig
new file mode 100644
index 0000000..c125a82
--- /dev/null
+++ b/guest-tools-iso/projects/qemu-ga/redhat/qemu-ga.sysconfig
@@ -0,0 +1,14 @@
+# This file configures the qemu-ga guest agent daemon
+
+# METHOD= Sets the type of channel being used.  The only type currently
+# supported is virtio-serial
+METHOD="virtio-serial"
+
+# PORT= Sets the name of the channel to use.  This should be a device file
+PORT="/dev/virtio-ports/org.qemu.guest_agent.0"
+
+# LOGFILE= Sets the path to the log file.
+LOGFILE="/var/log/qemu-ga.log"
+
+# Set options according to above
+OPTIONS="-m $METHOD -p $PORT -l $LOGFILE -d"
-- 
1.7.0.4

[Qemu-devel] [PATCH 5/5] guest tools: installer for qemu-ga

[Michael Roth]

Detects host distro/architecture, then chooses the appropriate binary
and init scripts to install. Support for more distros will be added as
they're tested. Currently RHEL6, RHEL5, Fedora 15, Ubuntu 10.04, and
Ubuntu 10.10 are known to be supported with qemu-ga binaries generated
on a RHEL5 build host. RHEL6 builds will not work on RHEL5 due to glibc
issues. These idiosyncrasies will need to be documented once everything
is fleshed out.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
 guest-tools-iso/projects/qemu-ga/install.linux |   61 ++++++++++++++++++++++++
 1 files changed, 61 insertions(+), 0 deletions(-)
 create mode 100755 guest-tools-iso/projects/qemu-ga/install.linux

diff --git a/guest-tools-iso/projects/qemu-ga/install.linux b/guest-tools-iso/projects/qemu-ga/install.linux
new file mode 100755
index 0000000..1f200b6
--- /dev/null
+++ b/guest-tools-iso/projects/qemu-ga/install.linux
@@ -0,0 +1,61 @@
+#!/bin/sh
+
+#arch=""
+#distro=""
+
+get_arch() {
+    machine=`uname --machine`
+    if [ "$machine" = "x86_64" ]; then
+        arch="x86_64"
+    elif [ "`echo $machine | sed 's/i[3456]86/x86/'`" = "x86" ]; then
+        arch="x86"
+    else
+        echo "unsupported architecture"
+        exit 1
+    fi
+}
+
+get_distro() {
+    if [ -f /etc/redhat-release ]; then
+        distro="redhat"
+    elif [ -f /etc/fedora-release ]; then
+        distro="fedora"
+    elif [ -f /etc/debian_version ]; then
+        distro_id=`lsb_release -i | awk '{print $3}'`
+        if [ "$distro_id" = "Ubuntu" ]; then
+            distro="ubuntu"
+        else
+            distro="debian"
+        fi
+    else
+        echo "unsupported distribution"
+        exit 1
+    fi
+}
+
+get_arch
+get_distro
+
+echo detected arch: $arch
+echo detected distro: $distro
+echo installing...
+
+install -D -p -m 0755 qemu-ga.linux.$arch /usr/sbin/qemu-ga
+
+if [ "$distro" = "redhat" ] || [ "$distro" = "fedora" ]; then
+    install -D -p -m 0755 redhat/qemu-ga.init /etc/init.d/qemu-ga
+    install -D -p -m 0644 redhat/qemu-ga.sysconfig /etc/sysconfig/qemu-ga
+    /sbin/chkconfig --add qemu-ga
+    /sbin/chkconfig --level 2345 qemu-ga on
+    /sbin/service qemu-ga restart
+elif [ "$distro" = "ubuntu" ] || [ "$distro" = "debian" ]; then
+    install -D -p -m 0755 debian/qemu-ga.init /etc/init.d/qemu-ga
+    install -D -p -m 0644 debian/qemu-ga.sysconfig /etc/sysconfig/qemu-ga
+    /usr/sbin/update-rc.d qemu-ga defaults
+    /etc/init.d/qemu-ga restart
+else
+    "no associated init script for distro: $distro"
+    exit 1
+fi
+
+echo installation complete.
-- 
1.7.0.4