[Qemu-devel] RFC: Qemu Guest Tools ISO

[Qemu-devel] RFC: Qemu Guest Tools ISO

[Michael Roth]

Goal:

Provide a mechanism, similar to vmware and virtualbox guest tools ISOs, 
that allows us to easily distribute guest tools (and potentially 
drivers) for linux and windows guests.

Advantages (rough list to start the discussion, feel free to add/comment):

- Simplify deployment of guest additions. ISO-installable additions can 
be pulled from QEMU/KVM/virtio/etc upstream or external projects as 
needed rather than worked into distros as independent packages. Users do 
not need to worry about installing lists of packages for full support. 
Pre-made ISOs can be pulled into QEMU/KVM in a manner similar to 
BIOSs/option roms.

- Reduce complexity involved with needing to manage guests with 
outdated/missing tools or drivers. No need to rely on distros to pull 
drivers/features/bug fixes from upstream before relying on them; we can 
assume these fixes/features are immediately available from an upstream 
perspective, and distros can still maintain compatibility within a 
distro-centric environment by shipping specific versions of the guest 
tools ISO (hopefully the version committed to qemu.git at time of rebase 
or newer)

- Simplify updates: Hypervisor can push guest tools updates by building 
QMP/guest agent interfaces around an ISO.

- Extend support to older guests (and windows) where new repo packages 
are not a realistic option.

- ?

Disadvantages:

- Need to test changes to tools against supported distros/platforms 
rather than punting to / or leveraging distro maintainers. KVM Autotest 
would likely be a big part of this.

- Potentially less integration from a distro-centric perspective. 
Upstream mandates guest tools, distros need to keep up or rebase to 
remain in sync. Can still elect to support specific versions of a guest 
tools ISO, however.

- ?

Implementation:

I hope to follow-up in fairly short order with a basic prototype of the 
tools/workflow to create/install a guest additions ISO. A rough overview 
of the approach I'm currently pursuing:

- Use PyInstaller (built around pye2exe, linux/windows compatible, with 
logic to pull in required shared libs and windows/tcl/cmd.exe support as 
needed) to generate executables from python scripts.

- Each project exists as a free-form directory with source code, or 
32/64 bit pre-compiled binaries, windows-based installers, etc. To add 
to an ISO a symlink to this directory would be added along with a python 
installer script which accepts arch/distro as arguments. 
install/update/uninstall logic handled completely by this install script.

- Top-level installer will iterate through guest additions projects and 
execute installers in turn. (some basic dependency support or explicit 
ordered may be needed).

- Install scripts (top-level and per-project) will be run through a set 
of scripts built around PyInstaller to generate a group of executable 
installers for linux as well as for windows (installers can be 
do-nothings for unsupported platforms, or simply call out to other 
binaries if using, say, an MSI windows installer). Both will co-exist on 
the same ISO, and share the top-level projects directory containing the 
individual code/binaries for individual projects.

Thoughts?

Re: [Qemu-devel] RFC: Qemu Guest Tools ISO

[Alon Levy]

On Wed, Jun 22, 2011 at 01:55:25PM -0500, Michael Roth wrote:
> Goal:
> 
> Provide a mechanism, similar to vmware and virtualbox guest tools
> ISOs, that allows us to easily distribute guest tools (and
> potentially drivers) for linux and windows guests.

What would be the advantage for linux guests, with their package managers already
handling this task? I see how it would make testing easier with various linux
distributions, but for management I wonder if it won't be easier to use the
package management system to update the guests same as the hosts.

> 
> Advantages (rough list to start the discussion, feel free to add/comment):
> 
> - Simplify deployment of guest additions. ISO-installable additions
> can be pulled from QEMU/KVM/virtio/etc upstream or external projects
> as needed rather than worked into distros as independent packages.
> Users do not need to worry about installing lists of packages for
> full support. Pre-made ISOs can be pulled into QEMU/KVM in a manner
> similar to BIOSs/option roms.
> 
> - Reduce complexity involved with needing to manage guests with
> outdated/missing tools or drivers. No need to rely on distros to
> pull drivers/features/bug fixes from upstream before relying on
> them; we can assume these fixes/features are immediately available
> from an upstream perspective, and distros can still maintain
> compatibility within a distro-centric environment by shipping
> specific versions of the guest tools ISO (hopefully the version
> committed to qemu.git at time of rebase or newer)
> 
> - Simplify updates: Hypervisor can push guest tools updates by
> building QMP/guest agent interfaces around an ISO.
> 
> - Extend support to older guests (and windows) where new repo
> packages are not a realistic option.
> 

ok, this is relevant to linux guests too.

> - ?
> 
> Disadvantages:
> 
> - Need to test changes to tools against supported distros/platforms
> rather than punting to / or leveraging distro maintainers. KVM
> Autotest would likely be a big part of this.
> 
> - Potentially less integration from a distro-centric perspective.
> Upstream mandates guest tools, distros need to keep up or rebase to
> remain in sync. Can still elect to support specific versions of a
> guest tools ISO, however.
> 
> - ?
> 
> Implementation:
> 
> I hope to follow-up in fairly short order with a basic prototype of
> the tools/workflow to create/install a guest additions ISO. A rough
> overview of the approach I'm currently pursuing:
> 
> - Use PyInstaller (built around pye2exe, linux/windows compatible,
> with logic to pull in required shared libs and windows/tcl/cmd.exe
> support as needed) to generate executables from python scripts.
> 
> - Each project exists as a free-form directory with source code, or
> 32/64 bit pre-compiled binaries, windows-based installers, etc. To
> add to an ISO a symlink to this directory would be added along with
> a python installer script which accepts arch/distro as arguments.
> install/update/uninstall logic handled completely by this install
> script.
> 
> - Top-level installer will iterate through guest additions projects
> and execute installers in turn. (some basic dependency support or
> explicit ordered may be needed).

I'm not sure all drivers have installers. sometimes it will need to install
from inf I think. Should look at how the REHV-M iso does this.

> 
> - Install scripts (top-level and per-project) will be run through a
> set of scripts built around PyInstaller to generate a group of
> executable installers for linux as well as for windows (installers
> can be do-nothings for unsupported platforms, or simply call out to
> other binaries if using, say, an MSI windows installer). Both will
> co-exist on the same ISO, and share the top-level projects directory
> containing the individual code/binaries for individual projects.
> 
> Thoughts?
> 

Re: [Qemu-devel] RFC: Qemu Guest Tools ISO

[Stefan Hajnoczi]

On Thu, Jun 23, 2011 at 10:29 AM, Alon Levy <alevy@redhat.com> wrote:
> On Wed, Jun 22, 2011 at 01:55:25PM -0500, Michael Roth wrote:
>> Goal:
>>
>> Provide a mechanism, similar to vmware and virtualbox guest tools
>> ISOs, that allows us to easily distribute guest tools (and
>> potentially drivers) for linux and windows guests.
>
> What would be the advantage for linux guests, with their package managers already
> handling this task? I see how it would make testing easier with various linux
> distributions, but for management I wonder if it won't be easier to use the
> package management system to update the guests same as the hosts.

If the guest tools come from the host QEMU we don't need complicated
compatibility testing and fallbacks.  Guest and host will be in sync
and support the same features.

I'm not sure how we can update guest tools when the QEMU on the host
is updated though.

Stefan

Re: [Qemu-devel] RFC: Qemu Guest Tools ISO

[Daniel P. Berrange]

On Thu, Jun 23, 2011 at 12:08:26PM +0100, Stefan Hajnoczi wrote:
> On Thu, Jun 23, 2011 at 10:29 AM, Alon Levy <alevy@redhat.com> wrote:
> > On Wed, Jun 22, 2011 at 01:55:25PM -0500, Michael Roth wrote:
> >> Goal:
> >>
> >> Provide a mechanism, similar to vmware and virtualbox guest tools
> >> ISOs, that allows us to easily distribute guest tools (and
> >> potentially drivers) for linux and windows guests.
> >
> > What would be the advantage for linux guests, with their package managers already
> > handling this task? I see how it would make testing easier with various linux
> > distributions, but for management I wonder if it won't be easier to use the
> > package management system to update the guests same as the hosts.
> 
> If the guest tools come from the host QEMU we don't need complicated
> compatibility testing and fallbacks.  Guest and host will be in sync
> and support the same features.
> 
> I'm not sure how we can update guest tools when the QEMU on the host
> is updated though.

While an ISO is a good option for the initial deployment, since it can
be used to provide both the virtio drivers + agents together, once you
have the virtio stuff installed in the guest, it might be worth exposing
updated tools via a 9p filesystem mount. For Linux guest the 9p fs
exposed could include a YUM repo (or equiv) which the guest update tool
would automatically see and pull updates from, since 9pfs is "live"
unlike an ISO.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] RFC: Qemu Guest Tools ISO

[Michael Roth]

On 06/23/2011 06:38 AM, Daniel P. Berrange wrote:
> On Thu, Jun 23, 2011 at 12:08:26PM +0100, Stefan Hajnoczi wrote:
>> On Thu, Jun 23, 2011 at 10:29 AM, Alon Levy<alevy@redhat.com>  wrote:
>>> On Wed, Jun 22, 2011 at 01:55:25PM -0500, Michael Roth wrote:
>>>> Goal:
>>>>
>>>> Provide a mechanism, similar to vmware and virtualbox guest tools
>>>> ISOs, that allows us to easily distribute guest tools (and
>>>> potentially drivers) for linux and windows guests.
>>>
>>> What would be the advantage for linux guests, with their package managers already
>>> handling this task? I see how it would make testing easier with various linux
>>> distributions, but for management I wonder if it won't be easier to use the
>>> package management system to update the guests same as the hosts.
>>
>> If the guest tools come from the host QEMU we don't need complicated
>> compatibility testing and fallbacks.  Guest and host will be in sync
>> and support the same features.
>>
>> I'm not sure how we can update guest tools when the QEMU on the host
>> is updated though.
>
> While an ISO is a good option for the initial deployment, since it can
> be used to provide both the virtio drivers + agents together, once you
> have the virtio stuff installed in the guest, it might be worth exposing
> updated tools via a 9p filesystem mount. For Linux guest the 9p fs
> exposed could include a YUM repo (or equiv) which the guest update tool
> would automatically see and pull updates from, since 9pfs is "live"
> unlike an ISO.
>
> Regards,
> Daniel

Definitely, or at least some form of automating the install beyond the 
initial one. One approach would be extending the guest RPC agent that 
would ship with the ISO to be able to kick off a future update. 
Implementing in would be pretty simple too:

Guest agent: add RPC will execute the ISO installer
QMP: add command that will mount latest ISO as a cdrom then initiate the 
update via the guest agent RPC.

We could even take the ISO out of the equation after the initial 
install, since files can be pushed to the guest via guest 
agent/virtio-serial

Re: [Qemu-devel] RFC: Qemu Guest Tools ISO

[Avi Kivity]

On 06/23/2011 02:08 PM, Stefan Hajnoczi wrote:
> On Thu, Jun 23, 2011 at 10:29 AM, Alon Levy<alevy@redhat.com>  wrote:
> >  On Wed, Jun 22, 2011 at 01:55:25PM -0500, Michael Roth wrote:
> >>  Goal:
> >>
> >>  Provide a mechanism, similar to vmware and virtualbox guest tools
> >>  ISOs, that allows us to easily distribute guest tools (and
> >>  potentially drivers) for linux and windows guests.
> >
> >  What would be the advantage for linux guests, with their package managers already
> >  handling this task? I see how it would make testing easier with various linux
> >  distributions, but for management I wonder if it won't be easier to use the
> >  package management system to update the guests same as the hosts.
>
> If the guest tools come from the host QEMU we don't need complicated
> compatibility testing and fallbacks.  Guest and host will be in sync
> and support the same features.
>

Even building the tools would be very hard.  In general if you build 
against libc version y, you cannot expect your code to work against libc 
version y-1, unless you take special measures.  With other libraries the 
"special measures" may not even be possible.

(I'm assuming statically linking the binaries is out of the question)

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] RFC: Qemu Guest Tools ISO

[Michael Roth]

On 06/23/2011 07:00 AM, Avi Kivity wrote:
> On 06/23/2011 02:08 PM, Stefan Hajnoczi wrote:
>> On Thu, Jun 23, 2011 at 10:29 AM, Alon Levy<alevy@redhat.com> wrote:
>> > On Wed, Jun 22, 2011 at 01:55:25PM -0500, Michael Roth wrote:
>> >> Goal:
>> >>
>> >> Provide a mechanism, similar to vmware and virtualbox guest tools
>> >> ISOs, that allows us to easily distribute guest tools (and
>> >> potentially drivers) for linux and windows guests.
>> >
>> > What would be the advantage for linux guests, with their package
>> managers already
>> > handling this task? I see how it would make testing easier with
>> various linux
>> > distributions, but for management I wonder if it won't be easier to
>> use the
>> > package management system to update the guests same as the hosts.
>>
>> If the guest tools come from the host QEMU we don't need complicated
>> compatibility testing and fallbacks. Guest and host will be in sync
>> and support the same features.
>>
>
> Even building the tools would be very hard. In general if you build
> against libc version y, you cannot expect your code to work against libc
> version y-1, unless you take special measures. With other libraries the
> "special measures" may not even be possible.

Nvidia/ATI driver installers might be a good (or bad) precedent, I 
believe they ship a generic blob....need to confirm. We'd want to have a 
controlled environment that's used for building the tools we add to the 
ISO though.

>
> (I'm assuming statically linking the binaries is out of the question)
>

Not totally, but the build environment would need to be carefully 
crafted, and for something like the guest agent which pulls in glib/curl 
I'm not sure how viable that is.

Re: [Qemu-devel] RFC: Qemu Guest Tools ISO

[Avi Kivity]

On 06/23/2011 05:54 PM, Michael Roth wrote:
> On 06/23/2011 07:00 AM, Avi Kivity wrote:
>> On 06/23/2011 02:08 PM, Stefan Hajnoczi wrote:
>>> On Thu, Jun 23, 2011 at 10:29 AM, Alon Levy<alevy@redhat.com> wrote:
>>> > On Wed, Jun 22, 2011 at 01:55:25PM -0500, Michael Roth wrote:
>>> >> Goal:
>>> >>
>>> >> Provide a mechanism, similar to vmware and virtualbox guest tools
>>> >> ISOs, that allows us to easily distribute guest tools (and
>>> >> potentially drivers) for linux and windows guests.
>>> >
>>> > What would be the advantage for linux guests, with their package
>>> managers already
>>> > handling this task? I see how it would make testing easier with
>>> various linux
>>> > distributions, but for management I wonder if it won't be easier to
>>> use the
>>> > package management system to update the guests same as the hosts.
>>>
>>> If the guest tools come from the host QEMU we don't need complicated
>>> compatibility testing and fallbacks. Guest and host will be in sync
>>> and support the same features.
>>>
>>
>> Even building the tools would be very hard. In general if you build
>> against libc version y, you cannot expect your code to work against libc
>> version y-1, unless you take special measures. With other libraries the
>> "special measures" may not even be possible.
>
> Nvidia/ATI driver installers might be a good (or bad) precedent, I 
> believe they ship a generic blob....need to confirm.

The kernel keeps breaking it and they keep fixing it.  They're solving a 
much more difficult problem though (an explicitly unstable API).

> We'd want to have a controlled environment that's used for building 
> the tools we add to the ISO though.

I'm not even sure that such a least-common-denominator exists.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] RFC: Qemu Guest Tools ISO

[Michael Roth]

On 06/23/2011 10:09 AM, Avi Kivity wrote:
> On 06/23/2011 05:54 PM, Michael Roth wrote:
>> On 06/23/2011 07:00 AM, Avi Kivity wrote:
>>> On 06/23/2011 02:08 PM, Stefan Hajnoczi wrote:
>>>> On Thu, Jun 23, 2011 at 10:29 AM, Alon Levy<alevy@redhat.com> wrote:
>>>> > On Wed, Jun 22, 2011 at 01:55:25PM -0500, Michael Roth wrote:
>>>> >> Goal:
>>>> >>
>>>> >> Provide a mechanism, similar to vmware and virtualbox guest tools
>>>> >> ISOs, that allows us to easily distribute guest tools (and
>>>> >> potentially drivers) for linux and windows guests.
>>>> >
>>>> > What would be the advantage for linux guests, with their package
>>>> managers already
>>>> > handling this task? I see how it would make testing easier with
>>>> various linux
>>>> > distributions, but for management I wonder if it won't be easier to
>>>> use the
>>>> > package management system to update the guests same as the hosts.
>>>>
>>>> If the guest tools come from the host QEMU we don't need complicated
>>>> compatibility testing and fallbacks. Guest and host will be in sync
>>>> and support the same features.
>>>>
>>>
>>> Even building the tools would be very hard. In general if you build
>>> against libc version y, you cannot expect your code to work against libc
>>> version y-1, unless you take special measures. With other libraries the
>>> "special measures" may not even be possible.
>>
>> Nvidia/ATI driver installers might be a good (or bad) precedent, I
>> believe they ship a generic blob....need to confirm.
>
> The kernel keeps breaking it and they keep fixing it. They're solving a
> much more difficult problem though (an explicitly unstable API).
>

Yup, and their userspace stuff has some pretty complex library 
dependencies (pthreads, X11) that work well-enough for most. I'm sure 
we'll run into some issues, but if we limit ourselves to stable 
interfaces where possible they should be minimal, and if need be we can 
ship different binaries for certain arch/distro combos (though that can 
get out of hand fairly quickly and should be avoided)

>> We'd want to have a controlled environment that's used for building
>> the tools we add to the ISO though.
>
> I'm not even sure that such a least-common-denominator exists.
>

I mean mostly just to avoid having different guest tools in the ISO that 
were all built in different environments, since that might exacerbate 
library mismatches.

Re: [Qemu-devel] RFC: Qemu Guest Tools ISO

[Anthony Liguori]

On 06/23/2011 07:00 AM, Avi Kivity wrote:
> On 06/23/2011 02:08 PM, Stefan Hajnoczi wrote:
>> On Thu, Jun 23, 2011 at 10:29 AM, Alon Levy<alevy@redhat.com> wrote:
>> > On Wed, Jun 22, 2011 at 01:55:25PM -0500, Michael Roth wrote:
>> >> Goal:
>> >>
>> >> Provide a mechanism, similar to vmware and virtualbox guest tools
>> >> ISOs, that allows us to easily distribute guest tools (and
>> >> potentially drivers) for linux and windows guests.
>> >
>> > What would be the advantage for linux guests, with their package
>> managers already
>> > handling this task? I see how it would make testing easier with
>> various linux
>> > distributions, but for management I wonder if it won't be easier to
>> use the
>> > package management system to update the guests same as the hosts.
>>
>> If the guest tools come from the host QEMU we don't need complicated
>> compatibility testing and fallbacks. Guest and host will be in sync
>> and support the same features.
>>
>
> Even building the tools would be very hard. In general if you build
> against libc version y, you cannot expect your code to work against libc
> version y-1, unless you take special measures. With other libraries the
> "special measures" may not even be possible.

Good libraries provide strong ABI compatibility.

Something like glib clearly documents what version of the library 
functions are available in, if you still to responsibly common 
functions, ABI compatibility should be much of an issue.

Regards,

Anthony Liguori

>
> (I'm assuming statically linking the binaries is out of the question)
>

Re: [Qemu-devel] RFC: Qemu Guest Tools ISO

[Avi Kivity]

On 06/23/2011 06:25 PM, Anthony Liguori wrote:
>> Even building the tools would be very hard. In general if you build
>> against libc version y, you cannot expect your code to work against libc
>> version y-1, unless you take special measures. With other libraries the
>> "special measures" may not even be possible.
>
>
> Good libraries provide strong ABI compatibility.
>
> Something like glib clearly documents what version of the library 
> functions are available in, if you still to responsibly common 
> functions, ABI compatibility should be much of an issue.

I don't know about glib, but glibc only guarantees backward 
compatibility, not forward compatibility.  If you build against a newer 
glibc, your executable may not run with an older glibc, even if the 
symbols exist in both.

See [1] which touches on the issue; I don't have a better reference.

[1] ftp://ftp.kernel.org/pub/software/libs/glibc/hjl/compat/index.html

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] RFC: Qemu Guest Tools ISO

[Natalia Portillo]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Ok you're forgetting one thing:

90% of the devices we emulate are real physical ones.
The drivers for those devices in non-opensource guests already exist, and most of the times prevent we distributing them (read the EULA).

I think a "guest tools" with binary drivers for binary platforms (Windows, OS/2, NeXT, DOS) is a good idea.
One with binary drivers for open-source platforms (Linux, *BSD) is really stupid, will create hundreds of conflicts.
One with source drivers for open-source platforms is reinventing the wheel. The distros and systems already take care about drivers, including even drivers for other VMs.

So for any kind of usefulness we need to ask Creative Labs, Intel, AMD, VMWare, Realtek, so on, permissions to distribute their drivers.
I'm not sure they will allow us (pretty sure the VMWare video one will not be allowed ever).

In the case they allow us and create the guest tools disc, we can create a false device that a dormant process (TSR in DOS world) hears off.
That's what VMWare does, writes in that device requesting guest tools version (if none installed, noone answers), and if the provided one is better informs the user and offers to automount it.

Regards

El 23/06/2011, a las 16:52, Avi Kivity escribió:

> On 06/23/2011 06:25 PM, Anthony Liguori wrote:
>>> Even building the tools would be very hard. In general if you build
>>> against libc version y, you cannot expect your code to work against libc
>>> version y-1, unless you take special measures. With other libraries the
>>> "special measures" may not even be possible.
>> 
>> 
>> Good libraries provide strong ABI compatibility.
>> 
>> Something like glib clearly documents what version of the library functions are available in, if you still to responsibly common functions, ABI compatibility should be much of an issue.
> 
> I don't know about glib, but glibc only guarantees backward compatibility, not forward compatibility.  If you build against a newer glibc, your executable may not run with an older glibc, even if the symbols exist in both.
> 
> See [1] which touches on the issue; I don't have a better reference.
> 
> [1] ftp://ftp.kernel.org/pub/software/libs/glibc/hjl/compat/index.html
> 
> -- 
> error compiling committee.c: too many arguments to function
> 
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org

iF4EAREIAAYFAk4HzaQACgkQv/wfOsykIRTVoQD/SgniEhqOCIf5FIiBz7RT0GAc
g8aA4FGWdKQB6jrfbwwA/igtLCAjhCpF/lFJ6o+0dUU2r3tT+gvDdZeURG82Xr5Z
=2Z/r
-----END PGP SIGNATURE-----

Re: [Qemu-devel] RFC: Qemu Guest Tools ISO

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1460 bytes --]

On 2011-06-23 11:29, Alon Levy wrote:
>> Implementation:
>>
>> I hope to follow-up in fairly short order with a basic prototype of
>> the tools/workflow to create/install a guest additions ISO. A rough
>> overview of the approach I'm currently pursuing:
>>
>> - Use PyInstaller (built around pye2exe, linux/windows compatible,
>> with logic to pull in required shared libs and windows/tcl/cmd.exe
>> support as needed) to generate executables from python scripts.
>>
>> - Each project exists as a free-form directory with source code, or
>> 32/64 bit pre-compiled binaries, windows-based installers, etc. To
>> add to an ISO a symlink to this directory would be added along with
>> a python installer script which accepts arch/distro as arguments.
>> install/update/uninstall logic handled completely by this install
>> script.
>>
>> - Top-level installer will iterate through guest additions projects
>> and execute installers in turn. (some basic dependency support or
>> explicit ordered may be needed).
> 
> I'm not sure all drivers have installers. sometimes it will need to install
> from inf I think. Should look at how the REHV-M iso does this.

Back in my old days, I wrote a tiny add-on for NSIS to install also
device drivers. Not sure if it still works, but it used to up to XP.
Someone create a wiki page with more information:

http://nsis.sourceforge.net/InstDrv_plug-in

In general, I liked NSIS a lot.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: [Qemu-devel] RFC: Qemu Guest Tools ISO

[Michael Roth]

On 06/23/2011 04:29 AM, Alon Levy wrote:
> On Wed, Jun 22, 2011 at 01:55:25PM -0500, Michael Roth wrote:
>> Goal:
>>
>> Provide a mechanism, similar to vmware and virtualbox guest tools
>> ISOs, that allows us to easily distribute guest tools (and
>> potentially drivers) for linux and windows guests.
>
> What would be the advantage for linux guests, with their package managers already
> handling this task? I see how it would make testing easier with various linux
> distributions, but for management I wonder if it won't be easier to use the
> package management system to update the guests same as the hosts.
>

The advantages listed below are in comparison to distro-managed 
packaging. The main one I think would be the support matrix for upstream 
code:

Certain versions of QEMU ship with certain versions of the ISO, and 
support guests with that particular version of the ISO installed.

This is a very enabling assumption to be able to make when working on 
new features (or deprecating old ones), and it's simply not one that can 
be made for distro-managed tools since there are a number of reasons why 
a distro won't update tools in a reasonable timeframe: maintaining 
compatability with a down-level host stack they support, distro-specific 
additions to various tools, support/packaging policies, etc.

Ideally we maintain backward compatibility with older versions the 
ISO/guest tools, but we can elect to break them if need be since we 
don't need to worry about supporting guests with older tools that 
haven't been updated by the distro; the hypervisor can push required 
updates/tools with minimal guest participation beyond the initial guest 
tools installation (updates can get pushed automatically from that point 
forward).

>>
>> Advantages (rough list to start the discussion, feel free to add/comment):
>>
>> - Simplify deployment of guest additions. ISO-installable additions
>> can be pulled from QEMU/KVM/virtio/etc upstream or external projects
>> as needed rather than worked into distros as independent packages.
>> Users do not need to worry about installing lists of packages for
>> full support. Pre-made ISOs can be pulled into QEMU/KVM in a manner
>> similar to BIOSs/option roms.
>>
>> - Reduce complexity involved with needing to manage guests with
>> outdated/missing tools or drivers. No need to rely on distros to
>> pull drivers/features/bug fixes from upstream before relying on
>> them; we can assume these fixes/features are immediately available
>> from an upstream perspective, and distros can still maintain
>> compatibility within a distro-centric environment by shipping
>> specific versions of the guest tools ISO (hopefully the version
>> committed to qemu.git at time of rebase or newer)
>>
>> - Simplify updates: Hypervisor can push guest tools updates by
>> building QMP/guest agent interfaces around an ISO.
>>
>> - Extend support to older guests (and windows) where new repo
>> packages are not a realistic option.
>>
>
> ok, this is relevant to linux guests too.
>
>> - ?
>>
>> Disadvantages:
>>
>> - Need to test changes to tools against supported distros/platforms
>> rather than punting to / or leveraging distro maintainers. KVM
>> Autotest would likely be a big part of this.
>>
>> - Potentially less integration from a distro-centric perspective.
>> Upstream mandates guest tools, distros need to keep up or rebase to
>> remain in sync. Can still elect to support specific versions of a
>> guest tools ISO, however.
>>
>> - ?
>>
>> Implementation:
>>
>> I hope to follow-up in fairly short order with a basic prototype of
>> the tools/workflow to create/install a guest additions ISO. A rough
>> overview of the approach I'm currently pursuing:
>>
>> - Use PyInstaller (built around pye2exe, linux/windows compatible,
>> with logic to pull in required shared libs and windows/tcl/cmd.exe
>> support as needed) to generate executables from python scripts.
>>
>> - Each project exists as a free-form directory with source code, or
>> 32/64 bit pre-compiled binaries, windows-based installers, etc. To
>> add to an ISO a symlink to this directory would be added along with
>> a python installer script which accepts arch/distro as arguments.
>> install/update/uninstall logic handled completely by this install
>> script.
>>
>> - Top-level installer will iterate through guest additions projects
>> and execute installers in turn. (some basic dependency support or
>> explicit ordered may be needed).
>
> I'm not sure all drivers have installers. sometimes it will need to install
> from inf I think. Should look at how the REHV-M iso does this.
>

By installers I mean the python install script you include with the 
project, so it's pretty free-form: you can simply copy files over, 
install an rpm/deb, execute a binary, make/make install, etc.

>>
>> - Install scripts (top-level and per-project) will be run through a
>> set of scripts built around PyInstaller to generate a group of
>> executable installers for linux as well as for windows (installers
>> can be do-nothings for unsupported platforms, or simply call out to
>> other binaries if using, say, an MSI windows installer). Both will
>> co-exist on the same ISO, and share the top-level projects directory
>> containing the individual code/binaries for individual projects.
>>
>> Thoughts?
>>

Re: [Qemu-devel] RFC: Qemu Guest Tools ISO

[Ronen Hod]

On 06/22/2011 09:55 PM, Michael Roth wrote:
> Goal:
>
> Provide a mechanism, similar to vmware and virtualbox guest tools 
> ISOs, that allows us to easily distribute guest tools (and potentially 
> drivers) for linux and windows guests.
>
> Advantages (rough list to start the discussion, feel free to 
> add/comment):
>
> - Simplify deployment of guest additions. ISO-installable additions 
> can be pulled from QEMU/KVM/virtio/etc upstream or external projects 
> as needed rather than worked into distros as independent packages. 
> Users do not need to worry about installing lists of packages for full 
> support. Pre-made ISOs can be pulled into QEMU/KVM in a manner similar 
> to BIOSs/option roms.
>
> - Reduce complexity involved with needing to manage guests with 
> outdated/missing tools or drivers. No need to rely on distros to pull 
> drivers/features/bug fixes from upstream before relying on them; we 
> can assume these fixes/features are immediately available from an 
> upstream perspective, and distros can still maintain compatibility 
> within a distro-centric environment by shipping specific versions of 
> the guest tools ISO (hopefully the version committed to qemu.git at 
> time of rebase or newer)
>
> - Simplify updates: Hypervisor can push guest tools updates by 
> building QMP/guest agent interfaces around an ISO.
>
> - Extend support to older guests (and windows) where new repo packages 
> are not a realistic option.
>
> - ?
>
> Disadvantages:
>
> - Need to test changes to tools against supported distros/platforms 
> rather than punting to / or leveraging distro maintainers. KVM 
> Autotest would likely be a big part of this.
>
> - Potentially less integration from a distro-centric perspective. 
> Upstream mandates guest tools, distros need to keep up or rebase to 
> remain in sync. Can still elect to support specific versions of a 
> guest tools ISO, however.
>
> - ?
>
> Implementation:
>
> I hope to follow-up in fairly short order with a basic prototype of 
> the tools/workflow to create/install a guest additions ISO. A rough 
> overview of the approach I'm currently pursuing:
>
> - Use PyInstaller (built around pye2exe, linux/windows compatible, 
> with logic to pull in required shared libs and windows/tcl/cmd.exe 
> support as needed) to generate executables from python scripts.
>
> - Each project exists as a free-form directory with source code, or 
> 32/64 bit pre-compiled binaries, windows-based installers, etc. To add 
> to an ISO a symlink to this directory would be added along with a 
> python installer script which accepts arch/distro as arguments. 
> install/update/uninstall logic handled completely by this install script.
>
> - Top-level installer will iterate through guest additions projects 
> and execute installers in turn. (some basic dependency support or 
> explicit ordered may be needed).
>
> - Install scripts (top-level and per-project) will be run through a 
> set of scripts built around PyInstaller to generate a group of 
> executable installers for linux as well as for windows (installers can 
> be do-nothings for unsupported platforms, or simply call out to other 
> binaries if using, say, an MSI windows installer). Both will co-exist 
> on the same ISO, and share the top-level projects directory containing 
> the individual code/binaries for individual projects.
>
> Thoughts?
>
The windows drivers are an issue. You do not want to compile them since 
you need the hard-to-get Microsoft certification. Now that you have to 
provide them in binary mode, the question is whether it makes sense to 
treat the Windows agent differently.
Other than building the windows drivers, I don't see an issue.
Ronen.