Re: pci-stub error and MSI-X for KVM guest

Daniel P. Berrange
2010-01-08T11:05:36Z

On Thu, Jan 07, 2010 at 04:50:03PM -0800, Chris Wright wrote:
> * Fischer, Anna (anna.fischer@hp.com) wrote:
> > So, when setting a breakpoint for the exit() call I'm getting a bit closer to figuring where it kills my guest.
>
> Thanks, this helps clarify what is happening.
>
> > Breakpoint 1, exit (status=1) at exit.c:99
> > 99 {
> > Current language: auto
> > The current source language is "auto; currently c".
> > (gdb) bt
> > #0 exit (status=1) at exit.c:99
> > #1 0x0000000000470c6e in assigned_dev_pci_read_config (d=0x259c6f0, address=64, len=4)
>
> assigned_dev_pci_read_config(..., 64, 4)
> ^^
> This is a libvirt issue. When you use virt-manager it has libvirtd
> fork/exec qemu-kvm. libvirtd will drop privileges and run qemu-kvm as
> user qemu (or perhaps root if you've edited qemu.conf). Regardless of
> the user, it clears capabilities. Reading PCI config space beyond just
> the header requires CAP_SYS_ADMIN. The above is reading the first 4
> bytes of device dependent config space, and the kernel is returning 0
> because qemu doesn't have CAP_SYS_ADMIN.

Hmm, libvirt also chown()'s the files in /sys/bus/pci/devices/<DEVICE>/*
to 'qemu' (and sets SELinux context) so that the unprivileged QEMU process
can have full read/write access to them. I would have hoped that would
avoid the need to have any capabilities like CAP_SYS_ADMIN :-(

> Basically, this means that device assignment w/ libvirt will break
> MSI/MSI-X because qemu will never be able to see that the host device
> has those PCI capabilities. This, in turn, renders VF device assignment
> useless (since a VF is required to support MSI and/or MSI-X).
>
> Granting CAP_SYS_ADMIN for each qemu instance that does device assignment
> would render the privilege reduction useless (CAP_SYS_ADMIN is the
> kitchen sink catchall of the Linux capability system).

Yeah that's pretty troublesome, even when libvirt runs QEMU as 'root', it will
remove all capabilities. Why is the 'CAP_SYS_ADMIN' check there - is it a
mistakenly over-zealous permission check that could be removed, just relying
on access controls on the sysfs /sys/bus/pci/devices/<DEVICE>/config
file ?

Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|