Re: [TESTDAY] PV / HVM pass-through works when IOMMU present; weird failures when not

[George Dunlap <george.dunlap@eu.citrix.com>]

On 28/06/13 17:00, Jan Beulich wrote:
>>>> On 28.06.13 at 17:37, George Dunlap <George.Dunlap@eu.citrix.com> wrote:
>> - For HVM guests, the only user-visible indication tha the IOMMU has
>> been disabled is the following error message on the command-line:
>>
>> # xl pci-attach h0 07:00.0
>> libxl: error: libxl_pci.c:949:do_pci_add: xc_assign_device failed
>>
>> However, the device itself ends up passed-through to the guest anyway;
>> the guest seems to be able to see it and interact with it normally.
>> This is particularly scary, as in theory this should not be possible
>> without a working IOMMU.
>>
>> I don't think this is a blocker for 4.3, but we should definitely
>> release note it, and for 4.4 add a check to see if there is a
>> functioning IOMMU and only add a device if there's an override set.
> To me this very much looks like a security problem (which I
> think we should fix asap).
>
> As I tried this the other day with a boot time assignment, and
> it prevented the guest from booting (which is how it should be)
> - are you also seeing the guest happily using such device when
> assigned via guest config file?
>
> Knowing that may hint at where to look for the actual problem.
>
> Also, I can't really see how the guest would be able to interact
> with a half way assigned device properly - I could imagine you
> being able to look at its config space, and perhaps load the
> driver, but I can't see an I/O to succeed, at least not as long
> as any bus mastering is being used the device (this ought to
> crash this guest, another guest, or the host, or deliver corrupt
> data). Purely port based I/O would likely work, but other than
> serial cards I can't think of many things that would do so.

I get basically the same results; adding "pci=['07:00.0']" to the config 
file:

# xl create h0
Parsing config from h0
xc: info: VIRTUAL MEMORY ARRANGEMENT:
   Loader:        0000000000100000->000000000019ee28
   Modules:       0000000000000000->0000000000000000
   TOTAL:         0000000000000000->00000001ff800000
   ENTRY ADDRESS: 0000000000100608
xc: info: PHYSICAL MEMORY ALLOCATION:
   4KB PAGES: 0x0000000000000200
   2MB PAGES: 0x00000000000009fb
   1GB PAGES: 0x0000000000000003
libxl: error: libxl_pci.c:949:do_pci_add: xc_assign_device failed: 
Function not implemented
Daemon running with PID 4346
kodo2:~# xl pci-assignable-list
kodo2:~# xl pci-list h0
Vdev Device
05.0 0000:07:00.0

And in the guest:

# lspci
[snip]
00:05.0 Ethernet controller: Intel Corporation 82575GB Gigabit Network 
Connection (rev 02)
# ifconfig -a
[snip]
eth2      Link encap:Ethernet  HWaddr 00:1b:21:3e:fe:90
           BROADCAST MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000
           RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
           Memory:f3200000-f3220000
[snip]
# ifup eth2
Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on LPF/eth2/00:1b:21:3e:fe:90
Sending on   LPF/eth2/00:1b:21:3e:fe:90
Sending on   Socket/fallback
DHCPDISCOVER on eth2 to 255.255.255.255 port 67 interval 8

Since the cable is not plugged in, the device never comes up.  So it's 
quite possible that since it's sending packets but not receiving 
anything, that it's either not doing any DMA, or that it's DMA'ing out 
junk but it doesn't matter.

Either way, if someone *is* passing through a device, it is probably a 
security issue.

  -George