Re: Determining iommu groups in Xen?

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 28/08/14 14:26, Peter Kay wrote:
> This should be a simple question, but I can't find the answer : how are iommu groups determined/found in Xen?
>
> I've used KVM before, and the use of the vfio framework makes it easy to find the iommu groups. Unless a 'will never be approved' patch is applied to the Linux kernel, it is impossible to pass through only one device to a KVM VM out of a group due to the lack of ACS (iommu protection between devices on the same bus segment).
>
> Xen does not seem to enforce this - why not, especially as it can cause security and stability issues?
>
> PK

>From memory, ACS is present to fix an interaction issue between PCI
Passthrough and peer-to-peer dma translations in the PCIe spec.  ACS
instructions a bridge/switch to forward the transaction to the upstream
port for translation by the IOMMU instead of resolving it privately as a
peer-to-peer transaction.

Xen uses the domain identifier to determine which iommu context to apply
to different dma requests.  Without ACS, it is not safe (or indeed
functional in general) to pass through different devices behind a PCIe
switch to different domains.  The problem is that if two different
domains program different devices with overlapping guest physical
addresses, the peer-to-peer option in PCIe causes the dma traffic to be
bounced between the two devices, rather than ending up being translated
into separate domains memory ranges.

Unfortunately, a lot of 1st era PCIe switches after ACS was specified
have errata, caused by an ambiguity in the spec, which means that
despite claiming ACS support, they don't function correctly.  In some
cases, there are workarounds, but in other cases there are not.

It is sad to say that passing through multiple devices behind a switch
to multiple domains doesn't work very well in a lot of cases.

Certainly within XenServer, we state that customers using PCIPassthrough
must trust the guest administrators, which 'fixes' the security aspect
of things from the point of view of malicious guests.

~Andrew