Re: [RFC/PATCH 0/16] Ops based MSI Implementation

[Benjamin Herrenschmidt <benh@kernel.crashing.org>]

> You can't implement isolation unless you 1) strictly control what
> devices can do to other devices on the PCI domain or 2) filter
> transactions in the PCI bridges so that PCI devices cannot send
> arbitrary junk to each other.
> 
> #2 is prohibitively expensive and complicated because it requires
> specialized hardware.  #1 is low cost in that all you need to do is
> make PCI config space accesses and MSI setup go through the
> hypervisor.  That's why systems implement #1 to give full isolation.

Well, on IBM machines, they do #2 :-) Or rather, they have a P2P bridge
for every slot pretty much ... but then, those are expensive
machinces :-)

Now, #1 is an acceptable solution up to a certain point... MSIs being
just normal upstream stores, while you can try to prevent the OS from
programming the config space for a device to use somebody else MSI
target address or values, you can't prevent the OS to program some
random device DMA engine to issue those same bogus cycles, unless you
have an iommu per slot, which is what IBM has :-)

So yes, #1 is probably the "easy" solution but is definitely not 100%
robust.

> That's why I think the whole MSI hypervisor thing done by RTAS is
> absolutely reasonable and something we should support.  It's NOT like
> TCP Offload Engines and the like, not at all, and it's quite upsetting
> to see Eric characterize it in that way.  It's a protection and
> isolation facility, not a way to hide hardware behind binary blobs.

I tend myself to go for a simpler reason which is that unlike TOE which
is a supposed performance improvement on already working hardware with a
huge impact on the linux TCP stack, MSIs "a-la-rtas" are a fairly low
impact on an overall small piece of code to support, and not doing so
means basically totally unuseable machines as soon as MSI-only PCIe
stuff shows up (and it does exist already).

Ben.