Re: IOMMU groups ... PEX8606 switch?

[Dana Goyette <DanaGoyette@gmail.com>]

On 12/29/2013 08:16 PM, Alex Williamson wrote:
> On Sat, 2013-12-28 at 23:32 -0800, Dana Goyette wrote:
>> On 12/28/2013 7:23 PM, Alex Williamson wrote:
>>> On Sat, 2013-12-28 at 18:31 -0800, Dana Goyette wrote:
>>>> I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to
>>>> soon decide which one to keep.
>>>>
>>>> The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX
>>>> PEX8066 switch, which claims to support ACS.  I'd expect the devices
>>>> downstream of the PLX switch to be in separate groups.
>>>>
>>>> With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities"
>>>> applied and set for the Intel root ports, the devices behind the switch
>>>> remain stuck in the same group.
>>>>
>>>> In terms of passing devices to different VMs, which is better: all
>>>> devices on different root ports, or all devices behind the one
>>>> ACS-supporting switch?
>>>
>>> Can you provide lspci -vvv info?  If you're getting that for groups
>>> either the switch has ACS capabilities, but doesn't support the features
>>> we need or we're doing something wrong.  Thanks,
>>>
>> I initially tried attaching the output as a .txt file, but it's too
>> large.  Anyway, here's the output of lspci -nnvvv (you may notice that I
>> moved the Radeon to a different slot).
>
> Well, something seems amiss since the downstream switch ports all seem
> to support and enable the correct set of ACS capabilities.  I'm tending
> to suspect something wrong with the ACS override patch or how it's being
> used since your IOMMU group is still based at the root port.  Each root
> port is isolated from the other root ports though, so something is
> happening with the override patch.  Can you provide the kernel command
> line you use to enable ACS overrides and the override patch you're
> using, as it applies to 3.13-rc5?  Thanks,
>
> Alex
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
I'm using the original acs-override patch from this post:
https://lkml.org/lkml/2013/5/30/513

Kernel parameter is:
pcie_acs_override=id:8086:8c10,id:8086:8c12,id:8086:8c16,id:8086:8c18

When booting a kernel without the override patch, the following devices 
are all in the same group: Intel Root Ports 1, 2, 4, 5; ASMedia SATA 
controller; PLX PEX8606 switch; Renesas USB controller; TI Firewire 
controller; Intel I210 Ethernet controller.