From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dana Goyette Subject: Re: IOMMU groups ... PEX8606 switch? Date: Mon, 30 Dec 2013 16:13:08 -0800 Message-ID: References: <1388287383.4981.16.camel@ul30vt.home> <1388377005.4981.69.camel@ul30vt.home> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit To: kvm@vger.kernel.org Return-path: Received: from plane.gmane.org ([80.91.229.3]:38851 "EHLO plane.gmane.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932332Ab3LaAN1 (ORCPT ); Mon, 30 Dec 2013 19:13:27 -0500 Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1VxmxZ-0007Fb-Em for kvm@vger.kernel.org; Tue, 31 Dec 2013 01:13:25 +0100 Received: from 64.89.225.17 ([64.89.225.17]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 31 Dec 2013 01:13:25 +0100 Received: from DanaGoyette by 64.89.225.17 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 31 Dec 2013 01:13:25 +0100 In-Reply-To: <1388377005.4981.69.camel@ul30vt.home> Sender: kvm-owner@vger.kernel.org List-ID: On 12/29/2013 08:16 PM, Alex Williamson wrote: > On Sat, 2013-12-28 at 23:32 -0800, Dana Goyette wrote: >> On 12/28/2013 7:23 PM, Alex Williamson wrote: >>> On Sat, 2013-12-28 at 18:31 -0800, Dana Goyette wrote: >>>> I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to >>>> soon decide which one to keep. >>>> >>>> The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX >>>> PEX8066 switch, which claims to support ACS. I'd expect the devices >>>> downstream of the PLX switch to be in separate groups. >>>> >>>> With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities" >>>> applied and set for the Intel root ports, the devices behind the switch >>>> remain stuck in the same group. >>>> >>>> In terms of passing devices to different VMs, which is better: all >>>> devices on different root ports, or all devices behind the one >>>> ACS-supporting switch? >>> >>> Can you provide lspci -vvv info? If you're getting that for groups >>> either the switch has ACS capabilities, but doesn't support the features >>> we need or we're doing something wrong. Thanks, >>> >> I initially tried attaching the output as a .txt file, but it's too >> large. Anyway, here's the output of lspci -nnvvv (you may notice that I >> moved the Radeon to a different slot). > > Well, something seems amiss since the downstream switch ports all seem > to support and enable the correct set of ACS capabilities. I'm tending > to suspect something wrong with the ACS override patch or how it's being > used since your IOMMU group is still based at the root port. Each root > port is isolated from the other root ports though, so something is > happening with the override patch. Can you provide the kernel command > line you use to enable ACS overrides and the override patch you're > using, as it applies to 3.13-rc5? Thanks, > > Alex > > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > I'm using the original acs-override patch from this post: https://lkml.org/lkml/2013/5/30/513 Kernel parameter is: pcie_acs_override=id:8086:8c10,id:8086:8c12,id:8086:8c16,id:8086:8c18 When booting a kernel without the override patch, the following devices are all in the same group: Intel Root Ports 1, 2, 4, 5; ASMedia SATA controller; PLX PEX8606 switch; Renesas USB controller; TI Firewire controller; Intel I210 Ethernet controller.