From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59734) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VOlZV-00015u-Ll for qemu-devel@nongnu.org; Wed, 25 Sep 2013 05:39:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VOlZP-0005Do-LU for qemu-devel@nongnu.org; Wed, 25 Sep 2013 05:39:49 -0400 Received: from mx1.redhat.com ([209.132.183.28]:6308) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VOlZP-0005Dh-D4 for qemu-devel@nongnu.org; Wed, 25 Sep 2013 05:39:43 -0400 Message-ID: <5242AF59.5060905@redhat.com> Date: Wed, 25 Sep 2013 05:39:37 -0400 From: Laine Stump MIME-Version: 1.0 References: <524162DE.1080705@redhat.com> <20130925070116.GA5436@redhat.com> <1380098908.1968.30.camel@localhost.localdomain> In-Reply-To: <1380098908.1968.30.camel@localhost.localdomain> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] Attaching PCI devices to the PCIe root complex List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: marcel.a@redhat.com Cc: qemu list , Marcel Apfelbaum , "Michael S. Tsirkin" On 09/25/2013 04:48 AM, Marcel Apfelbaum wrote: > On Wed, 2013-09-25 at 10:01 +0300, Michael S. Tsirkin wrote: >> On Tue, Sep 24, 2013 at 06:01:02AM -0400, Laine Stump wrote: >>> When I added support for the Q35-based machinetypes to libvirt, I >>> specifically prohibited attaching any PCI devices (with the exception of >>> graphics controllers) to the PCIe root complex, >> That's wrong I think. Anything attached to RC is an integrated >> endpoint, and these can be PCI devices. > I couldn't find on PCIe spec any mention that "Root Complex Integrated EndPoint" > must be PCIe. But, from spec 1.3.2.3: > - A Root Complex Integrated Endpoint must not require I/O resources claimed through BAR(s). > - A Root Complex Integrated Endpoint must not generate I/O Requests. > - A Root Complex Integrated Endpoint is required to support MSI or MSI-X or both if an > interrupt resource is requested. So much easier in the real world, where the rule is "if it fits in the socket and the pins match up, then it's okay" :-) >> IMO, we really need to grow an interface to query this kind of thing. > Basically libvirt needs to know: > 1. for (libvirt) controllers: what kind of devices can be plugged in The controllers also need a flag indicating if they supporting having devices hot-plugged into them. For example, as far as I understand, the PCI root complex ("pcie-root" in libvirt) doesn't support hot-plugging devices, nor does i82801b11-bridge ("dmi-to-pci-bridge" in libvirt), but pci-bridge, ioh3420 ("root-port" in libvirt), and xio3130-downstream ("downstream-switch-port" in libvirt) *do* support hot-plugging devices (as far as I know, none of these controllers can themselves be hot-plugged into another controller, but that could change in the future, e.g. I recall someone saying something about hot-plug of a pci-bridge being a future possibility) > 2. for devices (controller is also a device) > - to which controllers can it be plugged in > - does it support hot-plug? > 3. implicit controllers of the machine types (q35 - "pcie-root", i440fx - "pci-root") > All the above must be exported to libvirt > > Implementation options: > 1. Add a compliance field on PCI/PCIe devices and controllers stating if it supports > PCI/PCIe or both (and maybe hot-plug) > - consider plug type + compliance to figure out whether a plug can go into a socket > > 2. Use Markus Armbruster idea of introducing a concept of "plug and sockets": > - dividing the devices into adapters and plugs > - adding sockets to bridges(buses?). > In this way it would be clear which devices can connect to bridges However it is done, each controller will need to have two sets of flags - one for what it can plug into, and one for what can be plugged into it.