From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48333) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VOkln-0003am-Hg for qemu-devel@nongnu.org; Wed, 25 Sep 2013 04:48:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VOkle-0006ln-WB for qemu-devel@nongnu.org; Wed, 25 Sep 2013 04:48:27 -0400 Received: from mail-we0-x22f.google.com ([2a00:1450:400c:c03::22f]:47177) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VOkle-0006lW-Oi for qemu-devel@nongnu.org; Wed, 25 Sep 2013 04:48:18 -0400 Received: by mail-we0-f175.google.com with SMTP id q59so5677097wes.20 for ; Wed, 25 Sep 2013 01:48:17 -0700 (PDT) Message-ID: <1380098908.1968.30.camel@localhost.localdomain> From: Marcel Apfelbaum Date: Wed, 25 Sep 2013 11:48:28 +0300 In-Reply-To: <20130925070116.GA5436@redhat.com> References: <524162DE.1080705@redhat.com> <20130925070116.GA5436@redhat.com> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] Attaching PCI devices to the PCIe root complex Reply-To: marcel.a@redhat.com List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Michael S. Tsirkin" Cc: qemu list , Laine Stump On Wed, 2013-09-25 at 10:01 +0300, Michael S. Tsirkin wrote: > On Tue, Sep 24, 2013 at 06:01:02AM -0400, Laine Stump wrote: > > When I added support for the Q35-based machinetypes to libvirt, I > > specifically prohibited attaching any PCI devices (with the exception of > > graphics controllers) to the PCIe root complex, > > That's wrong I think. Anything attached to RC is an integrated > endpoint, and these can be PCI devices. I couldn't find on PCIe spec any mention that "Root Complex Integrated EndPoint" must be PCIe. But, from spec 1.3.2.3: - A Root Complex Integrated Endpoint must not require I/O resources claimed through BAR(s). - A Root Complex Integrated Endpoint must not generate I/O Requests. - A Root Complex Integrated Endpoint is required to support MSI or MSI-X or both if an interrupt resource is requested. I suppose that this restriction can be removed for PCI devices that 1. Actually work when plugged in into RC Integrated EndPoint 2. Respond to the above limitations > > > and had planned to > > prevent attaching them to PCIe root ports (ioh3420 device) and PCIe > > downstream switch ports (xio-3130 device) as well. I did this because, > > even though qemu currently allows attaching a normal PCI device in any > > of these three places, the restriction exists for real hardware and I > > didn't see any guarantee that qemu wouldn't add the restriction in the > > future in order to more closely emulate real hardware. > > > > However, since I did that, I've learned that many of the qemu "pci" > > devices really should be considered as "pci or pcie". Gerd Hoffman lists > > some of these cases in a bug he filed against libvirt: > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1003983 > > > > I would like to loosen up the restrictions in libvirt, but want to make > > sure that I don't allow something that could later be forbidden by qemu > > (thus creating a compatibility problem during upgrades). Beyond Gerd's > > specific requests to allow ehci, uhci, and hda controllers to attach to > > PCIe ports, are there any other devices that I specifically should or > > shouldn't allow? (I would rather be conservative in what I allow - it's > > easy to allow more things later, but nearly impossible to revoke > > permission once it's been allowed). For the moment I would not remove any restrictions, but only the ones requested and verified by somebody. > > IMO, we really need to grow an interface to query this kind of thing. Basically libvirt needs to know: 1. for (libvirt) controllers: what kind of devices can be plugged in 2. for devices (controller is also a device) - to which controllers can it be plugged in - does it support hot-plug? 3. implicit controllers of the machine types (q35 - "pcie-root", i440fx - "pci-root") All the above must be exported to libvirt Implementation options: 1. Add a compliance field on PCI/PCIe devices and controllers stating if it supports PCI/PCIe or both (and maybe hot-plug) - consider plug type + compliance to figure out whether a plug can go into a socket 2. Use Markus Armbruster idea of introducing a concept of "plug and sockets": - dividing the devices into adapters and plugs - adding sockets to bridges(buses?). In this way it would be clear which devices can connect to bridges Any thoughts? Thanks, Marcel >