From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:53744) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UcSoa-0002rx-6l for qemu-devel@nongnu.org; Tue, 14 May 2013 23:55:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UcSoW-0002Ax-0f for qemu-devel@nongnu.org; Tue, 14 May 2013 23:55:44 -0400 Received: from ozlabs.org ([2402:b800:7003:1:1::1]:41040) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UcSoV-0002Am-CT for qemu-devel@nongnu.org; Tue, 14 May 2013 23:55:39 -0400 Date: Wed, 15 May 2013 13:55:41 +1000 From: David Gibson Message-ID: <20130515035541.GR14944@truffula.fritz.box> References: <1368442465-14363-1-git-send-email-david@gibson.dropbear.id.au> <5190DB42.10002@redhat.com> <20130513131348.GD14944@truffula.fritz.box> <5190EAF2.6020606@redhat.com> <20130514023937.GH14944@truffula.fritz.box> <51920AB8.3050906@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="sDhuxz86kt5qlkuV" Content-Disposition: inline In-Reply-To: <51920AB8.3050906@redhat.com> Subject: Re: [Qemu-devel] [0/8] RFC: VFIO and guest side IOMMUs, revisited List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini Cc: aik@ozlabs.ru, alex.williamson@redhat.com, mst@redhat.com, qemu-devel@nongnu.org, agraf@suse.de --sDhuxz86kt5qlkuV Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, May 14, 2013 at 11:58:16AM +0200, Paolo Bonzini wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 >=20 > Il 14/05/2013 04:39, David Gibson ha scritto: > > On Mon, May 13, 2013 at 03:30:26PM +0200, Paolo Bonzini wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > >>=20 > >> Il 13/05/2013 15:13, David Gibson ha scritto: > >>> On Mon, May 13, 2013 at 02:23:30PM +0200, Paolo Bonzini wrote: > >>>> Il 13/05/2013 12:54, David Gibson ha scritto: > >>>>> Specifically the way the iommu is determined from a > >>>>> callback in the PCIBus means that it won't be assigned for > >>>>> devices under a PCI-PCI bridge. > >>>>=20 > >>>> Right. I saw the report from Alexey, but I am a bit wary of > >>>> touching it because it's not a regression. In fact there > >>>> is even a FIXME for it: > >>>>=20 > >>>> /* FIXME: inherit memory region from bus creator */ > >>>=20 > >>> Uh.. sort of. > >>>=20 > >>>> Perhaps we can make pci_iommu_as a Bus method, where the > >>>> default implementation looks up along the chain, and the end > >>>> of the recursion is in SysBus or in PCI buses that have set > >>>> the callback. > >>>=20 > >>> So, this is complicated by the fact that there are two cases, > >>> and they can both be found in existing hardware. > >>>=20 > >>> 1) One is where devices behind the bridge are not visible /=20 > >>> differentiable to the IOMMU, and so effectively all their DMAs > >>> originate from the bridge device itself. In this case the > >>> correct thing is to give all devices under the bridge the same > >>> DMA AddressSpace as the bridge device, as suggested by the > >>> FIXME. This will be typical behaviour for PCI-E to PCI > >>> bridges. > >>>=20 > >>> 2) The other case is where the bridge passes through RIDs, so > >>> that the IOMMU can still differentiate devices behind it. For > >>> this case, we really want the hook to be in the host bridge / > >>> root bus, and it can make a decision based on the full > >>> bus/dev/fn information. This will be typical for PCI-E to > >>> PCI-E bridges (or switches or nexuses or whatever they're > >>> usually called for PCI-E). This case will be very important as > >>> we start to model newer PCI-E based machines by default, where > >>> typically *all* devices are behind a logical p2p bridge inside > >>> the root complex (but are still differentiable by the Intel > >>> IOMMU amongst others). > >>>=20 > >>> I'm not sure at this stage how to properly handle both cases. > >>=20 > >> Suppose you have a host bridge pci_bus0 and a PCIE->PCIE bridge=20 > >> pci_bus1. pci_bus1 does not define a IOMMU callback, pci_bus0 > >> does. > >>=20 > >> Would it work to use the PCIBus callback provided by pci_bus0, > >> but invoke it as > >>=20 > >> pci_bus0->iommu_fn(pci_bus1, pci_bus0->iommu_opaque, devfn) > >=20 > > Hrm. I'm a bit nervous about that, because I think when writing > > an iommu_fn it would be very easy to forget that it could be called > > with a bus other than the one the hook is attached to - and e.g. > > assuming they can use bus->qbus.parent_dev to get to the host > > bridge. >=20 > I think we can fix that by removing the opaque, and just passing in > the PCIBus. >=20 > Then it's more obvious >=20 > pci_bus0->iommu_fn(pci_bus0, pci_bus1, devfn) Yeah, that's probably ok, especially if we can think of good names for the two bus parameters to make the distinction clear. > and almost the same, since the host bridge is just a container_of away > from pci_bus0. Well, bus->qbus.parent_dev and then one of the suitable class wrappers on container_of(). --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --sDhuxz86kt5qlkuV Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlGTBz0ACgkQaILKxv3ab8Z+3wCfWGFoKwpwgES49ETCrvaVWeT3 9f0AnjMTt1haPRyUa31fERQAFgHlIR16 =Audn -----END PGP SIGNATURE----- --sDhuxz86kt5qlkuV--