From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43896) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zext7-0000VB-2n for qemu-devel@nongnu.org; Thu, 24 Sep 2015 00:12:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Zext5-0003FV-U3 for qemu-devel@nongnu.org; Thu, 24 Sep 2015 00:12:05 -0400 Date: Thu, 24 Sep 2015 14:11:27 +1000 From: David Gibson Message-ID: <20150924041127.GO15944@voom.fritz.box> References: <1442647117-2726-1-git-send-email-david@gibson.dropbear.id.au> <1442647117-2726-12-git-send-email-david@gibson.dropbear.id.au> <1443029281.23936.481.camel@redhat.com> <20150924014959.GM15944@voom.fritz.box> <1443061176.23936.522.camel@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="V7BlxAaPrdhzdIM1" Content-Disposition: inline In-Reply-To: <1443061176.23936.522.camel@redhat.com> Subject: Re: [Qemu-devel] [RFC PATCH 11/14] spapr_pci: Allow EEH on spapr-pci-host-bridge List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alex Williamson Cc: lvivier@redhat.com, thuth@redhat.com, qemu-devel@nongnu.org, aik@ozlabs.ru, mdroth@linux.vnet.ibm.com, gwshan@linux.vnet.ibm.com, qemu-ppc@nongnu.org --V7BlxAaPrdhzdIM1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Sep 23, 2015 at 08:19:36PM -0600, Alex Williamson wrote: > On Thu, 2015-09-24 at 11:49 +1000, David Gibson wrote: > > On Wed, Sep 23, 2015 at 11:28:01AM -0600, Alex Williamson wrote: > > > On Sat, 2015-09-19 at 17:18 +1000, David Gibson wrote: > > > > The pseries machine type has two variants of the PCI Host Bridge de= vice: > > > > spapr-pci-host-bridge and spapr-pci-vfio-host-bridge. Originally, = only the > > > > latter could support VFIO devices, but we've now extended the VFIO = code so > > > > that they both can. > > > >=20 > > > > However, it's still only spapr-pci-vfio-host-bridge which supports = the PAPR > > > > Enhanced Error Handling (EEH) interfaces. The reason is that we do= n't yet > > > > have a way to determine the correct VFIOGroup for EEH operations on= the > > > > "plain" host bridge. > > > >=20 > > > > Handling this in general is problematic due to some limitations in = the > > > > current code, and some bugs in the kernel EEH interfaces. However,= it's > > > > easy enough to do for the unambiguous case: where there is only one= VFIO > > > > group used on the whole host bridge i.e. one Partitionable Endpoint= (PE). > > > >=20 > > > > This re-implements spapr_phb_check_vfio_group() in terms of the host > > > > bridge's Partitionable Endpoints, allowing EEH on any host bridge i= n the > > > > unambiguous case. This is enough to make spapr-pci-host-bridge sup= port > > > > EEH as well as spapr-pci-vfio-host-bridge, since the latter only su= pported > > > > devices from one IOMMU group anyway (although it didn't properly > > > > enforce that). > > > >=20 > > > > Signed-off-by: David Gibson > > > > --- > > > > hw/ppc/spapr_pci.c | 36 +++++++++++++++++++++++++++++++++= +++ > > > > hw/ppc/spapr_pci_vfio.c | 18 ------------------ > > > > include/hw/pci-host/spapr.h | 1 - > > > > 3 files changed, 36 insertions(+), 19 deletions(-) > > > >=20 > > > > diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c > > > > index 81ad3ae..5614b45 100644 > > > > --- a/hw/ppc/spapr_pci.c > > > > +++ b/hw/ppc/spapr_pci.c > > > > @@ -41,6 +41,8 @@ > > > > #include "hw/ppc/spapr_drc.h" > > > > #include "sysemu/device_tree.h" > > > > =20 > > > > +#include "hw/vfio/vfio-common.h" > > > > + > > > > /* Copied from the kernel arch/powerpc/platforms/pseries/msi.c */ > > > > #define RTAS_QUERY_FN 0 > > > > #define RTAS_CHANGE_FN 1 > > > > @@ -446,6 +448,40 @@ static sPAPRPHBGuestPE *spapr_phb_pe_by_device= (sPAPRPHBState *phb, > > > > return pe; > > > > } > > > > =20 > > > > +static int spapr_phb_check_vfio_group(sPAPRPHBState *phb, VFIOGrou= p **gpp) > > > > +{ > > > > + sPAPRPHBGuestPE *pe; > > > > + > > > > + if (QLIST_EMPTY(&phb->pe_list)) { > > > > + /* No EEH capable devices on this PHB */ > > > > + return RTAS_OUT_PARAM_ERROR; > > > > + } > > > > + > > > > + /* Limitations in both qemu and the kernel mean that, for now,= EEH > > > > + * won't work if there are devices from more than one PE > > > > + * (i.e. IOMMU group) on the same PHB */ > > > > + pe =3D QLIST_FIRST(&phb->pe_list); > > > > + if (QLIST_NEXT(pe, list)) { > > > > + error_report("spapr-pci-host-bridge: EEH attempted on PHB = with multiple" > > > > + " IOMMU groups"); > > >=20 > > > Don't wrap lines with search-able strings > >=20 > > Noted. > >=20 > > > > + return RTAS_OUT_HW_ERROR; > > > > + } > > > > + > > > > + if (!object_dynamic_cast(OBJECT(phb), "spapr-pci-vfio-host-bri= dge")) { > > > > + sPAPRPHBVFIOState *svphb =3D SPAPR_PCI_VFIO_HOST_BRIDGE(ph= b); > > > > + /* FIXME: this is an abstraction violation */ > > > > + if (pe->group->groupid !=3D svphb->iommugroupid) { > > > > + error_report("spapr-pci-vfio-host-bridge: Bad IOMMU gr= oup"); > > > > + return RTAS_OUT_HW_ERROR; > > > > + } > > > > + } > > > > + > > > > + if (gpp) { > > > > + *gpp =3D pe->group; > > > > + } > > > > + return RTAS_OUT_SUCCESS; > > >=20 > > >=20 > > > The structure of this function finally makes some sense once we get > > > here. > >=20 > > Yeah, I started off with something more like what you suggested in > > earlier comments, but I changed to this rather awkward structure to > > avoid churning the callers as this moves towards its final form > >=20 > > > I'm still not sure whether RTAS error codes make sense though, > > > but it's just a nit. > >=20 > > Well, (nearly) all the callers are in RTAS, and translating the error > > codes there seems pretty ugly too. > >=20 > > > I'd still prefer the function was renamed, "check" > > > indicates a verification, not a return. The primary purpose of calli= ng > > > this function is to get a group, not simply to check the validity. > >=20 > > Yeah, it's a crap name. How about spapr_phb_find_vfio_group(). >=20 > Why not just spapr_phb_get_vfio_group() since this started out as just a > wrapper for vfio_get_group() and that's still what it's doing even > though it's now cached on sPAPRPHBGuestPE rather than retrieved each > time. Well, "get" often implies taking a persistent reference in some way, which is why I avoided it. --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --V7BlxAaPrdhzdIM1 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJWA3fvAAoJEGw4ysog2bOSgjMP/iu9vaIcmmVRJtlstrgJRqLW Hriln8aBwhCydNjBvnNm/EmmRq1Zot1PcNMZO7LzFa9Z+VCeXitCgDXBWmWYi82/ +FoElHoQteubV9C1vUKELeKc7P8Yr5WYoK/RwMH7mZvL0Wg3Gr1RHA7XZYhm05MY D1Jf5m2mJx6dCwhySqb2k4qoLo8iAW4kJDzqHslzGA2Ox6uBRvyZA1hZ4MHYY5KM P+V04uyCMxCO6pONHb+oKmoBx8jP5tFRvwMtXe/pub40DwY0Sxw1wCL1yPdI+nrv 70C9MXRXj+eimC2jgUkeDBNl0q9jZ5M0rs4qIdVI7HHQHhwNlik74b+Iubjr82FM LoknbDsp6zPS9GfgDAbNJYEb+9CfZVnkVouyVissl5Q7spEPYVmaSgbjbAWd4YoK RUEC7hTXnyST83lK98v1on4mJlW+DfMuzAe519/wTJfoHHRsTULKqomu+6SSRlXr 09dyJhZcjS69E9BHF8oIwfS/BtraARGYmIZrFmCOC/hWco+gp17mpn4tsHQyHp6u Qof4FeG8DFwifbYRK4ThmqoK4CTbvTRav2FB3A3PTI0ft3gPd6OKYOtIiii6a3jl ykHuxHL4wgQlO0RLoJQCWVr8uqyvO7FapFoyNF8Zv1WDjS5DNNIpeyhR/fvJbH3Z lq9GIh6q8gCb6YUcya2Q =D7ze -----END PGP SIGNATURE----- --V7BlxAaPrdhzdIM1--