From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58130) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YbNQU-0004DE-9f for qemu-devel@nongnu.org; Fri, 27 Mar 2015 02:07:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YbNQS-0005W5-TP for qemu-devel@nongnu.org; Fri, 27 Mar 2015 02:07:26 -0400 Date: Fri, 27 Mar 2015 17:00:25 +1100 From: David Gibson Message-ID: <20150327060024.GF2900@voom.fritz.box> References: <1427348102-5879-1-git-send-email-gwshan@linux.vnet.ibm.com> <1427348102-5879-2-git-send-email-gwshan@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="hK8Uo4Yp55NZU70L" Content-Disposition: inline In-Reply-To: <1427348102-5879-2-git-send-email-gwshan@linux.vnet.ibm.com> Subject: Re: [Qemu-devel] [PATCH v3 1/2] VFIO: Clear stale MSIx table during EEH reset List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Gavin Shan Cc: alex.williamson@redhat.com, qemu-ppc@nongnu.org, qemu-devel@nongnu.org, agraf@suse.de --hK8Uo4Yp55NZU70L Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 26, 2015 at 04:35:01PM +1100, Gavin Shan wrote: > The PCI device MSIx table is cleaned out in hardware after EEH PE > reset. However, we still hold the stale MSIx entries in QEMU, which > should be cleared accordingly. Otherwise, we will run into another > (recursive) EEH error and the PCI devices contained in the PE have > to be offlined exceptionally. >=20 > The patch introduces function vfio_eeh_pe_reset(), which is called > by sPAPR when asserting hot or fundamental reset, to clear stale MSIx > table before EEH PE reset so that MSIx table could be restored properly > after EEH PE reset. >=20 > Signed-off-by: Gavin Shan > --- > hw/ppc/spapr_pci_vfio.c | 13 +++++++++---- > hw/vfio/Makefile.objs | 6 +++++- > hw/vfio/pci-stub.c | 16 ++++++++++++++++ > hw/vfio/pci.c | 36 ++++++++++++++++++++++++++++++++++++ > include/hw/vfio/vfio.h | 2 ++ > 5 files changed, 68 insertions(+), 5 deletions(-) > create mode 100644 hw/vfio/pci-stub.c >=20 > diff --git a/hw/ppc/spapr_pci_vfio.c b/hw/ppc/spapr_pci_vfio.c > index 99a1be5..6fa3afe 100644 > --- a/hw/ppc/spapr_pci_vfio.c > +++ b/hw/ppc/spapr_pci_vfio.c > @@ -151,19 +151,24 @@ static int spapr_phb_vfio_eeh_reset(sPAPRPHBState *= sphb, int option) > switch (option) { > case RTAS_SLOT_RESET_DEACTIVATE: > op.op =3D VFIO_EEH_PE_RESET_DEACTIVATE; > + ret =3D vfio_container_ioctl(&svphb->phb.iommu_as, > + svphb->iommugroupid, > + VFIO_EEH_PE_OP, &op); > break; > case RTAS_SLOT_RESET_HOT: > - op.op =3D VFIO_EEH_PE_RESET_HOT; > + ret =3D vfio_eeh_pe_reset(&svphb->phb.iommu_as, > + svphb->iommugroupid, > + VFIO_EEH_PE_RESET_HOT); > break; > case RTAS_SLOT_RESET_FUNDAMENTAL: > - op.op =3D VFIO_EEH_PE_RESET_FUNDAMENTAL; > + ret =3D vfio_eeh_pe_reset(&svphb->phb.iommu_as, > + svphb->iommugroupid, > + VFIO_EEH_PE_RESET_FUNDAMENTAL); > break; > default: > return RTAS_OUT_PARAM_ERROR; > } > =20 > - ret =3D vfio_container_ioctl(&svphb->phb.iommu_as, svphb->iommugroup= id, > - VFIO_EEH_PE_OP, &op); > if (ret < 0) { > return RTAS_OUT_HW_ERROR; > } > diff --git a/hw/vfio/Makefile.objs b/hw/vfio/Makefile.objs > index e31f30e..1b8a065 100644 > --- a/hw/vfio/Makefile.objs > +++ b/hw/vfio/Makefile.objs > @@ -1,4 +1,8 @@ > ifeq ($(CONFIG_LINUX), y) > obj-$(CONFIG_SOFTMMU) +=3D common.o > -obj-$(CONFIG_PCI) +=3D pci.o > +ifeq ($(CONFIG_PCI), y) > +obj-y +=3D pci.o > +else > +obj-y +=3D pci-stub.o > +endif > endif > diff --git a/hw/vfio/pci-stub.c b/hw/vfio/pci-stub.c > new file mode 100644 > index 0000000..f317c1e > --- /dev/null > +++ b/hw/vfio/pci-stub.c > @@ -0,0 +1,16 @@ > +/* > + * To include the file on !CONFIG_PCI > + * > + * This work is licensed under the terms of the GNU GPL, version 2. See > + * the COPYING file in the top-level directory. > + */ > + > +#include > + > +#include "exec/memory.h" > +#include "hw/vfio/vfio.h" > + > +int vfio_eeh_pe_reset(AddressSpace *as, int32_t groupid, uint32_t option) > +{ > + return -1; > +} This doesn't seem quite right. AFAICT the only caller of vfio_eeh_pe_reset() is in spapr_pci_vfio.c, which is only built if CONFIG_PCI is enabled. So if there needed to be !PCI stubs, I'd expect them further up the call stack. > +int vfio_eeh_pe_reset(AddressSpace *as, int32_t groupid, uint32_t option) > +{ > + VFIOGroup *group; > + VFIODevice *vbasedev; > + VFIOPCIDevice *vdev; > + struct vfio_eeh_pe_op op =3D { > + .argsz =3D sizeof(op), > + .op =3D option > + }; > + > + group =3D vfio_get_group(groupid, as); > + if (!group) { > + error_report("vfio: group %d not found\n", groupid); > + return -1; > + } > + > + /* > + * The MSIx table will be cleaned out by reset. We need > + * disable it so that it can be reenabled properly. Also, > + * the cached MSIx table should be cleared as it's not > + * reflecting the contents in hardware. > + */ > + QLIST_FOREACH(vbasedev, &group->device_list, next) { > + vdev =3D container_of(vbasedev, VFIOPCIDevice, vbasedev); > + if (msix_enabled(&vdev->pdev)) { > + vfio_disable_msix(vdev); > + } > + > + msix_reset(&vdev->pdev); > + } > + > + vfio_put_group(group); > + > + return vfio_container_ioctl(as, groupid, VFIO_EEH_PE_OP, &op); > +} This is much better than the vfio_eeh_event stuff(). --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --hK8Uo4Yp55NZU70L Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJVFPH4AAoJEGw4ysog2bOShm4P/Asb78AmQrV1DXYFtw8i2Oz1 yy3X4eSOL7I0njfutvb00tucy14uofPk86GqPkXis+NBpI0aoMx8TTFISW32Rprx qUkrIqtd1vnmQPQqvLGZxr6awFoeuL8kzdU/JofTKvjoggLR443UDwljjRxYN0S7 oNoF7X6FdYnb7JrivT0yTjNM9+t4yzYIDkjA+VgN0xpvckeWhi4yp8J6ABK8rUED cbz5zDo+kjckuIgYe3713iUcx1c2ToGdHKSWtDJK7xthrv4DwJlSLScfvAU6Fs6A Fp2DwIDYrcyxnhbAOA/AgKmQZzVYf+hEmyPQK9FMcFc1bRCJV8Z1YaTB2kDXhcuI berQAzjycM7uMqLLxVnEEvoVRqsztnsu3yZuYpnlGu6tVfRhAwa62nrCnt4fBe/q e5JDKNK0nzOhEGLZVBLNR3mxjexUQUuNrdN3pawT0fAKZmVXLOh57I5Rt1NqfOW8 N3pGCFFM9FQhkVedciB8UXHaWUGEZl+JdxKx7tUPHDB6D/5261aHpVfXYp/NRtQa Q3kU1yFvHeemKPwlshp5rnvDEsCxmxo8DBX0VFZ3+eSN1zWeUdhjrh7MDOQDXQdJ mzsHgGox4y83XQHGatwECu59p49KoUemSBDNpJr6bC5ddVmzrmt84mAVdPOvS395 4rw+DihoaSkn6yEzOujQ =3cBd -----END PGP SIGNATURE----- --hK8Uo4Yp55NZU70L--