From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47070) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YJENJ-0007nM-CK for qemu-devel@nongnu.org; Wed, 04 Feb 2015 23:49:10 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YJENF-0005C6-Dy for qemu-devel@nongnu.org; Wed, 04 Feb 2015 23:49:09 -0500 Date: Thu, 5 Feb 2015 15:49:27 +1100 From: David Gibson Message-ID: <20150205044927.GQ25675@voom.fritz.box> References: <1422523650-2888-1-git-send-email-aik@ozlabs.ru> <1422523650-2888-19-git-send-email-aik@ozlabs.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="SlnaBQtdWG0gYnqZ" Content-Disposition: inline In-Reply-To: <1422523650-2888-19-git-send-email-aik@ozlabs.ru> Subject: Re: [Qemu-devel] [PATCH v4 18/18] vfio: Enable in-kernel acceleration via VFIO KVM device List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alexey Kardashevskiy Cc: Alex Williamson , qemu-ppc@nongnu.org, qemu-devel@nongnu.org, Alexander Graf --SlnaBQtdWG0gYnqZ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jan 29, 2015 at 08:27:30PM +1100, Alexey Kardashevskiy wrote: > TCE hypercalls (H_PUT_TCE, H_PUT_TCE_INDIRECT, H_STUFF_TCE) use a logical= bus > number (LIOBN) to identify which TCE table the request is addressed to. > However VFIO kernel driver operates with IOMMU group IDs and has no idea > about which LIOBN corresponds to which group. If the host kernel supports > in-kernel acceleration for TCE calls, we have to provide the LIOBN to IOM= MU > mapping information. >=20 > This makes use of a VFIO KVM device's > KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN attribute to set the link > between LIOBN and IOMMU group. >=20 > The vfio_container_spapr_set_liobn() helper is implemented completely > in vfio.c because kvm_vfio_spapr_tce_liobn needs a group fd and > we do not want to share resources likes that outside vfio.c. I thought you'd moved away from the idea of in-kernel TCE acceleration, since big DMA windows made it unnecessary. > Signed-off-by: Alexey Kardashevskiy > --- > hw/ppc/spapr_iommu.c | 1 + > hw/ppc/spapr_pci_vfio.c | 11 +++++++++++ > hw/vfio/common.c | 33 +++++++++++++++++++++++++++++++++ > include/hw/vfio/vfio.h | 4 ++++ > 4 files changed, 49 insertions(+) >=20 > diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c > index 258f837..3de95d7 100644 > --- a/hw/ppc/spapr_iommu.c > +++ b/hw/ppc/spapr_iommu.c > @@ -142,6 +142,7 @@ static int spapr_tce_table_realize(DeviceState *dev) > if (!tcet->table) { > size_t table_size =3D tcet->nb_table * sizeof(uint64_t); > tcet->table =3D g_malloc0(table_size); > + tcet->vfio_accel =3D false; This should probably have a qdev prop so that it can be explicitly disabled on a per-device basis for testing. > } > =20 > trace_spapr_iommu_new_table(tcet->liobn, tcet, tcet->table, tcet->fd= ); > diff --git a/hw/ppc/spapr_pci_vfio.c b/hw/ppc/spapr_pci_vfio.c > index 257181d..2078187 100644 > --- a/hw/ppc/spapr_pci_vfio.c > +++ b/hw/ppc/spapr_pci_vfio.c > @@ -21,6 +21,7 @@ > #include "hw/pci-host/spapr.h" > #include "linux/vfio.h" > #include "hw/vfio/vfio.h" > +#include "qemu/error-report.h" > =20 > static Property spapr_phb_vfio_properties[] =3D { > DEFINE_PROP_INT32("iommu", sPAPRPHBVFIOState, iommugroupid, -1), > @@ -80,6 +81,16 @@ static int spapr_pci_vfio_ddw_create(sPAPRPHBState *sp= hb, uint32_t liobn, > memory_region_add_subregion(&sphb->iommu_root, (*ptcet)->bus_offset, > spapr_tce_get_iommu(*ptcet)); > =20 > + if (!(*ptcet)->vfio_accel) { > + return 0; > + } > + ret =3D vfio_container_spapr_set_liobn(&sphb->iommu_as, > + liobn, (*ptcet)->bus_offset); > + if (ret) { > + error_report("spapr-vfio: failed to create link to IOMMU"); > + ret =3D 0; > + } > + > return ret; > } > =20 > diff --git a/hw/vfio/common.c b/hw/vfio/common.c > index a26cbae..ec778d0 100644 > --- a/hw/vfio/common.c > +++ b/hw/vfio/common.c > @@ -1053,3 +1053,36 @@ int vfio_container_ioctl(AddressSpace *as, > =20 > return vfio_container_do_ioctl(as, req, param); > } > + > +int vfio_container_spapr_set_liobn(AddressSpace *as, > + uint64_t liobn, > + uint64_t start_addr) > +{ > +#ifdef CONFIG_KVM > + int ret; > + struct kvm_vfio_spapr_tce_liobn param =3D { > + .argsz =3D sizeof(param), > + .liobn =3D liobn, > + .start_addr =3D start_addr > + }; > + struct kvm_device_attr attr =3D { > + .group =3D KVM_DEV_VFIO_GROUP, > + .attr =3D KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN, > + .addr =3D (uint64_t)(unsigned long)¶m, > + }; > + > + if (vfio_kvm_device_fd < 0) { > + return 0; > + } > + > + ret =3D ioctl(vfio_kvm_device_fd, KVM_SET_DEVICE_ATTR, &attr); > + if (ret) { > + error_report("vfio: failed to setup liobn for a group: %s", > + strerror(errno)); > + } > + > + return ret; > +#else > + return 0; > +#endif > +} > diff --git a/include/hw/vfio/vfio.h b/include/hw/vfio/vfio.h > index 76b5744..8457933 100644 > --- a/include/hw/vfio/vfio.h > +++ b/include/hw/vfio/vfio.h > @@ -6,4 +6,8 @@ > extern int vfio_container_ioctl(AddressSpace *as, > int req, void *param); > =20 > +extern int vfio_container_spapr_set_liobn(AddressSpace *as, > + uint64_t liobn, > + uint64_t start_addr); > + > #endif --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --SlnaBQtdWG0gYnqZ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJU0vZXAAoJEGw4ysog2bOSZ64QANkvIAyfPhOkDsNbjxTUcF3z ao9REcxXa9IHbQeG0795wvcLl5Ya4+KsDcgG+lYCJuGvFFmETP9E3oXMpv118UQG bHnM6JLraZ1pAw1Mw4TG0cAr6/jkoW/qKaxHg+TLHtm8Dnbq6Wb2cJ05hd/PxcYr GJZoU+YKeePoXQAXFoh5Wt/qMcC7PZ3AfMQcsXvaoC9873fkav2+l80MyzCEgxDG YjcBFBYlgn/Ch+pu2CHQbxV9wLT0unMxU7wITOhtSRm/mcp9mxV7RoeTBlBiZ8z0 tpbBNe+IuIoTYpIhYPHp+r2Er9JsmJLG5pSiXx47FcMREVrBrBc4ovLFbElHF46T VGu0GBDRMC+Bw4MuZ7pNGXsuihjHyFvKIzvuC/Cuh8b1NPR+8fkNRQgJCJj5xNM3 6c4KcUUHvfZ20+LAxDb0Hn6WAjaBecrWvO+Fu7aSoH5hOOcXNmrWw592Rv+MDa6E AKK1KWdG/rRmWWmev7TunRTmkgpyYPsiJ8EMa7ohnmUCo2tyn78q79n5XP1Fn0HC SsjL9g2g/l+agBUbtkCf/suR8ZK/5TQ2uYvpPoECF3tdebuLuzHLHhSBWgMZfyAX a+/aCLwzFAMPEv/l34F6cv2DdPXR5o09rSCl5SrKn+QY0qcQNElEEGIk2PMD/5vI OpyDGsY22PjQt/p7sRBX =4wO+ -----END PGP SIGNATURE----- --SlnaBQtdWG0gYnqZ--