From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43129) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YIBDJ-0003X4-46 for qemu-devel@nongnu.org; Mon, 02 Feb 2015 02:14:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YIBDH-0003lP-Gj for qemu-devel@nongnu.org; Mon, 02 Feb 2015 02:14:29 -0500 Date: Mon, 2 Feb 2015 18:04:07 +1100 From: David Gibson Message-ID: <20150202070407.GN28703@voom.fritz.box> References: <1422523650-2888-1-git-send-email-aik@ozlabs.ru> <1422523650-2888-9-git-send-email-aik@ozlabs.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="PZYVFYZbFYjzBslI" Content-Disposition: inline In-Reply-To: <1422523650-2888-9-git-send-email-aik@ozlabs.ru> Subject: Re: [Qemu-devel] [PATCH v4 08/18] vfio: Add DMA memory registering List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alexey Kardashevskiy Cc: Alex Williamson , qemu-ppc@nongnu.org, qemu-devel@nongnu.org, Alexander Graf --PZYVFYZbFYjzBslI Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jan 29, 2015 at 08:27:20PM +1100, Alexey Kardashevskiy wrote: > This makes use of the new "memory registering" feature. The idea is > to provide the guest ability to notify the host kernel about pages which > are going to be used for DMA. Having this information, the host kernel > can pin them all, do locked pages accounting and not spent time on > doing that in real time with possible failures which cannot be handled > nicely in some cases. >=20 > This adds a memory listener which notifies a VFIO container about memory > which needs to be pinned/unpinned. VFIO MMIO regions are skipped. >=20 > Signed-off-by: Alexey Kardashevskiy > --- > hw/vfio/common.c | 99 +++++++++++++++++++++++++++++++++++++= +++++- > include/hw/vfio/vfio-common.h | 3 +- > 2 files changed, 100 insertions(+), 2 deletions(-) >=20 > diff --git a/hw/vfio/common.c b/hw/vfio/common.c > index cf483ff..c08f9ab 100644 > --- a/hw/vfio/common.c > +++ b/hw/vfio/common.c > @@ -485,6 +485,99 @@ void vfio_listener_release(VFIOContainer *container) > memory_listener_unregister(&container->iommu_data.type1.listener); > } > =20 > +static int vfio_mem_register(VFIOContainer *container, > + void *vaddr, ram_addr_t size) > +{ > + struct vfio_iommu_type1_register_memory reg =3D { > + .argsz =3D sizeof(reg), > + .vaddr =3D (__u64)(uintptr_t)vaddr, > + .size =3D size, > + }; > + long ret =3D ioctl(container->fd, VFIO_IOMMU_REGISTER_MEMORY, ®); > + > + DPRINTF("(%u) %s %u: va=3D%lx size=3D%lx, ret=3D%ld\n", getpid(), > + __func__, __LINE__, reg.vaddr, reg.size, ret); > + > + return ret; > +} > + > +static int vfio_mem_unregister(VFIOContainer *container, > + void *vaddr, ram_addr_t size) > +{ > + struct vfio_iommu_type1_unregister_memory reg =3D { > + .argsz =3D sizeof(reg), > + .vaddr =3D (__u64)(uintptr_t)vaddr, > + .size =3D size, > + }; > + long ret =3D ioctl(container->fd, VFIO_IOMMU_UNREGISTER_MEMORY, ®= ); > + > + DPRINTF("(%u) %s %u: va=3D%lx size=3D%lx, ret=3D%ld\n", getpid(), > + __func__, __LINE__, reg.vaddr, reg.size, ret); > + > + return ret; > +} > + > +static void vfio_ram_region_add(MemoryListener *listener, > + MemoryRegionSection *section) > +{ > + VFIOContainer *container =3D container_of(listener, VFIOContainer, > + iommu_data.type1.ramlistener= ); > + hwaddr end; > + Int128 llend; > + void *vaddr; > + > + if (!memory_region_is_ram(section->mr) || > + memory_region_is_skip_dump(section->mr)) { > + return; > + } > + > + llend =3D int128_make64(section->offset_within_address_space); > + llend =3D int128_add(llend, section->size); Does this need an add TARGET_PAGE_SIZE-1 in order to make it round up to a page boundary, rather than truncate to a page boundary? > + llend =3D int128_and(llend, int128_exts64(TARGET_PAGE_MASK)); > + > + memory_region_ref(section->mr); > + > + end =3D int128_get64(llend); > + vaddr =3D memory_region_get_ram_ptr(section->mr) + > + section->offset_within_region; > + vfio_mem_register(container, vaddr, end); > +} > + > +static void vfio_ram_region_del(MemoryListener *listener, > + MemoryRegionSection *section) > +{ > + VFIOContainer *container =3D container_of(listener, VFIOContainer, > + iommu_data.type1.ramlistener= ); > + hwaddr iova, end; > + void *vaddr; > + > + if (!memory_region_is_ram(section->mr) || > + memory_region_is_skip_dump(section->mr)) { > + return; > + } > + > + > + iova =3D TARGET_PAGE_ALIGN(section->offset_within_address_space); > + end =3D (section->offset_within_address_space + int128_get64(section= ->size)) & > + TARGET_PAGE_MASK; > + vaddr =3D memory_region_get_ram_ptr(section->mr) + > + section->offset_within_region + > + (iova - section->offset_within_address_space); It's not clear to me why region_add and region_del have a different set of address calculations. > + vfio_mem_unregister(container, vaddr, end - iova); > +} > + > +const MemoryListener vfio_ram_listener =3D { > + .region_add =3D vfio_ram_region_add, > + .region_del =3D vfio_ram_region_del, > +}; > + > +static void vfio_spapr_listener_release(VFIOContainer *container) > +{ > + memory_listener_unregister(&container->iommu_data.type1.listener); > + memory_listener_unregister(&container->iommu_data.type1.ramlistener); Accessing fields within type1 from a function whose name says it's spapr specific seems very wrong. > +} > + > int vfio_mmap_region(Object *obj, VFIORegion *region, > MemoryRegion *mem, MemoryRegion *submem, > void **map, size_t size, off_t offset, > @@ -705,6 +798,10 @@ static int vfio_connect_container(VFIOGroup *group, = AddressSpace *as) > goto free_container_exit; > } > =20 > + container->iommu_data.type1.ramlistener =3D vfio_ram_listener; > + memory_listener_register(&container->iommu_data.type1.ramlistene= r, > + &address_space_memory); Why two separate listeners, rather than doing both jobs from a single liste= ner? > /* > * The host kernel code implementing VFIO_IOMMU_DISABLE is called > * when container fd is closed so we do not call it explicitly > @@ -718,7 +815,7 @@ static int vfio_connect_container(VFIOGroup *group, A= ddressSpace *as) > } > =20 > container->iommu_data.type1.listener =3D vfio_memory_listener; > - container->iommu_data.release =3D vfio_listener_release; > + container->iommu_data.release =3D vfio_spapr_listener_release; > =20 > memory_listener_register(&container->iommu_data.type1.listener, > container->space->as); > diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h > index 1d5bfe8..3f91216 100644 > --- a/include/hw/vfio/vfio-common.h > +++ b/include/hw/vfio/vfio-common.h > @@ -66,6 +66,7 @@ struct VFIOGroup; > =20 > typedef struct VFIOType1 { > MemoryListener listener; > + MemoryListener ramlistener; > int error; > bool initialized; > } VFIOType1; > @@ -144,7 +145,7 @@ int vfio_get_device(VFIOGroup *group, const char *nam= e, > VFIODevice *vbasedev); > =20 > extern const MemoryRegionOps vfio_region_ops; > -extern const MemoryListener vfio_memory_listener; > +extern const MemoryListener vfio_memory_listener, vfio_ram_listener; > extern QLIST_HEAD(vfio_group_head, VFIOGroup) vfio_group_list; > extern QLIST_HEAD(vfio_as_head, VFIOAddressSpace) vfio_address_spaces; > =20 --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --PZYVFYZbFYjzBslI Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJUzyFnAAoJEGw4ysog2bOSW8kP/0n4tbAOzCagdIGFgweHV66x ySmVYQOkt4vOJW4v8op4sb9AWARKrXwHZb5q2v9Sl08l4KKsqLA82d/+Jo01xaae 4Ev944bSS/PfdVRNzd3BoBj9zE0ftE2MQgLqAl3CMl5fkfYmtwg4gAA4hX6SuEmg laJ+WFR2nZ9JoQYEt2P2Rv9bRFreuh7wU/HBwMKrnr048TyYVkBTUTxdAUP4xz/I ulRC1r1bEpEudJhAnKdGNl0Ikr6hkl4wlpqOsahztEqeIw//ku4TAUDieFBPuKFF FsOapzb0KKk0AH8/saxYzhfRxvNt1tkPb6wQ2zWIPY9uIRRgLnz7FQgHskdO3M24 XyfiPcmCnJKNfbh95PEpnxxoawW+/1lTZoHVFe8IMGlE00nn+DmW7Vo8XJLWv97h 7h63qvPi3MtkSsFZbExME10Z1vsDYAiO4/qos5LCLqhh4I9uJZoXPuEN8E477CBY wgU3dnL8Q9OMzvO7+jsG0O8TTDBPWf9Y9eomRJYb7q+7pwHMkJkBxmtKmcAgM2yk VqMsd6seZOp8DSAvEaqPWVZMlMnG1s/WqmwVC0CqipyXvab5QeMnieIWjvXRQsyW T47R1rfJglX8bEtZ8cLFWexL+AO77DEd5xk4rRi4TnOXHTcyEolmPdL6uSmN/hwR Hx+/lc864izxjcbMf5ZB =zKbh -----END PGP SIGNATURE----- --PZYVFYZbFYjzBslI--