From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33113) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a1FSe-00021R-Lg for qemu-devel@nongnu.org; Tue, 24 Nov 2015 10:24:58 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a1FSZ-0002CP-PE for qemu-devel@nongnu.org; Tue, 24 Nov 2015 10:24:52 -0500 Received: from mailout1.w1.samsung.com ([210.118.77.11]:17795) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a1FSZ-0002CK-JG for qemu-devel@nongnu.org; Tue, 24 Nov 2015 10:24:47 -0500 Received: from eucpsbgm2.samsung.com (unknown [203.254.199.245]) by mailout1.w1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTP id <0NYB000ITS58U530@mailout1.w1.samsung.com> for qemu-devel@nongnu.org; Tue, 24 Nov 2015 15:24:44 +0000 (GMT) From: Pavel Fedin References: <00fe01d1210c$1be12880$53a37980$@samsung.com> <1447884282.4697.111.camel@redhat.com> <013101d122b5$240ef500$6c2cdf00$@samsung.com> <1447976037.4697.205.camel@redhat.com> In-reply-to: <1447976037.4697.205.camel@redhat.com> Date: Tue, 24 Nov 2015 18:24:43 +0300 Message-id: <013801d126cc$3efcbdf0$bcf639d0$@samsung.com> MIME-version: 1.0 Content-type: text/plain; charset=UTF-8 Content-transfer-encoding: quoted-printable Content-language: ru Subject: Re: [Qemu-devel] [PATCH] vfio: Align iova also to IOMMU page size List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: 'Alex Williamson' Cc: 'Peter Maydell' , qemu-devel@nongnu.org Hello! > > So, i've got this problem on ARM64. On ARM64 we actually can never = have 1K pages. This page > > size was supported only by old 32-bit ARM CPUs, up to ARMv5 IIRC, = then it was dropped. Linux > > OS never even used it. > > But, since qemu can emulate those ancient CPUs, TARGET_PAGE_BITS is = defined to 10 for ARM. > > And, ARM64 and ARM32 is actually the same target for qemu, so this = is why we still get it. > > Perhaps, TARGET_PAGE_BITS should be a variable for ARM, and we = should set it according to > > the actual used CPU. Then this IOMMU alignment problem would = disappear automatically. What do > > you think? > > Cc'ed Peter since he is the main ARM guy here. >=20 > Do we only see these alignments when we're emulating those old 1k page > processors? No. We see them when we are emulating brand new ARM64. And we see them = only because TARGET_PAGE_BITS is still hardcoded to 10. > If not, should we really be telling a 4k page VM about 1k > aligned memory? Well, first of all, we are not telling about aligned memory at all. = I've researched for the origin of this address, it appears when we are = mapping MSI-X BAR of the VFIO device. My device defines this BAR to be of 2M size. In this case qemu splits = it up into three regions: 1) Region below the MSI-X table (it's called "mmap", for me it's empty = because table offset is 0) 2) MSI-X table itself (20 vectors =3D 0x00000140 bytes for me). 3) Region above the MSi-X table (it's called "mmap msix-hi"). Size for = my case is 2M - REAL_HOST_PAGE_ALIGN(0x140) =3D 0x1FF000. Regions (1) and (3) are passed through and directly mapped by VFIO, = region (2) is emulated. Now, we have PBA. The device says that PBA is placed in memory = specified by the same BAR as table, and its offset is 0x000f0000. PBA is = also emulated by qemu, and as a result it overlaps "mmap msix-hi" = region, which breaks up into two fragments as a consequence: 3a) offset 0x00000 size 0x0EF000 3b) offset 0xEF008 size 0x10FFF8 PBA sits between these fragments, having only 8 bytes in size. Attempt to map (3b) causes the problem. vfio_mmap_region() is expected = to align this up, and it does, but it does it to a minimum possible = granularity for ARM platform, which, as qemu thinks, is always 1K. > If that's all legit, maybe we should be aligning down > rather than up, we know the host memory is at least 4k pages, so > anything in the gap between those alignments should be backed by = memory, > right? You know, i also have this question. Why are we aligning up? What if = the device tries to access parts that we skipped by our alignment? Kind regards, Pavel Fedin Expert Engineer Samsung Electronics Research center Russia