From mboxrd@z Thu Jan 1 00:00:00 1970 From: Neo Jia Subject: Re: [RFC PATCH v2 3/3] VFIO: Type1 IOMMU mapping support for vGPU Date: Mon, 7 Mar 2016 16:31:39 -0800 Message-ID: <20160308003139.GA22106@nvidia.com> References: <1456244666-25369-1-git-send-email-kwankhede@nvidia.com> <1456244666-25369-3-git-send-email-kwankhede@nvidia.com> <56D6A68A.50004@intel.com> <20160304070025.GA32070@nvidia.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Cc: shuai.ruan@intel.com, Jike Song , kvm@vger.kernel.org, qemu-devel@nongnu.org, Kirti Wankhede , kevin.tian@intel.com, Alex Williamson , kraxel@redhat.com, pbonzini@redhat.com, zhiyuan.lv@intel.com To: Return-path: Content-Disposition: inline In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org Sender: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org List-Id: kvm.vger.kernel.org On Mon, Mar 07, 2016 at 02:07:15PM +0800, Jike Song wrote: > Hi Neo, > > On Fri, Mar 4, 2016 at 3:00 PM, Neo Jia wrote: > > On Wed, Mar 02, 2016 at 04:38:34PM +0800, Jike Song wrote: > >> On 02/24/2016 12:24 AM, Kirti Wankhede wrote: > >> > + vgpu_dma->size = map->size; > >> > + > >> > + vgpu_link_dma(vgpu_iommu, vgpu_dma); > >> > >> Hi Kirti & Neo, > >> > >> seems that no one actually setup mappings for IOMMU here? > >> > > > > Hi Jike, > > > > Yes. > > > > The actual mapping should be done by the host kernel driver after calling the > > translation/pinning API vgpu_dma_do_translate. > > Thanks for the reply. I mis-deleted the mail in my intel account, so > reply with private mail account, sorry for that. > > > In vgpu_dma_do_translate(): > > for (i = 0; i < count; i++) { > {snip} > dma_addr_t iova = gfn_buffer[i] << PAGE_SHIFT; > vgpu_dma = vgpu_find_dma(vgpu_iommu, iova, 0 /* size */); > > remote_vaddr = vgpu_dma->vaddr + iova - vgpu_dma->iova; > if (get_user_pages_unlocked(NULL, mm, remote_vaddr, 1, 1, 0, page) == 1) { > pfn = page_to_pfn(page[0]); > } > gfn_buffer[i] = pfn; > } > > If I understand correctly, the purpose of above code, is given an > array of gfns, try to pin & return associated pfns. There is still no > IOMMU mappings here. Yes. > Is it supposed to be the caller who should set > up IOMMU by DMA api such as dma_map_page(), after calling > vgpu_dma_do_translate()? > Don't think you need to call dma_map_page here. Once you have the pfn available to your GPU kernel driver, you can just go ahead to setup the mapping as you normally do such as calling pci_map_sg and its friends. Thanks, Neo > > -- > Thanks, > Jike From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43250) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ad5Yy-00087N-7e for qemu-devel@nongnu.org; Mon, 07 Mar 2016 19:31:49 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ad5Yt-00068u-3f for qemu-devel@nongnu.org; Mon, 07 Mar 2016 19:31:48 -0500 Received: from hqemgate15.nvidia.com ([216.228.121.64]:17438) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ad5Ys-000675-Rr for qemu-devel@nongnu.org; Mon, 07 Mar 2016 19:31:43 -0500 Date: Mon, 7 Mar 2016 16:31:39 -0800 From: Neo Jia Message-ID: <20160308003139.GA22106@nvidia.com> References: <1456244666-25369-1-git-send-email-kwankhede@nvidia.com> <1456244666-25369-3-git-send-email-kwankhede@nvidia.com> <56D6A68A.50004@intel.com> <20160304070025.GA32070@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: Subject: Re: [Qemu-devel] [RFC PATCH v2 3/3] VFIO: Type1 IOMMU mapping support for vGPU List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: jike.song@intel.com Cc: shuai.ruan@intel.com, kvm@vger.kernel.org, qemu-devel@nongnu.org, Kirti Wankhede , kevin.tian@intel.com, Alex Williamson , kraxel@redhat.com, pbonzini@redhat.com, zhiyuan.lv@intel.com On Mon, Mar 07, 2016 at 02:07:15PM +0800, Jike Song wrote: > Hi Neo, > > On Fri, Mar 4, 2016 at 3:00 PM, Neo Jia wrote: > > On Wed, Mar 02, 2016 at 04:38:34PM +0800, Jike Song wrote: > >> On 02/24/2016 12:24 AM, Kirti Wankhede wrote: > >> > + vgpu_dma->size = map->size; > >> > + > >> > + vgpu_link_dma(vgpu_iommu, vgpu_dma); > >> > >> Hi Kirti & Neo, > >> > >> seems that no one actually setup mappings for IOMMU here? > >> > > > > Hi Jike, > > > > Yes. > > > > The actual mapping should be done by the host kernel driver after calling the > > translation/pinning API vgpu_dma_do_translate. > > Thanks for the reply. I mis-deleted the mail in my intel account, so > reply with private mail account, sorry for that. > > > In vgpu_dma_do_translate(): > > for (i = 0; i < count; i++) { > {snip} > dma_addr_t iova = gfn_buffer[i] << PAGE_SHIFT; > vgpu_dma = vgpu_find_dma(vgpu_iommu, iova, 0 /* size */); > > remote_vaddr = vgpu_dma->vaddr + iova - vgpu_dma->iova; > if (get_user_pages_unlocked(NULL, mm, remote_vaddr, 1, 1, 0, page) == 1) { > pfn = page_to_pfn(page[0]); > } > gfn_buffer[i] = pfn; > } > > If I understand correctly, the purpose of above code, is given an > array of gfns, try to pin & return associated pfns. There is still no > IOMMU mappings here. Yes. > Is it supposed to be the caller who should set > up IOMMU by DMA api such as dma_map_page(), after calling > vgpu_dma_do_translate()? > Don't think you need to call dma_map_page here. Once you have the pfn available to your GPU kernel driver, you can just go ahead to setup the mapping as you normally do such as calling pci_map_sg and its friends. Thanks, Neo > > -- > Thanks, > Jike