From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jike Song Subject: Re: [RFC PATCH v2 3/3] VFIO: Type1 IOMMU mapping support for vGPU Date: Thu, 10 Mar 2016 11:10:10 +0800 Message-ID: <56E0E592.20008@intel.com> References: <1456244666-25369-1-git-send-email-kwankhede@nvidia.com> <1456244666-25369-3-git-send-email-kwankhede@nvidia.com> <56D6A68A.50004@intel.com> <20160304070025.GA32070@nvidia.com> <20160308003139.GA22106@nvidia.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Kirti Wankhede , Alex Williamson , pbonzini@redhat.com, kraxel@redhat.com, qemu-devel@nongnu.org, kvm@vger.kernel.org, kevin.tian@intel.com, shuai.ruan@intel.com, zhiyuan.lv@intel.com To: Neo Jia Return-path: Received: from mga11.intel.com ([192.55.52.93]:13764 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932750AbcCJDKF (ORCPT ); Wed, 9 Mar 2016 22:10:05 -0500 In-Reply-To: <20160308003139.GA22106@nvidia.com> Sender: kvm-owner@vger.kernel.org List-ID: On 03/08/2016 08:31 AM, Neo Jia wrote: > On Mon, Mar 07, 2016 at 02:07:15PM +0800, Jike Song wrote: >> Hi Neo, >> >> On Fri, Mar 4, 2016 at 3:00 PM, Neo Jia wrote: >>> On Wed, Mar 02, 2016 at 04:38:34PM +0800, Jike Song wrote: >>>> On 02/24/2016 12:24 AM, Kirti Wankhede wrote: >>>>> + vgpu_dma->size = map->size; >>>>> + >>>>> + vgpu_link_dma(vgpu_iommu, vgpu_dma); >>>> >>>> Hi Kirti & Neo, >>>> >>>> seems that no one actually setup mappings for IOMMU here? >>>> >>> >>> Hi Jike, >>> >>> Yes. >>> >>> The actual mapping should be done by the host kernel driver after calling the >>> translation/pinning API vgpu_dma_do_translate. >> >> Thanks for the reply. I mis-deleted the mail in my intel account, so >> reply with private mail account, sorry for that. >> >> >> In vgpu_dma_do_translate(): >> >> for (i = 0; i < count; i++) { >> {snip} >> dma_addr_t iova = gfn_buffer[i] << PAGE_SHIFT; >> vgpu_dma = vgpu_find_dma(vgpu_iommu, iova, 0 /* size */); >> >> remote_vaddr = vgpu_dma->vaddr + iova - vgpu_dma->iova; >> if (get_user_pages_unlocked(NULL, mm, remote_vaddr, 1, 1, 0, page) == 1) { >> pfn = page_to_pfn(page[0]); >> } >> gfn_buffer[i] = pfn; >> } >> >> If I understand correctly, the purpose of above code, is given an >> array of gfns, try to pin & return associated pfns. There is still no >> IOMMU mappings here. > > Yes. > Thanks for the conformation. >> Is it supposed to be the caller who should set >> up IOMMU by DMA api such as dma_map_page(), after calling >> vgpu_dma_do_translate()? >> > > Don't think you need to call dma_map_page here. Once you have the pfn available > to your GPU kernel driver, you can just go ahead to setup the mapping as you > normally do such as calling pci_map_sg and its friends. > Technically it's definitely OK to call DMA API from the caller rather than here, however personally I think it is a bit counter-intuitive: IOMMU page tables should be constructed within the VFIO IOMMU driver. > Thanks, > Neo -- Thanks, Jike From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44143) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1adqzU-0004TR-Tu for qemu-devel@nongnu.org; Wed, 09 Mar 2016 22:10:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1adqzS-0006cx-8z for qemu-devel@nongnu.org; Wed, 09 Mar 2016 22:10:20 -0500 Received: from mga03.intel.com ([134.134.136.65]:24972) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1adqzS-0006cG-2W for qemu-devel@nongnu.org; Wed, 09 Mar 2016 22:10:18 -0500 Message-ID: <56E0E592.20008@intel.com> Date: Thu, 10 Mar 2016 11:10:10 +0800 From: Jike Song MIME-Version: 1.0 References: <1456244666-25369-1-git-send-email-kwankhede@nvidia.com> <1456244666-25369-3-git-send-email-kwankhede@nvidia.com> <56D6A68A.50004@intel.com> <20160304070025.GA32070@nvidia.com> <20160308003139.GA22106@nvidia.com> In-Reply-To: <20160308003139.GA22106@nvidia.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC PATCH v2 3/3] VFIO: Type1 IOMMU mapping support for vGPU List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Neo Jia Cc: shuai.ruan@intel.com, kevin.tian@intel.com, Alex Williamson , kvm@vger.kernel.org, qemu-devel@nongnu.org, Kirti Wankhede , kraxel@redhat.com, pbonzini@redhat.com, zhiyuan.lv@intel.com On 03/08/2016 08:31 AM, Neo Jia wrote: > On Mon, Mar 07, 2016 at 02:07:15PM +0800, Jike Song wrote: >> Hi Neo, >> >> On Fri, Mar 4, 2016 at 3:00 PM, Neo Jia wrote: >>> On Wed, Mar 02, 2016 at 04:38:34PM +0800, Jike Song wrote: >>>> On 02/24/2016 12:24 AM, Kirti Wankhede wrote: >>>>> + vgpu_dma->size = map->size; >>>>> + >>>>> + vgpu_link_dma(vgpu_iommu, vgpu_dma); >>>> >>>> Hi Kirti & Neo, >>>> >>>> seems that no one actually setup mappings for IOMMU here? >>>> >>> >>> Hi Jike, >>> >>> Yes. >>> >>> The actual mapping should be done by the host kernel driver after calling the >>> translation/pinning API vgpu_dma_do_translate. >> >> Thanks for the reply. I mis-deleted the mail in my intel account, so >> reply with private mail account, sorry for that. >> >> >> In vgpu_dma_do_translate(): >> >> for (i = 0; i < count; i++) { >> {snip} >> dma_addr_t iova = gfn_buffer[i] << PAGE_SHIFT; >> vgpu_dma = vgpu_find_dma(vgpu_iommu, iova, 0 /* size */); >> >> remote_vaddr = vgpu_dma->vaddr + iova - vgpu_dma->iova; >> if (get_user_pages_unlocked(NULL, mm, remote_vaddr, 1, 1, 0, page) == 1) { >> pfn = page_to_pfn(page[0]); >> } >> gfn_buffer[i] = pfn; >> } >> >> If I understand correctly, the purpose of above code, is given an >> array of gfns, try to pin & return associated pfns. There is still no >> IOMMU mappings here. > > Yes. > Thanks for the conformation. >> Is it supposed to be the caller who should set >> up IOMMU by DMA api such as dma_map_page(), after calling >> vgpu_dma_do_translate()? >> > > Don't think you need to call dma_map_page here. Once you have the pfn available > to your GPU kernel driver, you can just go ahead to setup the mapping as you > normally do such as calling pci_map_sg and its friends. > Technically it's definitely OK to call DMA API from the caller rather than here, however personally I think it is a bit counter-intuitive: IOMMU page tables should be constructed within the VFIO IOMMU driver. > Thanks, > Neo -- Thanks, Jike