From mboxrd@z Thu Jan 1 00:00:00 1970 From: Thomas Monjalon Subject: Re: [PATCH 00/10] Infrastructure to detect iova mapping on the bus Date: Tue, 04 Jul 2017 12:10:44 +0200 Message-ID: <1695601.K0fVzJ8uRQ@xps> References: <20170608110513.22548-1-santosh.shukla@caviumnetworks.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7Bit Cc: dev@dpdk.org, bruce.richardson@intel.com, jerin.jacob@caviumnetworks.com, hemant.agrawal@nxp.com, shreyansh.jain@nxp.com, gaetan.rivet@6wind.com, jblunck@infradead.org, olivier.matz@6wind.com, jianbo.liu@linaro.org, anatoly.burakov@intel.com, sergio.gonzalez.monroy@intel.com To: Santosh Shukla Return-path: Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by dpdk.org (Postfix) with ESMTP id 9AB512935 for ; Tue, 4 Jul 2017 12:10:47 +0200 (CEST) In-Reply-To: <20170608110513.22548-1-santosh.shukla@caviumnetworks.com> List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Santosh, Let's try to make this proposal clearer in order to have some reviews. 08/06/2017 13:05, Santosh Shukla: > Q) Why do we need such infrastructure? > > A) Some NPU hardware like OCTEONTX follows push model to get the packet > from the pktio device. Where packet allocation and freeing done > by the HW. Since HW can operate only on IOVA with help of SMMU/IOMMU, Some readers may not know IOVA: IO Virtual address. Some explanations: https://www.kernel.org/doc/Documentation/Intel-IOMMU.txt http://vfio.blogspot.fr/2014/08/iommu-groups-inside-and-out.html It must be said that SMMU is equivalent to IOMMU for ARM: https://developer.arm.com/products/system-ip/system-controllers/system-memory-management-unit > when packet receives from the Ethernet device, it is the IOVA address > (which is PA in existing scheme). You mean that we are currently using only Physical Address (PA)? > Mapping IOVA as PA is expensive on those HW, where every packet > needs to be converted to VA from PA/IOVA. Please, could you explain how and where addresses are converted currently? > This patchset proposes the method to autodetect the preferred > IOVA mode for a device. Summary of IOVA scheme: > - If all the devices are iommu capable and support IOMMU > capable driver then selects IOVA_VA. > - If any of the devices are non-iommu then use default IOVA > scheme ie. IOVA_PA. > - If no device found then IOVA scheme would be > IOVA_DC (Don't care). I think you should better describe these modes and how they behave. > To achieve that, two global APIs introduced: > - rte_bus_get_iommu_class > - rte_pci_get_iommu_class > > Return values for those APIs are: > enum rte_iova_mod { > RTE_IOVA_DC, /* Don't care */ > RTE_IOVA_PA, > RTE_IOVA_VA > } > > Those are the bus policy for selecting IOVA mode. In case user > want to override bus IOVA mapping then added an EAL option > "--iova-mode=". User to pass string format 'pa' --> IOVA_PA, > 'va' --> IOVA_VA. > > To support new eal option, adding global API: > - rte_eal_iova_mode > > Patch Summary: > 2) 1st - 2th patch: Adds infrastructure in linuxapp and bsdapp > layer. > 1) 3rd patch: Introduces global bus api named rte_bus_get_iommu_class. > 3) 4th patch: Add new eal option called --iova-mode=. > 4) 5th - 6th patch: Logic to detect iova scheme. > 5) 9th patch: Check IOVA mode before programing vfio dma_map.iova. > Default scheme is IOVA_PA. > 6) 10th-12th patch: Check for IOVA_VA mode in below APIs > - rte_mem_virt2phy > - rte_mempool_virt2phy > - rte_malloc_virt2phy > If set then return paddr=vaddr, else return value from default > implementation.