From mboxrd@z Thu Jan 1 00:00:00 1970 From: santosh Subject: Re: [PATCH 00/10] Infrastructure to detect iova mapping on the bus Date: Tue, 4 Jul 2017 10:11:35 +0530 Message-ID: <2beb2e96-3938-05ab-d3e7-3f569f62d704@caviumnetworks.com> References: <20170608110513.22548-1-santosh.shukla@caviumnetworks.com> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Cc: jerin.jacob@caviumnetworks.com, hemant.agrawal@nxp.com, shreyansh.jain@nxp.com, gaetan.rivet@6wind.com To: thomas@monjalon.net, bruce.richardson@intel.com, dev@dpdk.org Return-path: Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-cys01nam02on0066.outbound.protection.outlook.com [104.47.37.66]) by dpdk.org (Postfix) with ESMTP id 8E0322935 for ; Tue, 4 Jul 2017 06:41:55 +0200 (CEST) In-Reply-To: <20170608110513.22548-1-santosh.shukla@caviumnetworks.com> List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, On Thursday 08 June 2017 04:35 PM, Santosh Shukla wrote: > Q) Why do we need such infrastructure? > > A) Some NPU hardware like OCTEONTX follows push model to get the packet > from the pktio device. Where packet allocation and freeing done > by the HW. Since HW can operate only on IOVA with help of SMMU/IOMMU, > when packet receives from the Ethernet device, it is the IOVA address > (which is PA in existing scheme). > > Mapping IOVA as PA is expensive on those HW, where every packet > needs to be converted to VA from PA/IOVA. > > This patchset proposes the method to autodetect the preferred > IOVA mode for a device. Summary of IOVA scheme: > - If all the devices are iommu capable and support IOMMU > capable driver then selects IOVA_VA. > - If any of the devices are non-iommu then use default IOVA > scheme ie. IOVA_PA. > - If no device found then IOVA scheme would be > IOVA_DC (Don't care). > > To achieve that, two global APIs introduced: > - rte_bus_get_iommu_class > - rte_pci_get_iommu_class > > Return values for those APIs are: > enum rte_iova_mod { > RTE_IOVA_DC, /* Don't care */ > RTE_IOVA_PA, > RTE_IOVA_VA > } > > Those are the bus policy for selecting IOVA mode. In case user > want to override bus IOVA mapping then added an EAL option > "--iova-mode=". User to pass string format 'pa' --> IOVA_PA, > 'va' --> IOVA_VA. > > To support new eal option, adding global API: > - rte_eal_iova_mode > > Patch Summary: > 2) 1st - 2th patch: Adds infrastructure in linuxapp and bsdapp > layer. > 1) 3rd patch: Introduces global bus api named rte_bus_get_iommu_class. > 3) 4th patch: Add new eal option called --iova-mode=. > 4) 5th - 6th patch: Logic to detect iova scheme. > 5) 9th patch: Check IOVA mode before programing vfio dma_map.iova. > Default scheme is IOVA_PA. > 6) 10th-12th patch: Check for IOVA_VA mode in below APIs > - rte_mem_virt2phy > - rte_mempool_virt2phy > - rte_malloc_virt2phy > If set then return paddr=vaddr, else return value from default > implementation. > > Test History: > - Tested for x86/XL710 40G NIC card for both modes (iova_va/pa). > - Tested for arm64/thunderx vNIC Integrated NIC for both modes > - Tested for arm64/Octeontx integrated NICs for only > Iova_va mode(It supports only one mode.) > - Ran standalone tests like mempool_autotest, mbuf_autotest. > - Verified for Doxygen. > > Work History: > Refer prev RFC proposal[1]. > > Noticed false positive checkpatch error: > - WARNING: quoted string split across lines > #60: FILE: lib/librte_eal/common/eal_common_bus.c:164: > + RTE_LOG(INFO, EAL, "Bus (%s) iommu class of" > + " devices not found.\n", bus->name); > > - WARNING: LINUX_VERSION_CODE should be avoided, code should be for the version to which it is merged > #86: FILE: lib/librte_eal/linuxapp/eal/eal_vfio.c:822: > +#if LINUX_VERSION_CODE < KERNEL_VERSION(4, 2, 0) > > Thanks. > > [1] http://dpdk.org/dev/patchwork/patch/24549/ > Ping? Thanks.