From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jerin Jacob Subject: Re: [PATCH] eal: disable IOVA mode detection by default Date: Wed, 1 Nov 2017 13:56:44 +0530 Message-ID: <20171101082642.GA30967@jerin> References: <20171101010726.17781-1-ferruh.yigit@intel.com> <20171101040251.GA13569@jerin> <80719d01-7859-e6ac-ee1e-964a04357098@intel.com> <20171101065431.GA25152@jerin> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Ferruh Yigit , Thomas Monjalon , Bruce Richardson , Sergio Gonzalez Monroy , dev@dpdk.org, Santosh Shukla To: "Tan, Jianfeng" Return-path: Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-cys01nam02on0064.outbound.protection.outlook.com [104.47.37.64]) by dpdk.org (Postfix) with ESMTP id 131181B299 for ; Wed, 1 Nov 2017 09:27:10 +0100 (CET) Content-Disposition: inline In-Reply-To: List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" -----Original Message----- > Date: Wed, 1 Nov 2017 15:39:01 +0800 > From: "Tan, Jianfeng" > To: Jerin Jacob , Ferruh Yigit > > CC: Thomas Monjalon , Bruce Richardson > , Sergio Gonzalez Monroy > , dev@dpdk.org, Santosh Shukla > > Subject: Re: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default > User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 > Thunderbird/45.8.0 > > > > On 11/1/2017 2:54 PM, Jerin Jacob wrote: > > -----Original Message----- > > > Date: Tue, 31 Oct 2017 23:21:18 -0700 > > > From: Ferruh Yigit > > > To: Jerin Jacob > > > CC: Thomas Monjalon , Bruce Richardson > > > , Sergio Gonzalez Monroy > > > , dev@dpdk.org, Jianfeng Tan > > > , Santosh Shukla > > > > > > Subject: Re: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default > > > User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 > > > Thunderbird/52.4.0 > > > > > > On 10/31/2017 9:02 PM, Jerin Jacob wrote: > > > > -----Original Message----- > > > > > Date: Wed, 1 Nov 2017 01:07:26 +0000 > > > > > From: Ferruh Yigit > > > > > To: Thomas Monjalon , Bruce Richardson > > > > > , Sergio Gonzalez Monroy > > > > > > > > > > CC: dev@dpdk.org, Ferruh Yigit , Jianfeng Tan > > > > > , Santosh Shukla > > > > > > > > > > Subject: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default > > > > > X-Mailer: git-send-email 2.13.6 > > > > > > > > > > Fix kernel crash with KNI because KNI requires physical addresses. > > > > The actual fix would be to make KNI IOMMU aware based on the DPDK mode. > > > > > > > > ie. On slow path, > > > > > > > > /* Get iommu domain for iova to physical addr conversion */ > > > > if (rte_eal_iova_mode() == RTE_IOVA_VA) > > > > kni->iommu_domain = iommu_get_domain_for_dev(dev); > > > > else > > > > kni->iommu_domain = NULL; > > > > > > > > On fast path, > > > > > > > > static inline u64 kni_iova_to_phys(struct ... *kni, dma_addr_t dma_addr) > > > > { > > > > /* Translation is installed only when IOMMU is present */ > > > > if (kni->iommu_domain) > > > > return iommu_iova_to_phys(kni->iommu_domain, dma_addr); > > > > return dma_addr; > > > > } > > > > > > > > > A config option introduced to disable IOVA mode detection and to set it > > > > > to physical address by default. Disabling config option will enable IOVA > > > > > mode detection. > > > > > > > > > > When there is no intension to use KNI, it is safe to enable detection. > > > > > > > > > > Config option disable IOVA mode detection by default to be sure only who > > > > > is aware of result enable it. > > > > > > > > > > Fixes: 72d013644bd6 ("mem: honor IOVA mode in malloc virt2phy") > > > > > > > > > > Signed-off-by: Ferruh Yigit > > > > > --- > > > > > Cc: Jianfeng Tan > > > > > Cc: Santosh Shukla > > > > > Cc: Thomas Monjalon > > > > > --- > > > > > config/common_base | 5 +++++ > > > > > lib/librte_eal/bsdapp/eal/eal.c | 4 ++++ > > > > > lib/librte_eal/linuxapp/eal/eal.c | 4 ++++ > > > > > 3 files changed, 13 insertions(+) > > > > > > > > > > diff --git a/config/common_base b/config/common_base > > > > > index 82ee75456..903e7685b 100644 > > > > > --- a/config/common_base > > > > > +++ b/config/common_base > > > > > @@ -107,6 +107,11 @@ CONFIG_RTE_MALLOC_DEBUG=n > > > > > CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n > > > > > # > > > > > +# Disabling PHYS_IOVA may crash kernel for KNI, use with caution > > > > > +# > > > > > +CONFIG_RTE_EAL_USE_PHYS_IOVA=y > > > > Defeat the purpose of all dynamic probing scheme. > > > > Either we can fix the KNI or revert the following patch for this release. > > > > > > > > http://dpdk.org/commit/f37dfab2 > > > This commit just enables IOVA VA mode for Intel drivers, that is how I can able > > > to observe the issue, but it is not the source of the problem. Reverting that > > > commit will not solve KNI crash with any other PMD that enables IOVA VA mode. > > I don't understand why a PMD needs to enable IOVA_VA if it can support IOVA_PA. > > IMO, IOVA_VA should be enabled only for those device it can WORK ONLY on > > IOVA_VA mode. > > IOVA_VA is also necessary if we don't want/have root privilege to run DPDK > app. OK. > > > Forget about KNI, If we set CONFIG_RTE_EAL_USE_PHYS_IOVA > > as y then the normal stuff wont work for if PMD can operate only in > > IOVA_VA mode(like octeontx). > > Yes, agreed. > > > Regarding the KNI crash, it can be avoid by first checking the exiting > > mode(rte_eal_iova_mode()). i.e since legacy driver like KNI need real > > physical address to work "now", it can grace full exit on the init time if > > mode == IOVA_VA; > > For the KNI crash, do we really need to correct the case: make those PMDs > (requires IOVA_VA) also work well with KNI? > - If no, we just need to report RTE_PCI_DRV_IOVA_AS_VA smartly. > - If yes, we shall make KNI work with VA. I would say "yes". But for this immediate release, I would say "no". > > Thanks, > Jianfeng >