From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jerin Jacob Subject: Re: [PATCH] eal: disable IOVA mode detection by default Date: Wed, 1 Nov 2017 12:24:32 +0530 Message-ID: <20171101065431.GA25152@jerin> References: <20171101010726.17781-1-ferruh.yigit@intel.com> <20171101040251.GA13569@jerin> <80719d01-7859-e6ac-ee1e-964a04357098@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Thomas Monjalon , Bruce Richardson , Sergio Gonzalez Monroy , dev@dpdk.org, Jianfeng Tan , Santosh Shukla To: Ferruh Yigit Return-path: Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on0056.outbound.protection.outlook.com [104.47.36.56]) by dpdk.org (Postfix) with ESMTP id 3521A1B213 for ; Wed, 1 Nov 2017 07:54:58 +0100 (CET) Content-Disposition: inline In-Reply-To: <80719d01-7859-e6ac-ee1e-964a04357098@intel.com> List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" -----Original Message----- > Date: Tue, 31 Oct 2017 23:21:18 -0700 > From: Ferruh Yigit > To: Jerin Jacob > CC: Thomas Monjalon , Bruce Richardson > , Sergio Gonzalez Monroy > , dev@dpdk.org, Jianfeng Tan > , Santosh Shukla > > Subject: Re: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default > User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 > Thunderbird/52.4.0 > > On 10/31/2017 9:02 PM, Jerin Jacob wrote: > > -----Original Message----- > >> Date: Wed, 1 Nov 2017 01:07:26 +0000 > >> From: Ferruh Yigit > >> To: Thomas Monjalon , Bruce Richardson > >> , Sergio Gonzalez Monroy > >> > >> CC: dev@dpdk.org, Ferruh Yigit , Jianfeng Tan > >> , Santosh Shukla > >> > >> Subject: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default > >> X-Mailer: git-send-email 2.13.6 > >> > >> Fix kernel crash with KNI because KNI requires physical addresses. > > > > The actual fix would be to make KNI IOMMU aware based on the DPDK mode. > > > > ie. On slow path, > > > > /* Get iommu domain for iova to physical addr conversion */ > > if (rte_eal_iova_mode() == RTE_IOVA_VA) > > kni->iommu_domain = iommu_get_domain_for_dev(dev); > > else > > kni->iommu_domain = NULL; > > > > On fast path, > > > > static inline u64 kni_iova_to_phys(struct ... *kni, dma_addr_t dma_addr) > > { > > /* Translation is installed only when IOMMU is present */ > > if (kni->iommu_domain) > > return iommu_iova_to_phys(kni->iommu_domain, dma_addr); > > return dma_addr; > > } > > > >> > >> A config option introduced to disable IOVA mode detection and to set it > >> to physical address by default. Disabling config option will enable IOVA > >> mode detection. > >> > >> When there is no intension to use KNI, it is safe to enable detection. > >> > >> Config option disable IOVA mode detection by default to be sure only who > >> is aware of result enable it. > >> > >> Fixes: 72d013644bd6 ("mem: honor IOVA mode in malloc virt2phy") > >> > >> Signed-off-by: Ferruh Yigit > >> --- > >> Cc: Jianfeng Tan > >> Cc: Santosh Shukla > >> Cc: Thomas Monjalon > >> --- > >> config/common_base | 5 +++++ > >> lib/librte_eal/bsdapp/eal/eal.c | 4 ++++ > >> lib/librte_eal/linuxapp/eal/eal.c | 4 ++++ > >> 3 files changed, 13 insertions(+) > >> > >> diff --git a/config/common_base b/config/common_base > >> index 82ee75456..903e7685b 100644 > >> --- a/config/common_base > >> +++ b/config/common_base > >> @@ -107,6 +107,11 @@ CONFIG_RTE_MALLOC_DEBUG=n > >> CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n > >> > >> # > >> +# Disabling PHYS_IOVA may crash kernel for KNI, use with caution > >> +# > >> +CONFIG_RTE_EAL_USE_PHYS_IOVA=y > > > > Defeat the purpose of all dynamic probing scheme. > > Either we can fix the KNI or revert the following patch for this release. > > > > http://dpdk.org/commit/f37dfab2 > > This commit just enables IOVA VA mode for Intel drivers, that is how I can able > to observe the issue, but it is not the source of the problem. Reverting that > commit will not solve KNI crash with any other PMD that enables IOVA VA mode. I don't understand why a PMD needs to enable IOVA_VA if it can support IOVA_PA. IMO, IOVA_VA should be enabled only for those device it can WORK ONLY on IOVA_VA mode. Forget about KNI, If we set CONFIG_RTE_EAL_USE_PHYS_IOVA as y then the normal stuff wont work for if PMD can operate only in IOVA_VA mode(like octeontx). Regarding the KNI crash, it can be avoid by first checking the exiting mode(rte_eal_iova_mode()). i.e since legacy driver like KNI need real physical address to work "now", it can grace full exit on the init time if mode == IOVA_VA; > > Related to the KNI, iommu is not involved at all, I am not clear with your above > suggestion, physical address is required for kernelspace - userspace communication. vhost-kernel addressed this case with IOVA as VA. I need to spend cycles on what takes to remove physical address dependency from KNI. But someone can add that support if it is required in future. > KNI uses physical address for two things: > > 1- Creates a buffer in userspace, a memzone, and shares its physical address to > the kernel, so that both kernel and app can access same buffer. > > The question is, even all devices supports IOVA VA mode, why creating a memzone > that has real physical address info is not possible anymore? > For KNI case this is not related at all with what devices supports. > > 2- For each mbuf that will be sent to kernel, dpdk app puts physical address of > that mbuf into shared buffer, so that kernel can access it. So that kernel can > access to mbuf data that can be coming from any mempool. > For this the physical address of the mbuf is required. > > > Overall KNI needs to know physical address of mbufs and memzone. > > This patch provides a way to have old behavior with a config option, so that KNI > can work and anyone who needs to create memzone with physical address can a way > to have it, and all PMDs will work fine. > > If there is no need for these, it is possible to disable config and dynamic > probing scheme is already there. > > Thanks, > ferruh