From mboxrd@z Thu Jan 1 00:00:00 1970 From: Shreyansh Jain Subject: Re: [PATCH 00/16] Support externally allocated memory in DPDK Date: Tue, 18 Sep 2018 17:59:56 +0530 Message-ID: References: <9c763791-2c10-fd36-7f50-3df5efbc20ab@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: "dev@dpdk.org" , "laszlo.madarassy@ericsson.com" , "laszlo.vadkerti@ericsson.com" , "andras.kovacs@ericsson.com" , "winnie.tian@ericsson.com" , "daniel.andrasi@ericsson.com" , "janos.kobor@ericsson.com" , "srinath.mannam@broadcom.com" , "scott.branden@broadcom.com" , "ajit.khaparde@broadcom.com" , "keith.wiles@intel.com" , "bruce.richardson@intel.com" , Thomas Monjalon To: "Burakov, Anatoly" , Shahaf Shuler Return-path: Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40043.outbound.protection.outlook.com [40.107.4.43]) by dpdk.org (Postfix) with ESMTP id 8BCBE1041 for ; Tue, 18 Sep 2018 14:30:26 +0200 (CEST) In-Reply-To: Content-Language: en-US List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Monday 17 September 2018 06:30 PM, Burakov, Anatoly wrote: > On 17-Sep-18 1:16 PM, Shahaf Shuler wrote: >> Monday, September 17, 2018 1:07 PM, Burakov, Anatoly: >>> Subject: Re: [dpdk-dev] [PATCH 00/16] Support externally allocated >>> memory >>> in DPDK >>> >>> On 13-Sep-18 8:44 AM, Shahaf Shuler wrote: >> >> [...] >> >>>>> The responsibility to ensure memory is accessible before using it is >>>>> on the shoulders of the user - there is no checking done with regards >>>>> to validity of the memory (nor could there be...). >>>> >>>> That makes sense. However who should be in-charge of mapping this >>> memory for dma access? >>>> The user or internally be the PMD when encounter the first packet or >>>> while >>> traversing the existing mempools? >>>> >>> Hi Shahaf, >>> >>> There are two ways this can be solved. The first way is to perform VFIO >>> mapping automatically on adding/attaching memory. The second is to force >>> user to do it manually. For now, the latter is chosen because user >>> knows best >>> if they intend to do DMA on that memory, but i'm open to suggestions. >> >> I agree with that approach, and will add not only if the mempool is >> for dma or not but also which ports will use this mempool (this can >> effect on the mapping). > > That is perhaps too hardware-specific - this should probably be handled > inside the driver callbacks. > >> However I don't think this is generic enough to use only VFIO. As you >> said, there are some devices not using VFIO for mapping rather some >> proprietary driver utility. >> IMO DPDK should introduce generic and device agnostic APIs to the user. >> >> My suggestion is instead of doing vfio_dma_map that or vfio_dma_unmap >> that have a generic dma_map(uint8_t port, address, len). Each driver >> will register with its own mapping callback (can be vfio_dma_map). >> It can be outside of this series, just wondering the people opinion on >> such approach. > > I don't disagree. I don't like bus/net/etc drivers doing their own thing > with regards to mapping, and i would by far prefer generic way to set up > DMA maps, to which VFIO will be a subscriber. > >> >>> >>> There is an issue with some devices and buses (i.e. bus/fslmc) >>> bypassing EAL >>> VFIO infrastructure and performing their own VFIO/DMA mapping magic, but >>> solving that problem is outside the scope of this patchset. Those >>> devices/buses should fix themselves :) DMA mapping is a very common principle and can be easily be a candidate for lets-make-generic-movement, but, being close to hardware (or hardware specific), it does require the driver to have some flexibility in terms of its eventual implementation. I maintain one of those drivers (bus/fslmc) in DPDK which needs to have special VFIO layer - and from that experience, I can say that VFIO mapping does require some flexibility. SoC semantics are sometimes too complex to pin to general-universally-agreed-standard concept. (or, one can easily call it a 'bug', while it is a 'feature' for others :D) In fact, NXP has another driver (bus/dpaa) which doesn't even work with VFIO - loves to work directly with Phys_addr. And, it is not at a lower priority than one with VFIO. Thus, I really don't think a strongly controlled VFIO mapping should be EAL's responsibility. Failure because of lack of mapping is a driver's problem. >>> >>> When not using VFIO, it's out of our hands anyway. >> >> Why? >> VFIO is not a must requirement for devices in DPDK. > > When i say "out of our hands", what i mean to say is, currently as far > as EAL API is concerned, there is no DMA mapping outside of VFIO. > >> >>> >>> -- >>> Thanks, >>> Anatoly > >