From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: Use of pci_map_page in nouveau, radeon TTM. Date: Tue, 1 Oct 2013 10:14:16 -0400 Message-ID: <20131001141416.GB5913@phenom.dumpdata.com> References: <524AA0F0.6000000@vmware.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from aserp1040.oracle.com (aserp1040.oracle.com [141.146.126.69]) by gabe.freedesktop.org (Postfix) with ESMTP id 9C461E7545 for ; Tue, 1 Oct 2013 07:14:20 -0700 (PDT) Content-Disposition: inline In-Reply-To: <524AA0F0.6000000@vmware.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dri-devel-bounces+sf-dri-devel=m.gmane.org@lists.freedesktop.org Errors-To: dri-devel-bounces+sf-dri-devel=m.gmane.org@lists.freedesktop.org To: Thomas Hellstrom Cc: Jerome Glisse , "dri-devel@lists.freedesktop.org" List-Id: dri-devel@lists.freedesktop.org On Tue, Oct 01, 2013 at 12:16:16PM +0200, Thomas Hellstrom wrote: > Jerome, Konrad > > Forgive an ignorant question, but it appears like both Nouveau and > Radeon may use pci_map_page() when populating TTMs on > pages obtained using the ordinary (not DMA pool). These pages will, > if I understand things correctly, not be pages allocated with > DMA_ALLOC_COHERENT. Not always. That depends if the SWIOTLB buffer has been enabled. Which happens if you have Calgary IOMMU, AMD GART and if you run under Xen. > > From what I understand, at least for the corresponding > dma_map_page() it's illegal for the CPU to access these pages > without calling > dma_sync_xx_for_cpu(). And before the device is allowed to access > them again, you need to call dma_sync_xx_for_device(). Correct. > So mapping for PCI really invalidates the TTM interleaved CPU / > device access model. Unless you use the TTM DMA one which allocates them from the coherent pool - in which case they are already mapped. Granted the part of using DMA export/import API is not finished (so moving from TTM pool to a V4L for example) and it will blow up with the right mix. > > Or did I miss something here? That is it. But for most of the use cases the drivers have been able to skirt this restriction b/c the pci_map_page/pci_unmap_page setup a DMA mapping that is static (until the pci_unmap_page) and on x86 the memory is coherent. So the map is good irregardless of the PCI devices. Naturally if you have multitple IOMMUs per bridge this all falls apart :-( This all falls flat also with non-coherent memory and I believe that is what some of the PA-RISC folks are hitting their heads against. And probably also on ARM once they start using these chipsets. > > Thanks, > Thomas