From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alexandre Courbot Subject: Re: [PATCH v2 2/3] drm/ttm: introduce dma cache sync helpers Date: Tue, 24 Jun 2014 19:55:12 +0900 Message-ID: <53A95910.20104@nvidia.com> References: <1403603667-11302-1-git-send-email-acourbot@nvidia.com> <1403603667-11302-3-git-send-email-acourbot@nvidia.com> <20140624100220.GK32514@n2100.arm.linux.org.uk> <53A953E6.2030503@nvidia.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <53A953E6.2030503-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nouveau-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "Nouveau" To: Russell King - ARM Linux Cc: David Airlie , "nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org" , "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org" , Ben Skeggs , "linux-tegra-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" List-Id: linux-tegra@vger.kernel.org T24gMDYvMjQvMjAxNCAwNzozMyBQTSwgQWxleGFuZHJlIENvdXJib3Qgd3JvdGU6Cj4gT24gMDYv MjQvMjAxNCAwNzowMiBQTSwgUnVzc2VsbCBLaW5nIC0gQVJNIExpbnV4IHdyb3RlOgo+PiBPbiBU dWUsIEp1biAyNCwgMjAxNCBhdCAwNjo1NDoyNlBNICswOTAwLCBBbGV4YW5kcmUgQ291cmJvdCB3 cm90ZToKPj4+IEZyb206IEx1Y2FzIFN0YWNoIDxkZXZAbHlueGV5ZS5kZT4KPj4+Cj4+PiBPbiBh cmNoaXRlY3R1cmVzIGZvciB3aGljaCBhY2Nlc3MgdG8gR1BVIG1lbW9yeSBpcyBub24tY29oZXJl bnQsCj4+PiBjYWNoZXMgbmVlZCB0byBiZSBmbHVzaGVkIGFuZCBpbnZhbGlkYXRlZCBleHBsaWNp dGx5IGF0IHRoZQo+Pj4gYXBwcm9wcmlhdGUgcGxhY2VzLiBJbnRyb2R1Y2UgdHdvIHNtYWxsIGhl bHBlcnMgdG8gbWFrZSB0aGluZ3MKPj4+IGVhc3kgZm9yIFRUTS1iYXNlZCBkcml2ZXJzLgo+Pgo+ PiBIYXZlIHlvdSBydW4gdGhpcyB3aXRoIERNQSBBUEkgZGVidWdnaW5nIGVuYWJsZWQ/ICBJIHN1 c3BlY3QgeW91IGhhdmVuJ3QsCj4+IGFuZCBJIHJlY29tbWVuZCB0aGF0IHlvdSBkby4KPgo+ICMg Y2F0IC9zeXMva2VybmVsL2RlYnVnL2RtYS1hcGkvZXJyb3JfY291bnQKPiAxNjI2MjEKPgo+ICji la/CsOKWocKw77yJ4pWv77i1IOKUu+KUgeKUuykKCipwdXRzIHRhYmxlIGJhY2sgb24gaXRzIGZl ZXQqCgpTbywgeWVhaCAtIFRUTSBtZW1vcnkgaXMgbm90IGFsbG9jYXRlZCB1c2luZyB0aGUgRE1B IEFQSSwgaGVuY2Ugd2UgCmNhbm5vdCB1c2UgdGhlIERNQSBBUEkgdG8gc3luYyBpdC4gVGhhbmtz IFJ1c3NlbGwgZm9yIHBvaW50aW5nIGl0IG91dC4KClRoZSBvbmx5IGFsdGVybmF0aXZlIEkgc2Vl IGhlcmUgaXMgdG8gZmx1c2ggdGhlIENQVSBjYWNoZXMgd2hlbiBzeW5jaW5nIApmb3IgdGhlIGRl dmljZSwgYW5kIGludmFsaWRhdGUgdGhlbSBmb3IgdGhlIG90aGVyIGRpcmVjdGlvbi4gT2YgY291 cnNlIAppZiB0aGUgZGV2aWNlIGhhcyBjYWNoZXMgb24gaXRzIHNpZGUgYXMgd2VsbCB0aGUgb3Bw b3NpdGUgb3BlcmF0aW9uIG11c3QgCmFsc28gYmUgZG9uZSBmb3IgaXQuIEd1ZXNzIHRoZSBvbmx5 IHdheSBpcyB0byBoYW5kbGUgaXQgYWxsIGJ5IG91cnNlbHZlcyAKaGVyZS4gOi8KX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KTm91dmVhdSBtYWlsaW5nIGxp c3QKTm91dmVhdUBsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cDovL2xpc3RzLmZyZWVkZXNrdG9w Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL25vdXZlYXUK From mboxrd@z Thu Jan 1 00:00:00 1970 From: acourbot@nvidia.com (Alexandre Courbot) Date: Tue, 24 Jun 2014 19:55:12 +0900 Subject: [PATCH v2 2/3] drm/ttm: introduce dma cache sync helpers In-Reply-To: <53A953E6.2030503@nvidia.com> References: <1403603667-11302-1-git-send-email-acourbot@nvidia.com> <1403603667-11302-3-git-send-email-acourbot@nvidia.com> <20140624100220.GK32514@n2100.arm.linux.org.uk> <53A953E6.2030503@nvidia.com> Message-ID: <53A95910.20104@nvidia.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On 06/24/2014 07:33 PM, Alexandre Courbot wrote: > On 06/24/2014 07:02 PM, Russell King - ARM Linux wrote: >> On Tue, Jun 24, 2014 at 06:54:26PM +0900, Alexandre Courbot wrote: >>> From: Lucas Stach >>> >>> On architectures for which access to GPU memory is non-coherent, >>> caches need to be flushed and invalidated explicitly at the >>> appropriate places. Introduce two small helpers to make things >>> easy for TTM-based drivers. >> >> Have you run this with DMA API debugging enabled? I suspect you haven't, >> and I recommend that you do. > > # cat /sys/kernel/debug/dma-api/error_count > 162621 > > (??????? ???) *puts table back on its feet* So, yeah - TTM memory is not allocated using the DMA API, hence we cannot use the DMA API to sync it. Thanks Russell for pointing it out. The only alternative I see here is to flush the CPU caches when syncing for the device, and invalidate them for the other direction. Of course if the device has caches on its side as well the opposite operation must also be done for it. Guess the only way is to handle it all by ourselves here. :/ From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753541AbaFXKzS (ORCPT ); Tue, 24 Jun 2014 06:55:18 -0400 Received: from hqemgate15.nvidia.com ([216.228.121.64]:19265 "EHLO hqemgate15.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751758AbaFXKzQ convert rfc822-to-8bit (ORCPT ); Tue, 24 Jun 2014 06:55:16 -0400 X-PGP-Universal: processed; by hqnvupgp08.nvidia.com on Tue, 24 Jun 2014 03:49:13 -0700 Message-ID: <53A95910.20104@nvidia.com> Date: Tue, 24 Jun 2014 19:55:12 +0900 From: Alexandre Courbot Organization: NVIDIA User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Russell King - ARM Linux CC: David Airlie , Ben Skeggs , Lucas Stach , Thierry Reding , "gnurou@gmail.com" , "nouveau@lists.freedesktop.org" , "linux-kernel@vger.kernel.org" , "dri-devel@lists.freedesktop.org" , "linux-tegra@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" Subject: Re: [PATCH v2 2/3] drm/ttm: introduce dma cache sync helpers References: <1403603667-11302-1-git-send-email-acourbot@nvidia.com> <1403603667-11302-3-git-send-email-acourbot@nvidia.com> <20140624100220.GK32514@n2100.arm.linux.org.uk> <53A953E6.2030503@nvidia.com> In-Reply-To: <53A953E6.2030503@nvidia.com> X-NVConfidentiality: public Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8BIT Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 06/24/2014 07:33 PM, Alexandre Courbot wrote: > On 06/24/2014 07:02 PM, Russell King - ARM Linux wrote: >> On Tue, Jun 24, 2014 at 06:54:26PM +0900, Alexandre Courbot wrote: >>> From: Lucas Stach >>> >>> On architectures for which access to GPU memory is non-coherent, >>> caches need to be flushed and invalidated explicitly at the >>> appropriate places. Introduce two small helpers to make things >>> easy for TTM-based drivers. >> >> Have you run this with DMA API debugging enabled? I suspect you haven't, >> and I recommend that you do. > > # cat /sys/kernel/debug/dma-api/error_count > 162621 > > (╯°□°）╯︵ ┻━┻) *puts table back on its feet* So, yeah - TTM memory is not allocated using the DMA API, hence we cannot use the DMA API to sync it. Thanks Russell for pointing it out. The only alternative I see here is to flush the CPU caches when syncing for the device, and invalidate them for the other direction. Of course if the device has caches on its side as well the opposite operation must also be done for it. Guess the only way is to handle it all by ourselves here. :/