From mboxrd@z Thu Jan 1 00:00:00 1970 From: Maarten Lankhorst Subject: Re: [Nouveau] [PATCH v2 2/3] drm/ttm: introduce dma cache sync helpers Date: Tue, 24 Jun 2014 14:27:49 +0200 Message-ID: <53A96EC5.3030701@canonical.com> References: <1403603667-11302-1-git-send-email-acourbot@nvidia.com> <1403603667-11302-3-git-send-email-acourbot@nvidia.com> <20140624100220.GK32514@n2100.arm.linux.org.uk> <53A953E6.2030503@nvidia.com> <53A95910.20104@nvidia.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: Alexandre Courbot , Alexandre Courbot Cc: Russell King - ARM Linux , "nouveau@lists.freedesktop.org" , "linux-kernel@vger.kernel.org" , "dri-devel@lists.freedesktop.org" , Ben Skeggs , "linux-tegra@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" List-Id: linux-tegra@vger.kernel.org b3AgMjQtMDYtMTQgMTQ6MjMsIEFsZXhhbmRyZSBDb3VyYm90IHNjaHJlZWY6Cj4gT24gVHVlLCBK dW4gMjQsIDIwMTQgYXQgNzo1NSBQTSwgQWxleGFuZHJlIENvdXJib3QgPGFjb3VyYm90QG52aWRp YS5jb20+IHdyb3RlOgo+PiBPbiAwNi8yNC8yMDE0IDA3OjMzIFBNLCBBbGV4YW5kcmUgQ291cmJv dCB3cm90ZToKPj4+IE9uIDA2LzI0LzIwMTQgMDc6MDIgUE0sIFJ1c3NlbGwgS2luZyAtIEFSTSBM aW51eCB3cm90ZToKPj4+PiBPbiBUdWUsIEp1biAyNCwgMjAxNCBhdCAwNjo1NDoyNlBNICswOTAw LCBBbGV4YW5kcmUgQ291cmJvdCB3cm90ZToKPj4+Pj4gRnJvbTogTHVjYXMgU3RhY2ggPGRldkBs eW54ZXllLmRlPgo+Pj4+Pgo+Pj4+PiBPbiBhcmNoaXRlY3R1cmVzIGZvciB3aGljaCBhY2Nlc3Mg dG8gR1BVIG1lbW9yeSBpcyBub24tY29oZXJlbnQsCj4+Pj4+IGNhY2hlcyBuZWVkIHRvIGJlIGZs dXNoZWQgYW5kIGludmFsaWRhdGVkIGV4cGxpY2l0bHkgYXQgdGhlCj4+Pj4+IGFwcHJvcHJpYXRl IHBsYWNlcy4gSW50cm9kdWNlIHR3byBzbWFsbCBoZWxwZXJzIHRvIG1ha2UgdGhpbmdzCj4+Pj4+ IGVhc3kgZm9yIFRUTS1iYXNlZCBkcml2ZXJzLgo+Pj4+Cj4+Pj4gSGF2ZSB5b3UgcnVuIHRoaXMg d2l0aCBETUEgQVBJIGRlYnVnZ2luZyBlbmFibGVkPyAgSSBzdXNwZWN0IHlvdSBoYXZlbid0LAo+ Pj4+IGFuZCBJIHJlY29tbWVuZCB0aGF0IHlvdSBkby4KPj4+Cj4+PiAjIGNhdCAvc3lzL2tlcm5l bC9kZWJ1Zy9kbWEtYXBpL2Vycm9yX2NvdW50Cj4+PiAxNjI2MjEKPj4+Cj4+PiAo4pWvwrDilqHC sO+8ieKVr++4tSDilLvilIHilLspCj4+Cj4+ICpwdXRzIHRhYmxlIGJhY2sgb24gaXRzIGZlZXQq Cj4+Cj4+IFNvLCB5ZWFoIC0gVFRNIG1lbW9yeSBpcyBub3QgYWxsb2NhdGVkIHVzaW5nIHRoZSBE TUEgQVBJLCBoZW5jZSB3ZSBjYW5ub3QKPj4gdXNlIHRoZSBETUEgQVBJIHRvIHN5bmMgaXQuIFRo YW5rcyBSdXNzZWxsIGZvciBwb2ludGluZyBpdCBvdXQuCj4+Cj4+IFRoZSBvbmx5IGFsdGVybmF0 aXZlIEkgc2VlIGhlcmUgaXMgdG8gZmx1c2ggdGhlIENQVSBjYWNoZXMgd2hlbiBzeW5jaW5nIGZv cgo+PiB0aGUgZGV2aWNlLCBhbmQgaW52YWxpZGF0ZSB0aGVtIGZvciB0aGUgb3RoZXIgZGlyZWN0 aW9uLiBPZiBjb3Vyc2UgaWYgdGhlCj4+IGRldmljZSBoYXMgY2FjaGVzIG9uIGl0cyBzaWRlIGFz IHdlbGwgdGhlIG9wcG9zaXRlIG9wZXJhdGlvbiBtdXN0IGFsc28gYmUKPj4gZG9uZSBmb3IgaXQu IEd1ZXNzIHRoZSBvbmx5IHdheSBpcyB0byBoYW5kbGUgaXQgYWxsIGJ5IG91cnNlbHZlcyBoZXJl LiA6Lwo+IC4uLiBhbmQgaXQgcmVhbGx5IHN1Y2tzLiBCYXNpY2FsbHkgaWYgd2UgY2Fubm90IHVz ZSB0aGUgRE1BIEFQSSBoZXJlCj4gd2Ugd2lsbCBsb3NlIHRoZSBjb252ZW5pZW5jZSBvZiBoYXZp bmcgYSBwb3J0YWJsZSBBUEkgdGhhdCBkb2VzIGp1c3QKPiB0aGUgcmlnaHQgdGhpbmcgZm9yIHRo ZSB1bmRlcmx5aW5nIHBsYXRmb3JtLiBXaXRob3V0IGl0IHdlIHdvdWxkIGhhdmUKPiB0byBkdXBs aWNhdGUgYXJtX2lvbW11X3N5bmNfc2luZ2xlX2Zvcl9jcHUvZGV2aWNlKCkgYW5kIHdlIHdvdWxk IG9ubHkKPiBoYXZlIHN1cHBvcnQgZm9yIEFSTS4KPgo+IFRoZSB1c2FnZSBvZiB0aGUgRE1BIEFQ SSB0aGF0IHdlIGFyZSBkb2luZyBtaWdodCBiZSBpbGxlZ2FsLCBidXQgaW4KPiBlc3NlbmNlIGl0 IGRvZXMgZXhhY3RseSB3aGF0IHdlIG5lZWQgLSBhdCBsZWFzdCBmb3IgQVJNLiBXaGF0IGFyZSB0 aGUKPiBhbHRlcm5hdGl2ZXM/CkNvbnZlcnQgVFRNIHRvIHVzZSB0aGUgZG1hIGFwaT8gOi0pCgp+ TWFhcnRlbgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpk cmktZGV2ZWwgbWFpbGluZyBsaXN0CmRyaS1kZXZlbEBsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0 cDovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2RyaS1kZXZlbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 From: maarten.lankhorst@canonical.com (Maarten Lankhorst) Date: Tue, 24 Jun 2014 14:27:49 +0200 Subject: [Nouveau] [PATCH v2 2/3] drm/ttm: introduce dma cache sync helpers In-Reply-To: References: <1403603667-11302-1-git-send-email-acourbot@nvidia.com> <1403603667-11302-3-git-send-email-acourbot@nvidia.com> <20140624100220.GK32514@n2100.arm.linux.org.uk> <53A953E6.2030503@nvidia.com> <53A95910.20104@nvidia.com> Message-ID: <53A96EC5.3030701@canonical.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org op 24-06-14 14:23, Alexandre Courbot schreef: > On Tue, Jun 24, 2014 at 7:55 PM, Alexandre Courbot wrote: >> On 06/24/2014 07:33 PM, Alexandre Courbot wrote: >>> On 06/24/2014 07:02 PM, Russell King - ARM Linux wrote: >>>> On Tue, Jun 24, 2014 at 06:54:26PM +0900, Alexandre Courbot wrote: >>>>> From: Lucas Stach >>>>> >>>>> On architectures for which access to GPU memory is non-coherent, >>>>> caches need to be flushed and invalidated explicitly at the >>>>> appropriate places. Introduce two small helpers to make things >>>>> easy for TTM-based drivers. >>>> >>>> Have you run this with DMA API debugging enabled? I suspect you haven't, >>>> and I recommend that you do. >>> >>> # cat /sys/kernel/debug/dma-api/error_count >>> 162621 >>> >>> (??????? ???) >> >> *puts table back on its feet* >> >> So, yeah - TTM memory is not allocated using the DMA API, hence we cannot >> use the DMA API to sync it. Thanks Russell for pointing it out. >> >> The only alternative I see here is to flush the CPU caches when syncing for >> the device, and invalidate them for the other direction. Of course if the >> device has caches on its side as well the opposite operation must also be >> done for it. Guess the only way is to handle it all by ourselves here. :/ > ... and it really sucks. Basically if we cannot use the DMA API here > we will lose the convenience of having a portable API that does just > the right thing for the underlying platform. Without it we would have > to duplicate arm_iommu_sync_single_for_cpu/device() and we would only > have support for ARM. > > The usage of the DMA API that we are doing might be illegal, but in > essence it does exactly what we need - at least for ARM. What are the > alternatives? Convert TTM to use the dma api? :-) ~Maarten From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753741AbaFXM15 (ORCPT ); Tue, 24 Jun 2014 08:27:57 -0400 Received: from youngberry.canonical.com ([91.189.89.112]:44846 "EHLO youngberry.canonical.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752798AbaFXM14 (ORCPT ); Tue, 24 Jun 2014 08:27:56 -0400 Message-ID: <53A96EC5.3030701@canonical.com> Date: Tue, 24 Jun 2014 14:27:49 +0200 From: Maarten Lankhorst User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Alexandre Courbot , Alexandre Courbot CC: David Airlie , "nouveau@lists.freedesktop.org" , "linux-kernel@vger.kernel.org" , "dri-devel@lists.freedesktop.org" , Ben Skeggs , "linux-tegra@vger.kernel.org" , Russell King - ARM Linux , "linux-arm-kernel@lists.infradead.org" Subject: Re: [Nouveau] [PATCH v2 2/3] drm/ttm: introduce dma cache sync helpers References: <1403603667-11302-1-git-send-email-acourbot@nvidia.com> <1403603667-11302-3-git-send-email-acourbot@nvidia.com> <20140624100220.GK32514@n2100.arm.linux.org.uk> <53A953E6.2030503@nvidia.com> <53A95910.20104@nvidia.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org op 24-06-14 14:23, Alexandre Courbot schreef: > On Tue, Jun 24, 2014 at 7:55 PM, Alexandre Courbot wrote: >> On 06/24/2014 07:33 PM, Alexandre Courbot wrote: >>> On 06/24/2014 07:02 PM, Russell King - ARM Linux wrote: >>>> On Tue, Jun 24, 2014 at 06:54:26PM +0900, Alexandre Courbot wrote: >>>>> From: Lucas Stach >>>>> >>>>> On architectures for which access to GPU memory is non-coherent, >>>>> caches need to be flushed and invalidated explicitly at the >>>>> appropriate places. Introduce two small helpers to make things >>>>> easy for TTM-based drivers. >>>> >>>> Have you run this with DMA API debugging enabled? I suspect you haven't, >>>> and I recommend that you do. >>> >>> # cat /sys/kernel/debug/dma-api/error_count >>> 162621 >>> >>> (╯°□°）╯︵ ┻━┻) >> >> *puts table back on its feet* >> >> So, yeah - TTM memory is not allocated using the DMA API, hence we cannot >> use the DMA API to sync it. Thanks Russell for pointing it out. >> >> The only alternative I see here is to flush the CPU caches when syncing for >> the device, and invalidate them for the other direction. Of course if the >> device has caches on its side as well the opposite operation must also be >> done for it. Guess the only way is to handle it all by ourselves here. :/ > ... and it really sucks. Basically if we cannot use the DMA API here > we will lose the convenience of having a portable API that does just > the right thing for the underlying platform. Without it we would have > to duplicate arm_iommu_sync_single_for_cpu/device() and we would only > have support for ARM. > > The usage of the DMA API that we are doing might be illegal, but in > essence it does exactly what we need - at least for ARM. What are the > alternatives? Convert TTM to use the dma api? :-) ~Maarten