From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08D2EC433E0 for ; Wed, 27 Jan 2021 12:12:04 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8FC6620780 for ; Wed, 27 Jan 2021 12:12:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8FC6620780 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D71906E7DC; Wed, 27 Jan 2021 12:12:02 +0000 (UTC) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7C4356E7D7 for ; Wed, 27 Jan 2021 12:12:01 +0000 (UTC) IronPort-SDR: zPyKT3qFFshXkX0fU32pSlrkLazOf/MfHoKIn1o814DKJzNg5x+SA/o/TX5KyrMFJyCr5mLop3 kEtsuq+dDiCg== X-IronPort-AV: E=McAfee;i="6000,8403,9876"; a="244135047" X-IronPort-AV: E=Sophos;i="5.79,379,1602572400"; d="scan'208";a="244135047" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2021 04:12:00 -0800 IronPort-SDR: meMHncnvbafvUEf9hMslrOzVEv2YrjcESH/4LLcraCtfZ5SfEGaNYLXB+pkcAbi0B08ClPRJtk P7aPBswCKo5w== X-IronPort-AV: E=Sophos;i="5.79,379,1602572400"; d="scan'208";a="362412240" Received: from msazhin-mobl1.ccr.corp.intel.com (HELO [10.252.54.105]) ([10.252.54.105]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2021 04:11:58 -0800 Subject: Re: [PATCH] RFC: dma-fence: Document recoverable page fault implications To: Felix Kuehling , Daniel Vetter , DRI Development References: <20210121194056.1734409-1-daniel.vetter@ffwll.ch> <6d373177-2645-1d67-9c14-dcad87c4f4d9@amd.com> From: Maarten Lankhorst Message-ID: <68740fcf-530e-b929-1c98-5810fc97ed23@linux.intel.com> Date: Wed, 27 Jan 2021 13:11:56 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: <6d373177-2645-1d67-9c14-dcad87c4f4d9@amd.com> Content-Language: en-US X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linaro-mm-sig@lists.linaro.org, Jerome Glisse , =?UTF-8?Q?Thomas_Hellstr=c3=b6m?= , Daniel Vetter , =?UTF-8?Q?Christian_K=c3=b6nig?= , linux-media@vger.kernel.org Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" T3AgMjctMDEtMjAyMSBvbSAwMToyMiBzY2hyZWVmIEZlbGl4IEt1ZWhsaW5nOgo+IEFtIDIwMjEt MDEtMjEgdW0gMjo0MCBwLm0uIHNjaHJpZWIgRGFuaWVsIFZldHRlcjoKPj4gUmVjZW50bHkgdGhl cmUgd2FzIGEgZmFpcmx5IGxvbmcgdGhyZWFkIGFib3V0IHJlY29yZWFibGUgaGFyZHdhcmUgcGFn ZQo+PiBmYXVsdHMsIGhvdyB0aGV5IGNhbiBkZWFkbG9jaywgYW5kIHdoYXQgdG8gZG8gYWJvdXQg dGhhdC4KPj4KPj4gV2hpbGUgdGhlIGRpc2N1c3Npb24gaXMgc3RpbGwgZnJlc2ggSSBmaWd1cmVk IGdvb2QgdGltZSB0byB0cnkgYW5kCj4+IGRvY3VtZW50IHRoZSBjb25jbHVzaW9ucyBhIGJpdC4K Pj4KPj4gUmVmZXJlbmNlczogaHR0cHM6Ly9sb3JlLmtlcm5lbC5vcmcvZHJpLWRldmVsLzIwMjEw MTA3MDMwMTI3LjIwMzkzLTEtRmVsaXguS3VlaGxpbmdAYW1kLmNvbS8KPj4gQ2M6IE1hYXJ0ZW4g TGFua2hvcnN0IDxtYWFydGVuLmxhbmtob3JzdEBsaW51eC5pbnRlbC5jb20+Cj4+IENjOiBUaG9t YXMgSGVsbHN0csO2bSA8dGhvbWFzLmhlbGxzdHJvbUBpbnRlbC5jb20+Cj4+IENjOiAiQ2hyaXN0 aWFuIEvDtm5pZyIgPGNocmlzdGlhbi5rb2VuaWdAYW1kLmNvbT4KPj4gQ2M6IEplcm9tZSBHbGlz c2UgPGpnbGlzc2VAcmVkaGF0LmNvbT4KPj4gQ2M6IEZlbGl4IEt1ZWhsaW5nIDxmZWxpeC5rdWVo bGluZ0BhbWQuY29tPgo+PiBTaWduZWQtb2ZmLWJ5OiBEYW5pZWwgVmV0dGVyIDxkYW5pZWwudmV0 dGVyQGludGVsLmNvbT4KPj4gQ2M6IFN1bWl0IFNlbXdhbCA8c3VtaXQuc2Vtd2FsQGxpbmFyby5v cmc+Cj4+IENjOiBsaW51eC1tZWRpYUB2Z2VyLmtlcm5lbC5vcmcKPj4gQ2M6IGxpbmFyby1tbS1z aWdAbGlzdHMubGluYXJvLm9yZwo+PiAtLQo+PiBJJ2xsIGJlIGF3YXkgbmV4dCB3ZWVrLCBidXQg ZmlndXJlZCBJJ2xsIHR5cGUgdGhpcyB1cCBxdWlja2x5IGZvciBzb21lCj4+IGNvbW1lbnRzIGFu ZCB0byBjaGVjayB3aGV0aGVyIEkgZ290IHRoaXMgYWxsIHJvdWdobHkgcmlnaHQuCj4+Cj4+IENy aXRpcXVlIHZlcnkgbXVjaCB3YW50ZWQgb24gdGhpcywgc28gdGhhdCB3ZSBjYW4gbWFrZSBzdXJl IGh3IHdoaWNoCj4+IGNhbid0IHByZWVtcHQgKHdpdGggcGFnZWZhdWx0cyBwZW5kaW5nKSBsaWtl IGdmeDEwIGhhcyBhIGNsZWFyIHBhdGggdG8KPj4gc3VwcG9ydCBwYWdlIGZhdWx0cyBpbiB1cHN0 cmVhbS4gU28gYW55dGhpbmcgSSBtaXNzZWQsIGdvdCB3cm9uZyBvcgo+PiBsaWtlIHRoYXQgd291 bGQgYmUgZ29vZC4KPj4gLURhbmllbAo+PiAtLS0KPj4gIERvY3VtZW50YXRpb24vZHJpdmVyLWFw aS9kbWEtYnVmLnJzdCB8IDY2ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysKPj4gIDEgZmls ZSBjaGFuZ2VkLCA2NiBpbnNlcnRpb25zKCspCj4+Cj4+IGRpZmYgLS1naXQgYS9Eb2N1bWVudGF0 aW9uL2RyaXZlci1hcGkvZG1hLWJ1Zi5yc3QgYi9Eb2N1bWVudGF0aW9uL2RyaXZlci1hcGkvZG1h LWJ1Zi5yc3QKPj4gaW5kZXggYTIxMzNkNjk4NzJjLi5lOTI0YzFlNGY3YTMgMTAwNjQ0Cj4+IC0t LSBhL0RvY3VtZW50YXRpb24vZHJpdmVyLWFwaS9kbWEtYnVmLnJzdAo+PiArKysgYi9Eb2N1bWVu dGF0aW9uL2RyaXZlci1hcGkvZG1hLWJ1Zi5yc3QKPj4gQEAgLTI1NywzICsyNTcsNjkgQEAgZmVu Y2VzIGluIHRoZSBrZXJuZWwuIFRoaXMgbWVhbnM6Cj4+ICAgIHVzZXJzcGFjZSBpcyBhbGxvd2Vk IHRvIHVzZSB1c2Vyc3BhY2UgZmVuY2luZyBvciBsb25nIHJ1bm5pbmcgY29tcHV0ZQo+PiAgICB3 b3JrbG9hZHMuIFRoaXMgYWxzbyBtZWFucyBubyBpbXBsaWNpdCBmZW5jaW5nIGZvciBzaGFyZWQg YnVmZmVycyBpbiB0aGVzZQo+PiAgICBjYXNlcy4KPj4gKwo+PiArUmVjb3ZlcmFibGUgSGFyZHdh cmUgUGFnZSBGYXVsdHMgSW1wbGljYXRpb25zCj4+ICt+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+ fn5+fn5+fn5+fn5+fn5+fn5+fn4KPj4gKwo+PiArTW9kZXJuIGhhcmR3YXJlIHN1cHBvcnRzIHJl Y292ZXJhYmxlIHBhZ2UgZmF1bHRzLCB3aGljaCBoYXMgYSBsb3Qgb2YKPj4gK2ltcGxpY2F0aW9u cyBmb3IgRE1BIGZlbmNlcy4KPj4gKwo+PiArRmlyc3QsIGEgcGVuZGluZyBwYWdlIGZhdWx0IG9i dmlvdXNseSBob2xkcyB1cCB0aGUgd29yayB0aGF0J3MgcnVubmluZyBvbiB0aGUKPj4gK2FjY2Vs ZXJhdG9yIGFuZCBhIG1lbW9yeSBhbGxvY2F0aW9uIGlzIHVzdWFsbHkgcmVxdWlyZWQgdG8gcmVz b2x2ZSB0aGUgZmF1bHQuCj4+ICtCdXQgbWVtb3J5IGFsbG9jYXRpb25zIGFyZSBub3QgYWxsb3dl ZCB0byBnYXRlIGNvbXBsZXRpb24gb2YgRE1BIGZlbmNlcywgd2hpY2gKPj4gK21lYW5zIGFueSB3 b3JrbG9hZCB1c2luZyByZWNvdmVyYWJsZSBwYWdlIGZhdWx0cyBjYW5ub3QgdXNlIERNQSBmZW5j ZXMgZm9yCj4+ICtzeW5jaHJvbml6YXRpb24uIFN5bmNocm9uaXphdGlvbiBmZW5jZXMgY29udHJv bGxlZCBieSB1c2Vyc3BhY2UgbXVzdCBiZSB1c2VkCj4+ICtpbnN0ZWFkLgo+PiArCj4+ICtPbiBH UFVzIHRoaXMgcG9zZXMgYSBwcm9ibGVtLCBiZWNhdXNlIGN1cnJlbnQgZGVza3RvcCBjb21wb3Np dG9yIHByb3RvY29scyBvbgo+PiArTGludXMgcmVseSBvbiBETUEgZmVuY2VzLCB3aGljaCBtZWFu cyB3aXRob3V0IGFuIGVudGlyZWx5IG5ldyB1c2Vyc3BhY2Ugc3RhY2sKPj4gK2J1aWx0IG9uIHRv cCBvZiB1c2Vyc3BhY2UgZmVuY2VzLCB0aGV5IGNhbm5vdCBiZW5lZml0IGZyb20gcmVjb3ZlcmFi bGUgcGFnZQo+PiArZmF1bHRzLiBUaGUgZXhjZXB0aW9uIGlzIHdoZW4gcGFnZSBmYXVsdHMgYXJl IG9ubHkgdXNlZCBhcyBtaWdyYXRpb24gaGludHMgYW5kCj4+ICtuZXZlciB0byBvbi1kZW1hbmQg ZmlsbCBhIG1lbW9yeSByZXF1ZXN0LiBGb3Igbm93IHRoaXMgbWVhbnMgcmVjb3ZlcmFibGUgcGFn ZQo+PiArZmF1bHRzIG9uIEdQVXMgYXJlIGxpbWl0ZWQgdG8gcHVyZSBjb21wdXRlIHdvcmtsb2Fk cy4KPj4gKwo+PiArRnVydGhlcm1vcmUgR1BVcyB1c3VhbGx5IGhhdmUgc2hhcmVkIHJlc291cmNl cyBiZXR3ZWVuIHRoZSAzRCByZW5kZXJpbmcgYW5kCj4+ICtjb21wdXRlIHNpZGUsIGxpa2UgY29t cHV0ZSB1bml0cyBvciBjb21tYW5kIHN1Ym1pc3Npb24gZW5naW5lcy4gSWYgYm90aCBhIDNECj4+ ICtqb2Igd2l0aCBhIERNQSBmZW5jZSBhbmQgYSBjb21wdXRlIHdvcmtsb2FkIHVzaW5nIHJlY292 ZXJhYmxlIHBhZ2UgZmF1bHRzIGFyZQo+PiArcGVuZGluZyB0aGV5IGNvdWxkIGRlYWRsb2NrOgo+ PiArCj4+ICstIFRoZSAzRCB3b3JrbG9hZCBtaWdodCBuZWVkIHRvIHdhaXQgZm9yIHRoZSBjb21w dXRlIGpvYiB0byBmaW5pc2ggYW5kIHJlbGVhc2UKPj4gKyAgaGFyZHdhcmUgcmVzb3VyY2VzIGZp cnN0Lgo+PiArCj4+ICstIFRoZSBjb21wdXRlIHdvcmtsb2FkIG1pZ2h0IGJlIHN0dWNrIGluIGEg cGFnZSBmYXVsdCwgYmVjYXVzZSB0aGUgbWVtb3J5Cj4+ICsgIGFsbG9jYXRpb24gaXMgd2FpdGlu ZyBmb3IgdGhlIERNQSBmZW5jZSBvZiB0aGUgM0Qgd29ya2xvYWQgdG8gY29tcGxldGUuCj4+ICsK Pj4gK1RoZXJlIGFyZSBhIGZldyB3YXlzIHRvIHByZXZlbnQgdGhpcyBwcm9ibGVtOgo+PiArCj4+ ICstIENvbXB1dGUgd29ya2xvYWRzIGNhbiBhbHdheXMgYmUgcHJlZW1wdGVkLCBldmVuIHdoZW4g YSBwYWdlIGZhdWx0IGlzIHBlbmRpbmcKPj4gKyAgYW5kIG5vdCB5ZXQgcmVwYWlyZWQuIE5vdCBh bGwgaGFyZHdhcmUgc3VwcG9ydHMgdGhpcy4KPj4gKwo+PiArLSBETUEgZmVuY2Ugd29ya2xvYWRz IGFuZCB3b3JrbG9hZHMgd2hpY2ggbmVlZCBwYWdlIGZhdWx0IGhhbmRsaW5nIGhhdmUKPj4gKyAg aW5kZXBlbmRlbnQgaGFyZHdhcmUgcmVzb3VyY2VzIHRvIGd1YXJhbnRlZSBmb3J3YXJkIHByb2dy ZXNzLiBUaGlzIGNvdWxkIGJlCj4+ICsgIGFjaGlldmVkIHRocm91Z2ggZS5nLiB0aHJvdWdoIGRl ZGljYXRlZCBlbmdpbmVzIGFuZCBtaW5pbWFsIGNvbXB1dGUgdW5pdAo+PiArICByZXNlcnZhdGlv bnMgZm9yIERNQSBmZW5jZSB3b3JrbG9hZHMuCj4+ICsKPj4gKy0gVGhlIHJlc2VydmF0aW9uIGFw cHJvYWNoIGNvdWxkIGJlIGZ1cnRoZXIgcmVmaW5lZCBieSBvbmx5IHJlc2VydmluZyB0aGUKPj4g KyAgaGFyZHdhcmUgcmVzb3VyY2VzIGZvciBETUEgZmVuY2Ugd29ya2xvYWRzIHdoZW4gdGhleSBh cmUgaW4tZmxpZ2h0LiBUaGlzIG11c3QKPj4gKyAgY292ZXIgdGhlIHRpbWUgZnJvbSB3aGVuIHRo ZSBETUEgZmVuY2UgaXMgdmlzaWJsZSB0byBvdGhlciB0aHJlYWRzIHVwIHRvCj4+ICsgIG1vbWVu dCB3aGVuIGZlbmNlIGlzIGNvbXBsZXRlZCB0aHJvdWdoIGRtYV9mZW5jZV9zaWduYWwoKS4KPj4g Kwo+PiArLSBBcyBhIGxhc3QgcmVzb3J0LCBpZiB0aGUgaGFyZHdhcmUgcHJvdmlkZXMgbm8gdXNl ZnVsIHJlc2VydmF0aW9uIG1lY2hhbmljcywKPj4gKyAgYWxsIHdvcmtsb2FkcyBtdXN0IGJlIGZs dXNoZWQgZnJvbSB0aGUgR1BVIHdoZW4gc3dpdGNoaW5nIGJldHdlZW4gam9icwo+PiArICByZXF1 aXJpbmcgRE1BIGZlbmNlcyBvciBqb2JzIHJlcXVpcmluZyBwYWdlIGZhdWx0IGhhbmRsaW5nOiBU aGlzIG1lYW5zIGFsbCBETUEKPj4gKyAgZmVuY2VzIG11c3QgY29tcGxldGUgYmVmb3JlIGEgY29t cHV0ZSBqb2Igd2l0aCBwYWdlIGZhdWx0IGhhbmRsaW5nIGNhbiBiZQo+PiArICBpbnNlcnRlZCBp bnRvIHRoZSBzY2hlZHVsZXIgcXVldWUuIEFuZCB2aWNlIHZlcnNhLCBiZWZvcmUgYSBETUEgZmVu Y2UgY2FuIGJlCj4+ICsgIG1hZGUgdmlzaWJsZSBhbnl3aGVyZSBpbiB0aGUgc3lzdGVtLCBhbGwg Y29tcHV0ZSB3b3JrbG9hZHMgbXVzdCBiZSBwcmVlbXB0ZWQKPj4gKyAgdG8gZ3VhcmFudGVlIGFs bCBwZW5kaW5nIEdQVSBwYWdlIGZhdWx0cyBhcmUgZmx1c2hlZC4KPiBJIHRob3VnaHQgb2YgYW5v dGhlciBwb3NzaWJsZSB3b3JrYXJvdW5kOgo+Cj4gICAqIFBhcnRpdGlvbiB0aGUgbWVtb3J5LiBT ZXJ2aWNpbmcgb2YgcGFnZSBmYXVsdHMgd2lsbCB1c2UgYSBzZXBhcmF0ZQo+ICAgICBtZW1vcnkg cG9vbCB0aGF0IGNhbiBhbHdheXMgYmUgYWxsb2NhdGVkIGZyb20gd2l0aG91dCB3YWl0aW5nIGZv cgo+ICAgICBmZW5jZXMuIFRoaXMgaW5jbHVkZXMgbWVtb3J5IGZvciBwYWdlIHRhYmxlcyBhbmQg bWVtb3J5IGZvcgo+ICAgICBtaWdyYXRpbmcgZGF0YSB0by4gWW91IG1heSBzdGVhbCBtZW1vcnkg ZnJvbSBvdGhlciBwcm9jZXNzZXMgdGhhdAo+ICAgICBjYW4gcGFnZSBmYXVsdCwgc28gbm8gZmVu Y2Ugd2FpdGluZyBpcyBuZWNlc3NhcnkuIEJlaW5nIGFibGUgdG8KPiAgICAgc3RlYWwgbWVtb3J5 IGF0IGFueSB0aW1lIGFsc28gbWVhbnMgdGhlcmUgYXJlIGJhc2ljYWxseSBubwo+ICAgICBvdXQt b2YtbWVtb3J5IHNpdHVhdGlvbnMgeW91IG5lZWQgdG8gd29ycnkgYWJvdXQuIEV2ZW4gcGFnZSB0 YWJsZXMKPiAgICAgKGV4Y2VwdCB0aGUgcm9vdCBwYWdlIGRpcmVjdG9yeSBvZiBlYWNoIHByb2Nl c3MpIGNhbiBiZSBzdG9sZW4gaW4KPiAgICAgdGhlIHdvcnN0IGNhc2UuCgpJIHRoaW5rICdvdmVy Y29tbWl0JyB3b3VsZCBiZSBhIG5pY2Ugd2F5IHRvIGRlc2NyaWJlIHRoaXMuIEJ1dCBJJ20gbm90 CnN1cmUgaG93IGVhc3kgdGhpcyBpcyB0byBpbXBsZW1lbnQgaW4gcHJhY3RpY2UuIFlvdSB3b3Vs ZCBiYXNpY2FsbHkgbmVlZAp0byBjcmVhdGUgeW91ciBvd24gbWVtb3J5IG1hbmFnZXIgZm9yIHRo aXMuCgpCdXQgZnJvbSBhIGRlc2lnbiBwb2ludCBvZiB2aWV3LCBkZWZpbml0ZWx5IGEgdmFsaWQg c29sdXRpb24uCgpCdXQgdGhpcyBsb29rcyBnb29kLCB0aG9zZSBzb2x1dGlvbnMgYXJlIGRlZmlu aXRlbHkgdGhlIHZhbGlkIG9wdGlvbnMgd2UKY2FuIGNob29zZSBmcm9tLgoKfk1hYXJ0ZW4KCl9f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmRyaS1kZXZlbCBt YWlsaW5nIGxpc3QKZHJpLWRldmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9yZwpodHRwczovL2xpc3Rz LmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2RyaS1kZXZlbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A034C433DB for ; Wed, 27 Jan 2021 12:16:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DF1362078E for ; Wed, 27 Jan 2021 12:16:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237265AbhA0MPe (ORCPT ); Wed, 27 Jan 2021 07:15:34 -0500 Received: from mga06.intel.com ([134.134.136.31]:31181 "EHLO mga06.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237635AbhA0MN6 (ORCPT ); Wed, 27 Jan 2021 07:13:58 -0500 IronPort-SDR: T4yBvBEXhw7OFVUj9zQcC0cpl/BKeEqD8OaCqUt2B3YlII2t5QeKG8LaFPpAWgzgZoxM1N1nsU mLvGF6ngX6nw== X-IronPort-AV: E=McAfee;i="6000,8403,9876"; a="241587601" X-IronPort-AV: E=Sophos;i="5.79,379,1602572400"; d="scan'208";a="241587601" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2021 04:12:00 -0800 IronPort-SDR: meMHncnvbafvUEf9hMslrOzVEv2YrjcESH/4LLcraCtfZ5SfEGaNYLXB+pkcAbi0B08ClPRJtk P7aPBswCKo5w== X-IronPort-AV: E=Sophos;i="5.79,379,1602572400"; d="scan'208";a="362412240" Received: from msazhin-mobl1.ccr.corp.intel.com (HELO [10.252.54.105]) ([10.252.54.105]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2021 04:11:58 -0800 Subject: Re: [PATCH] RFC: dma-fence: Document recoverable page fault implications To: Felix Kuehling , Daniel Vetter , DRI Development Cc: =?UTF-8?Q?Thomas_Hellstr=c3=b6m?= , =?UTF-8?Q?Christian_K=c3=b6nig?= , Jerome Glisse , Daniel Vetter , Sumit Semwal , linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org References: <20210121194056.1734409-1-daniel.vetter@ffwll.ch> <6d373177-2645-1d67-9c14-dcad87c4f4d9@amd.com> From: Maarten Lankhorst Message-ID: <68740fcf-530e-b929-1c98-5810fc97ed23@linux.intel.com> Date: Wed, 27 Jan 2021 13:11:56 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: <6d373177-2645-1d67-9c14-dcad87c4f4d9@amd.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Op 27-01-2021 om 01:22 schreef Felix Kuehling: > Am 2021-01-21 um 2:40 p.m. schrieb Daniel Vetter: >> Recently there was a fairly long thread about recoreable hardware page >> faults, how they can deadlock, and what to do about that. >> >> While the discussion is still fresh I figured good time to try and >> document the conclusions a bit. >> >> References: https://lore.kernel.org/dri-devel/20210107030127.20393-1-Felix.Kuehling@amd.com/ >> Cc: Maarten Lankhorst >> Cc: Thomas Hellström >> Cc: "Christian König" >> Cc: Jerome Glisse >> Cc: Felix Kuehling >> Signed-off-by: Daniel Vetter >> Cc: Sumit Semwal >> Cc: linux-media@vger.kernel.org >> Cc: linaro-mm-sig@lists.linaro.org >> -- >> I'll be away next week, but figured I'll type this up quickly for some >> comments and to check whether I got this all roughly right. >> >> Critique very much wanted on this, so that we can make sure hw which >> can't preempt (with pagefaults pending) like gfx10 has a clear path to >> support page faults in upstream. So anything I missed, got wrong or >> like that would be good. >> -Daniel >> --- >> Documentation/driver-api/dma-buf.rst | 66 ++++++++++++++++++++++++++++ >> 1 file changed, 66 insertions(+) >> >> diff --git a/Documentation/driver-api/dma-buf.rst b/Documentation/driver-api/dma-buf.rst >> index a2133d69872c..e924c1e4f7a3 100644 >> --- a/Documentation/driver-api/dma-buf.rst >> +++ b/Documentation/driver-api/dma-buf.rst >> @@ -257,3 +257,69 @@ fences in the kernel. This means: >> userspace is allowed to use userspace fencing or long running compute >> workloads. This also means no implicit fencing for shared buffers in these >> cases. >> + >> +Recoverable Hardware Page Faults Implications >> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> + >> +Modern hardware supports recoverable page faults, which has a lot of >> +implications for DMA fences. >> + >> +First, a pending page fault obviously holds up the work that's running on the >> +accelerator and a memory allocation is usually required to resolve the fault. >> +But memory allocations are not allowed to gate completion of DMA fences, which >> +means any workload using recoverable page faults cannot use DMA fences for >> +synchronization. Synchronization fences controlled by userspace must be used >> +instead. >> + >> +On GPUs this poses a problem, because current desktop compositor protocols on >> +Linus rely on DMA fences, which means without an entirely new userspace stack >> +built on top of userspace fences, they cannot benefit from recoverable page >> +faults. The exception is when page faults are only used as migration hints and >> +never to on-demand fill a memory request. For now this means recoverable page >> +faults on GPUs are limited to pure compute workloads. >> + >> +Furthermore GPUs usually have shared resources between the 3D rendering and >> +compute side, like compute units or command submission engines. If both a 3D >> +job with a DMA fence and a compute workload using recoverable page faults are >> +pending they could deadlock: >> + >> +- The 3D workload might need to wait for the compute job to finish and release >> + hardware resources first. >> + >> +- The compute workload might be stuck in a page fault, because the memory >> + allocation is waiting for the DMA fence of the 3D workload to complete. >> + >> +There are a few ways to prevent this problem: >> + >> +- Compute workloads can always be preempted, even when a page fault is pending >> + and not yet repaired. Not all hardware supports this. >> + >> +- DMA fence workloads and workloads which need page fault handling have >> + independent hardware resources to guarantee forward progress. This could be >> + achieved through e.g. through dedicated engines and minimal compute unit >> + reservations for DMA fence workloads. >> + >> +- The reservation approach could be further refined by only reserving the >> + hardware resources for DMA fence workloads when they are in-flight. This must >> + cover the time from when the DMA fence is visible to other threads up to >> + moment when fence is completed through dma_fence_signal(). >> + >> +- As a last resort, if the hardware provides no useful reservation mechanics, >> + all workloads must be flushed from the GPU when switching between jobs >> + requiring DMA fences or jobs requiring page fault handling: This means all DMA >> + fences must complete before a compute job with page fault handling can be >> + inserted into the scheduler queue. And vice versa, before a DMA fence can be >> + made visible anywhere in the system, all compute workloads must be preempted >> + to guarantee all pending GPU page faults are flushed. > I thought of another possible workaround: > > * Partition the memory. Servicing of page faults will use a separate > memory pool that can always be allocated from without waiting for > fences. This includes memory for page tables and memory for > migrating data to. You may steal memory from other processes that > can page fault, so no fence waiting is necessary. Being able to > steal memory at any time also means there are basically no > out-of-memory situations you need to worry about. Even page tables > (except the root page directory of each process) can be stolen in > the worst case. I think 'overcommit' would be a nice way to describe this. But I'm not sure how easy this is to implement in practice. You would basically need to create your own memory manager for this. But from a design point of view, definitely a valid solution. But this looks good, those solutions are definitely the valid options we can choose from. ~Maarten