From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 37DCAC25B10 for ; Fri, 10 May 2024 04:29:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To: Subject:MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=cooFSL9ykpGPdP6mzgFOfn6c+K9GCAUmkjiM8f2jQo0=; b=caIjgJb9Ru7sha 603cEL0DftxXFSZHHBK6N1T7d/hqgs1Ec1hVPmAQvVAojkUosITcl2tu1GSTM41SMLJw3oBXiBcA0 AEJjulXNgPJGN4sA78aTzZvrBFAXCwEQF6jpKJdztnTkBtEqFyo8AYM/4TIPxbt/sNele19rz/Rsx Ywic4yJQj8CR+sCACWw433FnFc07DqWPbz6YqFJIYJbEuhYQqG08QOxxD5O774Q/nFbGxks6btBeP Ts1jU51hc705C+EEXeaBV8WbpCv99+Lib7Q3I9t1n3TnyjgV7wZNFPmsvj1dgyofpq3bhzTZMLf/2 PM7V6oqn+BqW9g7dn/sg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5Hsi-00000003zn4-1Xm1; Fri, 10 May 2024 04:29:16 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5Hsc-00000003zlL-3Ua0 for linux-arm-kernel@lists.infradead.org; Fri, 10 May 2024 04:29:14 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 33638106F; Thu, 9 May 2024 21:29:26 -0700 (PDT) Received: from [10.163.37.102] (unknown [10.163.37.102]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3FFBF3F762; Thu, 9 May 2024 21:28:47 -0700 (PDT) Message-ID: <42837c05-c111-49fc-bf19-e690608f66da@arm.com> Date: Fri, 10 May 2024 09:58:53 +0530 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] arm64: mm: force write fault for atomic RMW instructions Content-Language: en-US To: Yang Shi , catalin.marinas@arm.com, will@kernel.org, scott@os.amperecomputing.com, cl@gentwo.org Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org References: <20240507223558.3039562-1-yang@os.amperecomputing.com> <5e6158aa-09d3-4665-878e-17358aee10cb@arm.com> <328c4c86-96c8-4896-8b6d-94f2facdac9a@os.amperecomputing.com> From: Anshuman Khandual In-Reply-To: <328c4c86-96c8-4896-8b6d-94f2facdac9a@os.amperecomputing.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240509_212911_008117_A064177D X-CRM114-Status: GOOD ( 32.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org CgpPbiA1LzEwLzI0IDAzOjE2LCBZYW5nIFNoaSB3cm90ZToKPiAKPiAKPiBPbiA1LzgvMjQgOToz MSBQTSwgQW5zaHVtYW4gS2hhbmR1YWwgd3JvdGU6Cj4+Cj4+IE9uIDUvOS8yNCAwMDowNywgWWFu ZyBTaGkgd3JvdGU6Cj4+Pgo+Pj4gT24gNS83LzI0IDExOjQ1IFBNLCBBbnNodW1hbiBLaGFuZHVh bCB3cm90ZToKPj4+PiBIZWxsbyBZYW5nLAo+Pj4+Cj4+Pj4gT24gNS84LzI0IDA0OjA1LCBZYW5n IFNoaSB3cm90ZToKPj4+Pj4gVGhlIGF0b21pYyBSTVcgaW5zdHJ1Y3Rpb25zLCBmb3IgZXhhbXBs ZSwgbGRhZGQsIGFjdHVhbGx5IGRvZXMgbG9hZCArCj4+Pj4+IGFkZCArIHN0b3JlIGluIG9uZSBp bnN0cnVjdGlvbiwgaXQgbWF5IHRyaWdnZXIgdHdvIHBhZ2UgZmF1bHRzLCB0aGUKPj4+Pj4gZmly c3QgZmF1bHQgaXMgYSByZWFkIGZhdWx0LCB0aGUgc2Vjb25kIGZhdWx0IGlzIGEgd3JpdGUgZmF1 bHQuCj4+Pj4gSXQgbWF5IG9yIGl0IHdpbGwgZGVmaW5pdGVseSBjcmVhdGUgdHdvIGNvbnNlY3V0 aXZlIHBhZ2UgZmF1bHRzLiBXaGF0Cj4+Pj4gaWYgdGhlIHNlY29uZCB3cml0ZSBmYXVsdCBuZXZl ciBjYW1lIGFib3V0LiBJbiB0aGF0IGNhc2UgYW4gd3JpdGFibGUKPj4+PiBwYWdlIHRhYmxlIGVu dHJ5IHdvdWxkIGJlIGNyZWF0ZWQgdW5uZWNlc3NhcmlseSAob3IgZXZlbiB3cm9uZ2Z1bGx5KSwK Pj4+PiB0aHVzIGJyZWFraW5nIHRoZSBDb1cuCj4+Pj4KPj4+PiBKdXN0IHRyeWluZyB0byB1bmRl cnN0YW5kLCBpcyB0aGUgZG91YmxlIHBhZ2UgZmF1bHQgYSBwb3NzaWJpbGl0eSBvcgo+Pj4+IGEg Y2VydGFpbnR5LiBEb2VzIHRoYXQgZGVwZW5kIG9uIGFyY2hpdGVjdHVyZSAocGxlYXNlIGRvIHBy b3ZpZGUgc29tZQo+Pj4+IGxpbmtzKSBvciBpcyBpdCBpbXBsZW1lbnRhdGlvbiBkZWZpbmVkLgo+ Pj4gQ2hyaXN0b3BoZXIgaGVscGVkIGFuc3dlciBzb21lIHF1ZXN0aW9ucywgSSB3aWxsIHNraXAg dGhvc2UgaWYgSSBoYXZlIG5vdGhpbmcgdG8gYWRkLgo+Pj4KPj4+IEl0IGlzIGRlZmluZWQgaW4g QVJNIGFyY2hpdGVjdHVyZSByZWZlcmVuY2UgbWFudWFsLCBzbyBpdCBpcyBub3QgaW1wbGVtZW50 YXRpb24gZGVmaW5lZC4KPj4gU3VyZSwgYnV0IHBsZWFzZSByZXBsYWNlIHRoZSAibWF5IHRyaWdn ZXIiIHBocmFzZSBhYm92ZSBhcyBhcHByb3ByaWF0ZS4KPiAKPiBZZWFoLCBzdXJlLgo+IAo+Pgo+ Pj4+PiBTb21lIGFwcGxpY2F0aW9ucyB1c2UgYXRvbWljIFJNVyBpbnN0cnVjdGlvbnMgdG8gcG9w dWxhdGUgbWVtb3J5LCBmb3IKPj4+Pj4gZXhhbXBsZSwgb3BlbmpkayB1c2VzIGF0b21pYy1hZGQt MCB0byBkbyBwcmV0b3VjaCAocG9wdWxhdGUgaGVhcCBtZW1vcnkKPj4+PiBCdXQgd2h5IGNhbm5v dCBub3JtYWwgc3RvcmUgb3BlcmF0aW9uIGlzIHN1ZmZpY2llbnQgZm9yIHByZS10b3VjaGluZwo+ Pj4+IHRoZSBoZWFwIG1lbW9yeSwgd2h5IHJlYWQtbW9kaWZ5LXdyaXRlIChSTVcpIGlzIHJlcXVp cmVkIGluc3RlYWQgPwo+Pj4gTWVtb3J5IHdyaXRlIGlzIGZpbmUsIGJ1dCBpdCBkZXBlbmRzIG9u IGFwcGxpY2F0aW9ucy4gRm9yIGV4YW1wbGUsIEpWTSBtYXkgd2FudCB0byAicGVybWl0IHVzZSBv ZiBtZW1vcnkgY29uY3VycmVudGx5IHdpdGggcHJldG91Y2giLiBTbyB0aGV5IGNob3NlIHVzZSBh dG9taWMgaW5zdGVhZCBvZiBtZW1vcnkgd3JpdGUuCj4+Pgo+Pj4+PiBhdCBsYXVuY2ggdGltZSkg YmV0d2VlbiB2MTggYW5kIHYyMi4KPj4+PiBWMTgsIFYyMiA/Cj4+PiB2MTgvdjE5L3YyMC92MjEv djIyCj4+Pgo+Pj4+PiBCdXQgdGhlIGRvdWJsZSBwYWdlIGZhdWx0IGhhcyBzb21lIHByb2JsZW1z Ogo+Pj4+Pgo+Pj4+PiAxLiBOb3RpY2VhYmxlIFRMQiBvdmVyaGVhZC7CoCBUaGUga2VybmVsIGFj dHVhbGx5IGluc3RhbGxzIHplcm8gcGFnZSB3aXRoCj4+Pj4+IMKgwqDCoMKgIHJlYWRvbmx5IFBU RSBmb3IgdGhlIHJlYWQgZmF1bHQuwqAgVGhlIHdyaXRlIGZhdWx0IHdpbGwgdHJpZ2dlciBhCj4+ Pj4+IMKgwqDCoMKgIHdyaXRlLXByb3RlY3Rpb24gZmF1bHQgKENvVykuwqAgVGhlIENvVyB3aWxs IGFsbG9jYXRlIGEgbmV3IHBhZ2UgYW5kCj4+Pj4+IMKgwqDCoMKgIG1ha2UgdGhlIFBURSBwb2lu dCB0byB0aGUgbmV3IHBhZ2UsIHRoaXMgbmVlZHMgVExCIGludmFsaWRhdGlvbnMuwqAgVGhlCj4+ Pj4+IMKgwqDCoMKgIHRsYiBpbnZhbGlkYXRpb24gYW5kIHRoZSBtYW5kYXRvcnkgbWVtb3J5IGJh cnJpZXJzIG1heSBpbmN1cgo+Pj4+PiDCoMKgwqDCoCBzaWduaWZpY2FudCBvdmVyaGVhZCwgcGFy dGljdWxhcmx5IG9uIHRoZSBtYWNoaW5lcyB3aXRoIG1hbnkgY29yZXMuCj4+Pj4+Cj4+Pj4+IDIu IEJyZWFrIHVwIGh1Z2UgcGFnZXMuwqAgSWYgVEhQIGlzIG9uIHRoZSByZWFkIGZhdWx0IHdpbGwg aW5zdGFsbCBodWdlCj4+Pj4+IMKgwqDCoMKgIHplcm8gcGFnZXMuwqAgVGhlIGxhdGVyIENvVyB3 aWxsIGJyZWFrIHVwIHRoZSBodWdlIHBhZ2UgYW5kIGFsbG9jYXRlCj4+Pj4+IMKgwqDCoMKgIGJh c2UgcGFnZXMgaW5zdGVhZCBvZiBodWdlIHBhZ2UuwqAgVGhlIGFwcGxpY2F0aW9ucyBoYXZlIHRv IHJlbHkgb24KPj4+Pj4gwqDCoMKgwqAga2h1Z2VwYWdlZCAoa2VybmVsIHRocmVhZCkgdG8gY29s bGFwc2UgaHVnZSBwYWdlcyBhc3luY2hyb25vdXNseS4KPj4+Pj4gwqDCoMKgwqAgVGhpcyBhbHNv IGluY3VycyBub3RpY2VhYmxlIHBlcmZvcm1hbmNlIHBlbmFsdHkuCj4+Pj4+Cj4+Pj4+IDMuIDUx MnggcGFnZSBmYXVsdHMgd2l0aCBodWdlIHBhZ2UuwqAgRHVlIHRvICMyLCB0aGUgYXBwbGljYXRp b25zIGhhdmUgdG8KPj4+Pj4gwqDCoMKgwqAgaGF2ZSBwYWdlIGZhdWx0cyBmb3IgZXZlcnkgNEsg YXJlYSBmb3IgdGhlIHdyaXRlLCB0aGlzIG1ha2VzIHRoZSBzcGVlZAo+Pj4+PiDCoMKgwqDCoCB1 cCBieSB1c2luZyBodWdlIHBhZ2UgYWN0dWFsbHkgZ29uZS4KPj4+PiBUaGUgcHJvYmxlbXMgbWVu dGlvbmVkIGFib3ZlIGFyZSByZWFzb25hYmxlIGFuZCBleHBlY3RlZC4KPj4+PiDCoMKgIElmIHRo ZSBtZW1vcnkgYWRkcmVzcyBoYXMgc29tZSB2YWxpZCBkYXRhLCBpdCBtdXN0IGhhdmUgYWxyZWFk eSByZWFjaGVkIHRoZXJlCj4+Pj4gdmlhIGEgcHJldmlvdXMgd3JpdGUgYWNjZXNzLCB3aGljaCB3 b3VsZCBoYXZlIGNhdXNlZCBpbml0aWFsIENvVyB0cmFuc2l0aW9uID8KPj4+PiBJZiB0aGUgbWVt b3J5IGFkZHJlc3MgaGFzIG5vIHZhbGlkIGRhdGEgdG8gYmVnaW4gd2l0aCwgd2h5IGV2ZW4gdXNl IFJNVyA/Cj4+Pj4KPj4+Pj4gU28gaXQgc291bmRzIHBvaW50bGVzcyB0byBoYXZlIHR3byBwYWdl IGZhdWx0cyBzaW5jZSB3ZSBrbm93IHRoZSBtZW1vcnkKPj4+Pj4gd2lsbCBiZSBkZWZpbml0ZWx5 IHdyaXR0ZW4gdmVyeSBzb29uLsKgIEZvcmNpbmcgd3JpdGUgZmF1bHQgZm9yIGF0b21pYyBSTVcK Pj4+Pj4gaW5zdHJ1Y3Rpb24gbWFrZXMgc29tZSBzZW5zZSBhbmQgaXQgY2FuIHNvbHZlIHRoZSBh Zm9yZW1lbnRpb25lZCBwcm9ibGVtczoKPj4+Pj4KPj4+Pj4gRmlyc3RseSwgaXQganVzdCBhbGxv Y2F0ZXMgemVybydlZCBwYWdlLCBubyB0bGIgaW52YWxpZGF0aW9uIGFuZCBtZW1vcnkKPj4+Pj4g YmFycmllcnMgYW55bW9yZS4KPj4+Pj4gU2Vjb25kbHksIGl0IGNhbiBwb3B1bGF0ZSB3cml0YWJs ZSBodWdlIHBhZ2VzIGluIHRoZSBmaXJzdCBwbGFjZSBhbmQKPj4+Pj4gZG9uJ3QgYnJlYWsgdGhl bSB1cC7CoCBKdXN0IG9uZSBwYWdlIGZhdWx0IGlzIG5lZWRlZCBmb3IgMk0gYXJlYSBpbnN0cmFk Cj4+Pj4+IG9mIDUxMiBmYXVsdHMgYW5kIGFsc28gc2F2ZSBjcHUgdGltZSBieSBub3QgdXNpbmcg a2h1Z2VwYWdlZC4KPj4+Pj4KPj4+Pj4gQSBzaW1wbGUgbWljcm8gYmVuY2htYXJrIHdoaWNoIHBv cHVsYXRlcyAxRyBtZW1vcnkgc2hvd3MgdGhlIG51bWJlciBvZgo+Pj4+PiBwYWdlIGZhdWx0cyBp cyByZWR1Y2VkIGJ5IGhhbGYgYW5kIHRoZSB0aW1lIHNwZW50IGJ5IHN5c3RlbSBpcyByZWR1Y2Vk Cj4+Pj4+IGJ5IDYwJSBvbiBhIFZNIHJ1bm5pbmcgb24gQW1wZXJlIEFsdHJhIHBsYXRmb3JtLgo+ Pj4+Pgo+Pj4+PiBBbmQgdGhlIGJlbmNobWFyayBmb3IgYW5vbnltb3VzIHJlYWQgZmF1bHQgb24g MUcgbWVtb3J5LCBmaWxlIHJlYWQgZmF1bHQKPj4+Pj4gb24gMUcgZmlsZSAoY29sZCBwYWdlIGNh Y2hlIGFuZCB3YXJtIHBhZ2UgY2FjaGUpIGRvbid0IHNob3cgbm90aWNlYWJsZQo+Pj4+PiByZWdy ZXNzaW9uLgo+Pj4+Pgo+Pj4+PiBTb21lIG90aGVyIGFyY2hpdGVjdHVyZXMgYWxzbyBoYXZlIGNv ZGUgaW5zcGVjdGlvbiBpbiBwYWdlIGZhdWx0IHBhdGgsCj4+Pj4+IGZvciBleGFtcGxlLCBTUEFS QyBhbmQgeDg2Lgo+Pj4+IE9rYXksIEkgd2FzIGFib3V0IHRvIGFzaywgYnV0IGlzIG5vdCBjYWxs aW5nIGdldF91c2VyKCkgZm9yIGFsbCBkYXRhCj4+Pj4gcmVhZCBwYWdlIGZhdWx0cyBpbmNyZWFz ZSB0aGUgY29zdCBmb3IgYSBob3QgY29kZSBwYXRoIGluIGdlbmVyYWwgZm9yCj4+Pj4gc29tZSBw b3RlbnRpYWwgc2F2aW5ncyBmb3IgYSB2ZXJ5IHNwZWNpZmljIHVzZSBjYXNlLiBOb3Qgc3VyZSBp ZiB0aGF0Cj4+Pj4gaXMgd29ydGggdGhlIHRyYWRlLW9mZi4KPj4+IEkgdGVzdGVkIHJlYWQgZmF1 bHQgbGF0ZW5jeSAoYW5vbnltb3VzIHJlYWQgZmF1bHQgYW5kIGZpbGUgcmVhZCBmYXVsdCksIEkg ZGlkbid0IHNlZSBub3RpY2VhYmxlIHJlZ3Jlc3Npb24uCj4+IENvdWxkIHlvdSBwbGVhc2UgcnVu IGEgbXVsdGkgdGhyZWFkZWQgYXBwbGljYXRpb24gYWNjZXNzaW5nIG9uZSBjb21tb24KPj4gYnVm ZmVyIHdoaWxlIHJ1bm5pbmcgdGhlc2UgYXRvbWljIG9wZXJhdGlvbnMuIFdlIGp1c3QgbmVlZCB0 byBlbnN1cmUKPj4gdGhhdCBwYWdlZmF1bHRfZGlzYWJsZSgpLWVuYWJsZSgpIHdpbmRvdyBpcyBu b3QgcHJldmVudGluZyBjb25jdXJyZW50Cj4+IHBhZ2UgZmF1bHRzIGFuZCBhZGRpbmcgYWNjZXNz IGxhdGVuY3kgdG8gb3RoZXIgdGhyZWFkcy4KPiAKPiBJIG1vZGlmaWVkIHBhZ2VfZmF1bHQxIHRl c3QgaW4gd2lsbC1pdC1zY2FsZSB0byBtYWtlIGl0IGp1c3QgZ2VuZXJhdGUgcmVhZCBmYXVsdCAo dGhlIG9yaWdpbmFsIGNvZGUgZ2VuZXJhdGVkIHdyaXRlIGZhdWx0KSwgYW5kIGFub255bW91cyBy ZWFkIGZhdWx0IHNob3VsZCBiZSB0aGUgbW9zdCBzZW5zaXRpdmUgY2FzZSB0byB0aGlzIGNoYW5n ZS4gVGhlbiBJIHJhbiB0aGUgdGVzdCB3aXRoIGRpZmZlcmVudCBudW1iZXIgb2YgdGhyZWFkcyAo MSAtIDE2MCAKClJpZ2h0LCBvbmx5IHdpdGggcmVhZCBkYXRhIGZhdWx0cyBpLmUgKCFGQVVMVF9G TEFHX1dSSVRFIGFuZCAhRkFVTFRfRkxBR19JTlNUUlVDVElPTikKY29kZSBwYXRoIGVudGVycyB0 aGUgcGFnZWZhdWx0X2Rpc2FibGUvZW5hYmxlKCkgd2luZG93LCBidXQgYWxsIG90aGVycyB3aWxs IHNraXAgaXQuCgpiZWNhdXNlIHRvdGFsIDE2MCBjb3JlcyBvbiBteSB0ZXN0IG1hY2hpbmUpLCBw bGVhc2Ugc2VlIHRoZSBiZWxvdyB0YWJsZSAoaG9wZWZ1bGx5IG15IGVtYWlsIGNsaWVudCB3b24n dCBtZXNzIGl0KQoKVGhhbmtzIGZvciBwcm92aWRpbmcgdGhlIHRlc3QgcmVzdWx0cy4KCj4gCj4g bnJfdGhyZWFkc8KgwqDCoMKgwqDCoMKgwqDCoMKgIGJlZm9yZcKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCBhZnRlcsKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgKy8tCj4gMcKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAyMDU2OTk2wqDCoMKgwqDCoMKgwqDCoMKg wqDCoCAyMDQ4MDMwwqDCoMKgwqDCoMKgwqAgLTAuNCUKPiAyMMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgIDE3ODM2NDIywqDCoMKgwqDCoMKgwqDCoMKgIDE2NzE4NjA2wqDC oMKgwqDCoCAtNi4yNyUKPiA0MMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg IDI4NTM2MjM3wqDCoMKgwqDCoMKgwqDCoMKgIDI3OTU4ODc1wqDCoMKgwqDCoCAtMi4wMyUKPiA2 MMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDM1OTQ3ODU0wqDCoMKgwqDC oMKgwqDCoMKgIDM1MjM2ODg0wqDCoMKgwqDCoCAtMiUKPiA4MMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgIDMxNjQ2NjMywqDCoMKgwqDCoMKgwqDCoMKgIDM5MjA5NjY1wqDC oMKgwqDCoCArMjQlCj4gMTAwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAyMDgz NjE0MsKgwqDCoMKgwqDCoMKgwqDCoCAyMTAxNzc5NsKgwqDCoMKgwqAgKzAuOSUKPiAxMjDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIDIwMzUwOTgwwqDCoMKgwqDCoMKgwqDCoMKg IDIwNjM1NjAzwqDCoMKgwqDCoCArMS40JQo+IDE0MMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqAgMjAwNDE5MjDCoMKgwqDCoMKgwqDCoMKgwqAgMTk5MDQwMTXCoMKgwqDCoMKgIC0w LjclCj4gMTYwwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAxOTU2MTkwOMKgwqDC oMKgwqDCoMKgwqDCoCAyMDI2NDM2MMKgwqDCoMKgwqAgKzMuNiUKPiAKPiBTb21ldGltZXMgdGhl IGFmdGVyIGlzIGJldHRlciB0aGFuIHRoZSBiZWZvcmUsIHNvbWV0aW1lcyBvcHBvc2l0ZS4gVGhl cmUgYXJlIHR3byBvdXRsaWVycywgb3RoZXIgdGhhbiB0aGVtIHRoZXJlIGlzIG5vdCBub3RpY2Vh YmxlIHJlZ3Jlc3Npb24uCgpUaGlzIGRvZXMgbm90IGxvb2sgdGhhdCBiYWQsIGJ1dCB3aWxsIHBy b2JhYmx5IGxldCBvdGhlcnMgd2VpZ2ggaW4uCgo+IAo+IFRvIHJ1bGUgb3V0IHRoZSB3b3JzdCBj YXNlLCBJIGFsc28gcmFuIHRoZSB0ZXN0IDEwMCBpdGVyYXRpb25zIHdpdGggMTYwIHRocmVhZHMg dGhlbiBjb21wYXJlZCB0aGUgd29yc3QgY2FzZToKPiAKPiDCoMKgwqAgTsKgwqDCoMKgwqDCoMKg wqDCoMKgIE1pbsKgwqDCoMKgwqDCoMKgwqDCoMKgIE1heMKgwqDCoMKgwqDCoMKgIE1lZGlhbsKg wqDCoMKgwqDCoMKgwqDCoMKgIEF2ZyBTdGRkZXYKPiDCoDEwMMKgwqDCoMKgwqDCoMKgwqAgMzQ3 NzDCoMKgwqDCoMKgwqDCoMKgIDg0OTc5wqDCoMKgwqDCoMKgwqDCoCA2NTUzNsKgwqDCoMKgwqDC oCA2MzUzNy43IDEwMzU4Ljg3Mwo+IMKgMTAwwqDCoMKgwqDCoMKgwqDCoCAzODA3N8KgwqDCoMKg wqDCoMKgwqAgODc2NTLCoMKgwqDCoMKgwqDCoMKgIDY1NTM2wqDCoMKgwqDCoCA2MzExOS4wMiA4 NzkyLjczOTkKPiAKPiBTdGlsbCBubyBub3RpY2VhYmxlIHJlZ3Jlc3Npb24uCgpJIGd1ZXNzIHRv IG1ha2UgdGhpbmdzIGJldHRlciwgcHJvYmFibHkgcGFnZWZhdWx0X2VuYWJsZSgpIGNvdWxkIGJl IG1vdmVkCmJlZm9yZSBhYXJjaDY0X2luc25faXNfY2xhc3NfYXRvbWljKCkgd2hpY2ggbWlnaHQg bm90IG5lZWQgcGFnZSBmYXVsdHMgdG8KYmUgZGlzYWJsZWQgPyBBbHNvIHdoYXQgYWJvdXQgbm9u IHVzZXIgbW9kZSBhdG9taWMgaW5zdHJ1Y3Rpb25zLCBjYXVzaW5nCnNpbWlsYXIgc2NlbmFyaW9z ID8gQmVjYXVzZSBnZXRfdXNlcigpIHdpbGwgbm90IGJlIGFibGUgdG8gZmV0Y2ggdGhvc2UuIAoK PiAKPj4KPj4+Pj4gU2lnbmVkLW9mZi1ieTogWWFuZyBTaGkgPHlhbmdAb3MuYW1wZXJlY29tcHV0 aW5nLmNvbT4KPj4+Pj4gLS0tCj4+Pj4+IMKgwqAgYXJjaC9hcm02NC9pbmNsdWRlL2FzbS9pbnNu LmggfMKgIDEgKwo+Pj4+PiDCoMKgIGFyY2gvYXJtNjQvbW0vZmF1bHQuY8KgwqDCoMKgwqDCoMKg wqAgfCAxOSArKysrKysrKysrKysrKysrKysrCj4+Pj4+IMKgwqAgMiBmaWxlcyBjaGFuZ2VkLCAy MCBpbnNlcnRpb25zKCspCj4+Pj4+Cj4+Pj4+IGRpZmYgLS1naXQgYS9hcmNoL2FybTY0L2luY2x1 ZGUvYXNtL2luc24uaCBiL2FyY2gvYXJtNjQvaW5jbHVkZS9hc20vaW5zbi5oCj4+Pj4+IGluZGV4 IGRiMWFlYWNkNGNkOS4uNWQ1YTNmYmVlY2MwIDEwMDY0NAo+Pj4+PiAtLS0gYS9hcmNoL2FybTY0 L2luY2x1ZGUvYXNtL2luc24uaAo+Pj4+PiArKysgYi9hcmNoL2FybTY0L2luY2x1ZGUvYXNtL2lu c24uaAo+Pj4+PiBAQCAtMzE5LDYgKzMxOSw3IEBAIHN0YXRpYyBfX2Fsd2F5c19pbmxpbmUgdTMy IGFhcmNoNjRfaW5zbl9nZXRfIyNhYmJyIyNfdmFsdWUodm9pZCnCoMKgwqAgXAo+Pj4+PiDCoMKg wqAgKiAiLSIgbWVhbnMgImRvbid0IGNhcmUiCj4+Pj4+IMKgwqDCoCAqLwo+Pj4+PiDCoMKgIF9f QUFSQ0g2NF9JTlNOX0ZVTkNTKGNsYXNzX2JyYW5jaF9zeXMswqDCoMKgIDB4MWMwMDAwMDAsIDB4 MTQwMDAwMDApCj4+Pj4+ICtfX0FBUkNINjRfSU5TTl9GVU5DUyhjbGFzc19hdG9taWMswqDCoMKg IDB4M2IyMDBjMDAsIDB4MzgyMDAwMDApCj4+Pj4+IMKgwqAgwqAgX19BQVJDSDY0X0lOU05fRlVO Q1MoYWRyLMKgwqDCoCAweDlGMDAwMDAwLCAweDEwMDAwMDAwKQo+Pj4+PiDCoMKgIF9fQUFSQ0g2 NF9JTlNOX0ZVTkNTKGFkcnAswqDCoMKgIDB4OUYwMDAwMDAsIDB4OTAwMDAwMDApCj4+Pj4+IGRp ZmYgLS1naXQgYS9hcmNoL2FybTY0L21tL2ZhdWx0LmMgYi9hcmNoL2FybTY0L21tL2ZhdWx0LmMK Pj4+Pj4gaW5kZXggODI1MWUyZmVhOWM3Li5mN2JjZWVkZjVlZjMgMTAwNjQ0Cj4+Pj4+IC0tLSBh L2FyY2gvYXJtNjQvbW0vZmF1bHQuYwo+Pj4+PiArKysgYi9hcmNoL2FybTY0L21tL2ZhdWx0LmMK Pj4+Pj4gQEAgLTUyOSw2ICs1MjksNyBAQCBzdGF0aWMgaW50IF9fa3Byb2JlcyBkb19wYWdlX2Zh dWx0KHVuc2lnbmVkIGxvbmcgZmFyLCB1bnNpZ25lZCBsb25nIGVzciwKPj4+Pj4gwqDCoMKgwqDC oMKgIHVuc2lnbmVkIGludCBtbV9mbGFncyA9IEZBVUxUX0ZMQUdfREVGQVVMVDsKPj4+Pj4gwqDC oMKgwqDCoMKgIHVuc2lnbmVkIGxvbmcgYWRkciA9IHVudGFnZ2VkX2FkZHIoZmFyKTsKPj4+Pj4g wqDCoMKgwqDCoMKgIHN0cnVjdCB2bV9hcmVhX3N0cnVjdCAqdm1hOwo+Pj4+PiArwqDCoMKgIHVu c2lnbmVkIGludCBpbnNuOwo+Pj4+PiDCoMKgIMKgwqDCoMKgwqAgaWYgKGtwcm9iZV9wYWdlX2Zh dWx0KHJlZ3MsIGVzcikpCj4+Pj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgIHJldHVybiAwOwo+Pj4+ PiBAQCAtNTg2LDYgKzU4NywyNCBAQCBzdGF0aWMgaW50IF9fa3Byb2JlcyBkb19wYWdlX2ZhdWx0 KHVuc2lnbmVkIGxvbmcgZmFyLCB1bnNpZ25lZCBsb25nIGVzciwKPj4+Pj4gwqDCoMKgwqDCoMKg IGlmICghdm1hKQo+Pj4+PiDCoMKgwqDCoMKgwqDCoMKgwqDCoCBnb3RvIGxvY2tfbW1hcDsKPj4+ Pj4gwqDCoCArwqDCoMKgIGlmIChtbV9mbGFncyAmIChGQVVMVF9GTEFHX1dSSVRFIHwgRkFVTFRf RkxBR19JTlNUUlVDVElPTikpCj4+Pj4+ICvCoMKgwqDCoMKgwqDCoCBnb3RvIGNvbnRpbnVlX2Zh dWx0Owo+Pj4+PiArCj4+Pj4+ICvCoMKgwqAgcGFnZWZhdWx0X2Rpc2FibGUoKTsKPj4+Pj4gKwo+ Pj4+PiArwqDCoMKgIGlmIChnZXRfdXNlcihpbnNuLCAodW5zaWduZWQgaW50IF9fdXNlciAqKSBp bnN0cnVjdGlvbl9wb2ludGVyKHJlZ3MpKSkgewo+Pj4+PiArwqDCoMKgwqDCoMKgwqAgcGFnZWZh dWx0X2VuYWJsZSgpOwo+Pj4+PiArwqDCoMKgwqDCoMKgwqAgZ290byBjb250aW51ZV9mYXVsdDsK Pj4+Pj4gK8KgwqDCoCB9Cj4+Pj4+ICsKPj4+Pj4gK8KgwqDCoCBpZiAoYWFyY2g2NF9pbnNuX2lz X2NsYXNzX2F0b21pYyhpbnNuKSkgewo+Pj4+PiArwqDCoMKgwqDCoMKgwqAgdm1fZmxhZ3MgPSBW TV9XUklURTsKPj4+Pj4gK8KgwqDCoMKgwqDCoMKgIG1tX2ZsYWdzIHw9IEZBVUxUX0ZMQUdfV1JJ VEU7Cj4+Pj4+ICvCoMKgwqAgfQo+Pj4+PiArCj4+Pj4+ICvCoMKgwqAgcGFnZWZhdWx0X2VuYWJs ZSgpOwo+Pj4+PiArCj4+Pj4+ICtjb250aW51ZV9mYXVsdDoKPj4+Pj4gwqDCoMKgwqDCoMKgIGlm ICghKHZtYS0+dm1fZmxhZ3MgJiB2bV9mbGFncykpIHsKPj4+Pj4gwqDCoMKgwqDCoMKgwqDCoMKg wqAgdm1hX2VuZF9yZWFkKHZtYSk7Cj4+Pj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgIGdvdG8gbG9j a19tbWFwOwo+IAoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X18KbGludXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5p bmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8v bGludXgtYXJtLWtlcm5lbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 240C180046 for ; Fri, 10 May 2024 04:29:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715315345; cv=none; b=RL9DEfmkeY8KiOMQb+rrXu9qCcAvqHmBALkrs9AU91S4WzyPvJ7oRIM7CKodCasJ8+JCXh5G8UNaXKSCoWrlXBldAxUka/vF0EHp/dJCVXvx/+SyBkz/z33SqLLVWOyyAKDwrk3ZivzHN4/EHon13qHSP3GwuaaZrWlchtvZ5og= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715315345; c=relaxed/simple; bh=O840MrsrAQB+7MaiiLKUB8HaKsvkvNE2dTnR+IXHXbQ=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=hlhRMg2yvigdpQi65XhhOyzjDq5Rd3wj2d/rNkI7vvKHpOZMBIuuvtNnh6+Fo5EwGrNH6o7cNCowcnU3XbtBbVLEtiNY1BV381Wpfqyug9GE7DjIMxM7JbY+lbrAfF+BoIZe4KtR3uVnX5BrYiN+A0rS2HD+jzfHldBMQOKZSYU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 33638106F; Thu, 9 May 2024 21:29:26 -0700 (PDT) Received: from [10.163.37.102] (unknown [10.163.37.102]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3FFBF3F762; Thu, 9 May 2024 21:28:47 -0700 (PDT) Message-ID: <42837c05-c111-49fc-bf19-e690608f66da@arm.com> Date: Fri, 10 May 2024 09:58:53 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] arm64: mm: force write fault for atomic RMW instructions Content-Language: en-US To: Yang Shi , catalin.marinas@arm.com, will@kernel.org, scott@os.amperecomputing.com, cl@gentwo.org Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org References: <20240507223558.3039562-1-yang@os.amperecomputing.com> <5e6158aa-09d3-4665-878e-17358aee10cb@arm.com> <328c4c86-96c8-4896-8b6d-94f2facdac9a@os.amperecomputing.com> From: Anshuman Khandual In-Reply-To: <328c4c86-96c8-4896-8b6d-94f2facdac9a@os.amperecomputing.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit On 5/10/24 03:16, Yang Shi wrote: > > > On 5/8/24 9:31 PM, Anshuman Khandual wrote: >> >> On 5/9/24 00:07, Yang Shi wrote: >>> >>> On 5/7/24 11:45 PM, Anshuman Khandual wrote: >>>> Hello Yang, >>>> >>>> On 5/8/24 04:05, Yang Shi wrote: >>>>> The atomic RMW instructions, for example, ldadd, actually does load + >>>>> add + store in one instruction, it may trigger two page faults, the >>>>> first fault is a read fault, the second fault is a write fault. >>>> It may or it will definitely create two consecutive page faults. What >>>> if the second write fault never came about. In that case an writable >>>> page table entry would be created unnecessarily (or even wrongfully), >>>> thus breaking the CoW. >>>> >>>> Just trying to understand, is the double page fault a possibility or >>>> a certainty. Does that depend on architecture (please do provide some >>>> links) or is it implementation defined. >>> Christopher helped answer some questions, I will skip those if I have nothing to add. >>> >>> It is defined in ARM architecture reference manual, so it is not implementation defined. >> Sure, but please replace the "may trigger" phrase above as appropriate. > > Yeah, sure. > >> >>>>> Some applications use atomic RMW instructions to populate memory, for >>>>> example, openjdk uses atomic-add-0 to do pretouch (populate heap memory >>>> But why cannot normal store operation is sufficient for pre-touching >>>> the heap memory, why read-modify-write (RMW) is required instead ? >>> Memory write is fine, but it depends on applications. For example, JVM may want to "permit use of memory concurrently with pretouch". So they chose use atomic instead of memory write. >>> >>>>> at launch time) between v18 and v22. >>>> V18, V22 ? >>> v18/v19/v20/v21/v22 >>> >>>>> But the double page fault has some problems: >>>>> >>>>> 1. Noticeable TLB overhead.  The kernel actually installs zero page with >>>>>      readonly PTE for the read fault.  The write fault will trigger a >>>>>      write-protection fault (CoW).  The CoW will allocate a new page and >>>>>      make the PTE point to the new page, this needs TLB invalidations.  The >>>>>      tlb invalidation and the mandatory memory barriers may incur >>>>>      significant overhead, particularly on the machines with many cores. >>>>> >>>>> 2. Break up huge pages.  If THP is on the read fault will install huge >>>>>      zero pages.  The later CoW will break up the huge page and allocate >>>>>      base pages instead of huge page.  The applications have to rely on >>>>>      khugepaged (kernel thread) to collapse huge pages asynchronously. >>>>>      This also incurs noticeable performance penalty. >>>>> >>>>> 3. 512x page faults with huge page.  Due to #2, the applications have to >>>>>      have page faults for every 4K area for the write, this makes the speed >>>>>      up by using huge page actually gone. >>>> The problems mentioned above are reasonable and expected. >>>>    If the memory address has some valid data, it must have already reached there >>>> via a previous write access, which would have caused initial CoW transition ? >>>> If the memory address has no valid data to begin with, why even use RMW ? >>>> >>>>> So it sounds pointless to have two page faults since we know the memory >>>>> will be definitely written very soon.  Forcing write fault for atomic RMW >>>>> instruction makes some sense and it can solve the aforementioned problems: >>>>> >>>>> Firstly, it just allocates zero'ed page, no tlb invalidation and memory >>>>> barriers anymore. >>>>> Secondly, it can populate writable huge pages in the first place and >>>>> don't break them up.  Just one page fault is needed for 2M area instrad >>>>> of 512 faults and also save cpu time by not using khugepaged. >>>>> >>>>> A simple micro benchmark which populates 1G memory shows the number of >>>>> page faults is reduced by half and the time spent by system is reduced >>>>> by 60% on a VM running on Ampere Altra platform. >>>>> >>>>> And the benchmark for anonymous read fault on 1G memory, file read fault >>>>> on 1G file (cold page cache and warm page cache) don't show noticeable >>>>> regression. >>>>> >>>>> Some other architectures also have code inspection in page fault path, >>>>> for example, SPARC and x86. >>>> Okay, I was about to ask, but is not calling get_user() for all data >>>> read page faults increase the cost for a hot code path in general for >>>> some potential savings for a very specific use case. Not sure if that >>>> is worth the trade-off. >>> I tested read fault latency (anonymous read fault and file read fault), I didn't see noticeable regression. >> Could you please run a multi threaded application accessing one common >> buffer while running these atomic operations. We just need to ensure >> that pagefault_disable()-enable() window is not preventing concurrent >> page faults and adding access latency to other threads. > > I modified page_fault1 test in will-it-scale to make it just generate read fault (the original code generated write fault), and anonymous read fault should be the most sensitive case to this change. Then I ran the test with different number of threads (1 - 160 Right, only with read data faults i.e (!FAULT_FLAG_WRITE and !FAULT_FLAG_INSTRUCTION) code path enters the pagefault_disable/enable() window, but all others will skip it. because total 160 cores on my test machine), please see the below table (hopefully my email client won't mess it) Thanks for providing the test results. > > nr_threads           before                after            +/- > 1                      2056996            2048030        -0.4% > 20                    17836422          16718606      -6.27% > 40                    28536237          27958875      -2.03% > 60                    35947854          35236884      -2% > 80                    31646632          39209665      +24% > 100                  20836142          21017796      +0.9% > 120                  20350980          20635603      +1.4% > 140                  20041920          19904015      -0.7% > 160                  19561908          20264360      +3.6% > > Sometimes the after is better than the before, sometimes opposite. There are two outliers, other than them there is not noticeable regression. This does not look that bad, but will probably let others weigh in. > > To rule out the worst case, I also ran the test 100 iterations with 160 threads then compared the worst case: > >     N           Min           Max        Median           Avg Stddev >  100         34770         84979         65536       63537.7 10358.873 >  100         38077         87652         65536      63119.02 8792.7399 > > Still no noticeable regression. I guess to make things better, probably pagefault_enable() could be moved before aarch64_insn_is_class_atomic() which might not need page faults to be disabled ? Also what about non user mode atomic instructions, causing similar scenarios ? Because get_user() will not be able to fetch those. > >> >>>>> Signed-off-by: Yang Shi >>>>> --- >>>>>    arch/arm64/include/asm/insn.h |  1 + >>>>>    arch/arm64/mm/fault.c         | 19 +++++++++++++++++++ >>>>>    2 files changed, 20 insertions(+) >>>>> >>>>> diff --git a/arch/arm64/include/asm/insn.h b/arch/arm64/include/asm/insn.h >>>>> index db1aeacd4cd9..5d5a3fbeecc0 100644 >>>>> --- a/arch/arm64/include/asm/insn.h >>>>> +++ b/arch/arm64/include/asm/insn.h >>>>> @@ -319,6 +319,7 @@ static __always_inline u32 aarch64_insn_get_##abbr##_value(void)    \ >>>>>     * "-" means "don't care" >>>>>     */ >>>>>    __AARCH64_INSN_FUNCS(class_branch_sys,    0x1c000000, 0x14000000) >>>>> +__AARCH64_INSN_FUNCS(class_atomic,    0x3b200c00, 0x38200000) >>>>>      __AARCH64_INSN_FUNCS(adr,    0x9F000000, 0x10000000) >>>>>    __AARCH64_INSN_FUNCS(adrp,    0x9F000000, 0x90000000) >>>>> diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c >>>>> index 8251e2fea9c7..f7bceedf5ef3 100644 >>>>> --- a/arch/arm64/mm/fault.c >>>>> +++ b/arch/arm64/mm/fault.c >>>>> @@ -529,6 +529,7 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, >>>>>        unsigned int mm_flags = FAULT_FLAG_DEFAULT; >>>>>        unsigned long addr = untagged_addr(far); >>>>>        struct vm_area_struct *vma; >>>>> +    unsigned int insn; >>>>>          if (kprobe_page_fault(regs, esr)) >>>>>            return 0; >>>>> @@ -586,6 +587,24 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, >>>>>        if (!vma) >>>>>            goto lock_mmap; >>>>>    +    if (mm_flags & (FAULT_FLAG_WRITE | FAULT_FLAG_INSTRUCTION)) >>>>> +        goto continue_fault; >>>>> + >>>>> +    pagefault_disable(); >>>>> + >>>>> +    if (get_user(insn, (unsigned int __user *) instruction_pointer(regs))) { >>>>> +        pagefault_enable(); >>>>> +        goto continue_fault; >>>>> +    } >>>>> + >>>>> +    if (aarch64_insn_is_class_atomic(insn)) { >>>>> +        vm_flags = VM_WRITE; >>>>> +        mm_flags |= FAULT_FLAG_WRITE; >>>>> +    } >>>>> + >>>>> +    pagefault_enable(); >>>>> + >>>>> +continue_fault: >>>>>        if (!(vma->vm_flags & vm_flags)) { >>>>>            vma_end_read(vma); >>>>>            goto lock_mmap; >