From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 56A0C10A2F for ; Thu, 14 Sep 2023 17:37:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9B111C433C8; Thu, 14 Sep 2023 17:37:14 +0000 (UTC) Date: Thu, 14 Sep 2023 18:37:12 +0100 From: Catalin Marinas To: Kuan-Ying Lee =?utf-8?B?KOadjuWGoOepjik=?= Cc: "dietmar.eggemann@arm.com" , "hughd@google.com" , "peterz@infradead.org" , "maz@kernel.org" , "rostedt@goodmis.org" , "rppt@kernel.org" , "yuzenghui@huawei.com" , "james.morse@arm.com" , "vschneid@redhat.com" , "bristot@redhat.com" , "juri.lelli@redhat.com" , "alexandru.elisei@arm.com" , "suzuki.poulose@arm.com" , "mingo@redhat.com" , "akpm@linux-foundation.org" , "mhiramat@kernel.org" , "bsegall@google.com" , "mgorman@suse.de" , "arnd@arndb.de" , "oliver.upton@linux.dev" , "vincent.guittot@linaro.org" , "will@kernel.org" , "linux-kernel@vger.kernel.org" , "linux-trace-kernel@vger.kernel.org" , Qun-wei Lin =?utf-8?B?KOael+e+pOW0tCk=?= , "linux-mm@kvack.org" , "hyesoo.yu@samsung.com" , "kcc@google.com" , "kvmarm@lists.linux.dev" , "david@redhat.com" , Casper Li =?utf-8?B?KOadjuS4reamrik=?= , "steven.price@arm.com" , Chinwen Chang =?utf-8?B?KOW8temMpuaWhyk=?= , "eugenis@google.com" , "linux-arm-kernel@lists.infradead.org" , "pcc@google.com" , "vincenzo.frascino@arm.com" , "linux-arch@vger.kernel.org" , "linux-fsdevel@vger.kernel.org" , "anshuman.khandual@arm.com" Subject: Re: [PATCH RFC 00/37] Add support for arm64 MTE dynamic tag storage reuse Message-ID: References: <20230823131350.114942-1-alexandru.elisei@arm.com> <0dc2afaf8a976ef8eb9af711fd941f1bbfd71321.camel@mediatek.com> Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <0dc2afaf8a976ef8eb9af711fd941f1bbfd71321.camel@mediatek.com> Hi Kuan-Ying, On Wed, Sep 13, 2023 at 08:11:40AM +0000, Kuan-Ying Lee (李冠穎) wrote: > On Wed, 2023-08-23 at 14:13 +0100, Alexandru Elisei wrote: > > diff --git a/arch/arm64/boot/dts/arm/fvp-base-revc.dts > > b/arch/arm64/boot/dts/arm/fvp-base-revc.dts > > index 60472d65a355..bd050373d6cf 100644 > > --- a/arch/arm64/boot/dts/arm/fvp-base-revc.dts > > +++ b/arch/arm64/boot/dts/arm/fvp-base-revc.dts > > @@ -165,10 +165,28 @@ C1_L2: l2-cache1 { > > }; > > }; > > > > - memory@80000000 { > > + memory0: memory@80000000 { > > device_type = "memory"; > > - reg = <0x00000000 0x80000000 0 0x80000000>, > > - <0x00000008 0x80000000 0 0x80000000>; > > + reg = <0x00 0x80000000 0x00 0x7c000000>; > > + }; > > + > > + metadata0: metadata@c0000000 { > > + compatible = "arm,mte-tag-storage"; > > + reg = <0x00 0xfc000000 0x00 0x3e00000>; > > + block-size = <0x1000>; > > + memory = <&memory0>; > > + }; > > + > > + memory1: memory@880000000 { > > + device_type = "memory"; > > + reg = <0x08 0x80000000 0x00 0x7c000000>; > > + }; > > + > > + metadata1: metadata@8c0000000 { > > + compatible = "arm,mte-tag-storage"; > > + reg = <0x08 0xfc000000 0x00 0x3e00000>; > > + block-size = <0x1000>; > > + memory = <&memory1>; > > }; > > > > AFAIK, the above memory configuration means that there are two region > of dram(0x80000000-0xfc000000 and 0x8_80000000-0x8_fc0000000) and this > is called PDD memory map. > > Document[1] said there are some constraints of tag memory as below. > > | The following constraints apply to the tag regions in DRAM: > | 1. The tag region cannot be interleaved with the data region. > | The tag region must also be above the data region within DRAM. > | > | 2.The tag region in the physical address space cannot straddle > | multiple regions of a memory map. > | > | PDD memory map is not allowed to have part of the tag region between > | 2GB-4GB and another part between 34GB-64GB. > > I'm not sure if we can separate tag memory with the above > configuration. Or do I miss something? > > [1] https://developer.arm.com/documentation/101569/0300/?lang=en > (Section 5.4.6.1) Good point, thanks. The above dts some random layout we picked as an example, it doesn't match any real hardware and we didn't pay attention to the interconnect limitations (we fake the tag storage on the model). I'll try to dig out how the mtu_tag_addr_shutter registers work and how the sparse DRAM space is compressed to a smaller tag range. But that's something done by firmware and the kernel only learns the tag storage location from the DT (provided by firmware). We also don't need to know the fine-grained mapping between 32 bytes of data and 1 byte (2 tags) in the tag storage, only the block size in the tag storage space that covers all interleaving done by the interconnect (it can be from 1 byte to something larger like a page; the kernel will then use the lowest common multiple between a page size and this tag block size to figure out how many pages to reserve). -- Catalin From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 331E7EEAA5E for ; Thu, 14 Sep 2023 17:37:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=mTDbKgCGbXYB9XXzI/76SW1HOMYtAKrVM5WXEKnYRcw=; b=MXY1Clblbv1oHf XQuU0eOACaBREIrAPJgnJjW/tCKzsTFsy7BYOUm65Ma+TOu12o1MYsm3o48y8dKB0PPkF/gvSTjQi kobuJMNtP8Y09JJwcxFHB+zY8LDjlM5FrpAcbcZxfoykNBcCo4giOoJJa04CghwM+mFZlZh9LAQHS TZPI7e9Xh0QN9D+Q1q0ALgOC5UYMGM8AYYqlaNS8hmsyrCDOkucNjXyYhfk9w+tKVHqzWRtGZ6Bt3 Sqwx1naV8hPmmhFvRCPCrAUDhGzQtqu94+GGqbjy4XhnNXQVSCXU9gJFppfle+pMu+3izILwlwibK uTaNPqacjZSX0MWWzmhA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qgqHN-0090C4-1k; Thu, 14 Sep 2023 17:37:25 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qgqHK-0090Af-2j for linux-arm-kernel@lists.infradead.org; Thu, 14 Sep 2023 17:37:24 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7DD5761E40; Thu, 14 Sep 2023 17:37:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9B111C433C8; Thu, 14 Sep 2023 17:37:14 +0000 (UTC) Date: Thu, 14 Sep 2023 18:37:12 +0100 From: Catalin Marinas To: Kuan-Ying Lee =?utf-8?B?KOadjuWGoOepjik=?= Cc: "dietmar.eggemann@arm.com" , "hughd@google.com" , "peterz@infradead.org" , "maz@kernel.org" , "rostedt@goodmis.org" , "rppt@kernel.org" , "yuzenghui@huawei.com" , "james.morse@arm.com" , "vschneid@redhat.com" , "bristot@redhat.com" , "juri.lelli@redhat.com" , "alexandru.elisei@arm.com" , "suzuki.poulose@arm.com" , "mingo@redhat.com" , "akpm@linux-foundation.org" , "mhiramat@kernel.org" , "bsegall@google.com" , "mgorman@suse.de" , "arnd@arndb.de" , "oliver.upton@linux.dev" , "vincent.guittot@linaro.org" , "will@kernel.org" , "linux-kernel@vger.kernel.org" , "linux-trace-kernel@vger.kernel.org" , Qun-wei Lin =?utf-8?B?KOael+e+pOW0tCk=?= , "linux-mm@kvack.org" , "hyesoo.yu@samsung.com" , "kcc@google.com" , "kvmarm@lists.linux.dev" , "david@redhat.com" , Casper Li =?utf-8?B?KOadjuS4reamrik=?= , "steven.price@arm.com" , Chinwen Chang =?utf-8?B?KOW8temMpuaWhyk=?= , "eugenis@google.com" , "linux-arm-kernel@lists.infradead.org" , "pcc@google.com" , "vincenzo.frascino@arm.com" , "linux-arch@vger.kernel.org" , "linux-fsdevel@vger.kernel.org" , "anshuman.khandual@arm.com" Subject: Re: [PATCH RFC 00/37] Add support for arm64 MTE dynamic tag storage reuse Message-ID: References: <20230823131350.114942-1-alexandru.elisei@arm.com> <0dc2afaf8a976ef8eb9af711fd941f1bbfd71321.camel@mediatek.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <0dc2afaf8a976ef8eb9af711fd941f1bbfd71321.camel@mediatek.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230914_103722_969784_EA3AFDD2 X-CRM114-Status: GOOD ( 26.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org SGkgS3Vhbi1ZaW5nLAoKT24gV2VkLCBTZXAgMTMsIDIwMjMgYXQgMDg6MTE6NDBBTSArMDAwMCwg S3Vhbi1ZaW5nIExlZSAo5p2O5Yag56mOKSB3cm90ZToKPiBPbiBXZWQsIDIwMjMtMDgtMjMgYXQg MTQ6MTMgKzAxMDAsIEFsZXhhbmRydSBFbGlzZWkgd3JvdGU6Cj4gPiBkaWZmIC0tZ2l0IGEvYXJj aC9hcm02NC9ib290L2R0cy9hcm0vZnZwLWJhc2UtcmV2Yy5kdHMKPiA+IGIvYXJjaC9hcm02NC9i b290L2R0cy9hcm0vZnZwLWJhc2UtcmV2Yy5kdHMKPiA+IGluZGV4IDYwNDcyZDY1YTM1NS4uYmQw NTAzNzNkNmNmIDEwMDY0NAo+ID4gLS0tIGEvYXJjaC9hcm02NC9ib290L2R0cy9hcm0vZnZwLWJh c2UtcmV2Yy5kdHMKPiA+ICsrKyBiL2FyY2gvYXJtNjQvYm9vdC9kdHMvYXJtL2Z2cC1iYXNlLXJl dmMuZHRzCj4gPiBAQCAtMTY1LDEwICsxNjUsMjggQEAgQzFfTDI6IGwyLWNhY2hlMSB7Cj4gPiAg ICAgICAgICAgICAgICAgfTsKPiA+ICAgICAgICAgfTsKPiA+ICAKPiA+IC0gICAgICAgbWVtb3J5 QDgwMDAwMDAwIHsKPiA+ICsgICAgICAgbWVtb3J5MDogbWVtb3J5QDgwMDAwMDAwIHsKPiA+ICAg ICAgICAgICAgICAgICBkZXZpY2VfdHlwZSA9ICJtZW1vcnkiOwo+ID4gLSAgICAgICAgICAgICAg IHJlZyA9IDwweDAwMDAwMDAwIDB4ODAwMDAwMDAgMCAweDgwMDAwMDAwPiwKPiA+IC0gICAgICAg ICAgICAgICAgICAgICA8MHgwMDAwMDAwOCAweDgwMDAwMDAwIDAgMHg4MDAwMDAwMD47Cj4gPiAr ICAgICAgICAgICAgICAgcmVnID0gPDB4MDAgMHg4MDAwMDAwMCAweDAwIDB4N2MwMDAwMDA+Owo+ ID4gKyAgICAgICB9Owo+ID4gKwo+ID4gKyAgICAgICBtZXRhZGF0YTA6IG1ldGFkYXRhQGMwMDAw MDAwICB7Cj4gPiArICAgICAgICAgICAgICAgY29tcGF0aWJsZSA9ICJhcm0sbXRlLXRhZy1zdG9y YWdlIjsKPiA+ICsgICAgICAgICAgICAgICByZWcgPSA8MHgwMCAweGZjMDAwMDAwIDB4MDAgMHgz ZTAwMDAwPjsKPiA+ICsgICAgICAgICAgICAgICBibG9jay1zaXplID0gPDB4MTAwMD47Cj4gPiAr ICAgICAgICAgICAgICAgbWVtb3J5ID0gPCZtZW1vcnkwPjsKPiA+ICsgICAgICAgfTsKPiA+ICsK PiA+ICsgICAgICAgbWVtb3J5MTogbWVtb3J5QDg4MDAwMDAwMCB7Cj4gPiArICAgICAgICAgICAg ICAgZGV2aWNlX3R5cGUgPSAibWVtb3J5IjsKPiA+ICsgICAgICAgICAgICAgICByZWcgPSA8MHgw OCAweDgwMDAwMDAwIDB4MDAgMHg3YzAwMDAwMD47Cj4gPiArICAgICAgIH07Cj4gPiArCj4gPiAr ICAgICAgIG1ldGFkYXRhMTogbWV0YWRhdGFAOGMwMDAwMDAwICB7Cj4gPiArICAgICAgICAgICAg ICAgY29tcGF0aWJsZSA9ICJhcm0sbXRlLXRhZy1zdG9yYWdlIjsKPiA+ICsgICAgICAgICAgICAg ICByZWcgPSA8MHgwOCAweGZjMDAwMDAwIDB4MDAgMHgzZTAwMDAwPjsKPiA+ICsgICAgICAgICAg ICAgICBibG9jay1zaXplID0gPDB4MTAwMD47Cj4gPiArICAgICAgICAgICAgICAgbWVtb3J5ID0g PCZtZW1vcnkxPjsKPiA+ICAgICAgICAgfTsKPiA+ICAKPiAKPiBBRkFJSywgdGhlIGFib3ZlIG1l bW9yeSBjb25maWd1cmF0aW9uIG1lYW5zIHRoYXQgdGhlcmUgYXJlIHR3byByZWdpb24KPiBvZiBk cmFtKDB4ODAwMDAwMDAtMHhmYzAwMDAwMCBhbmQgMHg4XzgwMDAwMDAwLTB4OF9mYzAwMDAwMDAp IGFuZCB0aGlzCj4gaXMgY2FsbGVkIFBERCBtZW1vcnkgbWFwLgo+IAo+IERvY3VtZW50WzFdIHNh aWQgdGhlcmUgYXJlIHNvbWUgY29uc3RyYWludHMgb2YgdGFnIG1lbW9yeSBhcyBiZWxvdy4KPiAK PiB8IFRoZSBmb2xsb3dpbmcgY29uc3RyYWludHMgYXBwbHkgdG8gdGhlIHRhZyByZWdpb25zIGlu IERSQU06Cj4gfCAxLiBUaGUgdGFnIHJlZ2lvbiBjYW5ub3QgYmUgaW50ZXJsZWF2ZWQgd2l0aCB0 aGUgZGF0YSByZWdpb24uCj4gfCBUaGUgdGFnIHJlZ2lvbiBtdXN0IGFsc28gYmUgYWJvdmUgdGhl IGRhdGEgcmVnaW9uIHdpdGhpbiBEUkFNLgo+IHwKPiB8IDIuVGhlIHRhZyByZWdpb24gaW4gdGhl IHBoeXNpY2FsIGFkZHJlc3Mgc3BhY2UgY2Fubm90IHN0cmFkZGxlCj4gfCBtdWx0aXBsZSByZWdp b25zIG9mIGEgbWVtb3J5IG1hcC4KPiB8Cj4gfCBQREQgbWVtb3J5IG1hcCBpcyBub3QgYWxsb3dl ZCB0byBoYXZlIHBhcnQgb2YgdGhlIHRhZyByZWdpb24gYmV0d2Vlbgo+IHwgMkdCLTRHQiBhbmQg YW5vdGhlciBwYXJ0IGJldHdlZW4gMzRHQi02NEdCLgo+IAo+IEknbSBub3Qgc3VyZSBpZiB3ZSBj YW4gc2VwYXJhdGUgdGFnIG1lbW9yeSB3aXRoIHRoZSBhYm92ZQo+IGNvbmZpZ3VyYXRpb24uIE9y IGRvIEkgbWlzcyBzb21ldGhpbmc/Cj4gCj4gWzFdIGh0dHBzOi8vZGV2ZWxvcGVyLmFybS5jb20v ZG9jdW1lbnRhdGlvbi8xMDE1NjkvMDMwMC8/bGFuZz1lbgo+IChTZWN0aW9uIDUuNC42LjEpCgpH b29kIHBvaW50LCB0aGFua3MuIFRoZSBhYm92ZSBkdHMgc29tZSByYW5kb20gbGF5b3V0IHdlIHBp Y2tlZCBhcyBhbgpleGFtcGxlLCBpdCBkb2Vzbid0IG1hdGNoIGFueSByZWFsIGhhcmR3YXJlIGFu ZCB3ZSBkaWRuJ3QgcGF5IGF0dGVudGlvbgp0byB0aGUgaW50ZXJjb25uZWN0IGxpbWl0YXRpb25z ICh3ZSBmYWtlIHRoZSB0YWcgc3RvcmFnZSBvbiB0aGUgbW9kZWwpLgoKSSdsbCB0cnkgdG8gZGln IG91dCBob3cgdGhlIG10dV90YWdfYWRkcl9zaHV0dGVyIHJlZ2lzdGVycyB3b3JrIGFuZCBob3cK dGhlIHNwYXJzZSBEUkFNIHNwYWNlIGlzIGNvbXByZXNzZWQgdG8gYSBzbWFsbGVyIHRhZyByYW5n ZS4gQnV0IHRoYXQncwpzb21ldGhpbmcgZG9uZSBieSBmaXJtd2FyZSBhbmQgdGhlIGtlcm5lbCBv bmx5IGxlYXJucyB0aGUgdGFnIHN0b3JhZ2UKbG9jYXRpb24gZnJvbSB0aGUgRFQgKHByb3ZpZGVk IGJ5IGZpcm13YXJlKS4gV2UgYWxzbyBkb24ndCBuZWVkIHRvIGtub3cKdGhlIGZpbmUtZ3JhaW5l ZCBtYXBwaW5nIGJldHdlZW4gMzIgYnl0ZXMgb2YgZGF0YSBhbmQgMSBieXRlICgyIHRhZ3MpIGlu CnRoZSB0YWcgc3RvcmFnZSwgb25seSB0aGUgYmxvY2sgc2l6ZSBpbiB0aGUgdGFnIHN0b3JhZ2Ug c3BhY2UgdGhhdApjb3ZlcnMgYWxsIGludGVybGVhdmluZyBkb25lIGJ5IHRoZSBpbnRlcmNvbm5l Y3QgKGl0IGNhbiBiZSBmcm9tIDEgYnl0ZQp0byBzb21ldGhpbmcgbGFyZ2VyIGxpa2UgYSBwYWdl OyB0aGUga2VybmVsIHdpbGwgdGhlbiB1c2UgdGhlIGxvd2VzdApjb21tb24gbXVsdGlwbGUgYmV0 d2VlbiBhIHBhZ2Ugc2l6ZSBhbmQgdGhpcyB0YWcgYmxvY2sgc2l6ZSB0byBmaWd1cmUKb3V0IGhv dyBtYW55IHBhZ2VzIHRvIHJlc2VydmUpLgoKLS0gCkNhdGFsaW4KCl9fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJuZWwgbWFpbGluZyBs aXN0CmxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5m cmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK