From mboxrd@z Thu Jan 1 00:00:00 1970 From: Catalin Marinas Subject: [PATCH 10/22] arm64: mte: Tags-aware copy_page() implementation Date: Wed, 11 Dec 2019 18:40:15 +0000 Message-ID: <20191211184027.20130-11-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, Richard Earnshaw , Szabolcs Nagy , Marc Zyngier , Kevin Brodsky , linux-mm@kvack.org, Andrey Konovalov , Vincenzo Frascino , Will Deacon List-Id: linux-arch.vger.kernel.org From: Vincenzo Frascino When the Memory Tagging Extension is enabled, the tags need to be preserved across page copy (e.g. for copy-on-write). Introduce MTE-aware copy_page() which preserves the tags across page copy. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas --- arch/arm64/lib/copy_page.S | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/arch/arm64/lib/copy_page.S b/arch/arm64/lib/copy_page.S index bbb8562396af..970b7a20da70 100644 --- a/arch/arm64/lib/copy_page.S +++ b/arch/arm64/lib/copy_page.S @@ -25,6 +25,29 @@ alternative_if ARM64_HAS_NO_HW_PREFETCH prfm pldl1strm, [x1, #384] alternative_else_nop_endif +#ifdef CONFIG_ARM64_MTE +alternative_if_not ARM64_MTE + b 2f +alternative_else_nop_endif + /* + * Copy tags if MTE has been enabled. + */ + mov x2, x0 + mov x3, x1 + + multitag_transfer_size x7, x5 +1: + ldgm x4, [x3] + stgm x4, [x2] + + add x2, x2, x7 + add x3, x3, x7 + + tst x2, #(PAGE_SIZE - 1) + b.ne 1b +2: +#endif + ldp x2, x3, [x1] ldp x4, x5, [x1, #16] ldp x6, x7, [x1, #32] From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from foss.arm.com ([217.140.110.172]:43126 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729879AbfLKSky (ORCPT ); Wed, 11 Dec 2019 13:40:54 -0500 From: Catalin Marinas Subject: [PATCH 10/22] arm64: mte: Tags-aware copy_page() implementation Date: Wed, 11 Dec 2019 18:40:15 +0000 Message-ID: <20191211184027.20130-11-catalin.marinas@arm.com> In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-arch-owner@vger.kernel.org List-ID: To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Message-ID: <20191211184015.SgGtJV4YiS3ESm_6oPphunbMpenIX5OrqG4mToWSTEs@z> From: Vincenzo Frascino When the Memory Tagging Extension is enabled, the tags need to be preserved across page copy (e.g. for copy-on-write). Introduce MTE-aware copy_page() which preserves the tags across page copy. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas --- arch/arm64/lib/copy_page.S | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/arch/arm64/lib/copy_page.S b/arch/arm64/lib/copy_page.S index bbb8562396af..970b7a20da70 100644 --- a/arch/arm64/lib/copy_page.S +++ b/arch/arm64/lib/copy_page.S @@ -25,6 +25,29 @@ alternative_if ARM64_HAS_NO_HW_PREFETCH prfm pldl1strm, [x1, #384] alternative_else_nop_endif +#ifdef CONFIG_ARM64_MTE +alternative_if_not ARM64_MTE + b 2f +alternative_else_nop_endif + /* + * Copy tags if MTE has been enabled. + */ + mov x2, x0 + mov x3, x1 + + multitag_transfer_size x7, x5 +1: + ldgm x4, [x3] + stgm x4, [x2] + + add x2, x2, x7 + add x3, x3, x7 + + tst x2, #(PAGE_SIZE - 1) + b.ne 1b +2: +#endif + ldp x2, x3, [x1] ldp x4, x5, [x1, #16] ldp x6, x7, [x1, #32]