From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 00611606A33B for ; Tue, 17 Nov 2026 07:06:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=JO1IWPiX; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=foss header.b=QtAAKAKJ DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Unsubscribe-Post: List-Owner; bh=RImp/fV3D3InxuUTVuaSs0rWb42FA9SO6Wnjy1jVERc=; b=JO1IWPiXhkO5+d BJYo6xflZ+59xsWjKibWeidO4EfwBjX9c1FYLu+ozXJuZVpicDtBE9R8mdNicT4pKLVIK18mMoUCH j5UjQSLiSXnrInXNI9Pq7YuF/rtRkQ8mgwrJsAD8NBJLsiKUgEOiw0fJIX6lwBIZU2LpG1KjB5Uh/ HAri7h/TEhqX5pZLsjhqRQka86rtHDhHH33IEi/EBQgWcvPj7/u6rOa+pYsIwAYvj2K38ibpaWJ7r 0/9g7xjULo+m1Lu9+6idiM5JO7dyAv3g+xsYG3CAOrKVThII6bp0xUE8fcnKnFd/D2q7KKayYFEiX YfLcm34A7IyUjx6RW/iQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.100.1 #2 (Red Hat Linux)) id 1xTDHF-00000004i3q-3JuK; Tue, 17 Nov 2026 07:06:49 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.100.1 #2 (Red Hat Linux)) id 1xTDH1-00000004hwj-1HzM for linux-arm-kernel@lists.infradead.org; Tue, 17 Nov 2026 07:06:36 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 68A041476; Mon, 16 Nov 2026 23:06:31 -0800 (PST) Received: from localhost (a085714.arm.com [10.164.19.28]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9F28E3F763; Mon, 16 Nov 2026 23:06:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1794899194; bh=DlYI5zPVBJe6EC0DN2RG1Fj9YuDqCOwNKOBIujdfKiw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QtAAKAKJrk1BBv8Kv36+Hc9xlP7OjDbU9LJUZh89SIlom0HQ7BYjyTQUeHt2gOV6d Pj3X+i8Bqc/q6SAE5tdkM+ASpfQxoMNDcPUktMfey+zOgzlDgF/4C/GkC3WO0l99y/ VcdEv3jZhSZIMTXJ6U3iaWhtZvXqNLZZ8E+gWvb8= From: Anshuman Khandual To: linux-arm-kernel@lists.infradead.org, catalin.marinas@arm.com, will@kernel.org Cc: Anshuman Khandual , Ryan Roberts , Mark Rutland , Lorenzo Stoakes , Andrew Morton , David Hildenbrand , Mike Rapoport , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH V3 03/14] arm64/mm: Route all pgtable writes via ptval_set() Date: Tue, 17 Nov 2026 12:36:02 +0530 Message-ID: <20261117070613.1183243-4-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20261117070613.1183243-1-anshuman.khandual@arm.com> References: <20261117070613.1183243-1-anshuman.khandual@arm.com> X-Archived-By: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20261116_230635_430692_3A3E12BE X-CRM114-Status: GOOD ( 12.53 ) X-Spam-Score: -4.4 (----) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=4.0.1 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on aws-us-west-2-korg-listserv-alma10-1 Currently ptval_set() is defined as WRITE_ONCE() but this will change for D128 pgtable builds, for which WRITE_ONCE() is not sufficient for single copy atomicity. In future this infrastructure can be used for D128 to maintain single copy atomicity semantics with inline asm blocks. Cc: Catalin Marinas Cc: Will Deacon Cc: Ryan Roberts Cc: Mark Rutland Cc: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org Reviewed-by: David Hildenbrand (Arm) Signed-off-by: Anshuman Khandual --- arch/arm64/include/asm/pgtable.h | 11 ++++++----- arch/arm64/mm/mmu.c | 4 ++-- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 1c982ccc1e5e3..0ce2e59e09198 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -85,6 +85,7 @@ static inline void arch_leave_lazy_mmu_mode(void) } #define ptval_get(x) READ_ONCE(x) +#define ptval_set(x, val) WRITE_ONCE(x, val) #define pmdp_get pmdp_get static inline pmd_t pmdp_get(const pmd_t *pmdp) @@ -386,7 +387,7 @@ static inline pte_t pte_clear_uffd(pte_t pte) static inline void __set_pte_nosync(pte_t *ptep, pte_t pte) { - WRITE_ONCE(*ptep, pte); + ptval_set(*ptep, pte); } static inline void __set_pte_complete(pte_t pte) @@ -815,7 +816,7 @@ static inline void set_pmd(pmd_t *pmdp, pmd_t pmd) } #endif /* __PAGETABLE_PMD_FOLDED */ - WRITE_ONCE(*pmdp, pmd); + ptval_set(*pmdp, pmd); if (pmd_valid(pmd)) queue_pte_barriers(); @@ -877,7 +878,7 @@ static inline void set_pud(pud_t *pudp, pud_t pud) return; } - WRITE_ONCE(*pudp, pud); + ptval_set(*pudp, pud); if (pud_valid(pud)) queue_pte_barriers(); @@ -956,7 +957,7 @@ static inline void set_p4d(p4d_t *p4dp, p4d_t p4d) return; } - WRITE_ONCE(*p4dp, p4d); + ptval_set(*p4dp, p4d); queue_pte_barriers(); } @@ -1081,7 +1082,7 @@ static inline void set_pgd(pgd_t *pgdp, pgd_t pgd) return; } - WRITE_ONCE(*pgdp, pgd); + ptval_set(*pgdp, pgd); queue_pte_barriers(); } diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index d0a4b8961c7d1..b5bff68dff764 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -75,7 +75,7 @@ void noinstr set_swapper_pgd(pgd_t *pgdp, pgd_t pgd) * writable in the kernel mapping. */ if (rodata_is_rw) { - WRITE_ONCE(*pgdp, pgd); + ptval_set(*pgdp, pgd); dsb(ishst); isb(); return; @@ -83,7 +83,7 @@ void noinstr set_swapper_pgd(pgd_t *pgdp, pgd_t pgd) spin_lock(&swapper_pgdir_lock); fixmap_pgdp = pgd_set_fixmap(__pa_symbol(pgdp)); - WRITE_ONCE(*fixmap_pgdp, pgd); + ptval_set(*fixmap_pgdp, pgd); /* * We need dsb(ishst) here to ensure the page-table-walker sees * our new entry before set_p?d() returns. The fixmap's -- 2.43.0