From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 64081C5DF84 for ; Tue, 18 Aug 2026 14:12:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=+iTHskaaics2YOb7mV8HW+F0QqHNBq2e6dwMrWUoe7A=; b=xXIu2QOEdB6FZMVxNZSEcGQ9Zf uX/ckq4nIoSd9P44Vbe1Xa5BTSWHI7tcH3jWGsEHITK7FYUHBTltGvdDvAw+es/8AyizH8OxLMpFT Z52KGPd0yUPptD2m0QmuONNwdx11lGrd16J8xOJ57JL1sFilrha3my5cSBgLaQjC4uruApkex5O0H 6pcRZDQo7fh+Y4D+RIHvvkd5Dixl3WwaB2xp28F1p3rJwQkYOzintBxZCotcCdRUHtz7TTPifFjDI 9ry4nYkHWkA7QLcCwXeqg2DlM12iktCjRfit40OAWFgikliX3/hglTqeWoxSFDMJf9oIkIXr9jiMX 21xjwwuA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.99.1 #2 (Red Hat Linux)) id 1wwKYN-000000087SZ-0h5T; Tue, 18 Aug 2026 14:12:35 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.99.1 #2 (Red Hat Linux)) id 1wwKYF-000000087Ji-3HOB for linux-arm-kernel@lists.infradead.org; Tue, 18 Aug 2026 14:12:30 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E5BE019F6; Tue, 18 Aug 2026 07:12:22 -0700 (PDT) Received: from localhost.localdomain (e123572-lin.cambridge.arm.com [10.2.208.46]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2DB773F85F; Tue, 18 Aug 2026 07:12:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1787062346; bh=UOSa72V7kKY4U1NE8IVeRcWRuq2og9Q1IdwEs06tE+M=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=OtW0CLTC1Rl3v+mFIeIbmL9SP3PFT6q+Yzw0Yul71S3Dg9s8AEUxGtiWdWJ75Zc7m GjLsxq8urpItPmMec24ABJaiZG/s+rA544vQe6QXzhiDqgDc0FvQx/LjxCz812YO8L 7WsIjwCUnEUfPJWmjSH8G4j2HVHqbjFjEBkGOrLk= From: Kevin Brodsky Date: Tue, 18 Aug 2026 15:08:55 +0100 Subject: [PATCH RFC v9 13/25] mm: kpkeys: Introduce early page table allocator MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260818-kpkeys-v9-13-743ad31b2c8f@arm.com> References: <20260818-kpkeys-v9-0-743ad31b2c8f@arm.com> In-Reply-To: <20260818-kpkeys-v9-0-743ad31b2c8f@arm.com> To: linux-hardening@vger.kernel.org Cc: Kevin Brodsky , Andrew Morton , Andy Lutomirski , Catalin Marinas , Dave Hansen , "David Hildenbrand (Arm)" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linu Cherian , Linus Walleij , Marc Zyngier , Mark Brown , Matthew Wilcox , Maxwell Bland , "Mike Rapoport (IBM)" , Peter Zijlstra , Pierre Langlois , =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= , Quentin Perret , Rick Edgecombe , Ryan Roberts , Vlastimil Babka , Will Deacon , Yang Shi , Yeoreum Yun , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org, Ira Weiny , Lorenzo Stoakes , Thomas Gleixner X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1787062276; l=6609; i=kevin.brodsky@arm.com; s=20260427; h=from:subject:message-id; bh=UOSa72V7kKY4U1NE8IVeRcWRuq2og9Q1IdwEs06tE+M=; b=jAm+phCdu4Mt53rUVdHDwcVsszaCBzeso2mFftAdJqvKcJ846KaIFHot07GHWpB22/RqDatQT kZAYTLS4BwoCWBG0h6rw8zKfTuLp752UziJbRJHktjH/gTeyNNMzMpY X-Developer-Key: i=kevin.brodsky@arm.com; a=ed25519; pk=N2QG+eJKrvkNovwhhwJhnJ4+ScVfsGCHldmqLfcMTFs= X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260818_071227_968853_D46F8A8A X-CRM114-Status: GOOD ( 25.57 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The kpkeys_hardened_pgtables feature aims to protect all page table pages (PTPs) by mapping them with a privileged pkey. This is primarily handled by kpkeys_pgtable_alloc(), called from pagetable_alloc(). However, this does not cover PTPs allocated early, before the buddy allocator is available. These PTPs are allocated by architecture code, either 1. from static pools or 2. using the memblock allocator, and should also be protected. This patch addresses the second category: PTPs allocated via memblock. Such PTPs are notably used to create the linear map. Protecting them as soon as they are allocated would require modifying the linear map while it is being created, which seems at best difficult. Instead, a simple allocator is introduced, obtaining pages from memblock and keeping track of all allocated ranges to set their pkey once it is safe to do so. PTPs allocated at that stage are not freed, so there is no need to manage a free list. Since kpkeys_hardened_pgtables currently requires the linear map to be PTE-mapped, we can directly allocate page by page using memblock, without intermediate cache. We rely on memblock allocating contiguous pages to minimise the number of tracked ranges. The number of PTPs required to create the linear map is proportional to the amount of available memory, which means it may be large. At such an early point, the memblock allocator may however only track a limited number of regions, and we size the tracking array (allocated_ranges) accordingly. The array may be quite large as a result (16KB on arm64), but it is discarded once boot has completed. Signed-off-by: Kevin Brodsky --- include/linux/kpkeys.h | 7 +++ mm/kpkeys_hardened_pgtables.c | 115 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 122 insertions(+) diff --git a/include/linux/kpkeys.h b/include/linux/kpkeys.h index 288c8853eed6..23526d3f9ab3 100644 --- a/include/linux/kpkeys.h +++ b/include/linux/kpkeys.h @@ -125,6 +125,8 @@ void kpkeys_pgtable_free(struct page *page, unsigned int order); */ void kpkeys_hardened_pgtables_init(void); +phys_addr_t kpkeys_physmem_pgtable_alloc(void); + #else /* CONFIG_KPKEYS_HARDENED_PGTABLES */ static inline bool kpkeys_hardened_pgtables_enabled(void) @@ -146,6 +148,11 @@ static inline void kpkeys_pgtable_free(struct page *page, unsigned int order) {} static inline void kpkeys_hardened_pgtables_init(void) {} +static inline phys_addr_t kpkeys_physmem_pgtable_alloc(void) +{ + return 0; +} + #endif /* CONFIG_KPKEYS_HARDENED_PGTABLES */ #endif /* _LINUX_KPKEYS_H */ diff --git a/mm/kpkeys_hardened_pgtables.c b/mm/kpkeys_hardened_pgtables.c index eca798207107..9e1e27c2cc96 100644 --- a/mm/kpkeys_hardened_pgtables.c +++ b/mm/kpkeys_hardened_pgtables.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0-only #include +#include #include #include @@ -30,6 +31,9 @@ static int set_pkey_default(struct page *page, unsigned int nr_pages) return ret; } +/* pkeys physmem allocator (PPA) - implemented below */ +static void ppa_finalize(void); + struct page *kpkeys_pgtable_alloc(gfp_t gfp, unsigned int order) { struct page *page; @@ -60,4 +64,115 @@ void __init kpkeys_hardened_pgtables_init(void) return; static_branch_enable(&kpkeys_hardened_pgtables_key); + + ppa_finalize(); +} + +/* + * pkeys physmem allocator (PPA): allocator for very early page tables + * (especially for creating the linear map), based on memblock. Allocated + * ranges are tracked so that their pkey can be set once it is safe to do so. + */ + +/* + * We may have to track many ranges when allocating page tables for the linear + * map, as their number grows with the amount of available memory. Assuming that + * memblock returns contiguous blocks whenever possible, the number of ranges + * to track cannot however exceed the number of regions that memblock itself + * tracks. memblock_allow_resize() hasn't been called yet at that point, so + * that limit is the size of the statically allocated array. + */ +#define PHYSMEM_MAX_RANGES INIT_MEMBLOCK_MEMORY_REGIONS + +struct physmem_range { + phys_addr_t addr; + phys_addr_t size; +}; + +struct pkeys_physmem_allocator { + struct physmem_range allocated_ranges[PHYSMEM_MAX_RANGES]; + unsigned int nr_allocated_ranges; +}; + +static struct pkeys_physmem_allocator pkeys_physmem_allocator __initdata; + +static int __init set_pkey_pgtable_phys(phys_addr_t pa, phys_addr_t size) +{ + unsigned long addr = (unsigned long)__va(pa); + int ret; + + ret = set_memory_pkey(addr, size / PAGE_SIZE, KPKEYS_PKEY_PGTABLES); + + WARN_ON(ret); + return ret; +} + +static bool __init ppa_try_extend_last_range(phys_addr_t addr, phys_addr_t size) +{ + struct pkeys_physmem_allocator *ppa = &pkeys_physmem_allocator; + struct physmem_range *range; + + if (!ppa->nr_allocated_ranges) + return false; + + range = &ppa->allocated_ranges[ppa->nr_allocated_ranges - 1]; + + /* Merge the new range into the last range if they are contiguous */ + if (addr == range->addr + range->size) { + range->size += size; + return true; + } else if (addr + size == range->addr) { + range->addr -= size; + range->size += size; + return true; + } + + return false; +} + +static void __init ppa_register_allocated_range(phys_addr_t addr, + phys_addr_t size) +{ + struct pkeys_physmem_allocator *ppa = &pkeys_physmem_allocator; + struct physmem_range *range; + + if (!addr) + return; + + if (ppa_try_extend_last_range(addr, size)) + return; + + /* Could not extend the last range, create a new one */ + if (WARN_ON(ppa->nr_allocated_ranges >= PHYSMEM_MAX_RANGES)) + return; + + range = &ppa->allocated_ranges[ppa->nr_allocated_ranges++]; + range->addr = addr; + range->size = size; +} + +static void __init ppa_finalize(void) +{ + struct pkeys_physmem_allocator *ppa = &pkeys_physmem_allocator; + + for (unsigned int i = 0; i < ppa->nr_allocated_ranges; i++) { + struct physmem_range *range = &ppa->allocated_ranges[i]; + + set_pkey_pgtable_phys(range->addr, range->size); + } +} + +phys_addr_t __ref kpkeys_physmem_pgtable_alloc(void) +{ + size_t size = PAGE_SIZE; + phys_addr_t addr; + + addr = memblock_phys_alloc_range(size, size, 0, + MEMBLOCK_ALLOC_NOLEAKTRACE); + if (!addr) + return addr; + + ppa_register_allocated_range(addr, size); + + return addr; } -- 2.51.2