From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5D313C4167B for ; Wed, 29 Nov 2023 11:19:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=PPROgBkmiA78p/rt1EkboVwcSaR2LA5m/1aznhkYFkc=; b=I0KSqT/iIrAjrd3yuAVWYMZhTl Udi/L+F86efJ5O+OXGIrLwAu3rtG9N+jc4FQIT1Kz3HrvAR9p4KjX8MX8DHivdf9TxVHsNYcrxDAK e80frjBBAti7ztSr8eTTPm+bsK1Wrh38Y2jYD/sAnBRIFIVhoX/s6B2BF0qYoqETVyTEIW3sUS0lm YQNtIhGyb20D6pqKuPYOHV2Jv1YU9WEv/b60K8LTYD95q1vN49ilcCK5L99h4+Gt8Nmsd2f7yNsah XFp86xbaDh2l6jJdMpzt712BkNvEUp9fEquPZuKbiAEYVwXneFAVhW/1UAcXwaufBPJ/RLKbOFEdz cmw8EjaA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1r8IbE-0080jI-2D; Wed, 29 Nov 2023 11:19:24 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1r8Iag-0080Fa-1A for linux-arm-kernel@bombadil.infradead.org; Wed, 29 Nov 2023 11:18:50 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: Message-ID:References:Mime-Version:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=okbm1hGtQ1g9Bp/OCoOpAcHZ+0GcwY8uwXUSlYDugV4=; b=df+XAX0Co2QDMP8uG3xsFU3Btn NlzRRElxSiqctHXS5u1Ee9vJ1etKyA10ncayqJY3Oz5VrX2r/wPzO73IJg1UGoAuiCMXEPYEbkIBP MP/yMjmbt2/nF/HyL/6ZKwWd1Q2xcqyLYO82XjFFlzy+wRdc/pdcgckrbnN0iQdDAMK+81Gi/m674 J+AtLmQywH52+iS5o3p0I0lHV4b0t5Fl9Wksm1PxlaqW89m0kDrx9mNYglt4IabzZWVRVyMFPUUNu dyizGc4RszX8K9PE1pr1ep85Ed2SFboL1KM1CPLzfd88TbMM2+fqmexSSksqOZg8XrSnZujymboKm YUf1qpYA==; Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by casper.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1r8Iad-00DKTV-0x for linux-arm-kernel@lists.infradead.org; Wed, 29 Nov 2023 11:18:49 +0000 Received: by mail-wr1-x449.google.com with SMTP id ffacd0b85a97d-332e61e2480so4211370f8f.0 for ; Wed, 29 Nov 2023 03:18:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1701256725; x=1701861525; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=okbm1hGtQ1g9Bp/OCoOpAcHZ+0GcwY8uwXUSlYDugV4=; b=1JJltV7TEfmoJ8VP/Fk7EsBzQUHbUDZ1ChOmx/DUHPBdvh7g6UZfGa85NVIlOZEfBQ q4xY7qfnPVHAClnRP6IZH0UlyU5q9OrNn1fGwSUF8Q1957JKwyBVrzu0U/5jNhll/05l /PlEs/laA8a4Y5tjq82G6ymlGJBDnI7Q1lqYVMmHl5rCfk9LqKZdjWIraf3QPco+nXAv YVmWsj5bJbbwd9xiPUMskHePNrWp6pQ4nBf9+AIhQJLyscQXasZDfdXxMoP1shBsVC43 ++oayjyCFOPPVMBo6ek2BCD3khUgvwXbaulirOpts8jUW76LXet0/530knC5QhrwLk7B 4v0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701256725; x=1701861525; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=okbm1hGtQ1g9Bp/OCoOpAcHZ+0GcwY8uwXUSlYDugV4=; b=a2GOmCoU+4HUuGhXSvboL8r/tv+BG7oAfGAjaF0TmQO1hLuPA4SKAmmy58CL7/Yem2 ys/P6qxkjOjeshxH/NU0sy2EuAno8j14YhZyEEPPDyzesH1SSTMqfSoJo0SLZf59Q/H1 4iKZFd5xVZ9L+3ZdUWF4t3nqIf3HGCoYPnQKBQBGmNphPohQ9PWfqzLxc5PREEhUXkhb 6twvJfJHKlXbuyX05K5el+0yc7QxE1/MBfHmWZ/iIwsBqpnb6yLuZCsJRYmptRktQoZ8 p+D+0bCndepVatErbIlRod2PMqamLtDpsFD0cebzhs8pWDRhh+yn3SKuYKWtIqAju77H fJvA== X-Gm-Message-State: AOJu0YwkChqq+6SALCIbQl227Ir92h3xhrS6ihyoDPL0jYrAcHmPKatB rN2IMhNra9K0Ci6KQU0iqGv5KJj9zSuD0/O54Cfd1M0PUuTwBzT9b00dKua6BvqH6/XNXMVa1vp uEGMyIyrgkiPkWCAGvlCMmNek3zslXE7Z1bjerrfWt7sSpH3H5rc3+ure7X3NmCa4b8tSn725CF k= X-Google-Smtp-Source: AGHT+IHXr+HaSrAPtM+gM81IulUp9WJTj6DeH54V/EP7sS5drhAdfOfiTp5ohDs+DxjnVedDLZpdFEzw X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:adf:9c82:0:b0:332:f889:39f8 with SMTP id d2-20020adf9c82000000b00332f88939f8mr205053wre.3.1701256724520; Wed, 29 Nov 2023 03:18:44 -0800 (PST) Date: Wed, 29 Nov 2023 12:16:35 +0100 In-Reply-To: <20231129111555.3594833-43-ardb@google.com> Mime-Version: 1.0 References: <20231129111555.3594833-43-ardb@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2126; i=ardb@kernel.org; h=from:subject; bh=1/UrX1+wuipaCeD3191GpWD3IQxBEzmiLyVFVdrLQFM=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JITVddjq7c/MNtpBfHzpZ1bXsZpgK/lr/Q/fS5Tw9fj8D5 0m3/97vKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABOptWL4ZzHTegXrvaes0b/W 3eg2vL/IVLrw8vPclXx5e1NVerin8DAyTBJO+Knv1jW3sDoob62z/Z3SK5vrnws2/JY2PrfnLO9 +dgA= X-Mailer: git-send-email 2.43.0.rc1.413.gea7ed67945-goog Message-ID: <20231129111555.3594833-82-ardb@google.com> Subject: [PATCH v6 39/41] arm64: mm: avoid fixmap for early swapper_pg_dir updates From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland , Ryan Roberts , Anshuman Khandual , Kees Cook X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231129_111847_121490_0FCAB679 X-CRM114-Status: GOOD ( 14.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel Early in the boot, when .rodata is still writable, we can poke swapper_pg_dir entries directly, and there is no need to go through the fixmap. After a future patch, we will enter the kernel with swapper_pg_dir already active, and early swapper_pg_dir updates for creating the fixmap page table hierarchy itself cannot go through the fixmap for obvious reaons. So let's keep track of whether rodata is writable, and update the descriptor directly in that case. As the same reasoning applies to early KASAN init, make the function noinstr as well. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index cef7573d156d..5996b374ff8a 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -55,6 +55,8 @@ EXPORT_SYMBOL(kimage_voffset); u32 __boot_cpu_mode[] = { BOOT_CPU_MODE_EL2, BOOT_CPU_MODE_EL1 }; +static bool rodata_is_rw __ro_after_init = true; + /* * The booting CPU updates the failed status @__early_cpu_boot_status, * with MMU turned off. @@ -71,10 +73,21 @@ EXPORT_SYMBOL(empty_zero_page); static DEFINE_SPINLOCK(swapper_pgdir_lock); static DEFINE_MUTEX(fixmap_lock); -void set_swapper_pgd(pgd_t *pgdp, pgd_t pgd) +void noinstr set_swapper_pgd(pgd_t *pgdp, pgd_t pgd) { pgd_t *fixmap_pgdp; + /* + * Don't bother with the fixmap if swapper_pg_dir is still mapped + * writable in the kernel mapping. + */ + if (rodata_is_rw) { + WRITE_ONCE(*pgdp, pgd); + dsb(ishst); + isb(); + return; + } + spin_lock(&swapper_pgdir_lock); fixmap_pgdp = pgd_set_fixmap(__pa_symbol(pgdp)); WRITE_ONCE(*fixmap_pgdp, pgd); @@ -628,6 +641,7 @@ void mark_rodata_ro(void) * to cover NOTES and EXCEPTION_TABLE. */ section_size = (unsigned long)__init_begin - (unsigned long)__start_rodata; + WRITE_ONCE(rodata_is_rw, false); update_mapping_prot(__pa_symbol(__start_rodata), (unsigned long)__start_rodata, section_size, PAGE_KERNEL_RO); -- 2.43.0.rc1.413.gea7ed67945-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel