From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C5FECC07CB1 for ; Wed, 29 Nov 2023 11:17:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: Mime-Version:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=Bvs6p/fuNcPYyeHex/g19DAzCjtARLb647n5fMJuT0g=; b=GK+ I+HKZ83A5oU37arJZME2f9bjVGIS6vIiff8g/DPJ/tty+F2F8tEH7nLd3Hux0bFPpqz86e9fTwNnk X9lJLk/Yq8Gev3xxGdHyoOF+KzoKErPSTxL6eK6kLfLHdWqs1rd0Q9kJ+f5mQEZakRzgOt1MyAiei AqYxSa8LC8RvEhZmT0c4PHl7S3M8A1tq0l6lnp41RJQOjB7EGA9VtT07farhFSrzNd5c6zmqQjWre i8JXMFxt5CWxyg8jI05V4+01tBKMWN4LSrZ7F5PVN42YJ4Ysp0al+3xgrNphF0SJgUa4YHQsm+h3W lrHOpQffKBRGKkpV6x4VA7UeOVmxIiA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1r8IZF-007yug-1F; Wed, 29 Nov 2023 11:17:21 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1r8IZB-007yqV-38 for linux-arm-kernel@lists.infradead.org; Wed, 29 Nov 2023 11:17:19 +0000 Received: by mail-wr1-x44a.google.com with SMTP id ffacd0b85a97d-332f91f43d0so3325921f8f.1 for ; Wed, 29 Nov 2023 03:17:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1701256634; x=1701861434; darn=lists.infradead.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=TX5EHxaTqj24ud1i6FTWF2WeXcEV6QHlN/R8MH1D/5Y=; b=4nBwnDiH3xBSrfMd33psc2B82XyQDV7RgzlUHFgWrrrIkRBUBhzbSfHu5O8cqwmAzx 2hddv7vniAEnc5C6Z7nKsB6bloDqJl3Lp+PMtM4B7z0BpI4XzU26SezEttG2XrGHjeWu Bp+PlrPqPnkSkehPP+9tsySk+X2sKCr8z/t0Go84MK6W+Onl2REDfbf6ey/syER8dCwC ADe+dNzDi/Fu60doMU+tw+fXw7B6zzpcXFZVSfh7lxuJ4BcddhQ3AtQIAMRfzATmH1C+ zACZXkC12Xq4G68HJ6Tnof/aHxdsf+vLELwkwG1bxx4KSSDHIJLsAr6qtxnVBDB8tzXB 4IqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701256634; x=1701861434; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=TX5EHxaTqj24ud1i6FTWF2WeXcEV6QHlN/R8MH1D/5Y=; b=QTfZYUmZ1BQZdWBctlD7XrW4oFDZs+c9Bea6A+SDffzyZQlCyZXsMSp5/BJquBC4n8 EiQx49Zd4FK4pqHu2PSzN5eZ4GWNbmR5PNrWoRnp8MRloKD6mUseCO6c/TFykhRFJhW+ EdyTGadHD/kUtnWjR2SAYfotk5YlkZAn29RM3A5c/dfB9Tdeve1W8/+Rg6YxPtwnEp8B U4SkPKLbQzwwV7ZjeRq+6dwdBnaiuRd38YGJqcdTjFxD1MFEqgY0p0pplnaT0qkWJLM0 WfQjQ+rkgYSl0NcOY13CJT7Jn0/AYtEgKw8frDNQD/BiCTIHTmW/E2dTSdCkZCmvEynk PlZQ== X-Gm-Message-State: AOJu0YwY4QwFqH3lfD4TLiOtwgeKEKaIzq2fzKLHO5yhLbh8rre+ynhB b270t1C+uXFJEEbJftxewMUQPKZ5x+FlhO9aIxt+QogQOrIoBSUTHT6A7ySC4uxQ65TcKUp32uB b59Vd0sG18g/EnKMLESSkpT2Aoi7TMR9QIbkt96AV/h8lYfkjzVjBqWO2uFlV5DfdrFEZgjvvs2 U= X-Google-Smtp-Source: AGHT+IF1kTq9sUw78wfnxi9ifTnRGaUuhx+Q2jWtIPT4ltFnu91GJ0/egFhO/fFfnfKUdmH5EjcdieKx X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a5d:5249:0:b0:333:18c:5172 with SMTP id k9-20020a5d5249000000b00333018c5172mr112722wrc.6.1701256633931; Wed, 29 Nov 2023 03:17:13 -0800 (PST) Date: Wed, 29 Nov 2023 12:15:56 +0100 Mime-Version: 1.0 X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=8005; i=ardb@kernel.org; h=from:subject; bh=m8DKP/EDMM7TSEkybbvNlRy6SHcKVvOgC7d5Ys7el+4=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JITVdNqe6JCPx1ZfClqyL/899ythuHlPJxvh6dkvtPe6EW Xr/K3g6SlkYxDgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwES+OTIy3BfZVLdM3i1HtF3G lOO5uinDnqp95VY6Jft4m92XllQcZWT4YO3eorKJbZnstIMH+AxjQ7jfn49s2D7xmJOUd81mLzk 2AA== X-Mailer: git-send-email 2.43.0.rc1.413.gea7ed67945-goog Message-ID: <20231129111555.3594833-43-ardb@google.com> Subject: [PATCH v6 00/41] arm64: Reorganize kernel VA space for LPA2 From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland , Ryan Roberts , Anshuman Khandual , Kees Cook X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231129_031718_010207_D9CCF451 X-CRM114-Status: GOOD ( 26.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel At the request of Catalin, this series was split off from my LPA2 series [0] in order to make the changes a bit more manageable. This series reorganizes the kernel VA space, and refactors/replaces the early mapping code so that: - everything is done only once, in the appropriate order; - everything is done with the MMU and caches enabled (*) - everything is done from C code (notably, 100s of lines of incomprehensible asm code are removed from head.S). (*) the initial ID map will be populated with the MMU and caches disabled if that is how we entered from the bootloader. This is important for LPA2, but also for other future extensions to the page table format, as managing this entirely in early asm code as we do today would become intractable. This applies also to things such as copying the KAsan shadow or the fixmap from the early page tables into the permanent ones - this is all being removed by this series. This approach also ensures that we never execute from writable memory, or parse the DT (which is external input) while the text/rodata segments are mapped writable; this is an important general hardening principle, but also a prerequisite for adding WXN support (which is implemented in the second half of the series that has been omitted from this drop) Another notable difference implemented by this series is the fact that the permanent ID map always covers 48 bits of VA space, and is no longer tied to the size of the kernel VA space. This removes awkward logic to add a translation level above PGD level, and will be beneficial for other reasons too (it permits future changes in the EFI logic to get rid of SetVirtualAddressMap() entirely) Changes since v5 [1]: - add helpers to deal with CPU feature overrides, rather than applying the value and mask directly - this is necessary because an override may be invalid for the field that we care about or for another field that shares the same CPUID system register - add missing patch to make __cpu_replace_ttbr1() out of line and insert it in the correct place in the series to ensure bisectability - incorporate maz's strict type changes (and more) into the prel64 handling in the early idreg override code Changes since v4: - merge a couple of followup tweaks for issues that were reported while the v4 was briefly queued up and pulled into -next - rebase onto v6.7-rc1 - omit LVA/LPA2 and WXN related changes [0] https://lore.kernel.org/all/20230912141549.278777-63-ardb@google.com/ [1] https://lore.kernel.org/all/20231124101840.944737-41-ardb@google.com/ Cc: Catalin Marinas Cc: Will Deacon Cc: Marc Zyngier Cc: Mark Rutland Cc: Ryan Roberts Cc: Anshuman Khandual Cc: Kees Cook Ard Biesheuvel (41): arm64: kernel: Disable latent_entropy GCC plugin in early C runtime arm64: mm: Take potential load offset into account when KASLR is off arm64: mm: get rid of kimage_vaddr global variable arm64: mm: Move PCI I/O emulation region above the vmemmap region arm64: mm: Move fixmap region above vmemmap region arm64: ptdump: Allow all region boundaries to be defined at boot time arm64: ptdump: Discover start of vmemmap region at runtime arm64: vmemmap: Avoid base2 order of struct page size to dimension region arm64: mm: Reclaim unused vmemmap region for vmalloc use arm64: kaslr: Adjust randomization range dynamically arm64: kernel: Manage absolute relocations in code built under pi/ arm64: kernel: Don't rely on objcopy to make code under pi/ __init arm64: head: move relocation handling to C code arm64: idreg-override: Omit non-NULL checks for override pointer arm64: idreg-override: Prepare for place relative reloc patching arm64: idreg-override: Avoid parameq() and parameqn() arm64: idreg-override: avoid strlen() to check for empty strings arm64: idreg-override: Avoid sprintf() for simple string concatenation arm64: idreg-override: Avoid kstrtou64() to parse a single hex digit arm64/kernel: Move 'nokaslr' parsing out of early idreg code arm64: idreg-override: Move to early mini C runtime arm64: kernel: Remove early fdt remap code arm64: head: Clear BSS and the kernel page tables in one go arm64: Move feature overrides into the BSS section arm64: head: Run feature override detection before mapping the kernel arm64: head: move dynamic shadow call stack patching into early C runtime arm64: cpufeature: Add helper to test for CPU feature overrides arm64: kaslr: Use feature override instead of parsing the cmdline again arm64: idreg-override: Create a pseudo feature for rodata=off arm64: Add helpers to probe local CPU for PAC and BTI support arm64: head: allocate more pages for the kernel mapping arm64: head: move memstart_offset_seed handling to C code arm64: mm: Make kaslr_requires_kpti() a static inline arm64: mmu: Make __cpu_replace_ttbr1() out of line arm64: head: Move early kernel mapping routines into C code arm64: mm: Use 48-bit virtual addressing for the permanent ID map arm64: pgtable: Decouple PGDIR size macros from PGD/PUD/PMD levels arm64: kernel: Create initial ID map from C code arm64: mm: avoid fixmap for early swapper_pg_dir updates arm64: mm: omit redundant remap of kernel image arm64: Revert "mm: provide idmap pointer to cpu_replace_ttbr1()" arch/arm64/include/asm/archrandom.h | 2 - arch/arm64/include/asm/assembler.h | 14 - arch/arm64/include/asm/cpufeature.h | 77 ++++ arch/arm64/include/asm/fixmap.h | 1 - arch/arm64/include/asm/kasan.h | 2 - arch/arm64/include/asm/kernel-pgtable.h | 128 +++--- arch/arm64/include/asm/memory.h | 20 +- arch/arm64/include/asm/mmu.h | 40 +- arch/arm64/include/asm/mmu_context.h | 53 +-- arch/arm64/include/asm/pgtable.h | 10 +- arch/arm64/include/asm/scs.h | 36 +- arch/arm64/include/asm/setup.h | 3 - arch/arm64/kernel/Makefile | 7 +- arch/arm64/kernel/cpufeature.c | 65 +-- arch/arm64/kernel/head.S | 428 ++------------------ arch/arm64/kernel/image-vars.h | 33 ++ arch/arm64/kernel/kaslr.c | 11 +- arch/arm64/kernel/module.c | 2 +- arch/arm64/kernel/pi/Makefile | 28 +- arch/arm64/kernel/{ => pi}/idreg-override.c | 182 +++++---- arch/arm64/kernel/pi/kaslr_early.c | 78 +--- arch/arm64/kernel/pi/map_kernel.c | 186 +++++++++ arch/arm64/kernel/pi/map_range.c | 100 +++++ arch/arm64/kernel/{ => pi}/patch-scs.c | 36 +- arch/arm64/kernel/pi/pi.h | 36 ++ arch/arm64/kernel/pi/relacheck.c | 130 ++++++ arch/arm64/kernel/pi/relocate.c | 64 +++ arch/arm64/kernel/setup.c | 22 - arch/arm64/kernel/vmlinux.lds.S | 17 +- arch/arm64/kvm/mmu.c | 15 +- arch/arm64/mm/fixmap.c | 34 -- arch/arm64/mm/kasan_init.c | 19 +- arch/arm64/mm/mmu.c | 167 ++++---- arch/arm64/mm/proc.S | 13 +- arch/arm64/mm/ptdump.c | 56 ++- 35 files changed, 1070 insertions(+), 1045 deletions(-) rename arch/arm64/kernel/{ => pi}/idreg-override.c (59%) create mode 100644 arch/arm64/kernel/pi/map_kernel.c create mode 100644 arch/arm64/kernel/pi/map_range.c rename arch/arm64/kernel/{ => pi}/patch-scs.c (89%) create mode 100644 arch/arm64/kernel/pi/pi.h create mode 100644 arch/arm64/kernel/pi/relacheck.c create mode 100644 arch/arm64/kernel/pi/relocate.c -- 2.43.0.rc1.413.gea7ed67945-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel