From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A41DA51C3E for ; Tue, 13 Feb 2024 12:42:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707828133; cv=none; b=jYgNzlzhzsQ3gmOmtEuFUePCdyiGuDj1qV3DvBGeQmRwd5RabKglVRiA2riqM0nwfuSn+XMTQtk1m1qP2obHGF319X6mrOLeUZoNxb1kbir3gA1adRTLSiU06wt9pYt2qgfPrO00yGxnuDEJAi7xvnHq7y71KtlKSU1dnOvi1q8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707828133; c=relaxed/simple; bh=y9rxsCe7t3+BoRvciY7zzDK+lsJfA46YxMM4xo0hlg0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=clfEwnu0L5CF6M5fFvOm5EO9Mqkpirqbz+4AG3FGDkTZRVgt7icAL/EjgR9KH1UIqA8vKyJHsq/T+0IxwXXv2TWxgI041W8BG3tdtCzx2cWleZJSImptjMGb1OCM8tbnPGVbLayHbmrFxETVUkQiir/ZFeKaMddIeq5sltEQsl0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Yd/MZTKT; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Yd/MZTKT" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6077e501cecso16413607b3.2 for ; Tue, 13 Feb 2024 04:42:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1707828131; x=1708432931; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=SCp/zoIzhRhhb9JUvWveq8Y/lRzJCxLp1biyCfwnfq0=; b=Yd/MZTKThtJqPWTOS3XmUpbmUn0VZMroErz53jkBH//5NHHiSGgVLfrtaIY/CWZdID mHqtqvaFc0WHBpvrcxqwRkbIkFyCP9oRHhM+GlOIZquzfjFXN0axurZFXJzmSQ9zk6Ha rtIWSt9bZwRyJasLb2mKEwuSsOrddM/vLAppAmYjcAWnu8O+I7buJPdNik3YXw8yBpp7 blfTW2wlpD3fmKbTR9X3QSWI94NmIUqcGO4oR4AnCIAwwXctk8MQE7mCxFQ/j3oLjbrP hDeqfT92Wj8tlCrU6L55tXZWu7Wvih+80LI7CRsYrPqSgyfifMVD2kbS4GX59puGHUdB VVKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707828131; x=1708432931; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=SCp/zoIzhRhhb9JUvWveq8Y/lRzJCxLp1biyCfwnfq0=; b=wq4VxxgYofVJ5+YSuPjA1rD016Q+vZ9v2Wplh8pyMT2Z7RLh2ZqYCOdwp2lm0WIUQW jmBE4MwwNisCYtebNEslZhyPK8PmC0tVAwKGOaMyE/TbIuzfNIH6cKKOB1eyMTghVV4r FQt0PUxcXLzphCsNmqTo/DEgFMTIBVhaGizJTAibx3W2S5DVehoji+eOz5B5A/wlb3Kp OP2xT38dlFVsN0bt8W8KfhNWy9cRLq6vh5l4A97vd7LRAct6YWl2MlaEkAXlCvwOxgOv EkdelJDgsIRTwEokHRT9zKficyGU0eZ/goEidUDOe9n33EmwqSSF1w6t9PY9z08mFxcl puuQ== X-Gm-Message-State: AOJu0Yyb4oFlidDAaecz5gCH/CCeMvyX3/EsDfuCPQVuIg9Xb81NniRJ 1HVmETQt8z1F8BRRiski6qP8bZwa8Bp9UaqeYgV67+ckc8+9nkIURgpsIbWe4jAeTYhtiw== X-Google-Smtp-Source: AGHT+IGsW3keBZXPy7F4/9lZjwJJzWayDCo+seSOOyeUx9r/VQaHkToEqzRzO0F2HfL5SWo3gImg3hPS X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a81:6d05:0:b0:607:9963:2dd4 with SMTP id i5-20020a816d05000000b0060799632dd4mr90739ywc.9.1707828130803; Tue, 13 Feb 2024 04:42:10 -0800 (PST) Date: Tue, 13 Feb 2024 13:41:48 +0100 In-Reply-To: <20240213124143.1484862-13-ardb+git@google.com> Precedence: bulk X-Mailing-List: llvm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240213124143.1484862-13-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3321; i=ardb@kernel.org; h=from:subject; bh=z98YWbAEHVe7Ntbv2WAOVJQ9O8vTY0tfpKYTE0xX9/E=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIfV0ck841z3bDM4lraYBmROE2A0KGp8WRh3hbUm0T2C9Z bj2lENHKQuDGAeDrJgii8Dsv+92np4oVes8SxZmDisTyBAGLk4BmMj+XYwM8/bZqcV2fOwPq966 6h2LxWvenZlbOGWbAk95ne/M2CKXxshwU41PiU1nTfbReNcy8diKz7P1jp4JEnhypeRl+8m1ga9 ZAA== X-Mailer: git-send-email 2.43.0.687.g38aa6559b0-goog Message-ID: <20240213124143.1484862-17-ardb+git@google.com> Subject: [PATCH v4 04/11] x86/startup_64: Defer assignment of 5-level paging global variables From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Kees Cook , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev Content-Type: text/plain; charset="UTF-8" From: Ard Biesheuvel Assigning the 5-level paging related global variables from the earliest C code using explicit references that use the 1:1 translation of memory is unnecessary, as the startup code itself does not rely on them to create the initial page tables, and this is all it should be doing. So defer these assignments to the primary C entry code that executes via the ordinary kernel virtual mapping. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/head64.c | 44 +++++++------------- 1 file changed, 14 insertions(+), 30 deletions(-) diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 4b08e321d168..4bcbd4ae2dc6 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -23,6 +23,7 @@ #include #include +#include #include #include #include @@ -77,24 +78,11 @@ static struct desc_struct startup_gdt[GDT_ENTRIES] __initdata = { [GDT_ENTRY_KERNEL_DS] = GDT_ENTRY_INIT(DESC_DATA64, 0, 0xfffff), }; -#ifdef CONFIG_X86_5LEVEL -static void __head *fixup_pointer(void *ptr, unsigned long physaddr) -{ - return ptr - (void *)_text + (void *)physaddr; -} - -static unsigned long __head *fixup_long(void *ptr, unsigned long physaddr) +static inline bool check_la57_support(void) { - return fixup_pointer(ptr, physaddr); -} - -static unsigned int __head *fixup_int(void *ptr, unsigned long physaddr) -{ - return fixup_pointer(ptr, physaddr); -} + if (!IS_ENABLED(CONFIG_X86_5LEVEL)) + return false; -static bool __head check_la57_support(unsigned long physaddr) -{ /* * 5-level paging is detected and enabled at kernel decompression * stage. Only check if it has been enabled there. @@ -102,21 +90,8 @@ static bool __head check_la57_support(unsigned long physaddr) if (!(native_read_cr4() & X86_CR4_LA57)) return false; - *fixup_int(&__pgtable_l5_enabled, physaddr) = 1; - *fixup_int(&pgdir_shift, physaddr) = 48; - *fixup_int(&ptrs_per_p4d, physaddr) = 512; - *fixup_long(&page_offset_base, physaddr) = __PAGE_OFFSET_BASE_L5; - *fixup_long(&vmalloc_base, physaddr) = __VMALLOC_BASE_L5; - *fixup_long(&vmemmap_base, physaddr) = __VMEMMAP_BASE_L5; - return true; } -#else -static bool __head check_la57_support(unsigned long physaddr) -{ - return false; -} -#endif static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdval_t *pmd) { @@ -180,7 +155,7 @@ unsigned long __head __startup_64(unsigned long physaddr, bool la57; int i; - la57 = check_la57_support(physaddr); + la57 = check_la57_support(); /* Is the address too large? */ if (physaddr >> MAX_PHYSMEM_BITS) @@ -463,6 +438,15 @@ asmlinkage __visible void __init __noreturn x86_64_start_kernel(char * real_mode (__START_KERNEL & PGDIR_MASK))); BUILD_BUG_ON(__fix_to_virt(__end_of_fixed_addresses) <= MODULES_END); + if (check_la57_support()) { + __pgtable_l5_enabled = 1; + pgdir_shift = 48; + ptrs_per_p4d = 512; + page_offset_base = __PAGE_OFFSET_BASE_L5; + vmalloc_base = __VMALLOC_BASE_L5; + vmemmap_base = __VMEMMAP_BASE_L5; + } + cr4_init_shadow(); /* Kill off the identity-map trampoline */ -- 2.43.0.687.g38aa6559b0-goog