From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail.skyhub.de (unknown [5.9.137.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A2CB1168 for ; Mon, 20 Dec 2021 11:00:23 +0000 (UTC) Received: from zn.tnic (dslb-088-067-202-008.088.067.pools.vodafone-ip.de [88.67.202.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.skyhub.de (SuperMail on ZX Spectrum 128k) with ESMTPSA id 17CF11EC0567; Mon, 20 Dec 2021 12:00:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alien8.de; s=dkim; t=1639998004; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=NVH+9Xv+mhJGrugUsoTinsKz21v2dpO7caEXunwTj4g=; b=VgsHUwyOifCQpiR3cvL/hgK0X6tE2JBvQTuLOdR9LdDOOuGWs7/JnxvJMvgkOtZ6oiHb0Z AE03A/omyUE/lLZMpgo/06bCgr+EM2B3qjftAsMUYsy7dXKd52allhqwL4axTUP36a+beI L71ePkgkKAduC5cNNi3YL25LF0ITDmE= Date: Mon, 20 Dec 2021 12:00:06 +0100 From: Borislav Petkov To: x86@kernel.org Cc: Nathan Chancellor , Yin Fengwei , Carel Si , Joerg Roedel , LKML , lkp@lists.01.org, lkp@intel.com, bfields@fieldses.org, llvm@lists.linux.dev, Nick Desaulniers Subject: [PATCH] x86/mm: Prevent early boot triple-faults with instrumentation Message-ID: References: <20211215070012.GA26582@linux.intel.com> <009391a5-468b-2a5d-1f12-44d2e3104bd6@intel.com> <20211216115838.GA23522@linux.intel.com> Precedence: bulk X-Mailing-List: llvm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: From: Borislav Petkov Commit in Fixes added a global TLB flush on the early boot path, after the kernel switches off of the trampoline page table. Compiler profiling options add additional measurement code which needs to be initialized prior to use. The global flush in x86_64_start_kernel() happens before those initializations can happen, leading to accessing invalid memory. The second issue this fixes is with KASAN: for a similar reason, kasan_early_init() needs to have happened before KASAN-instrumented functions are called. Therefore, reorder the flush to happen after the KASAN early init and prevent the compilers from adding profiling instrumentation to native_write_cr4(). Fixes: f154f290855b ("x86/mm/64: Flush global TLB on boot and AP bringup") Reported-by: "J. Bruce Fields" Reported-by: kernel test robot Signed-off-by: Borislav Petkov Link: https://lore.kernel.org/r/20211209144141.GC25654@xsang-OptiPlex-9020 --- arch/x86/kernel/cpu/common.c | 2 +- arch/x86/kernel/head64.c | 16 ++++++++++++++-- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 0083464de5e3..79b3d67addcc 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -384,7 +384,7 @@ void native_write_cr0(unsigned long val) } EXPORT_SYMBOL(native_write_cr0); -void native_write_cr4(unsigned long val) +void __no_profile native_write_cr4(unsigned long val) { unsigned long bits_changed = 0; diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 75acb6027a87..f5e80a8377ad 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -483,10 +483,12 @@ asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data) /* Kill off the identity-map trampoline */ reset_early_page_tables(); - __native_tlb_flush_global(native_read_cr4()); - clear_bss(); + /* + * This needs to happen *before* kasan_early_init() because latter maps stuff + * into that page. + */ clear_page(init_top_pgt); /* @@ -498,6 +500,16 @@ asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data) kasan_early_init(); + /* + * Flush global TLB entries which could be left over from the trampoline page + * table. + * + * This needs to happen *after* kasan_early_init() as KASAN-enabled .configs + * instrument native_write_cr4() so KASAN must be initialized for that + * instrumentation to work. + */ + __native_tlb_flush_global(this_cpu_read(cpu_tlbstate.cr4)); + idt_setup_early_handler(); copy_bootdata(__va(real_mode_data)); -- 2.29.2 -- Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette