From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 43E9F37D101; Tue, 24 Mar 2026 18:16:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.12 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774376198; cv=none; b=tmdT3C/9zmQBC7duMqeCjNXEZ4GNsvIpI8WsltvQbeNPjqHW1dKkIePy5h6/UBnZJkwu9Ax6CcbiV4JMW1A5lacSZgIcHvhcwqP+WgBdOaJpyQbV1UJODNm6maOvKbUew+u50NtaLmIPwOPoR2uI5X5FqSSYzD/n9ykLX/DmwbM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774376198; c=relaxed/simple; bh=hpBLvfVKe2gXEk92KVwlFaLVWr1gtfKzkp/0ToQ1c6s=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=lqFbij9AxmGVu7AY5GP7jkpBpcaL4BoNYUd/ZuqpqHtSVcbpxbNYC7kgYRph4gClJIPgjbx5XJsDPRWOt/QKoXRhaI13jdBPE5q/uroaIQ2e2hmiYsrqQR7M5pwnbYntHKDBVrzoUDT0UwfoKj80Bepu2ZXcLECrMBhiS1xyS5I= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=bXgM8zAt; arc=none smtp.client-ip=198.175.65.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="bXgM8zAt" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1774376197; x=1805912197; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=hpBLvfVKe2gXEk92KVwlFaLVWr1gtfKzkp/0ToQ1c6s=; b=bXgM8zAtIDFIgmrGUcRxoiW3nNUfMBFgHFuZfPW8OUpQUB6MquhToITg oSYVp48rpGSxworWmSkqqvhomKZdn5rrlH3rDacxBVxxxiDpm4QznsGbF ghZsxl5cMwj7h3nKk2izhayD2UOKgzRVdZRov8bvsblIHaD8Vhr3nv5// T4IL/h/5gNDxYPnfNuAQm/YbpdLVwUIdpTuEhYqabxvq3+CNLBsSf+msc ryn4rd8IPkVPWncg8CBG/l8VGzDs/AgeXkh99sz2MYkO5CmdJp0q1SkCy +gGTAEVxY8qFMFoMRfEmqsTlSPzlTRisxn4YNoVuWGrZ1nUYArxgXwfU+ w==; X-CSE-ConnectionGUID: Lxfk0Yx/SZuv/sn18eXAug== X-CSE-MsgGUID: r2Jb4fZERhW5Ljrz6RVg9A== X-IronPort-AV: E=McAfee;i="6800,10657,11739"; a="86879393" X-IronPort-AV: E=Sophos;i="6.23,138,1770624000"; d="scan'208";a="86879393" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2026 11:16:37 -0700 X-CSE-ConnectionGUID: 12myZWtxTzeYodboi/+sXw== X-CSE-MsgGUID: qYXCfrdjSCqtsl1dk/Dksw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,138,1770624000"; d="scan'208";a="262362803" Received: from guptapa-desk.jf.intel.com (HELO desk) ([10.165.239.46]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2026 11:16:37 -0700 Date: Tue, 24 Mar 2026 11:16:36 -0700 From: Pawan Gupta To: x86@kernel.org, Jon Kohler , Nikolay Borisov , "H. Peter Anvin" , Josh Poimboeuf , David Kaplan , Sean Christopherson , Borislav Petkov , Dave Hansen , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , KP Singh , Jiri Olsa , "David S. Miller" , David Laight , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , David Ahern , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , Stanislav Fomichev , Hao Luo , Paolo Bonzini , Jonathan Corbet Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Asit Mallick , Tao Zhang , bpf@vger.kernel.org, netdev@vger.kernel.org, linux-doc@vger.kernel.org Subject: [PATCH v8 01/10] x86/bhi: x86/vmscape: Move LFENCE out of clear_bhb_loop() Message-ID: <20260324-vmscape-bhb-v8-1-68bb524b3ab9@linux.intel.com> X-Mailer: b4 0.15-dev References: <20260324-vmscape-bhb-v8-0-68bb524b3ab9@linux.intel.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260324-vmscape-bhb-v8-0-68bb524b3ab9@linux.intel.com> Currently, BHB clearing sequence is followed by an LFENCE to prevent transient execution of subsequent indirect branches prematurely. However, LFENCE barrier could be unnecessary in certain cases. For example, when kernel is using BHI_DIS_S mitigation, and BHB clearing is only needed for userspace. In such cases, LFENCE is redundant because ring transitions would provide the necessary serialization. Below is a quick recap of BHI mitigation options: On Alder Lake and newer - BHI_DIS_S: Hardware control to mitigate BHI in ring0. This has low performance overhead. - Long loop: Alternatively, longer version of BHB clearing sequence can be used to mitigate BHI. It can also be used to mitigate BHI variant of VMSCAPE. This is not yet implemented in Linux. On older CPUs - Short loop: Clears BHB at kernel entry and VMexit. The "Long loop" is effective on older CPUs as well, but should be avoided because of unnecessary overhead. On Alder Lake and newer CPUs, eIBRS isolates the indirect targets between guest and host. But when affected by the BHI variant of VMSCAPE, a guest's branch history may still influence indirect branches in userspace. This also means the big hammer IBPB could be replaced with a cheaper option that clears the BHB at exit-to-userspace after a VMexit. In preparation for adding the support for BHB sequence (without LFENCE) on newer CPUs, move the LFENCE to the caller side after clear_bhb_loop() is executed. Allow callers to decide whether they need the LFENCE or not. This adds a few extra bytes to the call sites, but it obviates the need for multiple variants of clear_bhb_loop(). Suggested-by: Dave Hansen Reviewed-by: Nikolay Borisov Signed-off-by: Pawan Gupta --- arch/x86/entry/entry_64.S | 5 ++++- arch/x86/include/asm/nospec-branch.h | 4 ++-- arch/x86/net/bpf_jit_comp.c | 2 ++ 3 files changed, 8 insertions(+), 3 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 42447b1e1dff..3a180a36ca0e 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -1528,6 +1528,9 @@ SYM_CODE_END(rewind_stack_and_make_dead) * refactored in the future if needed. The .skips are for safety, to ensure * that all RETs are in the second half of a cacheline to mitigate Indirect * Target Selection, rather than taking the slowpath via its_return_thunk. + * + * Note, callers should use a speculation barrier like LFENCE immediately after + * a call to this function to ensure BHB is cleared before indirect branches. */ SYM_FUNC_START(clear_bhb_loop) ANNOTATE_NOENDBR @@ -1562,7 +1565,7 @@ SYM_FUNC_START(clear_bhb_loop) sub $1, %ecx jnz 1b .Lret2: RET -5: lfence +5: pop %rbp RET SYM_FUNC_END(clear_bhb_loop) diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h index 4f4b5e8a1574..70b377fcbc1c 100644 --- a/arch/x86/include/asm/nospec-branch.h +++ b/arch/x86/include/asm/nospec-branch.h @@ -331,11 +331,11 @@ #ifdef CONFIG_X86_64 .macro CLEAR_BRANCH_HISTORY - ALTERNATIVE "", "call clear_bhb_loop", X86_FEATURE_CLEAR_BHB_LOOP + ALTERNATIVE "", "call clear_bhb_loop; lfence", X86_FEATURE_CLEAR_BHB_LOOP .endm .macro CLEAR_BRANCH_HISTORY_VMEXIT - ALTERNATIVE "", "call clear_bhb_loop", X86_FEATURE_CLEAR_BHB_VMEXIT + ALTERNATIVE "", "call clear_bhb_loop; lfence", X86_FEATURE_CLEAR_BHB_VMEXIT .endm #else #define CLEAR_BRANCH_HISTORY diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index e9b78040d703..63d6c9fa5e80 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -1624,6 +1624,8 @@ static int emit_spectre_bhb_barrier(u8 **pprog, u8 *ip, if (emit_call(&prog, func, ip)) return -EINVAL; + /* Don't speculate past this until BHB is cleared */ + EMIT_LFENCE(); EMIT1(0x59); /* pop rcx */ EMIT1(0x58); /* pop rax */ } -- 2.34.1