From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 173B7306754; Thu, 16 Jul 2026 14:14:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784211242; cv=none; b=pnqfJp8kcys4gIsMyc/td9tzUWVEVbQWArgeXYqjOcbGWPF5i10KJhepkLljdQdJYRJeIAhbtChDMjLmClnbyzQpMaQzcY+nvewiC/f8SWlUMwckBZ04zqsZEZu3a4Ezv/s5M9myL8p1Ka04a0URMVx2oleZp1xF3U9LHxX8mds= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784211242; c=relaxed/simple; bh=cnBi4c0OoYvYw8OSqxa1UfIXgtcf+xGrLV7oVUMojMU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KxyDR5LqXKL4ttZAKLP4sh0ntnsCGZNy/XsrO5FdTa9k4Zmd5yXiEBGK9oNkOAFviCAK9SU4GIXA53acPvgVHDMpk3QHTF7RBRsjX1maKyJzAGjhirEsGlTjJUrluXXRNydnIWNHkcA8r/4BmzB+yWc60deJ6bS3ETi3vfG0uJg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b=ooDJ2CuY; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b="ooDJ2CuY" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A400E1F000E9; Thu, 16 Jul 2026 14:14:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=korg; t=1784211241; bh=q9w/5WXK4bUiCiVy3cYt0+yW5TcypYMIFyY+2N7WaxQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ooDJ2CuYPnB2Y1QUYH6yU+gApTVppZXDd0I9m+O48n2el7rC3QxQ4DrU+iKO16ZIu SiGCfzO+EEEIcf+nSFuCTWASDk53y8bbxd1VaagBRI1Uj1Thj8TQNae3RybHmcE9Pp GBqYRzJEaEjWSy3duZvriRmJKMNdqqaCpn2SgOFg= From: Greg Kroah-Hartman To: stable@vger.kernel.org Cc: Greg Kroah-Hartman , patches@lists.linux.dev, Pawan Gupta , Daniel Borkmann , Dave Hansen , Sasha Levin Subject: [PATCH 6.18 326/480] x86/bugs: Enable IBPB flush on BPF JIT allocation Date: Thu, 16 Jul 2026 15:31:13 +0200 Message-ID: <20260716133051.871306332@linuxfoundation.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260716133044.672218725@linuxfoundation.org> References: <20260716133044.672218725@linuxfoundation.org> User-Agent: quilt/0.69 X-stable: review X-Patchwork-Hint: ignore Precedence: bulk X-Mailing-List: patches@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit 6.18-stable review patch. If anyone has any objections, please let me know. ------------------ From: Pawan Gupta commit a3af84b0fa00ead01fcd0e28b5d773ff25990a0d upstream. Enable hardening against JIT spraying when Spectre-v2 mitigations are in use. Specifically, issue an IBPB flush on BPF JIT memory reuse. Skip enabling the IBPB flush if the BPF dispatcher is already using a retpoline sequence. This hardening applies only when BPF-JIT is in use. Guard the enabling under CONFIG_BPF_JIT so that bugs.c still builds with CONFIG_BPF_JIT=n. Signed-off-by: Pawan Gupta Acked-by: Daniel Borkmann Acked-by: Dave Hansen Signed-off-by: Daniel Borkmann Signed-off-by: Sasha Levin --- arch/x86/include/asm/nospec-branch.h | 4 +++ arch/x86/kernel/cpu/bugs.c | 50 +++++++++++++++++++++++++--- 2 files changed, 49 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h index 08ed5a2e46a5fd..71e9861a16a896 100644 --- a/arch/x86/include/asm/nospec-branch.h +++ b/arch/x86/include/asm/nospec-branch.h @@ -386,6 +386,10 @@ extern void srso_alias_return_thunk(void); extern void entry_untrain_ret(void); extern void write_ibpb(void); +#ifdef CONFIG_BPF_JIT +extern void bpf_arch_ibpb(void); +#endif + #ifdef CONFIG_X86_64 extern void clear_bhb_loop(void); #endif diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index d7fa03bf51b451..fd0b7880cf7e5a 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include @@ -1796,8 +1797,21 @@ static inline const char *spectre_v2_module_string(void) { return spectre_v2_bad_module ? " - vulnerable module loaded" : ""; } + +/* + * The "retpoline sequence" is the "call;mov;ret" sequence that + * replaces normal indirect branch instructions. Differentiate + * *the* retpoline sequence from the LFENCE-prefixed indirect + * branches that simply use the retpoline infrastructure. + */ +static inline bool retpoline_seq_enabled(void) +{ + return boot_cpu_has(X86_FEATURE_RETPOLINE) && !boot_cpu_has(X86_FEATURE_RETPOLINE_LFENCE); +} + #else static inline const char *spectre_v2_module_string(void) { return ""; } +static inline bool retpoline_seq_enabled(void) { return false; } #endif #define SPECTRE_V2_LFENCE_MSG "WARNING: LFENCE mitigation is not recommended for this CPU, data leaks possible!\n" @@ -2240,8 +2254,7 @@ static void __init bhi_apply_mitigation(void) return; /* Retpoline mitigates against BHI unless the CPU has RRSBA behavior */ - if (boot_cpu_has(X86_FEATURE_RETPOLINE) && - !boot_cpu_has(X86_FEATURE_RETPOLINE_LFENCE)) { + if (retpoline_seq_enabled()) { spec_ctrl_disable_kernel_rrsba(); if (rrsba_disabled) return; @@ -2383,6 +2396,27 @@ static void __init spectre_v2_update_mitigation(void) pr_info("%s\n", spectre_v2_strings[spectre_v2_enabled]); } +#ifdef CONFIG_BPF_JIT +static void __bpf_arch_ibpb(void *unused) +{ + write_ibpb(); +} + +void bpf_arch_ibpb(void) +{ + on_each_cpu(__bpf_arch_ibpb, NULL, 1); +} + +static bool __init cpu_wants_ibpb_bpf(void) +{ + /* A genuine retpoline already neutralizes ring0 indirect predictions */ + if (retpoline_seq_enabled()) + return false; + + return boot_cpu_has(X86_FEATURE_IBPB); +} +#endif + static void __init spectre_v2_apply_mitigation(void) { if (spectre_v2_enabled == SPECTRE_V2_EIBRS && unprivileged_ebpf_enabled()) @@ -2459,6 +2493,14 @@ static void __init spectre_v2_apply_mitigation(void) setup_force_cpu_cap(X86_FEATURE_USE_IBRS_FW); pr_info("Enabling Restricted Speculation for firmware calls\n"); } + +#ifdef CONFIG_BPF_JIT + if (cpu_wants_ibpb_bpf()) { + static_call_update(bpf_arch_pred_flush, bpf_arch_ibpb); + static_branch_enable(&bpf_pred_flush_enabled); + pr_info("Enabling IBPB for BPF\n"); + } +#endif } static void update_stibp_msr(void * __unused) @@ -3544,9 +3586,7 @@ static const char *spectre_bhi_state(void) return "; BHI: BHI_DIS_S"; else if (boot_cpu_has(X86_FEATURE_CLEAR_BHB_LOOP)) return "; BHI: SW loop, KVM: SW loop"; - else if (boot_cpu_has(X86_FEATURE_RETPOLINE) && - !boot_cpu_has(X86_FEATURE_RETPOLINE_LFENCE) && - rrsba_disabled) + else if (retpoline_seq_enabled() && rrsba_disabled) return "; BHI: Retpoline"; else if (boot_cpu_has(X86_FEATURE_CLEAR_BHB_VMEXIT)) return "; BHI: Vulnerable, KVM: SW loop"; -- 2.53.0