From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F154D30AD05; Thu, 16 Jul 2026 14:13:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784211213; cv=none; b=BIhfR8s5KBag0sQ0myggJNi7Lf8vFGeyec2ovJW+6pEu1muDtVJqiToKqjBYjASpD0CEyFYT6lUb0kcaiBapnvqYjjR4V7aN+FT543A+feqvaOJCjbEqreNdDeA7isM2upTICoEnRCG+s9llOpfq1FLDI+Q7W2l5xSbUrjqn8D4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784211213; c=relaxed/simple; bh=zDn2yUl8rfoRHRhO5h+a+KkXrUJ+cxpnruTEJFmiyCQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UZCZgfRIgJVobnTd0V/aLCX6R18AEAEAI/8EkbLc1KIV0D0Pw0hWfi4FkizL5QTYLa6cixiLubkBAVslNVKHvFTlFR3yu0c1yT/nqtXSc62blLvR3rbjUm+TAXM1oC+UgR1i+Ann8O1sxsO/lALVWaQEyH2U8t1lYloBlpXkynA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b=2YefUxbM; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b="2YefUxbM" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 616261F00A3A; Thu, 16 Jul 2026 14:13:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=korg; t=1784211211; bh=E7nkXul6nxPW8zmLSAtsSVBO3WaG+3jA5DkR5eNP82g=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=2YefUxbMxbTw+LVrbpQ8BCmNSFjC1ljYGmUcIeRCZTfHxXZuytYTAQE1AgP7O2JUo f1Abjii+KnC06kHMYcMCNY4oFkx0BfaM+pHg3tM/3vz9m+0DIZSZE55F6vsTErb6xl mmwJor6XMlh3vNQ8SyKUoIwoEvhu7zrtd97xVoF8= From: Greg Kroah-Hartman To: stable@vger.kernel.org Cc: Greg Kroah-Hartman , patches@lists.linux.dev, Pawan Gupta , Daniel Borkmann , Sasha Levin Subject: [PATCH 6.18 325/480] bpf: Support for hardening against JIT spraying Date: Thu, 16 Jul 2026 15:31:12 +0200 Message-ID: <20260716133051.849217269@linuxfoundation.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260716133044.672218725@linuxfoundation.org> References: <20260716133044.672218725@linuxfoundation.org> User-Agent: quilt/0.69 X-stable: review X-Patchwork-Hint: ignore Precedence: bulk X-Mailing-List: patches@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit 6.18-stable review patch. If anyone has any objections, please let me know. ------------------ From: Pawan Gupta commit 96cce16e26dd02a8678f1e87f88a4b5cdb63b995 upstream. The BPF JIT allocator packs many small programs into larger executable allocations and reuses space within those allocations as programs are loaded and freed. When fresh code is written into space that a previous program occupied, an indirect jump into the new program can reuse a branch prediction left behind by the old one. Flush the indirect branch predictors before reusing JIT memory so that indirect jumps into a newly written program don't reuse predictions from an old program that occupied the same space. Introduce bpf_arch_pred_flush_enabled static key and bpf_arch_pred_flush static call for flushing the branch predictors on JIT memory reuse. Architectures that need a flush, can update it to a predictor flush function. By default, its a NOP and does not emit any CALL. Allocations larger than a pack are not covered by this flush. That is safe because cBPF programs (the unprivileged attack surface) are bounded well below a pack size. Issue a warning if this assumption is ever violated while the flush is active. Signed-off-by: Pawan Gupta Acked-by: Daniel Borkmann Signed-off-by: Daniel Borkmann Signed-off-by: Sasha Levin --- include/linux/filter.h | 10 ++++++++++ kernel/bpf/core.c | 19 +++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/include/linux/filter.h b/include/linux/filter.h index cf7a0bce1bb6d8..3c4dd718beced8 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -22,6 +22,7 @@ #include #include #include +#include #include #include @@ -1266,6 +1267,15 @@ extern long bpf_jit_limit_max; typedef void (*bpf_jit_fill_hole_t)(void *area, unsigned int size); +/* + * Flush the indirect branch predictors before reusing JIT memory, so that + * indirect jumps into a newly written program don't reuse predictions left + * behind by an old program that occupied the same space. + */ +void bpf_arch_pred_flush(void); +DECLARE_STATIC_CALL(bpf_arch_pred_flush, bpf_arch_pred_flush); +DECLARE_STATIC_KEY_FALSE(bpf_pred_flush_enabled); + void bpf_jit_fill_hole_with_zero(void *area, unsigned int size); struct bpf_binary_header * diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 931a4ddd8530ce..a043fccb917e86 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -864,6 +865,15 @@ void bpf_jit_fill_hole_with_zero(void *area, unsigned int size) memset(area, 0, size); } +DEFINE_STATIC_CALL_NULL(bpf_arch_pred_flush, bpf_arch_pred_flush); + +/* + * Enabled once bpf_arch_pred_flush points at a real flush routine. Lets the + * pack allocator test "is a predictor flush wired up at all" with a cheap + * static branch instead of repeatedly querying the static call target. + */ +DEFINE_STATIC_KEY_FALSE(bpf_pred_flush_enabled); + #define BPF_PROG_SIZE_TO_NBITS(size) (round_up(size, BPF_PROG_CHUNK_SIZE) / BPF_PROG_CHUNK_SIZE) static DEFINE_MUTEX(pack_mutex); @@ -923,6 +933,14 @@ void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns) mutex_lock(&pack_mutex); if (size > BPF_PROG_PACK_SIZE) { + /* + * Allocations larger than a pack get their own pages, and + * predictors are not flushed for such allocation. This is only + * safe because cBPF programs (the unprivileged attack surface) + * are bounded well below a pack size. + */ + if (static_branch_unlikely(&bpf_pred_flush_enabled)) + pr_warn_once("BPF: Predictors not flushed for allocations greater than BPF_PROG_PACK_SIZE\n"); size = round_up(size, PAGE_SIZE); ptr = bpf_jit_alloc_exec(size); if (ptr) { @@ -953,6 +971,7 @@ void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns) pos = 0; found_free_area: + static_call_cond(bpf_arch_pred_flush)(); bitmap_set(pack->bitmap, pos, nbits); ptr = (void *)(pack->ptr) + (pos << BPF_PROG_CHUNK_SHIFT); -- 2.53.0