From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7EF7F42B75A; Thu, 16 Jul 2026 14:14:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784211289; cv=none; b=kd9d6uQyO2HEqyKEz5XqPePEqYlxNYY+wuH2Uf4N7/mU++sZ+aAJlNAqdF7RpB/WiwS7Fl3T83U04Id05wNVrqOZx1UMaTvzZ68HUYtroO7kgz2NRdM8ugF5vy8fGFYiZcr9z4rvWFE1Mw42mLXhBZawZIJJ6h5q6ojtJYoe9Zc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784211289; c=relaxed/simple; bh=lO2K9OMZEiqvlJxL7QwxW/WsdLd5ocxUJ4D9pCRSDT8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sU0DGJh3wnXEUV9c1aVQv6eNZjZvLom6BMJYWBYNyc+WbFEtNZBbDsHJysKXIrCa43Utyo1vZzS/S6K8bsfwJu6xi3yMZNus9tDSXm8eSFn0XQRIDBVzrHFmEm54WUrNaLaRoJlNj5frjBKZCloJAtYY1tiIzsYorKM8tj6qfnI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b=OcqU+uyP; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b="OcqU+uyP" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E51701F000E9; Thu, 16 Jul 2026 14:14:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=korg; t=1784211288; bh=9p0OQq0XukttnjQSili1LoCqUfrohJ1E9zhuRs/iU18=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=OcqU+uyPTfjanonSoAMJ7bt0P6fLvt6OuR/gBDrDVdpuy6VUq5JAkF62gXwq9t10e zO+JmTMZAJBfxbMfB7FQGGVl1KbnmYaOAzXeD8HV9+PXwrttUX9ABRj9HbWFTqV/TP eGr27akcxUDakYwfX/alCWpxM4b212zjiJDW6lRE= From: Greg Kroah-Hartman To: stable@vger.kernel.org Cc: Greg Kroah-Hartman , patches@lists.linux.dev, Pawan Gupta , Daniel Borkmann , Sasha Levin Subject: [PATCH 6.18 328/480] bpf: Skip redundant IBPB in pack allocator Date: Thu, 16 Jul 2026 15:31:15 +0200 Message-ID: <20260716133051.913208705@linuxfoundation.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260716133044.672218725@linuxfoundation.org> References: <20260716133044.672218725@linuxfoundation.org> User-Agent: quilt/0.69 X-stable: review X-Patchwork-Hint: ignore Precedence: bulk X-Mailing-List: patches@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit 6.18-stable review patch. If anyone has any objections, please let me know. ------------------ From: Pawan Gupta commit a23c1c5396a91680703360d1ee28a44657c503c4 upstream. bpf_prog_pack_alloc() issues IBPB on all CPUs on every cBPF allocation, even when reusing chunks from an existing pack where no new memory was touched since the last IBPB. Since IBPB on all CPUs is heavy, Dave Hansen suggested to track allocation since last IBPB, and only issue IBPB at reuse for the chunks that have not seen an IBPB since they were last freed. Track per-pack whether an IBPB is needed via arch_flush_needed. Set it when allocating a chunk, reset on IBPB flush. On reuse, conditionally issue the flush. Since IBPB invalidates all BTB entries, clear the flag on all packs after flushing. Signed-off-by: Pawan Gupta Acked-by: Daniel Borkmann Signed-off-by: Daniel Borkmann Signed-off-by: Sasha Levin --- kernel/bpf/core.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 338a806e25c045..49ca543a41115a 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -857,6 +857,7 @@ int bpf_jit_add_poke_descriptor(struct bpf_prog *prog, struct bpf_prog_pack { struct list_head list; void *ptr; + bool arch_flush_needed; unsigned long bitmap[]; }; @@ -910,6 +911,8 @@ static struct bpf_prog_pack *alloc_new_pack(bpf_jit_fill_hole_t bpf_fill_ill_ins bpf_fill_ill_insns(pack->ptr, BPF_PROG_PACK_SIZE); bitmap_zero(pack->bitmap, BPF_PROG_PACK_SIZE / BPF_PROG_CHUNK_SIZE); + if (static_branch_unlikely(&bpf_pred_flush_enabled)) + pack->arch_flush_needed = true; set_vm_flush_reset_perms(pack->ptr); err = set_memory_rox((unsigned long)pack->ptr, BPF_PROG_PACK_SIZE / PAGE_SIZE); @@ -972,8 +975,15 @@ void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns, bool found_free_area: /* Flush only for cBPF as it may contain a crafted gadget */ - if (static_branch_unlikely(&bpf_pred_flush_enabled) && was_classic) + if (static_branch_unlikely(&bpf_pred_flush_enabled) && + pack->arch_flush_needed && + was_classic) { + struct bpf_prog_pack *p; + static_call_cond(bpf_arch_pred_flush)(); + list_for_each_entry(p, &pack_list, list) + p->arch_flush_needed = false; + } bitmap_set(pack->bitmap, pos, nbits); ptr = (void *)(pack->ptr) + (pos << BPF_PROG_CHUNK_SHIFT); @@ -1011,6 +1021,9 @@ void bpf_prog_pack_free(void *ptr, u32 size) "bpf_prog_pack bug: missing bpf_arch_text_invalidate?\n"); bitmap_clear(pack->bitmap, pos, nbits); + + if (static_branch_unlikely(&bpf_pred_flush_enabled)) + pack->arch_flush_needed = true; if (bitmap_find_next_zero_area(pack->bitmap, BPF_PROG_CHUNK_COUNT, 0, BPF_PROG_CHUNK_COUNT, 0) == 0) { list_del(&pack->list); -- 2.53.0