From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 467C22DF137; Thu, 16 Jul 2026 13:38:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784209096; cv=none; b=t4lRiIThOdNXI+MuOqhuC54a/0a/dVi7e9BlP54yZwEL7Fei0JkCTqoMPlySp6bnl/axqaip+7Hv9h8M2+YsLJVabO8Vx87BOGOuFr4mJJF0/Sd+hPfNzOQHct2CDK+QtrTSszGlPeulVlFHIvYVyZIuvWThtoic5QiKr4QE/UE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784209096; c=relaxed/simple; bh=sg6/hhhBh79ukfGQs39tKOuf5ZQDFlgD+ZUgjkr3j7I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mF/DgA61FlZnqzYHpiKkkcAH4Sm16/utN0L4PGx4wiFJkbEylzJkw+Ycd3Mc0f2fCSKVvvi0gskEVP3jMCYIHz3FZb/VFKqVXCQnnVyQuNJ6C8EJk3JVYdDRHS2C2HEiqI+VpouBIK8ZDIJEmgVwPPDU1a5bBzrmbU9fD+7rKYk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b=sIj0HMgw; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b="sIj0HMgw" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 91E1B1F000E9; Thu, 16 Jul 2026 13:38:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=korg; t=1784209095; bh=ofRkNzb9+VVAEY6190kK2h+4iT7R0MM+Tnp4EQPJC28=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=sIj0HMgwL4aSPVQ6hnG/Qnfo68JBXVXwpqlDMew7f736Rc5KoGyB5Q9hQDk/mgc4O AjhATBrK8E4qyDcB7lKcHvET3diOy2OvQgDxws8MPoe7Ac1J8Uph4sZ+jlJoPG8LxQ 6OrHEJHJx2ipsbUzQwNKSPkeckk8Rw69kqjuk0jw= From: Greg Kroah-Hartman To: stable@vger.kernel.org Cc: Greg Kroah-Hartman , patches@lists.linux.dev, Pawan Gupta , Daniel Borkmann , Sasha Levin Subject: [PATCH 7.1 028/518] bpf: Prefer packs that wont trigger an IBPB flush on allocation Date: Thu, 16 Jul 2026 15:24:56 +0200 Message-ID: <20260716133048.391809646@linuxfoundation.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260716133047.772246337@linuxfoundation.org> References: <20260716133047.772246337@linuxfoundation.org> User-Agent: quilt/0.69 X-stable: review X-Patchwork-Hint: ignore Precedence: bulk X-Mailing-List: patches@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit 7.1-stable review patch. If anyone has any objections, please let me know. ------------------ From: Pawan Gupta commit a9b1f19a6a673ba06820898d0f1ad02883ea1639 upstream. Currently BPF pack allocator picks the chunks from the first available pack. While this is okay, it naturally leads to more frequent flushes when there are multiple packs in the system that weren't used since the last flush. As an optimization prefer allocating the new programs from packs that are unused since last flush. When all packs are dirty, allocation forces a flush and marks all packs clean. Below are some future optimizations ideas: 1. Currently, the "dirty" tracking is only done at the pack-level. Flush frequency can further be reduced with chunk-level tracking. This requires a new bitmap per-pack to track the dirty state. 2. IBPB flush is done on all CPUs, even if only a single CPU ran the BPF program. On a system with hundreds of CPUs this could be a major bottleneck forcing hundreds of IPIs to deliver the flush. The solution is to track the CPUs where a BPF program ran, and issue IBPB only on those CPUs. 3. Avoid IBPB when flush is already done at other sources (e.g. context switch). Signed-off-by: Pawan Gupta Acked-by: Daniel Borkmann Signed-off-by: Daniel Borkmann Signed-off-by: Sasha Levin --- kernel/bpf/core.c | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 1bf4b8dffe9158..bba4acd61d4126 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -948,8 +948,8 @@ static struct bpf_prog_pack *alloc_new_pack(bpf_jit_fill_hole_t bpf_fill_ill_ins void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns, bool was_classic) { unsigned int nbits = BPF_PROG_SIZE_TO_NBITS(size); - struct bpf_prog_pack *pack; - unsigned long pos; + struct bpf_prog_pack *pack, *fallback_pack = NULL; + unsigned long pos, fallback_pos = 0; void *ptr = NULL; mutex_lock(&pack_mutex); @@ -981,8 +981,29 @@ void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns, bool list_for_each_entry(pack, &pack_list, list) { pos = bitmap_find_next_zero_area(pack->bitmap, BPF_PROG_CHUNK_COUNT, 0, nbits, 0); - if (pos < BPF_PROG_CHUNK_COUNT) + if (pos >= BPF_PROG_CHUNK_COUNT) + continue; + /* Flush not enabled, use any pack */ + if (!static_branch_unlikely(&bpf_pred_flush_enabled)) goto found_free_area; + /* + * cBPF reuse of a dirty pack triggers a flush, so prefer a + * clean pack for cBPF. eBPF never flushes, so pick the first + * free pack, dirty or clean. + */ + if (!was_classic || !pack->arch_flush_needed) + goto found_free_area; + if (!fallback_pack) { + fallback_pack = pack; + fallback_pos = pos; + } + } + + /* No preferred pack found */ + if (fallback_pack) { + pack = fallback_pack; + pos = fallback_pos; + goto found_free_area; } pack = alloc_new_pack(bpf_fill_ill_insns); -- 2.53.0