From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-oa1-f44.google.com (mail-oa1-f44.google.com [209.85.160.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B990F3FF1B4 for ; Mon, 6 Jul 2026 17:19:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.44 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783358382; cv=none; b=eSNm8CkBjRjshrhPCFegYyL4bnSL+ggnHu3JZAP7HkVdYu4DlK20lvT5TeHN+lWv5izsBxtFTYm+RlCU6dBEyIJ0zftEyzYNL4YP8lH4iqmNKINAURoEkLsoGURLiXAN3ppds9uTgR5tewJLKmTbKo5lo8GtvhQlDalCsQMLE0I= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783358382; c=relaxed/simple; bh=MPxEh0/3wooywxlth/LtHamyqmWIegcg6qFouIkhZo8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mktXFn5P/TKHdRl1z71JGvCwsr65ySZpnH4eppZ+CcllOYaz+ItXO3FDIydDX7FfYF/lfNzhLZrQK7gnLz7LsMM8kYBIuLFgqAubgVQKqDLJGCnjhbMSkuV3mcKWbQv2UA1Ju7fARywqV8yTe/ggIUygLvDI7WLSPnbzWyaHtAA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QYVz+hjV; arc=none smtp.client-ip=209.85.160.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QYVz+hjV" Received: by mail-oa1-f44.google.com with SMTP id 586e51a60fabf-448b89f700fso1692749fac.2 for ; Mon, 06 Jul 2026 10:19:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1783358379; x=1783963179; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AQWIBdtgCcOi2hQRn9XvaotUq6iAQz/8mFPmCGlz/9w=; b=QYVz+hjVpKwT6JiunkJUCpA/HOzeKaS1QAbDEg1/DQGAihgg0pLXvqOidmYvnXnG2O Dnm99XxHOn4tZOdaCFhhwPkx546xnRZKIjRChjOMU8vqrBdBo0QO3pHQ2QpocEmyWt/G EcQKhHS+yZgb7Wl0zvpXaIQ3iyha/P/Z7H0a9xp8oWNqSBMDHp/Lj8Bd7tSkLB0OPT7Y OZMW4Ln+lwVoB65t7UtaJHwJ6+P0P3KrpBw7ZvmGE79F8naTAybM7HBx0+hyDtkqi877 sRV6HSI81RTIxgn9mkK4ogyykzEzAR6yQVXrLLRPmxg0p2DyKCOhYRzDqI3w3PV62FMY cQJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783358379; x=1783963179; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=AQWIBdtgCcOi2hQRn9XvaotUq6iAQz/8mFPmCGlz/9w=; b=H3c4yLPpNwA/9b2VSmxOV/FozSAGCS7haKufPOSHPZZKzaNocR4wXMdyWwS1+eRG2G uj33Lrozqy98W1jQYpQxP0HhAo+IvsxMoQiEJARhf8FgC7xutZHipguwsIISgTMkirgi rGfOpONQREf4kPKLdoeEKMkfCQTg0ZAd+DE1x29EwH/RG5ehl7xyRWfSAR1QhtkZGwFo 3KSvPDyZSNUs6A2JoBgdfzXDZGSKnxb/6SClK/uBergOj4WxwnLJ8iYa/GalmfApDsmf VI80XdnMpTchWw2AtY9+5DWDgH5703+SooVEb5S1brTtItb4iUAbXdB8aZoNcb45aCs1 FzBw== X-Gm-Message-State: AOJu0YwgRrh842FI+ZBO8pvfMJM6z/tm9wXog1MoaMSg1eH9aYC6lhPl MrWAKaKBw2zLylmtEkM/kLg0Y6mJs8m987SL72BbpHX8DPvuFZwTVAYn X-Gm-Gg: AfdE7cnNla0m3fbPn0Q2vWVvfKh4eJ2/wyhKIQ43X+jex/jOnVJaT6/3+qeMpKLAnts xrLxFimnZqIxYXIrubtIPXaZPtyXIT5Yk5efs0gVtSKZI+7MxzGFIP2quqNLMSZA/Z0g8R6WCaF yRdVHvL10eqe0aPwgJ1vQhK3FWXdD81TC68Yqi/FK3lVcCWXG/edcENFcTTE5/VMFvwSkNPodeQ AZOJADfvg2F5VnKy4EmJmdTDKxY0bDNAyqmdcATGWjG0JBM+FHx4okk8T1rhhX7hWdlJb/7qxMs /UOXz5/QimBYajFmhj0O7m8Cqr2Dx9HkkOBBBNxwGDwSIj1t5LRvO8rCESoa/aMNdUZ/VNI6Bo9 5T/aIA1EM2dYfskdQSfAEl0bmbWJMlFVF3v9rrNJ4BB7HmrKN6YCXMSqGv70orkUHKBy7NlFHj/ UNoI4= X-Received: by 2002:a05:6870:9590:b0:43b:9b9e:dc28 with SMTP id 586e51a60fabf-451064f99a9mr888782fac.23.1783358379299; Mon, 06 Jul 2026 10:19:39 -0700 (PDT) Received: from localhost ([2a03:2880:ff:4c::]) by smtp.gmail.com with ESMTPSA id 586e51a60fabf-44cfb13e15esm11667837fac.3.2026.07.06.10.19.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jul 2026 10:19:38 -0700 (PDT) From: Amery Hung To: bpf@vger.kernel.org Cc: netdev@vger.kernel.org, alexei.starovoitov@gmail.com, andrii@kernel.org, daniel@iogearbox.net, eddyz87@gmail.com, memxor@gmail.com, martin.lau@kernel.org, shakeel.butt@linux.dev, roman.gushchin@linux.dev, kuniyu@google.com, kerneljasonxing@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v3 08/15] bpf: Add a few bpf_cgroup_array_* helper functions Date: Mon, 6 Jul 2026 10:19:10 -0700 Message-ID: <20260706171918.317102-9-ameryhung@gmail.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260706171918.317102-1-ameryhung@gmail.com> References: <20260706171918.317102-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Martin KaFai Lau In the upcoming patch, the array can store a struct_ops map. The array could have a cfi_stubs acting as a dummy instead of the dummy_bpf_prog. The array logic will need to skip the cfi_stubs also in order to support storing struct_ops map in the array. bpf_cgroup_array_length(), bpf_cgroup_array_copy_to_user(), and bpf_cgroup_array_delete_safe_at() are added as a preparation work to allow skipping the cfi_stubs in the upcoming patch. This patch only skips the dummy_bpf_prog which is the same as the existing behavior. The current bpf_prog_array_*() callers are changed to call the new bpf_cgroup_array_*(). This is a no-op change. Unlike bpf_prog_array_copy_to_user(), bpf_cgroup_array_copy_to_user() does not need a temporary buffer. The cgroup caller already holds cgroup_mutex and dereferences the effective array with rcu_dereference_protected(), so it does not copy to userspace from an RCU read-side critical section. Details in commit 0911287ce32b. Another addition is the bpf_cgroup_array_free(). This prepares the array to have a different rcu gp for the struct_ops use case, for example, a struct_ops could have mix of sleepable ops and non-sleepable ops. In this patch, bpf_cgroup_array_free() only goes through the regular rcu gp. This is a no-op change also. bpf_prog_dummy() is also added to return the global dummy_bpf_prog. bpf_cgroup_array_dummy() is added to decide the sentinel based on atype. It now always returns bpf_prog_dummy(). In the upcoming patch, it can return a cfi_stubs if the atype belongs to a struct_ops. Signed-off-by: Martin KaFai Lau Signed-off-by: Amery Hung --- include/linux/bpf.h | 1 + kernel/bpf/cgroup.c | 79 +++++++++++++++++++++++++++++++++++++++------ kernel/bpf/core.c | 5 +++ 3 files changed, 76 insertions(+), 9 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 047ffc029666..e371a4733135 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -2561,6 +2561,7 @@ int bpf_prog_array_copy(struct bpf_prog_array *old_array, struct bpf_prog *include_prog, u64 bpf_cookie, struct bpf_prog_array **new_array); +struct bpf_prog *bpf_prog_dummy(void); struct bpf_run_ctx {}; diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c index 7abbe12e108f..081d81de1816 100644 --- a/kernel/bpf/cgroup.c +++ b/kernel/bpf/cgroup.c @@ -319,6 +319,67 @@ static void bpf_cgroup_link_auto_detach(struct bpf_cgroup_link *link) link->cgroup = NULL; } +static void bpf_cgroup_array_free(struct bpf_prog_array *array) +{ + if (!array || array == &bpf_empty_prog_array) + return; + kfree_rcu(array, rcu); +} + +static void *bpf_cgroup_array_dummy(enum cgroup_bpf_attach_type atype) +{ + return bpf_prog_dummy(); +} + +static int bpf_cgroup_array_length(struct bpf_prog_array *array, + enum cgroup_bpf_attach_type atype) +{ + struct bpf_prog_array_item *item; + int cnt = 0; + + for (item = array->items; item->prog; item++) + if (item->prog != bpf_cgroup_array_dummy(atype)) + cnt++; + + return cnt; +} + +static int bpf_cgroup_array_copy_to_user(struct bpf_prog_array *array, + __u32 __user *prog_ids, int cnt, + enum cgroup_bpf_attach_type atype) +{ + struct bpf_prog_array_item *item; + int i = 0; + u32 id; + + for (item = array->items; item->prog && i < cnt; item++) { + if (item->prog == bpf_cgroup_array_dummy(atype)) + continue; + id = item->prog->aux->id; + if (copy_to_user(prog_ids + i, &id, sizeof(id))) + return -EFAULT; + i++; + } + return item->prog ? -ENOSPC : 0; +} + +static int bpf_cgroup_array_delete_safe_at(struct bpf_prog_array *array, + int index, enum cgroup_bpf_attach_type atype) +{ + struct bpf_prog_array_item *item; + + for (item = array->items; item->prog; item++) { + if (item->prog == bpf_cgroup_array_dummy(atype)) + continue; + if (!index) { + WRITE_ONCE(item->prog, bpf_cgroup_array_dummy(atype)); + return 0; + } + index--; + } + return -ENOENT; +} + /** * cgroup_bpf_release() - put references of all bpf programs and * release all cgroup bpf data @@ -356,7 +417,7 @@ static void cgroup_bpf_release(struct work_struct *work) old_array = rcu_dereference_protected( cgrp->bpf.effective[atype], lockdep_is_held(&cgroup_mutex)); - bpf_prog_array_free(old_array); + bpf_cgroup_array_free(old_array); } list_for_each_entry_safe(storage, stmp, storages, list_cg) { @@ -530,7 +591,7 @@ static void activate_effective_progs(struct cgroup *cgrp, /* free prog array after grace period, since __cgroup_bpf_run_*() * might be still walking the array */ - bpf_prog_array_free(old_array); + bpf_cgroup_array_free(old_array); } /** @@ -570,7 +631,7 @@ static int cgroup_bpf_inherit(struct cgroup *cgrp) return 0; cleanup: for (i = 0; i < NR; i++) - bpf_prog_array_free(arrays[i]); + bpf_cgroup_array_free(arrays[i]); for (p = cgroup_parent(cgrp); p; p = cgroup_parent(p)) cgroup_bpf_put(p); @@ -625,7 +686,7 @@ static int update_effective_progs(struct cgroup *cgrp, if (percpu_ref_is_zero(&desc->bpf.refcnt)) { if (unlikely(desc->bpf.inactive)) { - bpf_prog_array_free(desc->bpf.inactive); + bpf_cgroup_array_free(desc->bpf.inactive); desc->bpf.inactive = NULL; } continue; @@ -644,7 +705,7 @@ static int update_effective_progs(struct cgroup *cgrp, css_for_each_descendant_pre(css, &cgrp->self) { struct cgroup *desc = container_of(css, struct cgroup, self); - bpf_prog_array_free(desc->bpf.inactive); + bpf_cgroup_array_free(desc->bpf.inactive); desc->bpf.inactive = NULL; } @@ -1166,7 +1227,7 @@ static void purge_effective_progs(struct cgroup *cgrp, struct bpf_prog_list *pl, lockdep_is_held(&cgroup_mutex)); /* Remove the program from the array */ - WARN_ONCE(bpf_prog_array_delete_safe_at(progs, pos), + WARN_ONCE(bpf_cgroup_array_delete_safe_at(progs, pos, atype), "Failed to purge a prog from array at index %d", pos); } } @@ -1296,7 +1357,7 @@ static int __cgroup_bpf_query(struct cgroup *cgrp, const union bpf_attr *attr, if (effective_query) { effective = rcu_dereference_protected(cgrp->bpf.effective[atype], lockdep_is_held(&cgroup_mutex)); - total_cnt += bpf_prog_array_length(effective); + total_cnt += bpf_cgroup_array_length(effective, atype); } else { total_cnt += prog_list_length(&cgrp->bpf.progs[atype], NULL); } @@ -1326,8 +1387,8 @@ static int __cgroup_bpf_query(struct cgroup *cgrp, const union bpf_attr *attr, if (effective_query) { effective = rcu_dereference_protected(cgrp->bpf.effective[atype], lockdep_is_held(&cgroup_mutex)); - cnt = min_t(int, bpf_prog_array_length(effective), total_cnt); - ret = bpf_prog_array_copy_to_user(effective, prog_ids, cnt); + cnt = min_t(int, bpf_cgroup_array_length(effective, atype), total_cnt); + ret = bpf_cgroup_array_copy_to_user(effective, prog_ids, cnt, atype); } else { struct hlist_head *progs; struct bpf_prog_list *pl; diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 649cce41e13f..1837bb7bb4e9 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -2740,6 +2740,11 @@ void bpf_prog_array_free_sleepable(struct bpf_prog_array *progs) call_rcu_tasks_trace(&progs->rcu, __bpf_prog_array_free_sleepable_cb); } +struct bpf_prog *bpf_prog_dummy(void) +{ + return &dummy_bpf_prog.prog; +} + int bpf_prog_array_length(struct bpf_prog_array *array) { struct bpf_prog_array_item *item; -- 2.52.0