From: Amery Hung <ameryhung@gmail.com>
To: bpf@vger.kernel.org
Cc: netdev@vger.kernel.org, alexei.starovoitov@gmail.com,
andrii@kernel.org, daniel@iogearbox.net, memxor@gmail.com,
martin.lau@kernel.org, kpsingh@kernel.org,
yonghong.song@linux.dev, song@kernel.org, haoluo@google.com,
ameryhung@gmail.com, kernel-team@meta.com
Subject: [PATCH bpf-next v5 09/16] bpf: Prepare for bpf_selem_unlink_nofail()
Date: Sun, 1 Feb 2026 09:50:42 -0800 [thread overview]
Message-ID: <20260201175050.468601-10-ameryhung@gmail.com> (raw)
In-Reply-To: <20260201175050.468601-1-ameryhung@gmail.com>
The next patch will introduce bpf_selem_unlink_nofail() to handle
rqspinlock errors. bpf_selem_unlink_nofail() will allow an selem to be
partially unlinked from map or local storage. Save memory allocation
method in selem so that later an selem can be correctly freed even when
SDATA(selem)->smap is init to NULL.
In addition, keep track of memory charge to the owner in local storage
so that later bpf_selem_unlink_nofail() can return the correct memory
charge to the owner. Updating selems_size is protected by
local_storage->lock.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
---
include/linux/bpf_local_storage.h | 4 +++-
kernel/bpf/bpf_local_storage.c | 10 +++++++++-
2 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/include/linux/bpf_local_storage.h b/include/linux/bpf_local_storage.h
index fba3354988d3..6fb02ffdb8aa 100644
--- a/include/linux/bpf_local_storage.h
+++ b/include/linux/bpf_local_storage.h
@@ -80,7 +80,8 @@ struct bpf_local_storage_elem {
* after raw_spin_unlock
*/
};
- /* 8 bytes hole */
+ bool use_kmalloc_nolock;
+ /* 7 bytes hole */
/* The data is stored in another cacheline to minimize
* the number of cachelines access during a cache hit.
*/
@@ -96,6 +97,7 @@ struct bpf_local_storage {
*/
struct rcu_head rcu;
rqspinlock_t lock; /* Protect adding/removing from the "list" */
+ u64 selems_size; /* Total selem size. Protected by "lock" */
bool use_kmalloc_nolock;
};
diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c
index 3735f79a7b55..0b8d7eaa2f92 100644
--- a/kernel/bpf/bpf_local_storage.c
+++ b/kernel/bpf/bpf_local_storage.c
@@ -85,6 +85,7 @@ bpf_selem_alloc(struct bpf_local_storage_map *smap, void *owner,
if (selem) {
RCU_INIT_POINTER(SDATA(selem)->smap, smap);
+ selem->use_kmalloc_nolock = smap->use_kmalloc_nolock;
if (value) {
/* No need to call check_and_init_map_value as memory is zero init */
@@ -214,7 +215,7 @@ void bpf_selem_free(struct bpf_local_storage_elem *selem,
smap = rcu_dereference_check(SDATA(selem)->smap, bpf_rcu_lock_held());
- if (!smap->use_kmalloc_nolock) {
+ if (!selem->use_kmalloc_nolock) {
/*
* No uptr will be unpin even when reuse_now == false since uptr
* is only supported in task local storage, where
@@ -305,12 +306,19 @@ static bool bpf_selem_unlink_storage_nolock(struct bpf_local_storage *local_stor
if (rcu_access_pointer(local_storage->smap) == smap)
RCU_INIT_POINTER(local_storage->smap, NULL);
+ local_storage->selems_size -= smap->elem_size;
+
return free_local_storage;
}
void bpf_selem_link_storage_nolock(struct bpf_local_storage *local_storage,
struct bpf_local_storage_elem *selem)
{
+ struct bpf_local_storage_map *smap;
+
+ smap = rcu_dereference_check(SDATA(selem)->smap, bpf_rcu_lock_held());
+ local_storage->selems_size += smap->elem_size;
+
RCU_INIT_POINTER(selem->local_storage, local_storage);
hlist_add_head_rcu(&selem->snode, &local_storage->list);
}
--
2.47.3
next prev parent reply other threads:[~2026-02-01 17:51 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-02-01 17:50 [PATCH bpf-next v5 00/16] Remove task and cgroup local storage percpu counters Amery Hung
2026-02-01 17:50 ` [PATCH bpf-next v5 01/16] bpf: Select bpf_local_storage_map_bucket based on bpf_local_storage Amery Hung
2026-02-01 17:50 ` [PATCH bpf-next v5 02/16] bpf: Convert bpf_selem_unlink_map to failable Amery Hung
2026-02-01 17:50 ` [PATCH bpf-next v5 03/16] bpf: Convert bpf_selem_link_map " Amery Hung
2026-02-01 17:50 ` [PATCH bpf-next v5 04/16] bpf: Convert bpf_selem_unlink " Amery Hung
2026-02-01 17:50 ` [PATCH bpf-next v5 05/16] bpf: Change local_storage->lock and b->lock to rqspinlock Amery Hung
2026-02-01 17:50 ` [PATCH bpf-next v5 06/16] bpf: Remove task local storage percpu counter Amery Hung
2026-02-01 17:50 ` [PATCH bpf-next v5 07/16] bpf: Remove cgroup " Amery Hung
2026-02-01 17:50 ` [PATCH bpf-next v5 08/16] bpf: Remove unused percpu counter from bpf_local_storage_map_free Amery Hung
2026-02-01 17:50 ` Amery Hung [this message]
2026-02-01 17:50 ` [PATCH bpf-next v5 10/16] bpf: Support lockless unlink when freeing map or local storage Amery Hung
2026-02-01 18:22 ` bot+bpf-ci
2026-02-04 5:39 ` Martin KaFai Lau
2026-02-04 23:14 ` Amery Hung
2026-02-05 1:08 ` Martin KaFai Lau
2026-02-01 17:50 ` [PATCH bpf-next v5 11/16] bpf: Switch to bpf_selem_unlink_nofail in bpf_local_storage_{map_free, destroy} Amery Hung
2026-02-04 1:52 ` Martin KaFai Lau
2026-02-04 23:20 ` Amery Hung
2026-02-01 17:50 ` [PATCH bpf-next v5 12/16] selftests/bpf: Update sk_storage_omem_uncharge test Amery Hung
2026-02-01 17:50 ` [PATCH bpf-next v5 13/16] selftests/bpf: Update task_local_storage/recursion test Amery Hung
2026-02-01 17:50 ` [PATCH bpf-next v5 14/16] selftests/bpf: Update task_local_storage/task_storage_nodeadlock test Amery Hung
2026-02-01 17:50 ` [PATCH bpf-next v5 15/16] selftests/bpf: Remove test_task_storage_map_stress_lookup Amery Hung
2026-02-01 17:50 ` [PATCH bpf-next v5 16/16] selftests/bpf: Choose another percpu variable in bpf for btf_dump test Amery Hung
2026-02-01 23:29 ` [PATCH bpf-next v5 00/16] Remove task and cgroup local storage percpu counters Alexei Starovoitov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260201175050.468601-10-ameryhung@gmail.com \
--to=ameryhung@gmail.com \
--cc=alexei.starovoitov@gmail.com \
--cc=andrii@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=haoluo@google.com \
--cc=kernel-team@meta.com \
--cc=kpsingh@kernel.org \
--cc=martin.lau@kernel.org \
--cc=memxor@gmail.com \
--cc=netdev@vger.kernel.org \
--cc=song@kernel.org \
--cc=yonghong.song@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox