From: Martin KaFai Lau <martin.lau@linux.dev>
To: Kumar Kartikeya Dwivedi <memxor@gmail.com>, bpf@vger.kernel.org
Cc: Martin KaFai Lau <kafai@fb.com>, KP Singh <kpsingh@kernel.org>,
Alexei Starovoitov <ast@kernel.org>,
Andrii Nakryiko <andrii@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Dave Marchevsky <davemarchevsky@fb.com>,
Delyan Kratunov <delyank@fb.com>
Subject: Re: [PATCH RFC bpf-next v1 05/32] bpf: Support kptrs in local storage maps
Date: Thu, 8 Sep 2022 22:27:32 -0700 [thread overview]
Message-ID: <5f4423cb-76f9-4e30-695d-22b7e8ab6422@linux.dev> (raw)
In-Reply-To: <20220904204145.3089-6-memxor@gmail.com>
On 9/4/22 1:41 PM, Kumar Kartikeya Dwivedi wrote:
> diff --git a/include/linux/bpf_local_storage.h b/include/linux/bpf_local_storage.h
> index 7ea18d4da84b..6786d00f004e 100644
> --- a/include/linux/bpf_local_storage.h
> +++ b/include/linux/bpf_local_storage.h
> @@ -74,7 +74,7 @@ struct bpf_local_storage_elem {
> struct hlist_node snode; /* Linked to bpf_local_storage */
> struct bpf_local_storage __rcu *local_storage;
> struct rcu_head rcu;
> - /* 8 bytes hole */
> + struct bpf_map *map; /* Only set for bpf_selem_free_rcu */
Instead of adding another map ptr and using the last 8 bytes hole,
> /* The data is stored in another cacheline to minimize
> * the number of cachelines access during a cache hit.
> */
> diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c
> index 802fc15b0d73..4a725379d761 100644
> --- a/kernel/bpf/bpf_local_storage.c
> +++ b/kernel/bpf/bpf_local_storage.c
> @@ -74,7 +74,8 @@ bpf_selem_alloc(struct bpf_local_storage_map *smap, void *owner,
> gfp_flags | __GFP_NOWARN);
> if (selem) {
> if (value)
> - memcpy(SDATA(selem)->data, value, smap->map.value_size);
> + copy_map_value(&smap->map, SDATA(selem)->data, value);
> + /* No call to check_and_init_map_value as memory is zero init */
> return selem;
> }
>
> @@ -92,12 +93,27 @@ void bpf_local_storage_free_rcu(struct rcu_head *rcu)
> kfree_rcu(local_storage, rcu);
> }
>
> +static void check_and_free_fields(struct bpf_local_storage_elem *selem)
> +{
> + if (map_value_has_kptrs(selem->map))
could SDATA(selem)->smap->map be used here ?
> + bpf_map_free_kptrs(selem->map, SDATA(selem));
> +}
> +
> static void bpf_selem_free_rcu(struct rcu_head *rcu)
> {
> struct bpf_local_storage_elem *selem;
>
> selem = container_of(rcu, struct bpf_local_storage_elem, rcu);
> - kfree_rcu(selem, rcu);
> + check_and_free_fields(selem);
> + kfree(selem);
> +}
> +
> +static void bpf_selem_free_tasks_trace_rcu(struct rcu_head *rcu)
> +{
> + struct bpf_local_storage_elem *selem;
> +
> + selem = container_of(rcu, struct bpf_local_storage_elem, rcu);
> + call_rcu(&selem->rcu, bpf_selem_free_rcu);
> }
>
> /* local_storage->lock must be held and selem->local_storage == local_storage.
> @@ -150,10 +166,11 @@ bool bpf_selem_unlink_storage_nolock(struct bpf_local_storage *local_storage,
> SDATA(selem))
> RCU_INIT_POINTER(local_storage->cache[smap->cache_idx], NULL);
>
> + selem->map = &smap->map;
> if (use_trace_rcu)
> - call_rcu_tasks_trace(&selem->rcu, bpf_selem_free_rcu);
> + call_rcu_tasks_trace(&selem->rcu, bpf_selem_free_tasks_trace_rcu);
> else
> - kfree_rcu(selem, rcu);
> + call_rcu(&selem->rcu, bpf_selem_free_rcu);
>
next prev parent reply other threads:[~2022-09-09 5:27 UTC|newest]
Thread overview: 82+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-04 20:41 [PATCH RFC bpf-next v1 00/32] Local kptrs, BPF linked lists Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 01/32] bpf: Add copy_map_value_long to copy to remote percpu memory Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 02/32] bpf: Support kptrs in percpu arraymap Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 03/32] bpf: Add zero_map_value to zero map value with special fields Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 04/32] bpf: Support kptrs in percpu hashmap and percpu LRU hashmap Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 05/32] bpf: Support kptrs in local storage maps Kumar Kartikeya Dwivedi
2022-09-07 19:00 ` Alexei Starovoitov
2022-09-08 2:47 ` Kumar Kartikeya Dwivedi
2022-09-09 5:27 ` Martin KaFai Lau [this message]
2022-09-09 11:22 ` Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 06/32] bpf: Annotate data races in bpf_local_storage Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 07/32] bpf: Allow specifying volatile type modifier for kptrs Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 08/32] bpf: Add comment about kptr's PTR_TO_MAP_VALUE handling Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 09/32] bpf: Rewrite kfunc argument handling Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 10/32] bpf: Drop kfunc support from btf_check_func_arg_match Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 11/32] bpf: Support constant scalar arguments for kfuncs Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 12/32] bpf: Teach verifier about non-size constant arguments Kumar Kartikeya Dwivedi
2022-09-07 22:11 ` Alexei Starovoitov
2022-09-08 2:49 ` Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 13/32] bpf: Introduce bpf_list_head support for BPF maps Kumar Kartikeya Dwivedi
2022-09-07 22:46 ` Alexei Starovoitov
2022-09-08 2:58 ` Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 14/32] bpf: Introduce bpf_kptr_alloc helper Kumar Kartikeya Dwivedi
2022-09-07 23:30 ` Alexei Starovoitov
2022-09-08 3:01 ` Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 15/32] bpf: Add helper macro bpf_expr_for_each_reg_in_vstate Kumar Kartikeya Dwivedi
2022-09-07 23:48 ` Alexei Starovoitov
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 16/32] bpf: Introduce BPF memory object model Kumar Kartikeya Dwivedi
2022-09-08 0:34 ` Alexei Starovoitov
2022-09-08 2:39 ` Kumar Kartikeya Dwivedi
2022-09-08 3:37 ` Alexei Starovoitov
2022-09-08 11:50 ` Kumar Kartikeya Dwivedi
2022-09-08 14:18 ` Alexei Starovoitov
2022-09-08 14:45 ` Kumar Kartikeya Dwivedi
2022-09-08 15:11 ` Alexei Starovoitov
2022-09-08 15:37 ` Kumar Kartikeya Dwivedi
2022-09-08 15:59 ` Alexei Starovoitov
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 17/32] bpf: Support bpf_list_node in local kptrs Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 18/32] bpf: Support bpf_spin_lock " Kumar Kartikeya Dwivedi
2022-09-08 0:35 ` Alexei Starovoitov
2022-09-09 8:25 ` Dave Marchevsky
2022-09-09 11:20 ` Kumar Kartikeya Dwivedi
2022-09-09 14:26 ` Alexei Starovoitov
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 19/32] bpf: Support bpf_list_head " Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 20/32] bpf: Introduce bpf_kptr_free helper Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 21/32] bpf: Allow locking bpf_spin_lock global variables Kumar Kartikeya Dwivedi
2022-09-08 0:27 ` Alexei Starovoitov
2022-09-08 0:39 ` Kumar Kartikeya Dwivedi
2022-09-08 0:55 ` Alexei Starovoitov
2022-09-08 1:00 ` Kumar Kartikeya Dwivedi
2022-09-08 1:08 ` Alexei Starovoitov
2022-09-08 1:15 ` Kumar Kartikeya Dwivedi
2022-09-08 2:39 ` Alexei Starovoitov
2022-09-09 8:13 ` Dave Marchevsky
2022-09-09 11:05 ` Kumar Kartikeya Dwivedi
2022-09-09 14:24 ` Alexei Starovoitov
2022-09-09 14:50 ` Kumar Kartikeya Dwivedi
2022-09-09 14:58 ` Alexei Starovoitov
2022-09-09 18:32 ` Andrii Nakryiko
2022-09-09 19:25 ` Alexei Starovoitov
2022-09-09 20:21 ` Andrii Nakryiko
2022-09-09 20:57 ` Alexei Starovoitov
2022-09-10 0:21 ` Andrii Nakryiko
2022-09-11 22:31 ` Alexei Starovoitov
2022-09-20 20:55 ` Andrii Nakryiko
2022-10-18 4:06 ` Andrii Nakryiko
2022-09-09 22:30 ` Dave Marchevsky
2022-09-09 22:49 ` Kumar Kartikeya Dwivedi
2022-09-09 22:57 ` Alexei Starovoitov
2022-09-09 23:04 ` Kumar Kartikeya Dwivedi
2022-09-09 22:51 ` Alexei Starovoitov
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 22/32] bpf: Bump BTF_KFUNC_SET_MAX_CNT Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 23/32] bpf: Add single ownership BPF linked list API Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 24/32] bpf: Permit NULL checking pointer with non-zero fixed offset Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 25/32] bpf: Allow storing local kptrs in BPF maps Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 26/32] bpf: Wire up freeing of bpf_list_heads in maps Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 27/32] bpf: Add destructor for bpf_list_head in local kptr Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 28/32] bpf: Remove duplicate PTR_TO_BTF_ID RO check Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 29/32] libbpf: Add support for private BSS map section Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 30/32] selftests/bpf: Add BTF tag macros for local kptrs, BPF linked lists Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 31/32] selftests/bpf: Add BPF linked list API tests Kumar Kartikeya Dwivedi
2022-09-04 20:41 ` [PATCH RFC bpf-next v1 32/32] selftests/bpf: Add referenced local kptr tests Kumar Kartikeya Dwivedi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5f4423cb-76f9-4e30-695d-22b7e8ab6422@linux.dev \
--to=martin.lau@linux.dev \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=davemarchevsky@fb.com \
--cc=delyank@fb.com \
--cc=kafai@fb.com \
--cc=kpsingh@kernel.org \
--cc=memxor@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox