* [PATCH v4 bpf-next 0/6] bpf: add percpu stats for bpf_map
@ 2023-07-05 16:01 Anton Protopopov
2023-07-05 16:01 ` [PATCH v4 bpf-next 1/6] bpf: add percpu stats for bpf_map elements insertions/deletions Anton Protopopov
` (5 more replies)
0 siblings, 6 replies; 24+ messages in thread
From: Anton Protopopov @ 2023-07-05 16:01 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, John Fastabend,
Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song,
KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf
Cc: Anton Protopopov
This series adds a mechanism for maps to populate per-cpu counters on
insertions/deletions. The sum of these counters can be accessed by a new kfunc
from map iterator and tracing programs.
The following patches are present in the series:
* Patch 1 adds a generic per-cpu counter to struct bpf_map
* Patch 2 adds a new kfunc to access the sum of per-cpu counters
* Patch 3 utilizes this mechanism for hash-based maps
* Patch 4 extends the preloaded map iterator to dump the sum
* Patch 5 adds a self-test for the change
* Patch 6 patches map_ptr selftest to check that elem_count was initialized
The reason for adding this functionality in our case (Cilium) is to get signals
about how full some heavy-used maps are and what the actual dynamic profile of
map capacity is. In the case of LRU maps this is impossible to get this
information anyhow else. The original presentation can be found here [1].
[1] https://lpc.events/event/16/contributions/1368/
v3 -> v4:
* fix selftests:
* added test code for batch map operations
* added a test for BPF_MAP_TYPE_HASH_OF_MAPS (Hao)
* added tests for BPF_MAP_TYPE_LRU* with BPF_F_NO_COMMON_LRU (Hao)
* map_info was called multiple times unnecessarily (Hao)
* small fixes + some memory leaks (Hao)
* fixed wrong error path for freeing a non-prealloc map (Hao)
* fixed counters for batch delete operations (Hao)
v2 -> v3:
- split commits to better represent update logic (Alexei)
- remove filter from kfunc to allow all tracing programs (Alexei)
- extend selftests (Alexei)
v1 -> v2:
- make the counters generic part of struct bpf_map (Alexei)
- don't use map_info and /proc/self/fdinfo in favor of a kfunc (Alexei)
Anton Protopopov (6):
bpf: add percpu stats for bpf_map elements insertions/deletions
bpf: add a new kfunc to return current bpf_map elements count
bpf: populate the per-cpu insertions/deletions counters for hashmaps
bpf: make preloaded map iterators to display map elements count
selftests/bpf: test map percpu stats
selftests/bpf: check that ->elem_count is non-zero for the hash map
include/linux/bpf.h | 30 +
kernel/bpf/hashtab.c | 23 +-
kernel/bpf/map_iter.c | 39 +-
kernel/bpf/preload/iterators/iterators.bpf.c | 9 +-
.../iterators/iterators.lskel-little-endian.h | 526 +++++++++---------
.../bpf/map_tests/map_percpu_stats.c | 450 +++++++++++++++
.../selftests/bpf/progs/map_percpu_stats.c | 24 +
.../selftests/bpf/progs/map_ptr_kern.c | 3 +
8 files changed, 841 insertions(+), 263 deletions(-)
create mode 100644 tools/testing/selftests/bpf/map_tests/map_percpu_stats.c
create mode 100644 tools/testing/selftests/bpf/progs/map_percpu_stats.c
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread* [PATCH v4 bpf-next 1/6] bpf: add percpu stats for bpf_map elements insertions/deletions 2023-07-05 16:01 [PATCH v4 bpf-next 0/6] bpf: add percpu stats for bpf_map Anton Protopopov @ 2023-07-05 16:01 ` Anton Protopopov 2023-07-05 16:01 ` [PATCH v4 bpf-next 2/6] bpf: add a new kfunc to return current bpf_map elements count Anton Protopopov ` (4 subsequent siblings) 5 siblings, 0 replies; 24+ messages in thread From: Anton Protopopov @ 2023-07-05 16:01 UTC (permalink / raw) To: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf Cc: Anton Protopopov Add a generic percpu stats for bpf_map elements insertions/deletions in order to keep track of both, the current (approximate) number of elements in a map and per-cpu statistics on update/delete operations. To expose these stats a particular map implementation should initialize the counter and adjust it as needed using the 'bpf_map_*_elem_count' helpers provided by this commit. Signed-off-by: Anton Protopopov <aspsk@isovalent.com> --- include/linux/bpf.h | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index f58895830ada..360433f14496 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -275,6 +275,7 @@ struct bpf_map { } owner; bool bypass_spec_v1; bool frozen; /* write-once; write-protected by freeze_mutex */ + s64 __percpu *elem_count; }; static inline const char *btf_field_type_name(enum btf_field_type type) @@ -2040,6 +2041,35 @@ bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, size_t align, } #endif +static inline int +bpf_map_init_elem_count(struct bpf_map *map) +{ + size_t size = sizeof(*map->elem_count), align = size; + gfp_t flags = GFP_USER | __GFP_NOWARN; + + map->elem_count = bpf_map_alloc_percpu(map, size, align, flags); + if (!map->elem_count) + return -ENOMEM; + + return 0; +} + +static inline void +bpf_map_free_elem_count(struct bpf_map *map) +{ + free_percpu(map->elem_count); +} + +static inline void bpf_map_inc_elem_count(struct bpf_map *map) +{ + this_cpu_inc(*map->elem_count); +} + +static inline void bpf_map_dec_elem_count(struct bpf_map *map) +{ + this_cpu_dec(*map->elem_count); +} + extern int sysctl_unprivileged_bpf_disabled; static inline bool bpf_allow_ptr_leaks(void) -- 2.34.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v4 bpf-next 2/6] bpf: add a new kfunc to return current bpf_map elements count 2023-07-05 16:01 [PATCH v4 bpf-next 0/6] bpf: add percpu stats for bpf_map Anton Protopopov 2023-07-05 16:01 ` [PATCH v4 bpf-next 1/6] bpf: add percpu stats for bpf_map elements insertions/deletions Anton Protopopov @ 2023-07-05 16:01 ` Anton Protopopov 2023-07-05 16:01 ` [PATCH v4 bpf-next 3/6] bpf: populate the per-cpu insertions/deletions counters for hashmaps Anton Protopopov ` (3 subsequent siblings) 5 siblings, 0 replies; 24+ messages in thread From: Anton Protopopov @ 2023-07-05 16:01 UTC (permalink / raw) To: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf Cc: Anton Protopopov A bpf_map_sum_elem_count kfunc was added to simplify getting the sum of the map per-cpu element counters. If a map doesn't implement the counter, then the function will always return 0. Signed-off-by: Anton Protopopov <aspsk@isovalent.com> --- kernel/bpf/map_iter.c | 39 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-) diff --git a/kernel/bpf/map_iter.c b/kernel/bpf/map_iter.c index b0fa190b0979..d06d3b7150e5 100644 --- a/kernel/bpf/map_iter.c +++ b/kernel/bpf/map_iter.c @@ -93,7 +93,7 @@ static struct bpf_iter_reg bpf_map_reg_info = { .ctx_arg_info_size = 1, .ctx_arg_info = { { offsetof(struct bpf_iter__bpf_map, map), - PTR_TO_BTF_ID_OR_NULL }, + PTR_TO_BTF_ID_OR_NULL | PTR_TRUSTED }, }, .seq_info = &bpf_map_seq_info, }; @@ -193,3 +193,40 @@ static int __init bpf_map_iter_init(void) } late_initcall(bpf_map_iter_init); + +__diag_push(); +__diag_ignore_all("-Wmissing-prototypes", + "Global functions as their definitions will be in vmlinux BTF"); + +__bpf_kfunc s64 bpf_map_sum_elem_count(struct bpf_map *map) +{ + s64 *pcount; + s64 ret = 0; + int cpu; + + if (!map || !map->elem_count) + return 0; + + for_each_possible_cpu(cpu) { + pcount = per_cpu_ptr(map->elem_count, cpu); + ret += READ_ONCE(*pcount); + } + return ret; +} + +__diag_pop(); + +BTF_SET8_START(bpf_map_iter_kfunc_ids) +BTF_ID_FLAGS(func, bpf_map_sum_elem_count, KF_TRUSTED_ARGS) +BTF_SET8_END(bpf_map_iter_kfunc_ids) + +static const struct btf_kfunc_id_set bpf_map_iter_kfunc_set = { + .owner = THIS_MODULE, + .set = &bpf_map_iter_kfunc_ids, +}; + +static int init_subsystem(void) +{ + return register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &bpf_map_iter_kfunc_set); +} +late_initcall(init_subsystem); -- 2.34.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v4 bpf-next 3/6] bpf: populate the per-cpu insertions/deletions counters for hashmaps 2023-07-05 16:01 [PATCH v4 bpf-next 0/6] bpf: add percpu stats for bpf_map Anton Protopopov 2023-07-05 16:01 ` [PATCH v4 bpf-next 1/6] bpf: add percpu stats for bpf_map elements insertions/deletions Anton Protopopov 2023-07-05 16:01 ` [PATCH v4 bpf-next 2/6] bpf: add a new kfunc to return current bpf_map elements count Anton Protopopov @ 2023-07-05 16:01 ` Anton Protopopov 2023-07-06 1:24 ` Alexei Starovoitov 2023-07-05 16:01 ` [PATCH v4 bpf-next 4/6] bpf: make preloaded map iterators to display map elements count Anton Protopopov ` (2 subsequent siblings) 5 siblings, 1 reply; 24+ messages in thread From: Anton Protopopov @ 2023-07-05 16:01 UTC (permalink / raw) To: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf Cc: Anton Protopopov Initialize and utilize the per-cpu insertions/deletions counters for hash-based maps. Non-trivial changes apply to preallocated maps for which the {inc,dec}_elem_count functions are not called, as there's no need in counting elements to sustain proper map operations. To increase/decrease percpu counters for preallocated hash maps we add raw calls to the bpf_map_{inc,dec}_elem_count functions so that the impact is minimal. For dynamically allocated maps we add corresponding calls to the existing {inc,dec}_elem_count functions. For LRU maps bpf_map_{inc,dec}_elem_count added to the lru pop/free helpers. Signed-off-by: Anton Protopopov <aspsk@isovalent.com> --- kernel/bpf/hashtab.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 56d3da7d0bc6..c23557bf9a1a 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -302,6 +302,7 @@ static struct htab_elem *prealloc_lru_pop(struct bpf_htab *htab, void *key, struct htab_elem *l; if (node) { + bpf_map_inc_elem_count(&htab->map); l = container_of(node, struct htab_elem, lru_node); memcpy(l->key, key, htab->map.key_size); return l; @@ -581,10 +582,17 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) } } + err = bpf_map_init_elem_count(&htab->map); + if (err) + goto free_extra_elements; + return &htab->map; +free_extra_elements: + free_percpu(htab->extra_elems); free_prealloc: - prealloc_destroy(htab); + if (prealloc) + prealloc_destroy(htab); free_map_locked: if (htab->use_percpu_counter) percpu_counter_destroy(&htab->pcount); @@ -804,6 +812,7 @@ static bool htab_lru_map_delete_node(void *arg, struct bpf_lru_node *node) if (l == tgt_l) { hlist_nulls_del_rcu(&l->hash_node); check_and_free_fields(htab, l); + bpf_map_dec_elem_count(&htab->map); break; } @@ -900,6 +909,8 @@ static bool is_map_full(struct bpf_htab *htab) static void inc_elem_count(struct bpf_htab *htab) { + bpf_map_inc_elem_count(&htab->map); + if (htab->use_percpu_counter) percpu_counter_add_batch(&htab->pcount, 1, PERCPU_COUNTER_BATCH); else @@ -908,6 +919,8 @@ static void inc_elem_count(struct bpf_htab *htab) static void dec_elem_count(struct bpf_htab *htab) { + bpf_map_dec_elem_count(&htab->map); + if (htab->use_percpu_counter) percpu_counter_add_batch(&htab->pcount, -1, PERCPU_COUNTER_BATCH); else @@ -920,6 +933,7 @@ static void free_htab_elem(struct bpf_htab *htab, struct htab_elem *l) htab_put_fd_value(htab, l); if (htab_is_prealloc(htab)) { + bpf_map_dec_elem_count(&htab->map); check_and_free_fields(htab, l); __pcpu_freelist_push(&htab->freelist, &l->fnode); } else { @@ -1000,6 +1014,7 @@ static struct htab_elem *alloc_htab_elem(struct bpf_htab *htab, void *key, if (!l) return ERR_PTR(-E2BIG); l_new = container_of(l, struct htab_elem, fnode); + bpf_map_inc_elem_count(&htab->map); } } else { if (is_map_full(htab)) @@ -1168,6 +1183,7 @@ static long htab_map_update_elem(struct bpf_map *map, void *key, void *value, static void htab_lru_push_free(struct bpf_htab *htab, struct htab_elem *elem) { check_and_free_fields(htab, elem); + bpf_map_dec_elem_count(&htab->map); bpf_lru_push_free(&htab->lru, &elem->lru_node); } @@ -1357,8 +1373,10 @@ static long __htab_lru_percpu_map_update_elem(struct bpf_map *map, void *key, err: htab_unlock_bucket(htab, b, hash, flags); err_lock_bucket: - if (l_new) + if (l_new) { + bpf_map_dec_elem_count(&htab->map); bpf_lru_push_free(&htab->lru, &l_new->lru_node); + } return ret; } @@ -1523,6 +1541,7 @@ static void htab_map_free(struct bpf_map *map) prealloc_destroy(htab); } + bpf_map_free_elem_count(map); free_percpu(htab->extra_elems); bpf_map_area_free(htab->buckets); bpf_mem_alloc_destroy(&htab->pcpu_ma); -- 2.34.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 3/6] bpf: populate the per-cpu insertions/deletions counters for hashmaps 2023-07-05 16:01 ` [PATCH v4 bpf-next 3/6] bpf: populate the per-cpu insertions/deletions counters for hashmaps Anton Protopopov @ 2023-07-06 1:24 ` Alexei Starovoitov 2023-07-06 5:47 ` Anton Protopopov 0 siblings, 1 reply; 24+ messages in thread From: Alexei Starovoitov @ 2023-07-06 1:24 UTC (permalink / raw) To: Anton Protopopov Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf On Wed, Jul 5, 2023 at 9:00 AM Anton Protopopov <aspsk@isovalent.com> wrote: > > Initialize and utilize the per-cpu insertions/deletions counters for hash-based > maps. Non-trivial changes apply to preallocated maps for which the > {inc,dec}_elem_count functions are not called, as there's no need in counting > elements to sustain proper map operations. > > To increase/decrease percpu counters for preallocated hash maps we add raw > calls to the bpf_map_{inc,dec}_elem_count functions so that the impact is > minimal. For dynamically allocated maps we add corresponding calls to the > existing {inc,dec}_elem_count functions. > > For LRU maps bpf_map_{inc,dec}_elem_count added to the lru pop/free helpers. > > Signed-off-by: Anton Protopopov <aspsk@isovalent.com> > --- > kernel/bpf/hashtab.c | 23 +++++++++++++++++++++-- > 1 file changed, 21 insertions(+), 2 deletions(-) > > diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c > index 56d3da7d0bc6..c23557bf9a1a 100644 > --- a/kernel/bpf/hashtab.c > +++ b/kernel/bpf/hashtab.c > @@ -302,6 +302,7 @@ static struct htab_elem *prealloc_lru_pop(struct bpf_htab *htab, void *key, > struct htab_elem *l; > > if (node) { > + bpf_map_inc_elem_count(&htab->map); > l = container_of(node, struct htab_elem, lru_node); > memcpy(l->key, key, htab->map.key_size); > return l; > @@ -581,10 +582,17 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) > } > } > > + err = bpf_map_init_elem_count(&htab->map); > + if (err) > + goto free_extra_elements; > + > return &htab->map; > > +free_extra_elements: > + free_percpu(htab->extra_elems); > free_prealloc: > - prealloc_destroy(htab); > + if (prealloc) > + prealloc_destroy(htab); This is a bit difficult to read. I think the logic would be easier to understand if bpf_map_init_elem_count was done right before htab->buckets = bpf_map_area_alloc() and if (err) goto free_htab where you would add bpf_map_free_elem_count. ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 3/6] bpf: populate the per-cpu insertions/deletions counters for hashmaps 2023-07-06 1:24 ` Alexei Starovoitov @ 2023-07-06 5:47 ` Anton Protopopov 0 siblings, 0 replies; 24+ messages in thread From: Anton Protopopov @ 2023-07-06 5:47 UTC (permalink / raw) To: Alexei Starovoitov Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf On Wed, Jul 05, 2023 at 06:24:44PM -0700, Alexei Starovoitov wrote: > On Wed, Jul 5, 2023 at 9:00 AM Anton Protopopov <aspsk@isovalent.com> wrote: > > > > Initialize and utilize the per-cpu insertions/deletions counters for hash-based > > maps. Non-trivial changes apply to preallocated maps for which the > > {inc,dec}_elem_count functions are not called, as there's no need in counting > > elements to sustain proper map operations. > > > > To increase/decrease percpu counters for preallocated hash maps we add raw > > calls to the bpf_map_{inc,dec}_elem_count functions so that the impact is > > minimal. For dynamically allocated maps we add corresponding calls to the > > existing {inc,dec}_elem_count functions. > > > > For LRU maps bpf_map_{inc,dec}_elem_count added to the lru pop/free helpers. > > > > Signed-off-by: Anton Protopopov <aspsk@isovalent.com> > > --- > > kernel/bpf/hashtab.c | 23 +++++++++++++++++++++-- > > 1 file changed, 21 insertions(+), 2 deletions(-) > > > > diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c > > index 56d3da7d0bc6..c23557bf9a1a 100644 > > --- a/kernel/bpf/hashtab.c > > +++ b/kernel/bpf/hashtab.c > > @@ -302,6 +302,7 @@ static struct htab_elem *prealloc_lru_pop(struct bpf_htab *htab, void *key, > > struct htab_elem *l; > > > > if (node) { > > + bpf_map_inc_elem_count(&htab->map); > > l = container_of(node, struct htab_elem, lru_node); > > memcpy(l->key, key, htab->map.key_size); > > return l; > > @@ -581,10 +582,17 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) > > } > > } > > > > + err = bpf_map_init_elem_count(&htab->map); > > + if (err) > > + goto free_extra_elements; > > + > > return &htab->map; > > > > +free_extra_elements: > > + free_percpu(htab->extra_elems); > > free_prealloc: > > - prealloc_destroy(htab); > > + if (prealloc) > > + prealloc_destroy(htab); > > This is a bit difficult to read. > I think the logic would be easier to understand if bpf_map_init_elem_count > was done right before htab->buckets = bpf_map_area_alloc() > and if (err) goto free_htab > where you would add bpf_map_free_elem_count. Thanks, I will fix this. ^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH v4 bpf-next 4/6] bpf: make preloaded map iterators to display map elements count 2023-07-05 16:01 [PATCH v4 bpf-next 0/6] bpf: add percpu stats for bpf_map Anton Protopopov ` (2 preceding siblings ...) 2023-07-05 16:01 ` [PATCH v4 bpf-next 3/6] bpf: populate the per-cpu insertions/deletions counters for hashmaps Anton Protopopov @ 2023-07-05 16:01 ` Anton Protopopov 2023-07-05 16:01 ` [PATCH v4 bpf-next 5/6] selftests/bpf: test map percpu stats Anton Protopopov 2023-07-05 16:01 ` [PATCH v4 bpf-next 6/6] selftests/bpf: check that ->elem_count is non-zero for the hash map Anton Protopopov 5 siblings, 0 replies; 24+ messages in thread From: Anton Protopopov @ 2023-07-05 16:01 UTC (permalink / raw) To: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf Cc: Anton Protopopov Add another column to the /sys/fs/bpf/maps.debug iterator to display cur_entries, the current number of entries in the map as is returned by the bpf_map_sum_elem_count kfunc. Also fix formatting. Example: # cat /sys/fs/bpf/maps.debug id name max_entries cur_entries 2 iterator.rodata 1 0 125 cilium_auth_map 524288 666 126 cilium_runtime_ 256 0 127 cilium_signals 32 0 128 cilium_node_map 16384 1344 129 cilium_events 32 0 ... Signed-off-by: Anton Protopopov <aspsk@isovalent.com> --- kernel/bpf/preload/iterators/iterators.bpf.c | 9 +- .../iterators/iterators.lskel-little-endian.h | 526 +++++++++--------- 2 files changed, 275 insertions(+), 260 deletions(-) diff --git a/kernel/bpf/preload/iterators/iterators.bpf.c b/kernel/bpf/preload/iterators/iterators.bpf.c index 03af863314ea..b78968b63fab 100644 --- a/kernel/bpf/preload/iterators/iterators.bpf.c +++ b/kernel/bpf/preload/iterators/iterators.bpf.c @@ -73,6 +73,8 @@ static const char *get_name(struct btf *btf, long btf_id, const char *fallback) return str + name_off; } +__s64 bpf_map_sum_elem_count(struct bpf_map *map) __ksym; + SEC("iter/bpf_map") int dump_bpf_map(struct bpf_iter__bpf_map *ctx) { @@ -84,9 +86,12 @@ int dump_bpf_map(struct bpf_iter__bpf_map *ctx) return 0; if (seq_num == 0) - BPF_SEQ_PRINTF(seq, " id name max_entries\n"); + BPF_SEQ_PRINTF(seq, " id name max_entries cur_entries\n"); + + BPF_SEQ_PRINTF(seq, "%4u %-16s %10d %10lld\n", + map->id, map->name, map->max_entries, + bpf_map_sum_elem_count(map)); - BPF_SEQ_PRINTF(seq, "%4u %-16s%6d\n", map->id, map->name, map->max_entries); return 0; } diff --git a/kernel/bpf/preload/iterators/iterators.lskel-little-endian.h b/kernel/bpf/preload/iterators/iterators.lskel-little-endian.h index 70f236a82fe1..5b98ab02025e 100644 --- a/kernel/bpf/preload/iterators/iterators.lskel-little-endian.h +++ b/kernel/bpf/preload/iterators/iterators.lskel-little-endian.h @@ -1,5 +1,5 @@ /* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ -/* THIS FILE IS AUTOGENERATED! */ +/* THIS FILE IS AUTOGENERATED BY BPFTOOL! */ #ifndef __ITERATORS_BPF_SKEL_H__ #define __ITERATORS_BPF_SKEL_H__ @@ -18,8 +18,6 @@ struct iterators_bpf { int dump_bpf_map_fd; int dump_bpf_prog_fd; } links; - struct iterators_bpf__rodata { - } *rodata; }; static inline int @@ -68,7 +66,6 @@ iterators_bpf__destroy(struct iterators_bpf *skel) iterators_bpf__detach(skel); skel_closenz(skel->progs.dump_bpf_map.prog_fd); skel_closenz(skel->progs.dump_bpf_prog.prog_fd); - skel_free_map_data(skel->rodata, skel->maps.rodata.initial_value, 4096); skel_closenz(skel->maps.rodata.map_fd); skel_free(skel); } @@ -81,15 +78,6 @@ iterators_bpf__open(void) if (!skel) goto cleanup; skel->ctx.sz = (void *)&skel->links - (void *)skel; - skel->rodata = skel_prep_map_data((void *)"\ -\x20\x20\x69\x64\x20\x6e\x61\x6d\x65\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\ -\x20\x20\x20\x6d\x61\x78\x5f\x65\x6e\x74\x72\x69\x65\x73\x0a\0\x25\x34\x75\x20\ -\x25\x2d\x31\x36\x73\x25\x36\x64\x0a\0\x20\x20\x69\x64\x20\x6e\x61\x6d\x65\x20\ -\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x61\x74\x74\x61\x63\x68\x65\ -\x64\x0a\0\x25\x34\x75\x20\x25\x2d\x31\x36\x73\x20\x25\x73\x20\x25\x73\x0a\0", 4096, 98); - if (!skel->rodata) - goto cleanup; - skel->maps.rodata.initial_value = (__u64) (long) skel->rodata; return skel; cleanup: iterators_bpf__destroy(skel); @@ -103,7 +91,7 @@ iterators_bpf__load(struct iterators_bpf *skel) int err; opts.ctx = (struct bpf_loader_ctx *)skel; - opts.data_sz = 6056; + opts.data_sz = 6208; opts.data = (void *)"\ \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\ \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\ @@ -138,190 +126,197 @@ iterators_bpf__load(struct iterators_bpf *skel) \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\ \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\ \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x9f\xeb\x01\0\ -\x18\0\0\0\0\0\0\0\x1c\x04\0\0\x1c\x04\0\0\xf9\x04\0\0\0\0\0\0\0\0\0\x02\x02\0\ +\x18\0\0\0\0\0\0\0\x80\x04\0\0\x80\x04\0\0\x31\x05\0\0\0\0\0\0\0\0\0\x02\x02\0\ \0\0\x01\0\0\0\x02\0\0\x04\x10\0\0\0\x13\0\0\0\x03\0\0\0\0\0\0\0\x18\0\0\0\x04\ \0\0\0\x40\0\0\0\0\0\0\0\0\0\0\x02\x08\0\0\0\0\0\0\0\0\0\0\x02\x0d\0\0\0\0\0\0\ \0\x01\0\0\x0d\x06\0\0\0\x1c\0\0\0\x01\0\0\0\x20\0\0\0\0\0\0\x01\x04\0\0\0\x20\ -\0\0\x01\x24\0\0\0\x01\0\0\x0c\x05\0\0\0\xa3\0\0\0\x03\0\0\x04\x18\0\0\0\xb1\0\ -\0\0\x09\0\0\0\0\0\0\0\xb5\0\0\0\x0b\0\0\0\x40\0\0\0\xc0\0\0\0\x0b\0\0\0\x80\0\ -\0\0\0\0\0\0\0\0\0\x02\x0a\0\0\0\xc8\0\0\0\0\0\0\x07\0\0\0\0\xd1\0\0\0\0\0\0\ -\x08\x0c\0\0\0\xd7\0\0\0\0\0\0\x01\x08\0\0\0\x40\0\0\0\x94\x01\0\0\x03\0\0\x04\ -\x18\0\0\0\x9c\x01\0\0\x0e\0\0\0\0\0\0\0\x9f\x01\0\0\x11\0\0\0\x20\0\0\0\xa4\ -\x01\0\0\x0e\0\0\0\xa0\0\0\0\xb0\x01\0\0\0\0\0\x08\x0f\0\0\0\xb6\x01\0\0\0\0\0\ -\x01\x04\0\0\0\x20\0\0\0\xc3\x01\0\0\0\0\0\x01\x01\0\0\0\x08\0\0\x01\0\0\0\0\0\ -\0\0\x03\0\0\0\0\x10\0\0\0\x12\0\0\0\x10\0\0\0\xc8\x01\0\0\0\0\0\x01\x04\0\0\0\ -\x20\0\0\0\0\0\0\0\0\0\0\x02\x14\0\0\0\x2c\x02\0\0\x02\0\0\x04\x10\0\0\0\x13\0\ -\0\0\x03\0\0\0\0\0\0\0\x3f\x02\0\0\x15\0\0\0\x40\0\0\0\0\0\0\0\0\0\0\x02\x18\0\ -\0\0\0\0\0\0\x01\0\0\x0d\x06\0\0\0\x1c\0\0\0\x13\0\0\0\x44\x02\0\0\x01\0\0\x0c\ -\x16\0\0\0\x90\x02\0\0\x01\0\0\x04\x08\0\0\0\x99\x02\0\0\x19\0\0\0\0\0\0\0\0\0\ -\0\0\0\0\0\x02\x1a\0\0\0\xea\x02\0\0\x06\0\0\x04\x38\0\0\0\x9c\x01\0\0\x0e\0\0\ -\0\0\0\0\0\x9f\x01\0\0\x11\0\0\0\x20\0\0\0\xf7\x02\0\0\x1b\0\0\0\xc0\0\0\0\x08\ -\x03\0\0\x15\0\0\0\0\x01\0\0\x11\x03\0\0\x1d\0\0\0\x40\x01\0\0\x1b\x03\0\0\x1e\ -\0\0\0\x80\x01\0\0\0\0\0\0\0\0\0\x02\x1c\0\0\0\0\0\0\0\0\0\0\x0a\x10\0\0\0\0\0\ -\0\0\0\0\0\x02\x1f\0\0\0\0\0\0\0\0\0\0\x02\x20\0\0\0\x65\x03\0\0\x02\0\0\x04\ -\x08\0\0\0\x73\x03\0\0\x0e\0\0\0\0\0\0\0\x7c\x03\0\0\x0e\0\0\0\x20\0\0\0\x1b\ -\x03\0\0\x03\0\0\x04\x18\0\0\0\x86\x03\0\0\x1b\0\0\0\0\0\0\0\x8e\x03\0\0\x21\0\ -\0\0\x40\0\0\0\x94\x03\0\0\x23\0\0\0\x80\0\0\0\0\0\0\0\0\0\0\x02\x22\0\0\0\0\0\ -\0\0\0\0\0\x02\x24\0\0\0\x98\x03\0\0\x01\0\0\x04\x04\0\0\0\xa3\x03\0\0\x0e\0\0\ -\0\0\0\0\0\x0c\x04\0\0\x01\0\0\x04\x04\0\0\0\x15\x04\0\0\x0e\0\0\0\0\0\0\0\0\0\ -\0\0\0\0\0\x03\0\0\0\0\x1c\0\0\0\x12\0\0\0\x23\0\0\0\x8b\x04\0\0\0\0\0\x0e\x25\ -\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x03\0\0\0\0\x1c\0\0\0\x12\0\0\0\x0e\0\0\0\x9f\x04\ -\0\0\0\0\0\x0e\x27\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x03\0\0\0\0\x1c\0\0\0\x12\0\0\0\ -\x20\0\0\0\xb5\x04\0\0\0\0\0\x0e\x29\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x03\0\0\0\0\ -\x1c\0\0\0\x12\0\0\0\x11\0\0\0\xca\x04\0\0\0\0\0\x0e\x2b\0\0\0\0\0\0\0\0\0\0\0\ -\0\0\0\x03\0\0\0\0\x10\0\0\0\x12\0\0\0\x04\0\0\0\xe1\x04\0\0\0\0\0\x0e\x2d\0\0\ -\0\x01\0\0\0\xe9\x04\0\0\x04\0\0\x0f\x62\0\0\0\x26\0\0\0\0\0\0\0\x23\0\0\0\x28\ -\0\0\0\x23\0\0\0\x0e\0\0\0\x2a\0\0\0\x31\0\0\0\x20\0\0\0\x2c\0\0\0\x51\0\0\0\ -\x11\0\0\0\xf1\x04\0\0\x01\0\0\x0f\x04\0\0\0\x2e\0\0\0\0\0\0\0\x04\0\0\0\0\x62\ -\x70\x66\x5f\x69\x74\x65\x72\x5f\x5f\x62\x70\x66\x5f\x6d\x61\x70\0\x6d\x65\x74\ -\x61\0\x6d\x61\x70\0\x63\x74\x78\0\x69\x6e\x74\0\x64\x75\x6d\x70\x5f\x62\x70\ -\x66\x5f\x6d\x61\x70\0\x69\x74\x65\x72\x2f\x62\x70\x66\x5f\x6d\x61\x70\0\x30\ -\x3a\x30\0\x2f\x77\x2f\x6e\x65\x74\x2d\x6e\x65\x78\x74\x2f\x6b\x65\x72\x6e\x65\ -\x6c\x2f\x62\x70\x66\x2f\x70\x72\x65\x6c\x6f\x61\x64\x2f\x69\x74\x65\x72\x61\ -\x74\x6f\x72\x73\x2f\x69\x74\x65\x72\x61\x74\x6f\x72\x73\x2e\x62\x70\x66\x2e\ -\x63\0\x09\x73\x74\x72\x75\x63\x74\x20\x73\x65\x71\x5f\x66\x69\x6c\x65\x20\x2a\ -\x73\x65\x71\x20\x3d\x20\x63\x74\x78\x2d\x3e\x6d\x65\x74\x61\x2d\x3e\x73\x65\ -\x71\x3b\0\x62\x70\x66\x5f\x69\x74\x65\x72\x5f\x6d\x65\x74\x61\0\x73\x65\x71\0\ -\x73\x65\x73\x73\x69\x6f\x6e\x5f\x69\x64\0\x73\x65\x71\x5f\x6e\x75\x6d\0\x73\ -\x65\x71\x5f\x66\x69\x6c\x65\0\x5f\x5f\x75\x36\x34\0\x75\x6e\x73\x69\x67\x6e\ -\x65\x64\x20\x6c\x6f\x6e\x67\x20\x6c\x6f\x6e\x67\0\x30\x3a\x31\0\x09\x73\x74\ -\x72\x75\x63\x74\x20\x62\x70\x66\x5f\x6d\x61\x70\x20\x2a\x6d\x61\x70\x20\x3d\ -\x20\x63\x74\x78\x2d\x3e\x6d\x61\x70\x3b\0\x09\x69\x66\x20\x28\x21\x6d\x61\x70\ -\x29\0\x09\x5f\x5f\x75\x36\x34\x20\x73\x65\x71\x5f\x6e\x75\x6d\x20\x3d\x20\x63\ -\x74\x78\x2d\x3e\x6d\x65\x74\x61\x2d\x3e\x73\x65\x71\x5f\x6e\x75\x6d\x3b\0\x30\ -\x3a\x32\0\x09\x69\x66\x20\x28\x73\x65\x71\x5f\x6e\x75\x6d\x20\x3d\x3d\x20\x30\ -\x29\0\x09\x09\x42\x50\x46\x5f\x53\x45\x51\x5f\x50\x52\x49\x4e\x54\x46\x28\x73\ -\x65\x71\x2c\x20\x22\x20\x20\x69\x64\x20\x6e\x61\x6d\x65\x20\x20\x20\x20\x20\ -\x20\x20\x20\x20\x20\x20\x20\x20\x6d\x61\x78\x5f\x65\x6e\x74\x72\x69\x65\x73\ -\x5c\x6e\x22\x29\x3b\0\x62\x70\x66\x5f\x6d\x61\x70\0\x69\x64\0\x6e\x61\x6d\x65\ -\0\x6d\x61\x78\x5f\x65\x6e\x74\x72\x69\x65\x73\0\x5f\x5f\x75\x33\x32\0\x75\x6e\ -\x73\x69\x67\x6e\x65\x64\x20\x69\x6e\x74\0\x63\x68\x61\x72\0\x5f\x5f\x41\x52\ -\x52\x41\x59\x5f\x53\x49\x5a\x45\x5f\x54\x59\x50\x45\x5f\x5f\0\x09\x42\x50\x46\ -\x5f\x53\x45\x51\x5f\x50\x52\x49\x4e\x54\x46\x28\x73\x65\x71\x2c\x20\x22\x25\ -\x34\x75\x20\x25\x2d\x31\x36\x73\x25\x36\x64\x5c\x6e\x22\x2c\x20\x6d\x61\x70\ -\x2d\x3e\x69\x64\x2c\x20\x6d\x61\x70\x2d\x3e\x6e\x61\x6d\x65\x2c\x20\x6d\x61\ -\x70\x2d\x3e\x6d\x61\x78\x5f\x65\x6e\x74\x72\x69\x65\x73\x29\x3b\0\x7d\0\x62\ -\x70\x66\x5f\x69\x74\x65\x72\x5f\x5f\x62\x70\x66\x5f\x70\x72\x6f\x67\0\x70\x72\ -\x6f\x67\0\x64\x75\x6d\x70\x5f\x62\x70\x66\x5f\x70\x72\x6f\x67\0\x69\x74\x65\ -\x72\x2f\x62\x70\x66\x5f\x70\x72\x6f\x67\0\x09\x73\x74\x72\x75\x63\x74\x20\x62\ -\x70\x66\x5f\x70\x72\x6f\x67\x20\x2a\x70\x72\x6f\x67\x20\x3d\x20\x63\x74\x78\ -\x2d\x3e\x70\x72\x6f\x67\x3b\0\x09\x69\x66\x20\x28\x21\x70\x72\x6f\x67\x29\0\ -\x62\x70\x66\x5f\x70\x72\x6f\x67\0\x61\x75\x78\0\x09\x61\x75\x78\x20\x3d\x20\ -\x70\x72\x6f\x67\x2d\x3e\x61\x75\x78\x3b\0\x09\x09\x42\x50\x46\x5f\x53\x45\x51\ -\x5f\x50\x52\x49\x4e\x54\x46\x28\x73\x65\x71\x2c\x20\x22\x20\x20\x69\x64\x20\ -\x6e\x61\x6d\x65\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x61\x74\ -\x74\x61\x63\x68\x65\x64\x5c\x6e\x22\x29\x3b\0\x62\x70\x66\x5f\x70\x72\x6f\x67\ -\x5f\x61\x75\x78\0\x61\x74\x74\x61\x63\x68\x5f\x66\x75\x6e\x63\x5f\x6e\x61\x6d\ -\x65\0\x64\x73\x74\x5f\x70\x72\x6f\x67\0\x66\x75\x6e\x63\x5f\x69\x6e\x66\x6f\0\ -\x62\x74\x66\0\x09\x42\x50\x46\x5f\x53\x45\x51\x5f\x50\x52\x49\x4e\x54\x46\x28\ -\x73\x65\x71\x2c\x20\x22\x25\x34\x75\x20\x25\x2d\x31\x36\x73\x20\x25\x73\x20\ -\x25\x73\x5c\x6e\x22\x2c\x20\x61\x75\x78\x2d\x3e\x69\x64\x2c\0\x30\x3a\x34\0\ -\x30\x3a\x35\0\x09\x69\x66\x20\x28\x21\x62\x74\x66\x29\0\x62\x70\x66\x5f\x66\ -\x75\x6e\x63\x5f\x69\x6e\x66\x6f\0\x69\x6e\x73\x6e\x5f\x6f\x66\x66\0\x74\x79\ -\x70\x65\x5f\x69\x64\0\x30\0\x73\x74\x72\x69\x6e\x67\x73\0\x74\x79\x70\x65\x73\ -\0\x68\x64\x72\0\x62\x74\x66\x5f\x68\x65\x61\x64\x65\x72\0\x73\x74\x72\x5f\x6c\ -\x65\x6e\0\x09\x74\x79\x70\x65\x73\x20\x3d\x20\x62\x74\x66\x2d\x3e\x74\x79\x70\ -\x65\x73\x3b\0\x09\x62\x70\x66\x5f\x70\x72\x6f\x62\x65\x5f\x72\x65\x61\x64\x5f\ -\x6b\x65\x72\x6e\x65\x6c\x28\x26\x74\x2c\x20\x73\x69\x7a\x65\x6f\x66\x28\x74\ -\x29\x2c\x20\x74\x79\x70\x65\x73\x20\x2b\x20\x62\x74\x66\x5f\x69\x64\x29\x3b\0\ -\x09\x73\x74\x72\x20\x3d\x20\x62\x74\x66\x2d\x3e\x73\x74\x72\x69\x6e\x67\x73\ -\x3b\0\x62\x74\x66\x5f\x74\x79\x70\x65\0\x6e\x61\x6d\x65\x5f\x6f\x66\x66\0\x09\ -\x6e\x61\x6d\x65\x5f\x6f\x66\x66\x20\x3d\x20\x42\x50\x46\x5f\x43\x4f\x52\x45\ -\x5f\x52\x45\x41\x44\x28\x74\x2c\x20\x6e\x61\x6d\x65\x5f\x6f\x66\x66\x29\x3b\0\ -\x30\x3a\x32\x3a\x30\0\x09\x69\x66\x20\x28\x6e\x61\x6d\x65\x5f\x6f\x66\x66\x20\ -\x3e\x3d\x20\x62\x74\x66\x2d\x3e\x68\x64\x72\x2e\x73\x74\x72\x5f\x6c\x65\x6e\ -\x29\0\x09\x72\x65\x74\x75\x72\x6e\x20\x73\x74\x72\x20\x2b\x20\x6e\x61\x6d\x65\ -\x5f\x6f\x66\x66\x3b\0\x30\x3a\x33\0\x64\x75\x6d\x70\x5f\x62\x70\x66\x5f\x6d\ -\x61\x70\x2e\x5f\x5f\x5f\x66\x6d\x74\0\x64\x75\x6d\x70\x5f\x62\x70\x66\x5f\x6d\ -\x61\x70\x2e\x5f\x5f\x5f\x66\x6d\x74\x2e\x31\0\x64\x75\x6d\x70\x5f\x62\x70\x66\ -\x5f\x70\x72\x6f\x67\x2e\x5f\x5f\x5f\x66\x6d\x74\0\x64\x75\x6d\x70\x5f\x62\x70\ -\x66\x5f\x70\x72\x6f\x67\x2e\x5f\x5f\x5f\x66\x6d\x74\x2e\x32\0\x4c\x49\x43\x45\ -\x4e\x53\x45\0\x2e\x72\x6f\x64\x61\x74\x61\0\x6c\x69\x63\x65\x6e\x73\x65\0\0\0\ -\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x2d\x09\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x02\0\0\ -\0\x04\0\0\0\x62\0\0\0\x01\0\0\0\x80\x04\0\0\0\0\0\0\0\0\0\0\x69\x74\x65\x72\ -\x61\x74\x6f\x72\x2e\x72\x6f\x64\x61\x74\x61\0\0\0\0\0\0\0\0\0\0\0\0\0\x2f\0\0\ -\0\0\0\0\0\0\0\0\0\0\0\0\0\x20\x20\x69\x64\x20\x6e\x61\x6d\x65\x20\x20\x20\x20\ -\x20\x20\x20\x20\x20\x20\x20\x20\x20\x6d\x61\x78\x5f\x65\x6e\x74\x72\x69\x65\ -\x73\x0a\0\x25\x34\x75\x20\x25\x2d\x31\x36\x73\x25\x36\x64\x0a\0\x20\x20\x69\ -\x64\x20\x6e\x61\x6d\x65\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\ -\x61\x74\x74\x61\x63\x68\x65\x64\x0a\0\x25\x34\x75\x20\x25\x2d\x31\x36\x73\x20\ -\x25\x73\x20\x25\x73\x0a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\ -\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x47\x50\x4c\0\0\0\0\0\ -\x79\x12\0\0\0\0\0\0\x79\x26\0\0\0\0\0\0\x79\x17\x08\0\0\0\0\0\x15\x07\x1b\0\0\ -\0\0\0\x79\x11\0\0\0\0\0\0\x79\x11\x10\0\0\0\0\0\x55\x01\x08\0\0\0\0\0\xbf\xa4\ -\0\0\0\0\0\0\x07\x04\0\0\xe8\xff\xff\xff\xbf\x61\0\0\0\0\0\0\x18\x62\0\0\0\0\0\ -\0\0\0\0\0\0\0\0\0\xb7\x03\0\0\x23\0\0\0\xb7\x05\0\0\0\0\0\0\x85\0\0\0\x7e\0\0\ -\0\x61\x71\0\0\0\0\0\0\x7b\x1a\xe8\xff\0\0\0\0\xb7\x01\0\0\x04\0\0\0\xbf\x72\0\ -\0\0\0\0\0\x0f\x12\0\0\0\0\0\0\x7b\x2a\xf0\xff\0\0\0\0\x61\x71\x14\0\0\0\0\0\ -\x7b\x1a\xf8\xff\0\0\0\0\xbf\xa4\0\0\0\0\0\0\x07\x04\0\0\xe8\xff\xff\xff\xbf\ -\x61\0\0\0\0\0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\x23\0\0\0\xb7\x03\0\0\x0e\0\0\0\ -\xb7\x05\0\0\x18\0\0\0\x85\0\0\0\x7e\0\0\0\xb7\0\0\0\0\0\0\0\x95\0\0\0\0\0\0\0\ -\0\0\0\0\x07\0\0\0\0\0\0\0\x42\0\0\0\x7b\0\0\0\x1e\x3c\x01\0\x01\0\0\0\x42\0\0\ -\0\x7b\0\0\0\x24\x3c\x01\0\x02\0\0\0\x42\0\0\0\xee\0\0\0\x1d\x44\x01\0\x03\0\0\ -\0\x42\0\0\0\x0f\x01\0\0\x06\x4c\x01\0\x04\0\0\0\x42\0\0\0\x1a\x01\0\0\x17\x40\ -\x01\0\x05\0\0\0\x42\0\0\0\x1a\x01\0\0\x1d\x40\x01\0\x06\0\0\0\x42\0\0\0\x43\ -\x01\0\0\x06\x58\x01\0\x08\0\0\0\x42\0\0\0\x56\x01\0\0\x03\x5c\x01\0\x0f\0\0\0\ -\x42\0\0\0\xdc\x01\0\0\x02\x64\x01\0\x1f\0\0\0\x42\0\0\0\x2a\x02\0\0\x01\x6c\ -\x01\0\0\0\0\0\x02\0\0\0\x3e\0\0\0\0\0\0\0\x08\0\0\0\x08\0\0\0\x3e\0\0\0\0\0\0\ -\0\x10\0\0\0\x02\0\0\0\xea\0\0\0\0\0\0\0\x20\0\0\0\x02\0\0\0\x3e\0\0\0\0\0\0\0\ -\x28\0\0\0\x08\0\0\0\x3f\x01\0\0\0\0\0\0\x78\0\0\0\x0d\0\0\0\x3e\0\0\0\0\0\0\0\ -\x88\0\0\0\x0d\0\0\0\xea\0\0\0\0\0\0\0\xa8\0\0\0\x0d\0\0\0\x3f\x01\0\0\0\0\0\0\ -\x1a\0\0\0\x21\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\ -\0\0\0\0\0\0\0\0\0\0\0\x64\x75\x6d\x70\x5f\x62\x70\x66\x5f\x6d\x61\x70\0\0\0\0\ -\0\0\0\0\x1c\0\0\0\0\0\0\0\x08\0\0\0\0\0\0\0\0\0\0\0\x01\0\0\0\x10\0\0\0\0\0\0\ -\0\0\0\0\0\x0a\0\0\0\x01\0\0\0\0\0\0\0\x08\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\ -\0\x10\0\0\0\0\0\0\0\x62\x70\x66\x5f\x69\x74\x65\x72\x5f\x62\x70\x66\x5f\x6d\ -\x61\x70\0\0\0\0\0\0\0\0\x47\x50\x4c\0\0\0\0\0\x79\x12\0\0\0\0\0\0\x79\x26\0\0\ -\0\0\0\0\x79\x12\x08\0\0\0\0\0\x15\x02\x3c\0\0\0\0\0\x79\x11\0\0\0\0\0\0\x79\ -\x27\0\0\0\0\0\0\x79\x11\x10\0\0\0\0\0\x55\x01\x08\0\0\0\0\0\xbf\xa4\0\0\0\0\0\ -\0\x07\x04\0\0\xd0\xff\xff\xff\xbf\x61\0\0\0\0\0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\ -\x31\0\0\0\xb7\x03\0\0\x20\0\0\0\xb7\x05\0\0\0\0\0\0\x85\0\0\0\x7e\0\0\0\x7b\ -\x6a\xc8\xff\0\0\0\0\x61\x71\0\0\0\0\0\0\x7b\x1a\xd0\xff\0\0\0\0\xb7\x03\0\0\ -\x04\0\0\0\xbf\x79\0\0\0\0\0\0\x0f\x39\0\0\0\0\0\0\x79\x71\x28\0\0\0\0\0\x79\ -\x78\x30\0\0\0\0\0\x15\x08\x18\0\0\0\0\0\xb7\x02\0\0\0\0\0\0\x0f\x21\0\0\0\0\0\ -\0\x61\x11\x04\0\0\0\0\0\x79\x83\x08\0\0\0\0\0\x67\x01\0\0\x03\0\0\0\x0f\x13\0\ -\0\0\0\0\0\x79\x86\0\0\0\0\0\0\xbf\xa1\0\0\0\0\0\0\x07\x01\0\0\xf8\xff\xff\xff\ -\xb7\x02\0\0\x08\0\0\0\x85\0\0\0\x71\0\0\0\xb7\x01\0\0\0\0\0\0\x79\xa3\xf8\xff\ -\0\0\0\0\x0f\x13\0\0\0\0\0\0\xbf\xa1\0\0\0\0\0\0\x07\x01\0\0\xf4\xff\xff\xff\ -\xb7\x02\0\0\x04\0\0\0\x85\0\0\0\x71\0\0\0\xb7\x03\0\0\x04\0\0\0\x61\xa1\xf4\ -\xff\0\0\0\0\x61\x82\x10\0\0\0\0\0\x3d\x21\x02\0\0\0\0\0\x0f\x16\0\0\0\0\0\0\ -\xbf\x69\0\0\0\0\0\0\x7b\x9a\xd8\xff\0\0\0\0\x79\x71\x18\0\0\0\0\0\x7b\x1a\xe0\ -\xff\0\0\0\0\x79\x71\x20\0\0\0\0\0\x79\x11\0\0\0\0\0\0\x0f\x31\0\0\0\0\0\0\x7b\ -\x1a\xe8\xff\0\0\0\0\xbf\xa4\0\0\0\0\0\0\x07\x04\0\0\xd0\xff\xff\xff\x79\xa1\ -\xc8\xff\0\0\0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\x51\0\0\0\xb7\x03\0\0\x11\0\0\0\ -\xb7\x05\0\0\x20\0\0\0\x85\0\0\0\x7e\0\0\0\xb7\0\0\0\0\0\0\0\x95\0\0\0\0\0\0\0\ -\0\0\0\0\x17\0\0\0\0\0\0\0\x42\0\0\0\x7b\0\0\0\x1e\x80\x01\0\x01\0\0\0\x42\0\0\ -\0\x7b\0\0\0\x24\x80\x01\0\x02\0\0\0\x42\0\0\0\x60\x02\0\0\x1f\x88\x01\0\x03\0\ -\0\0\x42\0\0\0\x84\x02\0\0\x06\x94\x01\0\x04\0\0\0\x42\0\0\0\x1a\x01\0\0\x17\ -\x84\x01\0\x05\0\0\0\x42\0\0\0\x9d\x02\0\0\x0e\xa0\x01\0\x06\0\0\0\x42\0\0\0\ -\x1a\x01\0\0\x1d\x84\x01\0\x07\0\0\0\x42\0\0\0\x43\x01\0\0\x06\xa4\x01\0\x09\0\ -\0\0\x42\0\0\0\xaf\x02\0\0\x03\xa8\x01\0\x11\0\0\0\x42\0\0\0\x1f\x03\0\0\x02\ -\xb0\x01\0\x18\0\0\0\x42\0\0\0\x5a\x03\0\0\x06\x04\x01\0\x1b\0\0\0\x42\0\0\0\0\ -\0\0\0\0\0\0\0\x1c\0\0\0\x42\0\0\0\xab\x03\0\0\x0f\x10\x01\0\x1d\0\0\0\x42\0\0\ -\0\xc0\x03\0\0\x2d\x14\x01\0\x1f\0\0\0\x42\0\0\0\xf7\x03\0\0\x0d\x0c\x01\0\x21\ -\0\0\0\x42\0\0\0\0\0\0\0\0\0\0\0\x22\0\0\0\x42\0\0\0\xc0\x03\0\0\x02\x14\x01\0\ -\x25\0\0\0\x42\0\0\0\x1e\x04\0\0\x0d\x18\x01\0\x28\0\0\0\x42\0\0\0\0\0\0\0\0\0\ -\0\0\x29\0\0\0\x42\0\0\0\x1e\x04\0\0\x0d\x18\x01\0\x2c\0\0\0\x42\0\0\0\x1e\x04\ -\0\0\x0d\x18\x01\0\x2d\0\0\0\x42\0\0\0\x4c\x04\0\0\x1b\x1c\x01\0\x2e\0\0\0\x42\ -\0\0\0\x4c\x04\0\0\x06\x1c\x01\0\x2f\0\0\0\x42\0\0\0\x6f\x04\0\0\x0d\x24\x01\0\ -\x31\0\0\0\x42\0\0\0\x1f\x03\0\0\x02\xb0\x01\0\x40\0\0\0\x42\0\0\0\x2a\x02\0\0\ -\x01\xc0\x01\0\0\0\0\0\x14\0\0\0\x3e\0\0\0\0\0\0\0\x08\0\0\0\x08\0\0\0\x3e\0\0\ -\0\0\0\0\0\x10\0\0\0\x14\0\0\0\xea\0\0\0\0\0\0\0\x20\0\0\0\x14\0\0\0\x3e\0\0\0\ -\0\0\0\0\x28\0\0\0\x18\0\0\0\x3e\0\0\0\0\0\0\0\x30\0\0\0\x08\0\0\0\x3f\x01\0\0\ -\0\0\0\0\x88\0\0\0\x1a\0\0\0\x3e\0\0\0\0\0\0\0\x98\0\0\0\x1a\0\0\0\xea\0\0\0\0\ -\0\0\0\xb0\0\0\0\x1a\0\0\0\x52\x03\0\0\0\0\0\0\xb8\0\0\0\x1a\0\0\0\x56\x03\0\0\ -\0\0\0\0\xc8\0\0\0\x1f\0\0\0\x84\x03\0\0\0\0\0\0\xe0\0\0\0\x20\0\0\0\xea\0\0\0\ -\0\0\0\0\xf8\0\0\0\x20\0\0\0\x3e\0\0\0\0\0\0\0\x20\x01\0\0\x24\0\0\0\x3e\0\0\0\ -\0\0\0\0\x58\x01\0\0\x1a\0\0\0\xea\0\0\0\0\0\0\0\x68\x01\0\0\x20\0\0\0\x46\x04\ -\0\0\0\0\0\0\x90\x01\0\0\x1a\0\0\0\x3f\x01\0\0\0\0\0\0\xa0\x01\0\0\x1a\0\0\0\ -\x87\x04\0\0\0\0\0\0\xa8\x01\0\0\x18\0\0\0\x3e\0\0\0\0\0\0\0\x1a\0\0\0\x42\0\0\ -\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\ -\0\0\x64\x75\x6d\x70\x5f\x62\x70\x66\x5f\x70\x72\x6f\x67\0\0\0\0\0\0\0\x1c\0\0\ -\0\0\0\0\0\x08\0\0\0\0\0\0\0\0\0\0\0\x01\0\0\0\x10\0\0\0\0\0\0\0\0\0\0\0\x1a\0\ -\0\0\x01\0\0\0\0\0\0\0\x13\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x10\0\0\0\0\0\ -\0\0\x62\x70\x66\x5f\x69\x74\x65\x72\x5f\x62\x70\x66\x5f\x70\x72\x6f\x67\0\0\0\ -\0\0\0\0"; - opts.insns_sz = 2216; +\0\0\x01\x24\0\0\0\x01\0\0\x0c\x05\0\0\0\xb0\0\0\0\x03\0\0\x04\x18\0\0\0\xbe\0\ +\0\0\x09\0\0\0\0\0\0\0\xc2\0\0\0\x0b\0\0\0\x40\0\0\0\xcd\0\0\0\x0b\0\0\0\x80\0\ +\0\0\0\0\0\0\0\0\0\x02\x0a\0\0\0\xd5\0\0\0\0\0\0\x07\0\0\0\0\xde\0\0\0\0\0\0\ +\x08\x0c\0\0\0\xe4\0\0\0\0\0\0\x01\x08\0\0\0\x40\0\0\0\xae\x01\0\0\x03\0\0\x04\ +\x18\0\0\0\xb6\x01\0\0\x0e\0\0\0\0\0\0\0\xb9\x01\0\0\x11\0\0\0\x20\0\0\0\xbe\ +\x01\0\0\x0e\0\0\0\xa0\0\0\0\xca\x01\0\0\0\0\0\x08\x0f\0\0\0\xd0\x01\0\0\0\0\0\ +\x01\x04\0\0\0\x20\0\0\0\xdd\x01\0\0\0\0\0\x01\x01\0\0\0\x08\0\0\x01\0\0\0\0\0\ +\0\0\x03\0\0\0\0\x10\0\0\0\x12\0\0\0\x10\0\0\0\xe2\x01\0\0\0\0\0\x01\x04\0\0\0\ +\x20\0\0\0\0\0\0\0\x01\0\0\x0d\x14\0\0\0\x26\x05\0\0\x04\0\0\0\x2b\x02\0\0\0\0\ +\0\x08\x15\0\0\0\x31\x02\0\0\0\0\0\x01\x08\0\0\0\x40\0\0\x01\x3b\x02\0\0\x01\0\ +\0\x0c\x13\0\0\0\0\0\0\0\0\0\0\x02\x18\0\0\0\x52\x02\0\0\x02\0\0\x04\x10\0\0\0\ +\x13\0\0\0\x03\0\0\0\0\0\0\0\x65\x02\0\0\x19\0\0\0\x40\0\0\0\0\0\0\0\0\0\0\x02\ +\x1c\0\0\0\0\0\0\0\x01\0\0\x0d\x06\0\0\0\x1c\0\0\0\x17\0\0\0\x6a\x02\0\0\x01\0\ +\0\x0c\x1a\0\0\0\xb6\x02\0\0\x01\0\0\x04\x08\0\0\0\xbf\x02\0\0\x1d\0\0\0\0\0\0\ +\0\0\0\0\0\0\0\0\x02\x1e\0\0\0\x10\x03\0\0\x06\0\0\x04\x38\0\0\0\xb6\x01\0\0\ +\x0e\0\0\0\0\0\0\0\xb9\x01\0\0\x11\0\0\0\x20\0\0\0\x1d\x03\0\0\x1f\0\0\0\xc0\0\ +\0\0\x2e\x03\0\0\x19\0\0\0\0\x01\0\0\x37\x03\0\0\x21\0\0\0\x40\x01\0\0\x41\x03\ +\0\0\x22\0\0\0\x80\x01\0\0\0\0\0\0\0\0\0\x02\x20\0\0\0\0\0\0\0\0\0\0\x0a\x10\0\ +\0\0\0\0\0\0\0\0\0\x02\x23\0\0\0\0\0\0\0\0\0\0\x02\x24\0\0\0\x8b\x03\0\0\x02\0\ +\0\x04\x08\0\0\0\x99\x03\0\0\x0e\0\0\0\0\0\0\0\xa2\x03\0\0\x0e\0\0\0\x20\0\0\0\ +\x41\x03\0\0\x03\0\0\x04\x18\0\0\0\xac\x03\0\0\x1f\0\0\0\0\0\0\0\xb4\x03\0\0\ +\x25\0\0\0\x40\0\0\0\xba\x03\0\0\x27\0\0\0\x80\0\0\0\0\0\0\0\0\0\0\x02\x26\0\0\ +\0\0\0\0\0\0\0\0\x02\x28\0\0\0\xbe\x03\0\0\x01\0\0\x04\x04\0\0\0\xc9\x03\0\0\ +\x0e\0\0\0\0\0\0\0\x32\x04\0\0\x01\0\0\x04\x04\0\0\0\x3b\x04\0\0\x0e\0\0\0\0\0\ +\0\0\0\0\0\0\0\0\0\x03\0\0\0\0\x20\0\0\0\x12\0\0\0\x30\0\0\0\xb1\x04\0\0\0\0\0\ +\x0e\x29\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x03\0\0\0\0\x20\0\0\0\x12\0\0\0\x1a\0\0\0\ +\xc5\x04\0\0\0\0\0\x0e\x2b\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x03\0\0\0\0\x20\0\0\0\ +\x12\0\0\0\x20\0\0\0\xdb\x04\0\0\0\0\0\x0e\x2d\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x03\ +\0\0\0\0\x20\0\0\0\x12\0\0\0\x11\0\0\0\xf0\x04\0\0\0\0\0\x0e\x2f\0\0\0\0\0\0\0\ +\0\0\0\0\0\0\0\x03\0\0\0\0\x10\0\0\0\x12\0\0\0\x04\0\0\0\x07\x05\0\0\0\0\0\x0e\ +\x31\0\0\0\x01\0\0\0\x0f\x05\0\0\x01\0\0\x0f\x04\0\0\0\x36\0\0\0\0\0\0\0\x04\0\ +\0\0\x16\x05\0\0\x04\0\0\x0f\x7b\0\0\0\x2a\0\0\0\0\0\0\0\x30\0\0\0\x2c\0\0\0\ +\x30\0\0\0\x1a\0\0\0\x2e\0\0\0\x4a\0\0\0\x20\0\0\0\x30\0\0\0\x6a\0\0\0\x11\0\0\ +\0\x1e\x05\0\0\x01\0\0\x0f\x04\0\0\0\x32\0\0\0\0\0\0\0\x04\0\0\0\x26\x05\0\0\0\ +\0\0\x0e\x06\0\0\0\x01\0\0\0\0\x62\x70\x66\x5f\x69\x74\x65\x72\x5f\x5f\x62\x70\ +\x66\x5f\x6d\x61\x70\0\x6d\x65\x74\x61\0\x6d\x61\x70\0\x63\x74\x78\0\x69\x6e\ +\x74\0\x64\x75\x6d\x70\x5f\x62\x70\x66\x5f\x6d\x61\x70\0\x69\x74\x65\x72\x2f\ +\x62\x70\x66\x5f\x6d\x61\x70\0\x30\x3a\x30\0\x2f\x68\x6f\x6d\x65\x2f\x61\x73\ +\x70\x73\x6b\x2f\x73\x72\x63\x2f\x62\x70\x66\x2d\x6e\x65\x78\x74\x2f\x6b\x65\ +\x72\x6e\x65\x6c\x2f\x62\x70\x66\x2f\x70\x72\x65\x6c\x6f\x61\x64\x2f\x69\x74\ +\x65\x72\x61\x74\x6f\x72\x73\x2f\x69\x74\x65\x72\x61\x74\x6f\x72\x73\x2e\x62\ +\x70\x66\x2e\x63\0\x09\x73\x74\x72\x75\x63\x74\x20\x73\x65\x71\x5f\x66\x69\x6c\ +\x65\x20\x2a\x73\x65\x71\x20\x3d\x20\x63\x74\x78\x2d\x3e\x6d\x65\x74\x61\x2d\ +\x3e\x73\x65\x71\x3b\0\x62\x70\x66\x5f\x69\x74\x65\x72\x5f\x6d\x65\x74\x61\0\ +\x73\x65\x71\0\x73\x65\x73\x73\x69\x6f\x6e\x5f\x69\x64\0\x73\x65\x71\x5f\x6e\ +\x75\x6d\0\x73\x65\x71\x5f\x66\x69\x6c\x65\0\x5f\x5f\x75\x36\x34\0\x75\x6e\x73\ +\x69\x67\x6e\x65\x64\x20\x6c\x6f\x6e\x67\x20\x6c\x6f\x6e\x67\0\x30\x3a\x31\0\ +\x09\x73\x74\x72\x75\x63\x74\x20\x62\x70\x66\x5f\x6d\x61\x70\x20\x2a\x6d\x61\ +\x70\x20\x3d\x20\x63\x74\x78\x2d\x3e\x6d\x61\x70\x3b\0\x09\x69\x66\x20\x28\x21\ +\x6d\x61\x70\x29\0\x30\x3a\x32\0\x09\x5f\x5f\x75\x36\x34\x20\x73\x65\x71\x5f\ +\x6e\x75\x6d\x20\x3d\x20\x63\x74\x78\x2d\x3e\x6d\x65\x74\x61\x2d\x3e\x73\x65\ +\x71\x5f\x6e\x75\x6d\x3b\0\x09\x69\x66\x20\x28\x73\x65\x71\x5f\x6e\x75\x6d\x20\ +\x3d\x3d\x20\x30\x29\0\x09\x09\x42\x50\x46\x5f\x53\x45\x51\x5f\x50\x52\x49\x4e\ +\x54\x46\x28\x73\x65\x71\x2c\x20\x22\x20\x20\x69\x64\x20\x6e\x61\x6d\x65\x20\ +\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x6d\x61\x78\x5f\x65\x6e\x74\ +\x72\x69\x65\x73\x20\x20\x63\x75\x72\x5f\x65\x6e\x74\x72\x69\x65\x73\x5c\x6e\ +\x22\x29\x3b\0\x62\x70\x66\x5f\x6d\x61\x70\0\x69\x64\0\x6e\x61\x6d\x65\0\x6d\ +\x61\x78\x5f\x65\x6e\x74\x72\x69\x65\x73\0\x5f\x5f\x75\x33\x32\0\x75\x6e\x73\ +\x69\x67\x6e\x65\x64\x20\x69\x6e\x74\0\x63\x68\x61\x72\0\x5f\x5f\x41\x52\x52\ +\x41\x59\x5f\x53\x49\x5a\x45\x5f\x54\x59\x50\x45\x5f\x5f\0\x09\x42\x50\x46\x5f\ +\x53\x45\x51\x5f\x50\x52\x49\x4e\x54\x46\x28\x73\x65\x71\x2c\x20\x22\x25\x34\ +\x75\x20\x25\x2d\x31\x36\x73\x20\x20\x25\x31\x30\x64\x20\x20\x20\x25\x31\x30\ +\x6c\x6c\x64\x5c\x6e\x22\x2c\0\x7d\0\x5f\x5f\x73\x36\x34\0\x6c\x6f\x6e\x67\x20\ +\x6c\x6f\x6e\x67\0\x62\x70\x66\x5f\x6d\x61\x70\x5f\x73\x75\x6d\x5f\x65\x6c\x65\ +\x6d\x5f\x63\x6f\x75\x6e\x74\0\x62\x70\x66\x5f\x69\x74\x65\x72\x5f\x5f\x62\x70\ +\x66\x5f\x70\x72\x6f\x67\0\x70\x72\x6f\x67\0\x64\x75\x6d\x70\x5f\x62\x70\x66\ +\x5f\x70\x72\x6f\x67\0\x69\x74\x65\x72\x2f\x62\x70\x66\x5f\x70\x72\x6f\x67\0\ +\x09\x73\x74\x72\x75\x63\x74\x20\x62\x70\x66\x5f\x70\x72\x6f\x67\x20\x2a\x70\ +\x72\x6f\x67\x20\x3d\x20\x63\x74\x78\x2d\x3e\x70\x72\x6f\x67\x3b\0\x09\x69\x66\ +\x20\x28\x21\x70\x72\x6f\x67\x29\0\x62\x70\x66\x5f\x70\x72\x6f\x67\0\x61\x75\ +\x78\0\x09\x61\x75\x78\x20\x3d\x20\x70\x72\x6f\x67\x2d\x3e\x61\x75\x78\x3b\0\ +\x09\x09\x42\x50\x46\x5f\x53\x45\x51\x5f\x50\x52\x49\x4e\x54\x46\x28\x73\x65\ +\x71\x2c\x20\x22\x20\x20\x69\x64\x20\x6e\x61\x6d\x65\x20\x20\x20\x20\x20\x20\ +\x20\x20\x20\x20\x20\x20\x20\x61\x74\x74\x61\x63\x68\x65\x64\x5c\x6e\x22\x29\ +\x3b\0\x62\x70\x66\x5f\x70\x72\x6f\x67\x5f\x61\x75\x78\0\x61\x74\x74\x61\x63\ +\x68\x5f\x66\x75\x6e\x63\x5f\x6e\x61\x6d\x65\0\x64\x73\x74\x5f\x70\x72\x6f\x67\ +\0\x66\x75\x6e\x63\x5f\x69\x6e\x66\x6f\0\x62\x74\x66\0\x09\x42\x50\x46\x5f\x53\ +\x45\x51\x5f\x50\x52\x49\x4e\x54\x46\x28\x73\x65\x71\x2c\x20\x22\x25\x34\x75\ +\x20\x25\x2d\x31\x36\x73\x20\x25\x73\x20\x25\x73\x5c\x6e\x22\x2c\x20\x61\x75\ +\x78\x2d\x3e\x69\x64\x2c\0\x30\x3a\x34\0\x30\x3a\x35\0\x09\x69\x66\x20\x28\x21\ +\x62\x74\x66\x29\0\x62\x70\x66\x5f\x66\x75\x6e\x63\x5f\x69\x6e\x66\x6f\0\x69\ +\x6e\x73\x6e\x5f\x6f\x66\x66\0\x74\x79\x70\x65\x5f\x69\x64\0\x30\0\x73\x74\x72\ +\x69\x6e\x67\x73\0\x74\x79\x70\x65\x73\0\x68\x64\x72\0\x62\x74\x66\x5f\x68\x65\ +\x61\x64\x65\x72\0\x73\x74\x72\x5f\x6c\x65\x6e\0\x09\x74\x79\x70\x65\x73\x20\ +\x3d\x20\x62\x74\x66\x2d\x3e\x74\x79\x70\x65\x73\x3b\0\x09\x62\x70\x66\x5f\x70\ +\x72\x6f\x62\x65\x5f\x72\x65\x61\x64\x5f\x6b\x65\x72\x6e\x65\x6c\x28\x26\x74\ +\x2c\x20\x73\x69\x7a\x65\x6f\x66\x28\x74\x29\x2c\x20\x74\x79\x70\x65\x73\x20\ +\x2b\x20\x62\x74\x66\x5f\x69\x64\x29\x3b\0\x09\x73\x74\x72\x20\x3d\x20\x62\x74\ +\x66\x2d\x3e\x73\x74\x72\x69\x6e\x67\x73\x3b\0\x62\x74\x66\x5f\x74\x79\x70\x65\ +\0\x6e\x61\x6d\x65\x5f\x6f\x66\x66\0\x09\x6e\x61\x6d\x65\x5f\x6f\x66\x66\x20\ +\x3d\x20\x42\x50\x46\x5f\x43\x4f\x52\x45\x5f\x52\x45\x41\x44\x28\x74\x2c\x20\ +\x6e\x61\x6d\x65\x5f\x6f\x66\x66\x29\x3b\0\x30\x3a\x32\x3a\x30\0\x09\x69\x66\ +\x20\x28\x6e\x61\x6d\x65\x5f\x6f\x66\x66\x20\x3e\x3d\x20\x62\x74\x66\x2d\x3e\ +\x68\x64\x72\x2e\x73\x74\x72\x5f\x6c\x65\x6e\x29\0\x09\x72\x65\x74\x75\x72\x6e\ +\x20\x73\x74\x72\x20\x2b\x20\x6e\x61\x6d\x65\x5f\x6f\x66\x66\x3b\0\x30\x3a\x33\ +\0\x64\x75\x6d\x70\x5f\x62\x70\x66\x5f\x6d\x61\x70\x2e\x5f\x5f\x5f\x66\x6d\x74\ +\0\x64\x75\x6d\x70\x5f\x62\x70\x66\x5f\x6d\x61\x70\x2e\x5f\x5f\x5f\x66\x6d\x74\ +\x2e\x31\0\x64\x75\x6d\x70\x5f\x62\x70\x66\x5f\x70\x72\x6f\x67\x2e\x5f\x5f\x5f\ +\x66\x6d\x74\0\x64\x75\x6d\x70\x5f\x62\x70\x66\x5f\x70\x72\x6f\x67\x2e\x5f\x5f\ +\x5f\x66\x6d\x74\x2e\x32\0\x4c\x49\x43\x45\x4e\x53\x45\0\x2e\x6b\x73\x79\x6d\ +\x73\0\x2e\x72\x6f\x64\x61\x74\x61\0\x6c\x69\x63\x65\x6e\x73\x65\0\x64\x75\x6d\ +\x6d\x79\x5f\x6b\x73\x79\x6d\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\ +\xc9\x09\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x02\0\0\0\x04\0\0\0\x7b\0\0\0\x01\0\0\0\ +\x80\0\0\0\0\0\0\0\0\0\0\0\x69\x74\x65\x72\x61\x74\x6f\x72\x2e\x72\x6f\x64\x61\ +\x74\x61\0\0\0\0\0\0\0\0\0\0\0\0\0\x34\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x20\x20\ +\x69\x64\x20\x6e\x61\x6d\x65\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\ +\x20\x6d\x61\x78\x5f\x65\x6e\x74\x72\x69\x65\x73\x20\x20\x63\x75\x72\x5f\x65\ +\x6e\x74\x72\x69\x65\x73\x0a\0\x25\x34\x75\x20\x25\x2d\x31\x36\x73\x20\x20\x25\ +\x31\x30\x64\x20\x20\x20\x25\x31\x30\x6c\x6c\x64\x0a\0\x20\x20\x69\x64\x20\x6e\ +\x61\x6d\x65\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x61\x74\x74\ +\x61\x63\x68\x65\x64\x0a\0\x25\x34\x75\x20\x25\x2d\x31\x36\x73\x20\x25\x73\x20\ +\x25\x73\x0a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\ +\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x47\x50\x4c\0\0\0\0\0\x79\x12\0\0\0\ +\0\0\0\x79\x26\0\0\0\0\0\0\x79\x17\x08\0\0\0\0\0\x15\x07\x1d\0\0\0\0\0\x79\x21\ +\x10\0\0\0\0\0\x55\x01\x08\0\0\0\0\0\xbf\xa4\0\0\0\0\0\0\x07\x04\0\0\xe0\xff\ +\xff\xff\xbf\x61\0\0\0\0\0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\0\0\0\0\xb7\x03\0\0\ +\x30\0\0\0\xb7\x05\0\0\0\0\0\0\x85\0\0\0\x7e\0\0\0\x61\x71\0\0\0\0\0\0\x7b\x1a\ +\xe0\xff\0\0\0\0\xb7\x01\0\0\x04\0\0\0\xbf\x72\0\0\0\0\0\0\x0f\x12\0\0\0\0\0\0\ +\x7b\x2a\xe8\xff\0\0\0\0\x61\x71\x14\0\0\0\0\0\x7b\x1a\xf0\xff\0\0\0\0\xbf\x71\ +\0\0\0\0\0\0\x85\x20\0\0\0\0\0\0\x7b\x0a\xf8\xff\0\0\0\0\xbf\xa4\0\0\0\0\0\0\ +\x07\x04\0\0\xe0\xff\xff\xff\xbf\x61\0\0\0\0\0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\ +\x30\0\0\0\xb7\x03\0\0\x1a\0\0\0\xb7\x05\0\0\x20\0\0\0\x85\0\0\0\x7e\0\0\0\xb7\ +\0\0\0\0\0\0\0\x95\0\0\0\0\0\0\0\0\0\0\0\x07\0\0\0\0\0\0\0\x42\0\0\0\x88\0\0\0\ +\x1e\x44\x01\0\x01\0\0\0\x42\0\0\0\x88\0\0\0\x24\x44\x01\0\x02\0\0\0\x42\0\0\0\ +\xfb\0\0\0\x1d\x4c\x01\0\x03\0\0\0\x42\0\0\0\x1c\x01\0\0\x06\x54\x01\0\x04\0\0\ +\0\x42\0\0\0\x2b\x01\0\0\x1d\x48\x01\0\x05\0\0\0\x42\0\0\0\x50\x01\0\0\x06\x60\ +\x01\0\x07\0\0\0\x42\0\0\0\x63\x01\0\0\x03\x64\x01\0\x0e\0\0\0\x42\0\0\0\xf6\ +\x01\0\0\x02\x6c\x01\0\x21\0\0\0\x42\0\0\0\x29\x02\0\0\x01\x80\x01\0\0\0\0\0\ +\x02\0\0\0\x3e\0\0\0\0\0\0\0\x08\0\0\0\x08\0\0\0\x3e\0\0\0\0\0\0\0\x10\0\0\0\ +\x02\0\0\0\xf7\0\0\0\0\0\0\0\x20\0\0\0\x08\0\0\0\x27\x01\0\0\0\0\0\0\x70\0\0\0\ +\x0d\0\0\0\x3e\0\0\0\0\0\0\0\x80\0\0\0\x0d\0\0\0\xf7\0\0\0\0\0\0\0\xa0\0\0\0\ +\x0d\0\0\0\x27\x01\0\0\0\0\0\0\x1a\0\0\0\x23\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\ +\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x64\x75\x6d\x70\x5f\x62\ +\x70\x66\x5f\x6d\x61\x70\0\0\0\0\0\0\0\0\x1c\0\0\0\0\0\0\0\x08\0\0\0\0\0\0\0\0\ +\0\0\0\x01\0\0\0\x10\0\0\0\0\0\0\0\0\0\0\0\x09\0\0\0\x01\0\0\0\0\0\0\0\x07\0\0\ +\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x10\0\0\0\0\0\0\0\x62\x70\x66\x5f\x69\x74\ +\x65\x72\x5f\x62\x70\x66\x5f\x6d\x61\x70\0\0\0\0\0\0\0\0\x62\x70\x66\x5f\x6d\ +\x61\x70\x5f\x73\x75\x6d\x5f\x65\x6c\x65\x6d\x5f\x63\x6f\x75\x6e\x74\0\0\x47\ +\x50\x4c\0\0\0\0\0\x79\x12\0\0\0\0\0\0\x79\x26\0\0\0\0\0\0\x79\x11\x08\0\0\0\0\ +\0\x15\x01\x3b\0\0\0\0\0\x79\x17\0\0\0\0\0\0\x79\x21\x10\0\0\0\0\0\x55\x01\x08\ +\0\0\0\0\0\xbf\xa4\0\0\0\0\0\0\x07\x04\0\0\xd0\xff\xff\xff\xbf\x61\0\0\0\0\0\0\ +\x18\x62\0\0\0\0\0\0\0\0\0\0\x4a\0\0\0\xb7\x03\0\0\x20\0\0\0\xb7\x05\0\0\0\0\0\ +\0\x85\0\0\0\x7e\0\0\0\x7b\x6a\xc8\xff\0\0\0\0\x61\x71\0\0\0\0\0\0\x7b\x1a\xd0\ +\xff\0\0\0\0\xb7\x03\0\0\x04\0\0\0\xbf\x79\0\0\0\0\0\0\x0f\x39\0\0\0\0\0\0\x79\ +\x71\x28\0\0\0\0\0\x79\x78\x30\0\0\0\0\0\x15\x08\x18\0\0\0\0\0\xb7\x02\0\0\0\0\ +\0\0\x0f\x21\0\0\0\0\0\0\x61\x11\x04\0\0\0\0\0\x79\x83\x08\0\0\0\0\0\x67\x01\0\ +\0\x03\0\0\0\x0f\x13\0\0\0\0\0\0\x79\x86\0\0\0\0\0\0\xbf\xa1\0\0\0\0\0\0\x07\ +\x01\0\0\xf8\xff\xff\xff\xb7\x02\0\0\x08\0\0\0\x85\0\0\0\x71\0\0\0\xb7\x01\0\0\ +\0\0\0\0\x79\xa3\xf8\xff\0\0\0\0\x0f\x13\0\0\0\0\0\0\xbf\xa1\0\0\0\0\0\0\x07\ +\x01\0\0\xf4\xff\xff\xff\xb7\x02\0\0\x04\0\0\0\x85\0\0\0\x71\0\0\0\xb7\x03\0\0\ +\x04\0\0\0\x61\xa1\xf4\xff\0\0\0\0\x61\x82\x10\0\0\0\0\0\x3d\x21\x02\0\0\0\0\0\ +\x0f\x16\0\0\0\0\0\0\xbf\x69\0\0\0\0\0\0\x7b\x9a\xd8\xff\0\0\0\0\x79\x71\x18\0\ +\0\0\0\0\x7b\x1a\xe0\xff\0\0\0\0\x79\x71\x20\0\0\0\0\0\x79\x11\0\0\0\0\0\0\x0f\ +\x31\0\0\0\0\0\0\x7b\x1a\xe8\xff\0\0\0\0\xbf\xa4\0\0\0\0\0\0\x07\x04\0\0\xd0\ +\xff\xff\xff\x79\xa1\xc8\xff\0\0\0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\x6a\0\0\0\xb7\ +\x03\0\0\x11\0\0\0\xb7\x05\0\0\x20\0\0\0\x85\0\0\0\x7e\0\0\0\xb7\0\0\0\0\0\0\0\ +\x95\0\0\0\0\0\0\0\0\0\0\0\x1b\0\0\0\0\0\0\0\x42\0\0\0\x88\0\0\0\x1e\x94\x01\0\ +\x01\0\0\0\x42\0\0\0\x88\0\0\0\x24\x94\x01\0\x02\0\0\0\x42\0\0\0\x86\x02\0\0\ +\x1f\x9c\x01\0\x03\0\0\0\x42\0\0\0\xaa\x02\0\0\x06\xa8\x01\0\x04\0\0\0\x42\0\0\ +\0\xc3\x02\0\0\x0e\xb4\x01\0\x05\0\0\0\x42\0\0\0\x2b\x01\0\0\x1d\x98\x01\0\x06\ +\0\0\0\x42\0\0\0\x50\x01\0\0\x06\xb8\x01\0\x08\0\0\0\x42\0\0\0\xd5\x02\0\0\x03\ +\xbc\x01\0\x10\0\0\0\x42\0\0\0\x45\x03\0\0\x02\xc4\x01\0\x17\0\0\0\x42\0\0\0\ +\x80\x03\0\0\x06\x04\x01\0\x1a\0\0\0\x42\0\0\0\x45\x03\0\0\x02\xc4\x01\0\x1b\0\ +\0\0\x42\0\0\0\xd1\x03\0\0\x0f\x10\x01\0\x1c\0\0\0\x42\0\0\0\xe6\x03\0\0\x2d\ +\x14\x01\0\x1e\0\0\0\x42\0\0\0\x1d\x04\0\0\x0d\x0c\x01\0\x20\0\0\0\x42\0\0\0\ +\x45\x03\0\0\x02\xc4\x01\0\x21\0\0\0\x42\0\0\0\xe6\x03\0\0\x02\x14\x01\0\x24\0\ +\0\0\x42\0\0\0\x44\x04\0\0\x0d\x18\x01\0\x27\0\0\0\x42\0\0\0\x45\x03\0\0\x02\ +\xc4\x01\0\x28\0\0\0\x42\0\0\0\x44\x04\0\0\x0d\x18\x01\0\x2b\0\0\0\x42\0\0\0\ +\x44\x04\0\0\x0d\x18\x01\0\x2c\0\0\0\x42\0\0\0\x72\x04\0\0\x1b\x1c\x01\0\x2d\0\ +\0\0\x42\0\0\0\x72\x04\0\0\x06\x1c\x01\0\x2e\0\0\0\x42\0\0\0\x95\x04\0\0\x0d\ +\x24\x01\0\x30\0\0\0\x42\0\0\0\x45\x03\0\0\x02\xc4\x01\0\x3f\0\0\0\x42\0\0\0\ +\x29\x02\0\0\x01\xd4\x01\0\0\0\0\0\x18\0\0\0\x3e\0\0\0\0\0\0\0\x08\0\0\0\x08\0\ +\0\0\x3e\0\0\0\0\0\0\0\x10\0\0\0\x18\0\0\0\xf7\0\0\0\0\0\0\0\x20\0\0\0\x1c\0\0\ +\0\x3e\0\0\0\0\0\0\0\x28\0\0\0\x08\0\0\0\x27\x01\0\0\0\0\0\0\x80\0\0\0\x1e\0\0\ +\0\x3e\0\0\0\0\0\0\0\x90\0\0\0\x1e\0\0\0\xf7\0\0\0\0\0\0\0\xa8\0\0\0\x1e\0\0\0\ +\x78\x03\0\0\0\0\0\0\xb0\0\0\0\x1e\0\0\0\x7c\x03\0\0\0\0\0\0\xc0\0\0\0\x23\0\0\ +\0\xaa\x03\0\0\0\0\0\0\xd8\0\0\0\x24\0\0\0\xf7\0\0\0\0\0\0\0\xf0\0\0\0\x24\0\0\ +\0\x3e\0\0\0\0\0\0\0\x18\x01\0\0\x28\0\0\0\x3e\0\0\0\0\0\0\0\x50\x01\0\0\x1e\0\ +\0\0\xf7\0\0\0\0\0\0\0\x60\x01\0\0\x24\0\0\0\x6c\x04\0\0\0\0\0\0\x88\x01\0\0\ +\x1e\0\0\0\x27\x01\0\0\0\0\0\0\x98\x01\0\0\x1e\0\0\0\xad\x04\0\0\0\0\0\0\xa0\ +\x01\0\0\x1c\0\0\0\x3e\0\0\0\0\0\0\0\x1a\0\0\0\x41\0\0\0\0\0\0\0\0\0\0\0\0\0\0\ +\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x64\x75\x6d\x70\x5f\ +\x62\x70\x66\x5f\x70\x72\x6f\x67\0\0\0\0\0\0\0\x1c\0\0\0\0\0\0\0\x08\0\0\0\0\0\ +\0\0\0\0\0\0\x01\0\0\0\x10\0\0\0\0\0\0\0\0\0\0\0\x19\0\0\0\x01\0\0\0\0\0\0\0\ +\x12\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x10\0\0\0\0\0\0\0\x62\x70\x66\x5f\ +\x69\x74\x65\x72\x5f\x62\x70\x66\x5f\x70\x72\x6f\x67\0\0\0\0\0\0\0"; + opts.insns_sz = 2456; opts.insns = (void *)"\ \xbf\x16\0\0\0\0\0\0\xbf\xa1\0\0\0\0\0\0\x07\x01\0\0\x78\xff\xff\xff\xb7\x02\0\ \0\x88\0\0\0\xb7\x03\0\0\0\0\0\0\x85\0\0\0\x71\0\0\0\x05\0\x14\0\0\0\0\0\x61\ @@ -331,79 +326,83 @@ iterators_bpf__load(struct iterators_bpf *skel) \0\0\0\x85\0\0\0\xa8\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x61\x01\0\0\0\0\ \0\0\xd5\x01\x02\0\0\0\0\0\xbf\x19\0\0\0\0\0\0\x85\0\0\0\xa8\0\0\0\xbf\x70\0\0\ \0\0\0\0\x95\0\0\0\0\0\0\0\x61\x60\x08\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\ -\x48\x0e\0\0\x63\x01\0\0\0\0\0\0\x61\x60\x0c\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\ -\0\0\x44\x0e\0\0\x63\x01\0\0\0\0\0\0\x79\x60\x10\0\0\0\0\0\x18\x61\0\0\0\0\0\0\ -\0\0\0\0\x38\x0e\0\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\0\x05\0\0\ -\x18\x61\0\0\0\0\0\0\0\0\0\0\x30\x0e\0\0\x7b\x01\0\0\0\0\0\0\xb7\x01\0\0\x12\0\ -\0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\x30\x0e\0\0\xb7\x03\0\0\x1c\0\0\0\x85\0\0\0\ +\xe8\x0e\0\0\x63\x01\0\0\0\0\0\0\x61\x60\x0c\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\ +\0\0\xe4\x0e\0\0\x63\x01\0\0\0\0\0\0\x79\x60\x10\0\0\0\0\0\x18\x61\0\0\0\0\0\0\ +\0\0\0\0\xd8\x0e\0\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\0\x05\0\0\ +\x18\x61\0\0\0\0\0\0\0\0\0\0\xd0\x0e\0\0\x7b\x01\0\0\0\0\0\0\xb7\x01\0\0\x12\0\ +\0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\xd0\x0e\0\0\xb7\x03\0\0\x1c\0\0\0\x85\0\0\0\ \xa6\0\0\0\xbf\x07\0\0\0\0\0\0\xc5\x07\xd4\xff\0\0\0\0\x63\x7a\x78\xff\0\0\0\0\ -\x61\xa0\x78\xff\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x80\x0e\0\0\x63\x01\0\0\0\ +\x61\xa0\x78\xff\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x20\x0f\0\0\x63\x01\0\0\0\ \0\0\0\x61\x60\x1c\0\0\0\0\0\x15\0\x03\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\ -\x5c\x0e\0\0\x63\x01\0\0\0\0\0\0\xb7\x01\0\0\0\0\0\0\x18\x62\0\0\0\0\0\0\0\0\0\ -\0\x50\x0e\0\0\xb7\x03\0\0\x48\0\0\0\x85\0\0\0\xa6\0\0\0\xbf\x07\0\0\0\0\0\0\ +\xfc\x0e\0\0\x63\x01\0\0\0\0\0\0\xb7\x01\0\0\0\0\0\0\x18\x62\0\0\0\0\0\0\0\0\0\ +\0\xf0\x0e\0\0\xb7\x03\0\0\x48\0\0\0\x85\0\0\0\xa6\0\0\0\xbf\x07\0\0\0\0\0\0\ \xc5\x07\xc3\xff\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x63\x71\0\0\0\0\0\ -\0\x79\x63\x20\0\0\0\0\0\x15\x03\x08\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x98\ -\x0e\0\0\xb7\x02\0\0\x62\0\0\0\x61\x60\x04\0\0\0\0\0\x45\0\x02\0\x01\0\0\0\x85\ +\0\x79\x63\x20\0\0\0\0\0\x15\x03\x08\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x38\ +\x0f\0\0\xb7\x02\0\0\x7b\0\0\0\x61\x60\x04\0\0\0\0\0\x45\0\x02\0\x01\0\0\0\x85\ \0\0\0\x94\0\0\0\x05\0\x01\0\0\0\0\0\x85\0\0\0\x71\0\0\0\x18\x62\0\0\0\0\0\0\0\ -\0\0\0\0\0\0\0\x61\x20\0\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x08\x0f\0\0\x63\ -\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\0\x0f\0\0\x18\x61\0\0\0\0\0\0\0\0\ -\0\0\x10\x0f\0\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\x98\x0e\0\0\ -\x18\x61\0\0\0\0\0\0\0\0\0\0\x18\x0f\0\0\x7b\x01\0\0\0\0\0\0\xb7\x01\0\0\x02\0\ -\0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\x08\x0f\0\0\xb7\x03\0\0\x20\0\0\0\x85\0\0\0\ +\0\0\0\0\0\0\0\x61\x20\0\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\xc0\x0f\0\0\x63\ +\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\xb8\x0f\0\0\x18\x61\0\0\0\0\0\0\0\ +\0\0\0\xc8\x0f\0\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\x38\x0f\0\0\ +\x18\x61\0\0\0\0\0\0\0\0\0\0\xd0\x0f\0\0\x7b\x01\0\0\0\0\0\0\xb7\x01\0\0\x02\0\ +\0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\xc0\x0f\0\0\xb7\x03\0\0\x20\0\0\0\x85\0\0\0\ \xa6\0\0\0\xbf\x07\0\0\0\0\0\0\xc5\x07\x9f\xff\0\0\0\0\x18\x62\0\0\0\0\0\0\0\0\ -\0\0\0\0\0\0\x61\x20\0\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x28\x0f\0\0\x63\ -\x01\0\0\0\0\0\0\xb7\x01\0\0\x16\0\0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\x28\x0f\0\0\ +\0\0\0\0\0\0\x61\x20\0\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\xe0\x0f\0\0\x63\ +\x01\0\0\0\0\0\0\xb7\x01\0\0\x16\0\0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\xe0\x0f\0\0\ \xb7\x03\0\0\x04\0\0\0\x85\0\0\0\xa6\0\0\0\xbf\x07\0\0\0\0\0\0\xc5\x07\x92\xff\ -\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\x30\x0f\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\ -\x78\x11\0\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\x38\x0f\0\0\x18\ -\x61\0\0\0\0\0\0\0\0\0\0\x70\x11\0\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\ -\0\0\0\x40\x10\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\xb8\x11\0\0\x7b\x01\0\0\0\0\0\0\ -\x18\x60\0\0\0\0\0\0\0\0\0\0\x48\x10\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\xc8\x11\0\ -\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\xe8\x10\0\0\x18\x61\0\0\0\0\ -\0\0\0\0\0\0\xe8\x11\0\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\0\0\0\ -\0\x18\x61\0\0\0\0\0\0\0\0\0\0\xe0\x11\0\0\x7b\x01\0\0\0\0\0\0\x61\x60\x08\0\0\ -\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x80\x11\0\0\x63\x01\0\0\0\0\0\0\x61\x60\x0c\ -\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x84\x11\0\0\x63\x01\0\0\0\0\0\0\x79\x60\ -\x10\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x88\x11\0\0\x7b\x01\0\0\0\0\0\0\x61\ -\xa0\x78\xff\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\xb0\x11\0\0\x63\x01\0\0\0\0\0\ -\0\x18\x61\0\0\0\0\0\0\0\0\0\0\xf8\x11\0\0\xb7\x02\0\0\x11\0\0\0\xb7\x03\0\0\ +\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\xe8\x0f\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\ +\x20\x12\0\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\xf0\x0f\0\0\x18\ +\x61\0\0\0\0\0\0\0\0\0\0\x18\x12\0\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\ +\0\0\0\x08\x11\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x60\x12\0\0\x7b\x01\0\0\0\0\0\0\ +\x18\x60\0\0\0\0\0\0\0\0\0\0\x10\x11\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x70\x12\0\ +\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\xa0\x11\0\0\x18\x61\0\0\0\0\ +\0\0\0\0\0\0\x90\x12\0\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\0\0\0\ +\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x88\x12\0\0\x7b\x01\0\0\0\0\0\0\x61\x60\x08\0\0\ +\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x28\x12\0\0\x63\x01\0\0\0\0\0\0\x61\x60\x0c\ +\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x2c\x12\0\0\x63\x01\0\0\0\0\0\0\x79\x60\ +\x10\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x30\x12\0\0\x7b\x01\0\0\0\0\0\0\x61\ +\xa0\x78\xff\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x58\x12\0\0\x63\x01\0\0\0\0\0\ +\0\x18\x61\0\0\0\0\0\0\0\0\0\0\xa0\x12\0\0\xb7\x02\0\0\x11\0\0\0\xb7\x03\0\0\ \x0c\0\0\0\xb7\x04\0\0\0\0\0\0\x85\0\0\0\xa7\0\0\0\xbf\x07\0\0\0\0\0\0\xc5\x07\ -\x5c\xff\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\x68\x11\0\0\x63\x70\x6c\0\0\0\0\0\ -\x77\x07\0\0\x20\0\0\0\x63\x70\x70\0\0\0\0\0\xb7\x01\0\0\x05\0\0\0\x18\x62\0\0\ -\0\0\0\0\0\0\0\0\x68\x11\0\0\xb7\x03\0\0\x8c\0\0\0\x85\0\0\0\xa6\0\0\0\xbf\x07\ -\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\xd8\x11\0\0\x61\x01\0\0\0\0\0\0\xd5\ -\x01\x02\0\0\0\0\0\xbf\x19\0\0\0\0\0\0\x85\0\0\0\xa8\0\0\0\xc5\x07\x4a\xff\0\0\ -\0\0\x63\x7a\x80\xff\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\x10\x12\0\0\x18\x61\0\ -\0\0\0\0\0\0\0\0\0\x10\x17\0\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\ -\x18\x12\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x08\x17\0\0\x7b\x01\0\0\0\0\0\0\x18\ -\x60\0\0\0\0\0\0\0\0\0\0\x28\x14\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x50\x17\0\0\ -\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\x30\x14\0\0\x18\x61\0\0\0\0\0\ -\0\0\0\0\0\x60\x17\0\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\xd0\x15\ -\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x80\x17\0\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\ -\0\0\0\0\0\0\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x78\x17\0\0\x7b\x01\0\0\0\0\ -\0\0\x61\x60\x08\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x18\x17\0\0\x63\x01\0\0\ -\0\0\0\0\x61\x60\x0c\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x1c\x17\0\0\x63\x01\ -\0\0\0\0\0\0\x79\x60\x10\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x20\x17\0\0\x7b\ -\x01\0\0\0\0\0\0\x61\xa0\x78\xff\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x48\x17\0\ -\0\x63\x01\0\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x90\x17\0\0\xb7\x02\0\0\x12\ -\0\0\0\xb7\x03\0\0\x0c\0\0\0\xb7\x04\0\0\0\0\0\0\x85\0\0\0\xa7\0\0\0\xbf\x07\0\ -\0\0\0\0\0\xc5\x07\x13\xff\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\0\x17\0\0\x63\ -\x70\x6c\0\0\0\0\0\x77\x07\0\0\x20\0\0\0\x63\x70\x70\0\0\0\0\0\xb7\x01\0\0\x05\ -\0\0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\0\x17\0\0\xb7\x03\0\0\x8c\0\0\0\x85\0\0\0\ -\xa6\0\0\0\xbf\x07\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\x70\x17\0\0\x61\x01\ -\0\0\0\0\0\0\xd5\x01\x02\0\0\0\0\0\xbf\x19\0\0\0\0\0\0\x85\0\0\0\xa8\0\0\0\xc5\ -\x07\x01\xff\0\0\0\0\x63\x7a\x84\xff\0\0\0\0\x61\xa1\x78\xff\0\0\0\0\xd5\x01\ -\x02\0\0\0\0\0\xbf\x19\0\0\0\0\0\0\x85\0\0\0\xa8\0\0\0\x61\xa0\x80\xff\0\0\0\0\ -\x63\x06\x28\0\0\0\0\0\x61\xa0\x84\xff\0\0\0\0\x63\x06\x2c\0\0\0\0\0\x18\x61\0\ -\0\0\0\0\0\0\0\0\0\0\0\0\0\x61\x10\0\0\0\0\0\0\x63\x06\x18\0\0\0\0\0\xb7\0\0\0\ -\0\0\0\0\x95\0\0\0\0\0\0\0"; +\x5c\xff\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\x10\x12\0\0\x63\x70\x6c\0\0\0\0\0\ +\x77\x07\0\0\x20\0\0\0\x63\x70\x70\0\0\0\0\0\x18\x68\0\0\0\0\0\0\0\0\0\0\xa8\ +\x10\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\xb8\x12\0\0\xb7\x02\0\0\x17\0\0\0\xb7\x03\ +\0\0\x0c\0\0\0\xb7\x04\0\0\0\0\0\0\x85\0\0\0\xa7\0\0\0\xbf\x07\0\0\0\0\0\0\xc5\ +\x07\x4d\xff\0\0\0\0\x75\x07\x03\0\0\0\0\0\x62\x08\x04\0\0\0\0\0\x6a\x08\x02\0\ +\0\0\0\0\x05\0\x0a\0\0\0\0\0\x63\x78\x04\0\0\0\0\0\xbf\x79\0\0\0\0\0\0\x77\x09\ +\0\0\x20\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\0\x01\0\0\x63\x90\0\0\0\0\0\0\x55\ +\x09\x02\0\0\0\0\0\x6a\x08\x02\0\0\0\0\0\x05\0\x01\0\0\0\0\0\x6a\x08\x02\0\x40\ +\0\0\0\xb7\x01\0\0\x05\0\0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\x10\x12\0\0\xb7\x03\0\ +\0\x8c\0\0\0\x85\0\0\0\xa6\0\0\0\xbf\x07\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\ +\0\0\x01\0\0\x61\x01\0\0\0\0\0\0\xd5\x01\x02\0\0\0\0\0\xbf\x19\0\0\0\0\0\0\x85\ +\0\0\0\xa8\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\x80\x12\0\0\x61\x01\0\0\0\0\0\0\ +\xd5\x01\x02\0\0\0\0\0\xbf\x19\0\0\0\0\0\0\x85\0\0\0\xa8\0\0\0\xc5\x07\x2c\xff\ +\0\0\0\0\x63\x7a\x80\xff\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\xd0\x12\0\0\x18\ +\x61\0\0\0\0\0\0\0\0\0\0\xa8\x17\0\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\ +\0\0\0\xd8\x12\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\xa0\x17\0\0\x7b\x01\0\0\0\0\0\0\ +\x18\x60\0\0\0\0\0\0\0\0\0\0\xe0\x14\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\xe8\x17\0\ +\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\xe8\x14\0\0\x18\x61\0\0\0\0\ +\0\0\0\0\0\0\xf8\x17\0\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\x78\ +\x16\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x18\x18\0\0\x7b\x01\0\0\0\0\0\0\x18\x60\0\ +\0\0\0\0\0\0\0\0\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x10\x18\0\0\x7b\x01\0\0\ +\0\0\0\0\x61\x60\x08\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\xb0\x17\0\0\x63\x01\ +\0\0\0\0\0\0\x61\x60\x0c\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\xb4\x17\0\0\x63\ +\x01\0\0\0\0\0\0\x79\x60\x10\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\xb8\x17\0\0\ +\x7b\x01\0\0\0\0\0\0\x61\xa0\x78\xff\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\xe0\ +\x17\0\0\x63\x01\0\0\0\0\0\0\x18\x61\0\0\0\0\0\0\0\0\0\0\x28\x18\0\0\xb7\x02\0\ +\0\x12\0\0\0\xb7\x03\0\0\x0c\0\0\0\xb7\x04\0\0\0\0\0\0\x85\0\0\0\xa7\0\0\0\xbf\ +\x07\0\0\0\0\0\0\xc5\x07\xf5\xfe\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\x98\x17\0\ +\0\x63\x70\x6c\0\0\0\0\0\x77\x07\0\0\x20\0\0\0\x63\x70\x70\0\0\0\0\0\xb7\x01\0\ +\0\x05\0\0\0\x18\x62\0\0\0\0\0\0\0\0\0\0\x98\x17\0\0\xb7\x03\0\0\x8c\0\0\0\x85\ +\0\0\0\xa6\0\0\0\xbf\x07\0\0\0\0\0\0\x18\x60\0\0\0\0\0\0\0\0\0\0\x08\x18\0\0\ +\x61\x01\0\0\0\0\0\0\xd5\x01\x02\0\0\0\0\0\xbf\x19\0\0\0\0\0\0\x85\0\0\0\xa8\0\ +\0\0\xc5\x07\xe3\xfe\0\0\0\0\x63\x7a\x84\xff\0\0\0\0\x61\xa1\x78\xff\0\0\0\0\ +\xd5\x01\x02\0\0\0\0\0\xbf\x19\0\0\0\0\0\0\x85\0\0\0\xa8\0\0\0\x61\xa0\x80\xff\ +\0\0\0\0\x63\x06\x28\0\0\0\0\0\x61\xa0\x84\xff\0\0\0\0\x63\x06\x2c\0\0\0\0\0\ +\x18\x61\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x61\x10\0\0\0\0\0\0\x63\x06\x18\0\0\0\0\0\ +\xb7\0\0\0\0\0\0\0\x95\0\0\0\0\0\0\0"; err = bpf_load_and_run(&opts); if (err < 0) return err; - skel->rodata = skel_finalize_map_data(&skel->maps.rodata.initial_value, - 4096, PROT_READ, skel->maps.rodata.map_fd); - if (!skel->rodata) - return -ENOMEM; return 0; } @@ -422,4 +421,15 @@ iterators_bpf__open_and_load(void) return skel; } +__attribute__((unused)) static void +iterators_bpf__assert(struct iterators_bpf *s __attribute__((unused))) +{ +#ifdef __cplusplus +#define _Static_assert static_assert +#endif +#ifdef __cplusplus +#undef _Static_assert +#endif +} + #endif /* __ITERATORS_BPF_SKEL_H__ */ -- 2.34.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v4 bpf-next 5/6] selftests/bpf: test map percpu stats 2023-07-05 16:01 [PATCH v4 bpf-next 0/6] bpf: add percpu stats for bpf_map Anton Protopopov ` (3 preceding siblings ...) 2023-07-05 16:01 ` [PATCH v4 bpf-next 4/6] bpf: make preloaded map iterators to display map elements count Anton Protopopov @ 2023-07-05 16:01 ` Anton Protopopov 2023-07-06 10:49 ` Hou Tao 2023-07-05 16:01 ` [PATCH v4 bpf-next 6/6] selftests/bpf: check that ->elem_count is non-zero for the hash map Anton Protopopov 5 siblings, 1 reply; 24+ messages in thread From: Anton Protopopov @ 2023-07-05 16:01 UTC (permalink / raw) To: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf Cc: Anton Protopopov Add a new map test, map_percpu_stats.c, which is checking the correctness of map's percpu elements counters. For supported maps the test upserts a number of elements, checks the correctness of the counters, then deletes all the elements and checks again that the counters sum drops down to zero. The following map types are tested: * BPF_MAP_TYPE_HASH, BPF_F_NO_PREALLOC * BPF_MAP_TYPE_PERCPU_HASH, BPF_F_NO_PREALLOC * BPF_MAP_TYPE_HASH, * BPF_MAP_TYPE_PERCPU_HASH, * BPF_MAP_TYPE_LRU_HASH * BPF_MAP_TYPE_LRU_PERCPU_HASH * BPF_MAP_TYPE_LRU_HASH, BPF_F_NO_COMMON_LRU * BPF_MAP_TYPE_LRU_PERCPU_HASH, BPF_F_NO_COMMON_LRU * BPF_MAP_TYPE_HASH_OF_MAPS Signed-off-by: Anton Protopopov <aspsk@isovalent.com> --- .../bpf/map_tests/map_percpu_stats.c | 450 ++++++++++++++++++ .../selftests/bpf/progs/map_percpu_stats.c | 24 + 2 files changed, 474 insertions(+) create mode 100644 tools/testing/selftests/bpf/map_tests/map_percpu_stats.c create mode 100644 tools/testing/selftests/bpf/progs/map_percpu_stats.c diff --git a/tools/testing/selftests/bpf/map_tests/map_percpu_stats.c b/tools/testing/selftests/bpf/map_tests/map_percpu_stats.c new file mode 100644 index 000000000000..37df564d9604 --- /dev/null +++ b/tools/testing/selftests/bpf/map_tests/map_percpu_stats.c @@ -0,0 +1,450 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2023 Isovalent */ + +#include <errno.h> +#include <unistd.h> +#include <pthread.h> + +#include <bpf/bpf.h> +#include <bpf/libbpf.h> + +#include <bpf_util.h> +#include <test_maps.h> + +#include "map_percpu_stats.skel.h" + +#define MAX_ENTRIES 16384 +#define MAX_ENTRIES_HASH_OF_MAPS 64 +#define N_THREADS 8 +#define MAX_MAP_KEY_SIZE 4 + +static void map_info(int map_fd, struct bpf_map_info *info) +{ + __u32 len = sizeof(*info); + int ret; + + memset(info, 0, sizeof(*info)); + + ret = bpf_obj_get_info_by_fd(map_fd, info, &len); + CHECK(ret < 0, "bpf_obj_get_info_by_fd", "error: %s\n", strerror(errno)); +} + +static const char *map_type_to_s(__u32 type) +{ + switch (type) { + case BPF_MAP_TYPE_HASH: + return "HASH"; + case BPF_MAP_TYPE_PERCPU_HASH: + return "PERCPU_HASH"; + case BPF_MAP_TYPE_LRU_HASH: + return "LRU_HASH"; + case BPF_MAP_TYPE_LRU_PERCPU_HASH: + return "LRU_PERCPU_HASH"; + default: + return "<define-me>"; + } +} + +static __u32 map_count_elements(__u32 type, int map_fd) +{ + __u32 key = -1; + int n = 0; + + while (!bpf_map_get_next_key(map_fd, &key, &key)) + n++; + return n; +} + +#define BATCH true + +static void delete_and_lookup_batch(int map_fd, void *keys, __u32 count) +{ + static __u8 values[(8 << 10) * MAX_ENTRIES]; + void *in_batch = NULL, *out_batch; + __u32 save_count = count; + int ret; + + ret = bpf_map_lookup_and_delete_batch(map_fd, + &in_batch, &out_batch, + keys, values, &count, + NULL); + + /* + * Despite what uapi header says, lookup_and_delete_batch will return + * -ENOENT in case we successfully have deleted all elements, so check + * this separately + */ + CHECK(ret < 0 && (errno != ENOENT || !count), "bpf_map_lookup_and_delete_batch", + "error: %s\n", strerror(errno)); + + CHECK(count != save_count, + "bpf_map_lookup_and_delete_batch", + "deleted not all elements: removed=%u expected=%u\n", + count, save_count); +} + +static void delete_all_elements(__u32 type, int map_fd, bool batch) +{ + static __u8 val[8 << 10]; /* enough for 1024 CPUs */ + __u32 key = -1; + void *keys; + __u32 i, n; + int ret; + + keys = calloc(MAX_MAP_KEY_SIZE, MAX_ENTRIES); + CHECK(!keys, "calloc", "error: %s\n", strerror(errno)); + + for (n = 0; !bpf_map_get_next_key(map_fd, &key, &key); n++) + memcpy(keys + n*MAX_MAP_KEY_SIZE, &key, MAX_MAP_KEY_SIZE); + + if (batch) { + /* Can't mix delete_batch and delete_and_lookup_batch because + * they have different semantics in relation to the keys + * argument. However, delete_batch utilize map_delete_elem, + * so we actually test it in non-batch scenario */ + delete_and_lookup_batch(map_fd, keys, n); + } else { + /* Intentionally mix delete and lookup_and_delete so we can test both */ + for (i = 0; i < n; i++) { + void *keyp = keys + i*MAX_MAP_KEY_SIZE; + + if (i % 2 || type == BPF_MAP_TYPE_HASH_OF_MAPS) { + ret = bpf_map_delete_elem(map_fd, keyp); + CHECK(ret < 0, "bpf_map_delete_elem", + "error: key %u: %s\n", i, strerror(errno)); + } else { + ret = bpf_map_lookup_and_delete_elem(map_fd, keyp, val); + CHECK(ret < 0, "bpf_map_lookup_and_delete_elem", + "error: key %u: %s\n", i, strerror(errno)); + } + } + } + + free(keys); +} + +static bool is_lru(__u32 map_type) +{ + return map_type == BPF_MAP_TYPE_LRU_HASH || + map_type == BPF_MAP_TYPE_LRU_PERCPU_HASH; +} + +struct upsert_opts { + __u32 map_type; + int map_fd; + __u32 n; +}; + +static int create_small_hash(void) +{ + int map_fd; + + map_fd = bpf_map_create(BPF_MAP_TYPE_HASH, "small", 4, 4, 4, NULL); + CHECK(map_fd < 0, "bpf_map_create()", "error:%s (name=%s)\n", + strerror(errno), "small"); + + return map_fd; +} + +static void *patch_map_thread(void *arg) +{ + struct upsert_opts *opts = arg; + int val; + int ret; + int i; + + for (i = 0; i < opts->n; i++) { + if (opts->map_type == BPF_MAP_TYPE_HASH_OF_MAPS) + val = create_small_hash(); + else + val = rand(); + ret = bpf_map_update_elem(opts->map_fd, &i, &val, 0); + CHECK(ret < 0, "bpf_map_update_elem", "key=%d error: %s\n", i, strerror(errno)); + + if (opts->map_type == BPF_MAP_TYPE_HASH_OF_MAPS) + close(val); + } + return NULL; +} + +static void upsert_elements(struct upsert_opts *opts) +{ + pthread_t threads[N_THREADS]; + int ret; + int i; + + for (i = 0; i < ARRAY_SIZE(threads); i++) { + ret = pthread_create(&i[threads], NULL, patch_map_thread, opts); + CHECK(ret != 0, "pthread_create", "error: %s\n", strerror(ret)); + } + + for (i = 0; i < ARRAY_SIZE(threads); i++) { + ret = pthread_join(i[threads], NULL); + CHECK(ret != 0, "pthread_join", "error: %s\n", strerror(ret)); + } +} + +static __u32 read_cur_elements(int iter_fd) +{ + char buf[64]; + ssize_t n; + __u32 ret; + + n = read(iter_fd, buf, sizeof(buf)-1); + CHECK(n <= 0, "read", "error: %s\n", strerror(errno)); + buf[n] = '\0'; + + errno = 0; + ret = (__u32)strtol(buf, NULL, 10); + CHECK(errno != 0, "strtol", "error: %s\n", strerror(errno)); + + return ret; +} + +static __u32 get_cur_elements(int map_id) +{ + LIBBPF_OPTS(bpf_iter_attach_opts, opts); + union bpf_iter_link_info linfo; + struct map_percpu_stats *skel; + struct bpf_link *link; + __u32 n_elements; + int iter_fd; + int ret; + + opts.link_info = &linfo; + opts.link_info_len = sizeof(linfo); + + skel = map_percpu_stats__open(); + CHECK(skel == NULL, "map_percpu_stats__open", "error: %s", strerror(errno)); + + skel->bss->target_id = map_id; + + ret = map_percpu_stats__load(skel); + CHECK(ret != 0, "map_percpu_stats__load", "error: %s", strerror(errno)); + + link = bpf_program__attach_iter(skel->progs.dump_bpf_map, &opts); + CHECK(!link, "bpf_program__attach_iter", "error: %s\n", strerror(errno)); + + iter_fd = bpf_iter_create(bpf_link__fd(link)); + CHECK(iter_fd < 0, "bpf_iter_create", "error: %s\n", strerror(errno)); + + n_elements = read_cur_elements(iter_fd); + + close(iter_fd); + bpf_link__destroy(link); + map_percpu_stats__destroy(skel); + + return n_elements; +} + +static void check_expected_number_elements(__u32 n_inserted, int map_fd, + struct bpf_map_info *info) +{ + __u32 n_real; + __u32 n_iter; + + /* Count the current number of elements in the map by iterating through + * all the map keys via bpf_get_next_key + */ + n_real = map_count_elements(info->type, map_fd); + + /* The "real" number of elements should be the same as the inserted + * number of elements in all cases except LRU maps, where some elements + * may have been evicted + */ + if (n_inserted == 0 || !is_lru(info->type)) + CHECK(n_inserted != n_real, "map_count_elements", + "n_real(%u) != n_inserted(%u)\n", n_real, n_inserted); + + /* Count the current number of elements in the map using an iterator */ + n_iter = get_cur_elements(info->id); + + /* Both counts should be the same, as all updates are over */ + CHECK(n_iter != n_real, "get_cur_elements", + "n_iter=%u, expected %u (map_type=%s,map_flags=%08x)\n", + n_iter, n_real, map_type_to_s(info->type), info->map_flags); +} + +static void __test(int map_fd) +{ + struct upsert_opts opts = { + .map_fd = map_fd, + }; + struct bpf_map_info info; + + map_info(map_fd, &info); + opts.map_type = info.type; + opts.n = info.max_entries; + + /* Reduce the number of elements we are updating such that we don't + * bump into -E2BIG from non-preallocated hash maps, but still will + * have some evictions for LRU maps */ + if (opts.map_type != BPF_MAP_TYPE_HASH_OF_MAPS) + opts.n -= 512; + else + opts.n /= 2; + + /* + * Upsert keys [0, n) under some competition: with random values from + * N_THREADS threads. Check values, then delete all elements and check + * values again. + */ + upsert_elements(&opts); + check_expected_number_elements(opts.n, map_fd, &info); + delete_all_elements(info.type, map_fd, !BATCH); + check_expected_number_elements(0, map_fd, &info); + + /* Now do the same, but using batch delete operations */ + upsert_elements(&opts); + check_expected_number_elements(opts.n, map_fd, &info); + delete_all_elements(info.type, map_fd, BATCH); + check_expected_number_elements(0, map_fd, &info); + + close(map_fd); +} + +static int map_create_opts(__u32 type, const char *name, + struct bpf_map_create_opts *map_opts, + __u32 key_size, __u32 val_size) +{ + int max_entries; + int map_fd; + + if (type == BPF_MAP_TYPE_HASH_OF_MAPS) + max_entries = MAX_ENTRIES_HASH_OF_MAPS; + else + max_entries = MAX_ENTRIES; + + map_fd = bpf_map_create(type, name, key_size, val_size, max_entries, map_opts); + CHECK(map_fd < 0, "bpf_map_create()", "error:%s (name=%s)\n", + strerror(errno), name); + + return map_fd; +} + +static int map_create(__u32 type, const char *name, struct bpf_map_create_opts *map_opts) +{ + return map_create_opts(type, name, map_opts, sizeof(int), sizeof(int)); +} + +static int create_hash(void) +{ + struct bpf_map_create_opts map_opts = { + .sz = sizeof(map_opts), + .map_flags = BPF_F_NO_PREALLOC, + }; + + return map_create(BPF_MAP_TYPE_HASH, "hash", &map_opts); +} + +static int create_percpu_hash(void) +{ + struct bpf_map_create_opts map_opts = { + .sz = sizeof(map_opts), + .map_flags = BPF_F_NO_PREALLOC, + }; + + return map_create(BPF_MAP_TYPE_PERCPU_HASH, "percpu_hash", &map_opts); +} + +static int create_hash_prealloc(void) +{ + return map_create(BPF_MAP_TYPE_HASH, "hash", NULL); +} + +static int create_percpu_hash_prealloc(void) +{ + return map_create(BPF_MAP_TYPE_PERCPU_HASH, "percpu_hash_prealloc", NULL); +} + +static int create_lru_hash(__u32 type, __u32 map_flags) +{ + struct bpf_map_create_opts map_opts = { + .sz = sizeof(map_opts), + .map_flags = map_flags, + }; + + return map_create(type, "lru_hash", &map_opts); +} + +static int create_hash_of_maps(void) +{ + struct bpf_map_create_opts map_opts = { + .sz = sizeof(map_opts), + .map_flags = BPF_F_NO_PREALLOC, + .inner_map_fd = create_small_hash(), + }; + int ret; + + ret = map_create_opts(BPF_MAP_TYPE_HASH_OF_MAPS, "hash_of_maps", + &map_opts, sizeof(int), sizeof(int)); + close(map_opts.inner_map_fd); + return ret; +} + +static void map_percpu_stats_hash(void) +{ + __test(create_hash()); + printf("test_%s:PASS\n", __func__); +} + +static void map_percpu_stats_percpu_hash(void) +{ + __test(create_percpu_hash()); + printf("test_%s:PASS\n", __func__); +} + +static void map_percpu_stats_hash_prealloc(void) +{ + __test(create_hash_prealloc()); + printf("test_%s:PASS\n", __func__); +} + +static void map_percpu_stats_percpu_hash_prealloc(void) +{ + __test(create_percpu_hash_prealloc()); + printf("test_%s:PASS\n", __func__); +} + +static void map_percpu_stats_lru_hash(void) +{ + __test(create_lru_hash(BPF_MAP_TYPE_LRU_HASH, 0)); + printf("test_%s:PASS\n", __func__); +} + +static void map_percpu_stats_lru_hash_no_common(void) +{ + __test(create_lru_hash(BPF_MAP_TYPE_LRU_HASH, BPF_F_NO_COMMON_LRU)); + printf("test_%s:PASS\n", __func__); +} + +static void map_percpu_stats_percpu_lru_hash(void) +{ + __test(create_lru_hash(BPF_MAP_TYPE_LRU_PERCPU_HASH, 0)); + printf("test_%s:PASS\n", __func__); +} + +static void map_percpu_stats_percpu_lru_hash_no_common(void) +{ + __test(create_lru_hash(BPF_MAP_TYPE_LRU_PERCPU_HASH, BPF_F_NO_COMMON_LRU)); + printf("test_%s:PASS\n", __func__); +} + +static void map_percpu_stats_hash_of_maps(void) +{ + __test(create_hash_of_maps()); + printf("test_%s:PASS\n", __func__); +} + +void test_map_percpu_stats(void) +{ + map_percpu_stats_hash(); + map_percpu_stats_percpu_hash(); + map_percpu_stats_hash_prealloc(); + map_percpu_stats_percpu_hash_prealloc(); + map_percpu_stats_lru_hash(); + map_percpu_stats_lru_hash_no_common(); + map_percpu_stats_percpu_lru_hash(); + map_percpu_stats_percpu_lru_hash_no_common(); + map_percpu_stats_hash_of_maps(); +} diff --git a/tools/testing/selftests/bpf/progs/map_percpu_stats.c b/tools/testing/selftests/bpf/progs/map_percpu_stats.c new file mode 100644 index 000000000000..10b2325c1720 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/map_percpu_stats.c @@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2023 Isovalent */ + +#include "vmlinux.h" +#include <bpf/bpf_helpers.h> +#include <bpf/bpf_tracing.h> + +__u32 target_id; + +__s64 bpf_map_sum_elem_count(struct bpf_map *map) __ksym; + +SEC("iter/bpf_map") +int dump_bpf_map(struct bpf_iter__bpf_map *ctx) +{ + struct seq_file *seq = ctx->meta->seq; + struct bpf_map *map = ctx->map; + + if (map && map->id == target_id) + BPF_SEQ_PRINTF(seq, "%lld", bpf_map_sum_elem_count(map)); + + return 0; +} + +char _license[] SEC("license") = "GPL"; -- 2.34.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 5/6] selftests/bpf: test map percpu stats 2023-07-05 16:01 ` [PATCH v4 bpf-next 5/6] selftests/bpf: test map percpu stats Anton Protopopov @ 2023-07-06 10:49 ` Hou Tao 2023-07-06 11:54 ` Anton Protopopov 0 siblings, 1 reply; 24+ messages in thread From: Hou Tao @ 2023-07-06 10:49 UTC (permalink / raw) To: Anton Protopopov Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf Hi, On 7/6/2023 12:01 AM, Anton Protopopov wrote: > Add a new map test, map_percpu_stats.c, which is checking the correctness of > map's percpu elements counters. For supported maps the test upserts a number > of elements, checks the correctness of the counters, then deletes all the > elements and checks again that the counters sum drops down to zero. > > The following map types are tested: > > * BPF_MAP_TYPE_HASH, BPF_F_NO_PREALLOC > * BPF_MAP_TYPE_PERCPU_HASH, BPF_F_NO_PREALLOC > * BPF_MAP_TYPE_HASH, > * BPF_MAP_TYPE_PERCPU_HASH, > * BPF_MAP_TYPE_LRU_HASH > * BPF_MAP_TYPE_LRU_PERCPU_HASH > * BPF_MAP_TYPE_LRU_HASH, BPF_F_NO_COMMON_LRU > * BPF_MAP_TYPE_LRU_PERCPU_HASH, BPF_F_NO_COMMON_LRU > * BPF_MAP_TYPE_HASH_OF_MAPS > > Signed-off-by: Anton Protopopov <aspsk@isovalent.com> Acked-by: Hou Tao <houtao1@huawei.com> With two nits below. > + > +static const char *map_type_to_s(__u32 type) > +{ > + switch (type) { > + case BPF_MAP_TYPE_HASH: > + return "HASH"; > + case BPF_MAP_TYPE_PERCPU_HASH: > + return "PERCPU_HASH"; > + case BPF_MAP_TYPE_LRU_HASH: > + return "LRU_HASH"; > + case BPF_MAP_TYPE_LRU_PERCPU_HASH: > + return "LRU_PERCPU_HASH"; > + default: > + return "<define-me>"; > + } Missing BPF_MAP_TYPE_HASH_OF_MAPS ? > +} > + > +static __u32 map_count_elements(__u32 type, int map_fd) > +{ > + __u32 key = -1; > + int n = 0; > + > + while (!bpf_map_get_next_key(map_fd, &key, &key)) > + n++; > + return n; > +} > + > +#define BATCH true > + > +static void delete_and_lookup_batch(int map_fd, void *keys, __u32 count) > +{ > + static __u8 values[(8 << 10) * MAX_ENTRIES]; > + void *in_batch = NULL, *out_batch; > + __u32 save_count = count; > + int ret; > + > + ret = bpf_map_lookup_and_delete_batch(map_fd, > + &in_batch, &out_batch, > + keys, values, &count, > + NULL); > + > + /* > + * Despite what uapi header says, lookup_and_delete_batch will return > + * -ENOENT in case we successfully have deleted all elements, so check > + * this separately > + */ It seems it is a bug in __htab_map_lookup_and_delete_batch(). I could post a patch to fix it if you don't plan to do that by yourself. > + CHECK(ret < 0 && (errno != ENOENT || !count), "bpf_map_lookup_and_delete_batch", > + "error: %s\n", strerror(errno)); > + > + CHECK(count != save_count, > + "bpf_map_lookup_and_delete_batch", > + "deleted not all elements: removed=%u expected=%u\n", > + count, save_count); > +} > + SNIP > +static __u32 get_cur_elements(int map_id) > +{ > + LIBBPF_OPTS(bpf_iter_attach_opts, opts); > + union bpf_iter_link_info linfo; > + struct map_percpu_stats *skel; > + struct bpf_link *link; > + __u32 n_elements; > + int iter_fd; > + int ret; > + > + opts.link_info = &linfo; > + opts.link_info_len = sizeof(linfo); > + > + skel = map_percpu_stats__open(); > + CHECK(skel == NULL, "map_percpu_stats__open", "error: %s", strerror(errno)); > + > + skel->bss->target_id = map_id; > + > + ret = map_percpu_stats__load(skel); > + CHECK(ret != 0, "map_percpu_stats__load", "error: %s", strerror(errno)); > + > + link = bpf_program__attach_iter(skel->progs.dump_bpf_map, &opts); Instead of passing a uninitialized opts, I think using NULL will be fine here because there is no option for bpf map iterator now. ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 5/6] selftests/bpf: test map percpu stats 2023-07-06 10:49 ` Hou Tao @ 2023-07-06 11:54 ` Anton Protopopov 2023-07-06 12:21 ` Hou Tao 0 siblings, 1 reply; 24+ messages in thread From: Anton Protopopov @ 2023-07-06 11:54 UTC (permalink / raw) To: Hou Tao Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf On Thu, Jul 06, 2023 at 06:49:02PM +0800, Hou Tao wrote: > Hi, > > On 7/6/2023 12:01 AM, Anton Protopopov wrote: > > Add a new map test, map_percpu_stats.c, which is checking the correctness of > > map's percpu elements counters. For supported maps the test upserts a number > > of elements, checks the correctness of the counters, then deletes all the > > elements and checks again that the counters sum drops down to zero. > > > > The following map types are tested: > > > > * BPF_MAP_TYPE_HASH, BPF_F_NO_PREALLOC > > * BPF_MAP_TYPE_PERCPU_HASH, BPF_F_NO_PREALLOC > > * BPF_MAP_TYPE_HASH, > > * BPF_MAP_TYPE_PERCPU_HASH, > > * BPF_MAP_TYPE_LRU_HASH > > * BPF_MAP_TYPE_LRU_PERCPU_HASH > > * BPF_MAP_TYPE_LRU_HASH, BPF_F_NO_COMMON_LRU > > * BPF_MAP_TYPE_LRU_PERCPU_HASH, BPF_F_NO_COMMON_LRU > > * BPF_MAP_TYPE_HASH_OF_MAPS > > > > Signed-off-by: Anton Protopopov <aspsk@isovalent.com> > > Acked-by: Hou Tao <houtao1@huawei.com> > > With two nits below. Thanks, fixed both for v5. > > + > > +static const char *map_type_to_s(__u32 type) > > +{ > > + switch (type) { > > + case BPF_MAP_TYPE_HASH: > > + return "HASH"; > > + case BPF_MAP_TYPE_PERCPU_HASH: > > + return "PERCPU_HASH"; > > + case BPF_MAP_TYPE_LRU_HASH: > > + return "LRU_HASH"; > > + case BPF_MAP_TYPE_LRU_PERCPU_HASH: > > + return "LRU_PERCPU_HASH"; > > + default: > > + return "<define-me>"; > > + } > Missing BPF_MAP_TYPE_HASH_OF_MAPS ? > > +} > > + > > +static __u32 map_count_elements(__u32 type, int map_fd) > > +{ > > + __u32 key = -1; > > + int n = 0; > > + > > + while (!bpf_map_get_next_key(map_fd, &key, &key)) > > + n++; > > + return n; > > +} > > + > > +#define BATCH true > > + > > +static void delete_and_lookup_batch(int map_fd, void *keys, __u32 count) > > +{ > > + static __u8 values[(8 << 10) * MAX_ENTRIES]; > > + void *in_batch = NULL, *out_batch; > > + __u32 save_count = count; > > + int ret; > > + > > + ret = bpf_map_lookup_and_delete_batch(map_fd, > > + &in_batch, &out_batch, > > + keys, values, &count, > > + NULL); > > + > > + /* > > + * Despite what uapi header says, lookup_and_delete_batch will return > > + * -ENOENT in case we successfully have deleted all elements, so check > > + * this separately > > + */ > > It seems it is a bug in __htab_map_lookup_and_delete_batch(). I could > post a patch to fix it if you don't plan to do that by yourself. This should be as simple as @@ -1876,7 +1876,8 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map, total += bucket_cnt; batch++; if (batch >= htab->n_buckets) { - ret = -ENOENT; + if (!total) + ret = -ENOENT; goto after_loop; } goto again; However, this might be already utilized by some apps to check that they've read all entries. Two local examples are map_tests/map_in_map_batch_ops.c and map_tests/htab_map_batch_ops.c. Another example I know is from BCC tools: https://github.com/iovisor/bcc/blob/master/libbpf-tools/map_helpers.c#L58 Can we update comments in include/uapi/linux/bpf.h? > > + CHECK(ret < 0 && (errno != ENOENT || !count), "bpf_map_lookup_and_delete_batch", > > + "error: %s\n", strerror(errno)); > > + > > + CHECK(count != save_count, > > + "bpf_map_lookup_and_delete_batch", > > + "deleted not all elements: removed=%u expected=%u\n", > > + count, save_count); > > +} > > + > SNIP > > +static __u32 get_cur_elements(int map_id) > > +{ > > + LIBBPF_OPTS(bpf_iter_attach_opts, opts); > > + union bpf_iter_link_info linfo; > > + struct map_percpu_stats *skel; > > + struct bpf_link *link; > > + __u32 n_elements; > > + int iter_fd; > > + int ret; > > + > > + opts.link_info = &linfo; > > + opts.link_info_len = sizeof(linfo); > > + > > + skel = map_percpu_stats__open(); > > + CHECK(skel == NULL, "map_percpu_stats__open", "error: %s", strerror(errno)); > > + > > + skel->bss->target_id = map_id; > > + > > + ret = map_percpu_stats__load(skel); > > + CHECK(ret != 0, "map_percpu_stats__load", "error: %s", strerror(errno)); > > + > > + link = bpf_program__attach_iter(skel->progs.dump_bpf_map, &opts); > > Instead of passing a uninitialized opts, I think using NULL will be fine > here because there is no option for bpf map iterator now. > ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 5/6] selftests/bpf: test map percpu stats 2023-07-06 11:54 ` Anton Protopopov @ 2023-07-06 12:21 ` Hou Tao 2023-07-06 12:57 ` Anton Protopopov 0 siblings, 1 reply; 24+ messages in thread From: Hou Tao @ 2023-07-06 12:21 UTC (permalink / raw) To: Anton Protopopov Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf Hi, On 7/6/2023 7:54 PM, Anton Protopopov wrote: > On Thu, Jul 06, 2023 at 06:49:02PM +0800, Hou Tao wrote: >> Hi, >> >> On 7/6/2023 12:01 AM, Anton Protopopov wrote: >>> Add a new map test, map_percpu_stats.c, which is checking the correctness of >>> map's percpu elements counters. For supported maps the test upserts a number >>> of elements, checks the correctness of the counters, then deletes all the >>> elements and checks again that the counters sum drops down to zero. >>> >>> The following map types are tested: >>> >>> * BPF_MAP_TYPE_HASH, BPF_F_NO_PREALLOC >>> * BPF_MAP_TYPE_PERCPU_HASH, BPF_F_NO_PREALLOC >>> * BPF_MAP_TYPE_HASH, >>> * BPF_MAP_TYPE_PERCPU_HASH, >>> * BPF_MAP_TYPE_LRU_HASH >>> * BPF_MAP_TYPE_LRU_PERCPU_HASH >>> * BPF_MAP_TYPE_LRU_HASH, BPF_F_NO_COMMON_LRU >>> * BPF_MAP_TYPE_LRU_PERCPU_HASH, BPF_F_NO_COMMON_LRU >>> * BPF_MAP_TYPE_HASH_OF_MAPS >>> >>> Signed-off-by: Anton Protopopov <aspsk@isovalent.com> >> Acked-by: Hou Tao <houtao1@huawei.com> >> >> With two nits below. > Thanks, fixed both for v5. Great. SNIP > +static void delete_and_lookup_batch(int map_fd, void *keys, __u32 count) > +{ > + static __u8 values[(8 << 10) * MAX_ENTRIES]; > + void *in_batch = NULL, *out_batch; > + __u32 save_count = count; > + int ret; > + > + ret = bpf_map_lookup_and_delete_batch(map_fd, > + &in_batch, &out_batch, > + keys, values, &count, > + NULL); > + > + /* > + * Despite what uapi header says, lookup_and_delete_batch will return > + * -ENOENT in case we successfully have deleted all elements, so check > + * this separately > + */ >> It seems it is a bug in __htab_map_lookup_and_delete_batch(). I could >> post a patch to fix it if you don't plan to do that by yourself. > This should be as simple as > > @@ -1876,7 +1876,8 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map, > total += bucket_cnt; > batch++; > if (batch >= htab->n_buckets) { > - ret = -ENOENT; > + if (!total) > + ret = -ENOENT; > goto after_loop; > } > goto again; No. I think changing it to "if (max_count > total) ret = -ENOENT;" will be more appropriate, because it means the requested count couldn't been fulfilled and it is also consistent with the comments in include/uapi/linux/bpf.h > > However, this might be already utilized by some apps to check that they've read > all entries. Two local examples are map_tests/map_in_map_batch_ops.c and > map_tests/htab_map_batch_ops.c. Another example I know is from BCC tools: > https://github.com/iovisor/bcc/blob/master/libbpf-tools/map_helpers.c#L58 I think these use cases will be fine. Because when the last element has been successfully iterated and returned, the out_batch is also updated, so if the batch op is called again, -ENOENT will be returned. > > Can we update comments in include/uapi/linux/bpf.h? I think the comments are correct. > > > . ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 5/6] selftests/bpf: test map percpu stats 2023-07-06 12:21 ` Hou Tao @ 2023-07-06 12:57 ` Anton Protopopov 2023-07-07 1:41 ` Hou Tao 0 siblings, 1 reply; 24+ messages in thread From: Anton Protopopov @ 2023-07-06 12:57 UTC (permalink / raw) To: Hou Tao Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf On Thu, Jul 06, 2023 at 08:21:17PM +0800, Hou Tao wrote: > Hi, > > On 7/6/2023 7:54 PM, Anton Protopopov wrote: > > On Thu, Jul 06, 2023 at 06:49:02PM +0800, Hou Tao wrote: > >> Hi, > >> > >> On 7/6/2023 12:01 AM, Anton Protopopov wrote: > >>> Add a new map test, map_percpu_stats.c, which is checking the correctness of > >>> map's percpu elements counters. For supported maps the test upserts a number > >>> of elements, checks the correctness of the counters, then deletes all the > >>> elements and checks again that the counters sum drops down to zero. > >>> > >>> The following map types are tested: > >>> > >>> * BPF_MAP_TYPE_HASH, BPF_F_NO_PREALLOC > >>> * BPF_MAP_TYPE_PERCPU_HASH, BPF_F_NO_PREALLOC > >>> * BPF_MAP_TYPE_HASH, > >>> * BPF_MAP_TYPE_PERCPU_HASH, > >>> * BPF_MAP_TYPE_LRU_HASH > >>> * BPF_MAP_TYPE_LRU_PERCPU_HASH > >>> * BPF_MAP_TYPE_LRU_HASH, BPF_F_NO_COMMON_LRU > >>> * BPF_MAP_TYPE_LRU_PERCPU_HASH, BPF_F_NO_COMMON_LRU > >>> * BPF_MAP_TYPE_HASH_OF_MAPS > >>> > >>> Signed-off-by: Anton Protopopov <aspsk@isovalent.com> > >> Acked-by: Hou Tao <houtao1@huawei.com> > >> > >> With two nits below. > > Thanks, fixed both for v5. > Great. > SNIP > > +static void delete_and_lookup_batch(int map_fd, void *keys, __u32 count) > > +{ > > + static __u8 values[(8 << 10) * MAX_ENTRIES]; > > + void *in_batch = NULL, *out_batch; > > + __u32 save_count = count; > > + int ret; > > + > > + ret = bpf_map_lookup_and_delete_batch(map_fd, > > + &in_batch, &out_batch, > > + keys, values, &count, > > + NULL); > > + > > + /* > > + * Despite what uapi header says, lookup_and_delete_batch will return > > + * -ENOENT in case we successfully have deleted all elements, so check > > + * this separately > > + */ > >> It seems it is a bug in __htab_map_lookup_and_delete_batch(). I could > >> post a patch to fix it if you don't plan to do that by yourself. > > This should be as simple as > > > > @@ -1876,7 +1876,8 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map, > > total += bucket_cnt; > > batch++; > > if (batch >= htab->n_buckets) { > > - ret = -ENOENT; > > + if (!total) > > + ret = -ENOENT; > > goto after_loop; > > } > > goto again; > > No. I think changing it to "if (max_count > total) ret = -ENOENT;" will > be more appropriate, because it means the requested count couldn't been > fulfilled and it is also consistent with the comments in > include/uapi/linux/bpf.h Say, I have a map of size N and I don't know how many entries there are. Then I will do count=N lookup_and_delete(&count) In this case we will walk through the whole map, reach the 'batch >= htab->n_buckets', and set the count to the number of elements we read. (If, in opposite, there's no space to read a whole bucket, then we check this above and return -ENOSPC.) > > However, this might be already utilized by some apps to check that they've read > > all entries. Two local examples are map_tests/map_in_map_batch_ops.c and > > map_tests/htab_map_batch_ops.c. Another example I know is from BCC tools: > > https://github.com/iovisor/bcc/blob/master/libbpf-tools/map_helpers.c#L58 > I think these use cases will be fine. Because when the last element has > been successfully iterated and returned, the out_batch is also updated, > so if the batch op is called again, -ENOENT will be returned. > > > > Can we update comments in include/uapi/linux/bpf.h? > I think the comments are correct. Currently we return -ENOENT as an indicator that (a) 'in_batch' is out of bounds (b) we reached the end of map. So technically, this is an optimization, as if we read elements in a loop by passing 'in_batch', 'out_batch', even if we return 0 in case (b), the next syscall would return -ENOENT, because the new 'in_batch' would point to out of bounds. This also makes sense for a map which is empty: we reached the end of map, didn't find any elements, so we're returning -ENOENT (in contrast with saying "all is ok, we read 0 elements"). So from my point of view -ENOENT makes sense. However, comments say "Returns zero on success" which doesn't look true to me as I think that reading the whole map in one syscall is a success :) ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 5/6] selftests/bpf: test map percpu stats 2023-07-06 12:57 ` Anton Protopopov @ 2023-07-07 1:41 ` Hou Tao 2023-07-07 7:28 ` Anton Protopopov 0 siblings, 1 reply; 24+ messages in thread From: Hou Tao @ 2023-07-07 1:41 UTC (permalink / raw) To: Anton Protopopov Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf Hi, On 7/6/2023 8:57 PM, Anton Protopopov wrote: > On Thu, Jul 06, 2023 at 08:21:17PM +0800, Hou Tao wrote: >> Hi, >> >> On 7/6/2023 7:54 PM, Anton Protopopov wrote: >> SNIP >>> +static void delete_and_lookup_batch(int map_fd, void *keys, __u32 count) >>> +{ >>> + static __u8 values[(8 << 10) * MAX_ENTRIES]; >>> + void *in_batch = NULL, *out_batch; >>> + __u32 save_count = count; >>> + int ret; >>> + >>> + ret = bpf_map_lookup_and_delete_batch(map_fd, >>> + &in_batch, &out_batch, >>> + keys, values, &count, >>> + NULL); >>> + >>> + /* >>> + * Despite what uapi header says, lookup_and_delete_batch will return >>> + * -ENOENT in case we successfully have deleted all elements, so check >>> + * this separately >>> + */ >>>> It seems it is a bug in __htab_map_lookup_and_delete_batch(). I could >>>> post a patch to fix it if you don't plan to do that by yourself. >>> This should be as simple as >>> >>> @@ -1876,7 +1876,8 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map, >>> total += bucket_cnt; >>> batch++; >>> if (batch >= htab->n_buckets) { >>> - ret = -ENOENT; >>> + if (!total) >>> + ret = -ENOENT; >>> goto after_loop; >>> } >>> goto again; >> No. I think changing it to "if (max_count > total) ret = -ENOENT;" will >> be more appropriate, because it means the requested count couldn't been >> fulfilled and it is also consistent with the comments in >> include/uapi/linux/bpf.h > Say, I have a map of size N and I don't know how many entries there are. > Then I will do > > count=N > lookup_and_delete(&count) > > In this case we will walk through the whole map, reach the 'batch >= > htab->n_buckets', and set the count to the number of elements we read. > > (If, in opposite, there's no space to read a whole bucket, then we check this > above and return -ENOSPC.) > >>> However, this might be already utilized by some apps to check that they've read >>> all entries. Two local examples are map_tests/map_in_map_batch_ops.c and >>> map_tests/htab_map_batch_ops.c. Another example I know is from BCC tools: >>> https://github.com/iovisor/bcc/blob/master/libbpf-tools/map_helpers.c#L58 >> I think these use cases will be fine. Because when the last element has >> been successfully iterated and returned, the out_batch is also updated, >> so if the batch op is called again, -ENOENT will be returned. >>> Can we update comments in include/uapi/linux/bpf.h? >> I think the comments are correct. > Currently we return -ENOENT as an indicator that (a) 'in_batch' is out of > bounds (b) we reached the end of map. So technically, this is an optimization, > as if we read elements in a loop by passing 'in_batch', 'out_batch', even if we > return 0 in case (b), the next syscall would return -ENOENT, because the new > 'in_batch' would point to out of bounds. > > This also makes sense for a map which is empty: we reached the end of map, > didn't find any elements, so we're returning -ENOENT (in contrast with saying > "all is ok, we read 0 elements"). > > So from my point of view -ENOENT makes sense. However, comments say "Returns > zero on success" which doesn't look true to me as I think that reading the > whole map in one syscall is a success :) I get your point. The current implementation of BPF_MAP_LOOKUP_BATCH does the following two things: 1) returns 0 when the whole map has not been iterated but there is no space for current bucket. 2) doesn't return 0 when the whole map has been iterated successfully (and the requested count is fulfilled) For 1) I prefer to update the comments in uapi. If instead we fix the implementation, we may break the existed users which need to check ENOSPC to continue the batch op. For 2) I don't have a preference. Both updating the comments and implementation are fine to me. WDYT ? ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 5/6] selftests/bpf: test map percpu stats 2023-07-07 1:41 ` Hou Tao @ 2023-07-07 7:28 ` Anton Protopopov 2023-07-07 9:40 ` Hou Tao 0 siblings, 1 reply; 24+ messages in thread From: Anton Protopopov @ 2023-07-07 7:28 UTC (permalink / raw) To: Hou Tao Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf On Fri, Jul 07, 2023 at 09:41:03AM +0800, Hou Tao wrote: > Hi, > > On 7/6/2023 8:57 PM, Anton Protopopov wrote: > > On Thu, Jul 06, 2023 at 08:21:17PM +0800, Hou Tao wrote: > >> Hi, > >> > >> On 7/6/2023 7:54 PM, Anton Protopopov wrote: > >> > SNIP > >>> +static void delete_and_lookup_batch(int map_fd, void *keys, __u32 count) > >>> +{ > >>> + static __u8 values[(8 << 10) * MAX_ENTRIES]; > >>> + void *in_batch = NULL, *out_batch; > >>> + __u32 save_count = count; > >>> + int ret; > >>> + > >>> + ret = bpf_map_lookup_and_delete_batch(map_fd, > >>> + &in_batch, &out_batch, > >>> + keys, values, &count, > >>> + NULL); > >>> + > >>> + /* > >>> + * Despite what uapi header says, lookup_and_delete_batch will return > >>> + * -ENOENT in case we successfully have deleted all elements, so check > >>> + * this separately > >>> + */ > >>>> It seems it is a bug in __htab_map_lookup_and_delete_batch(). I could > >>>> post a patch to fix it if you don't plan to do that by yourself. > >>> This should be as simple as > >>> > >>> @@ -1876,7 +1876,8 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map, > >>> total += bucket_cnt; > >>> batch++; > >>> if (batch >= htab->n_buckets) { > >>> - ret = -ENOENT; > >>> + if (!total) > >>> + ret = -ENOENT; > >>> goto after_loop; > >>> } > >>> goto again; > >> No. I think changing it to "if (max_count > total) ret = -ENOENT;" will > >> be more appropriate, because it means the requested count couldn't been > >> fulfilled and it is also consistent with the comments in > >> include/uapi/linux/bpf.h > > Say, I have a map of size N and I don't know how many entries there are. > > Then I will do > > > > count=N > > lookup_and_delete(&count) > > > > In this case we will walk through the whole map, reach the 'batch >= > > htab->n_buckets', and set the count to the number of elements we read. > > > > (If, in opposite, there's no space to read a whole bucket, then we check this > > above and return -ENOSPC.) > > > >>> However, this might be already utilized by some apps to check that they've read > >>> all entries. Two local examples are map_tests/map_in_map_batch_ops.c and > >>> map_tests/htab_map_batch_ops.c. Another example I know is from BCC tools: > >>> https://github.com/iovisor/bcc/blob/master/libbpf-tools/map_helpers.c#L58 > >> I think these use cases will be fine. Because when the last element has > >> been successfully iterated and returned, the out_batch is also updated, > >> so if the batch op is called again, -ENOENT will be returned. > >>> Can we update comments in include/uapi/linux/bpf.h? > >> I think the comments are correct. > > Currently we return -ENOENT as an indicator that (a) 'in_batch' is out of > > bounds (b) we reached the end of map. So technically, this is an optimization, > > as if we read elements in a loop by passing 'in_batch', 'out_batch', even if we > > return 0 in case (b), the next syscall would return -ENOENT, because the new > > 'in_batch' would point to out of bounds. > > > > This also makes sense for a map which is empty: we reached the end of map, > > didn't find any elements, so we're returning -ENOENT (in contrast with saying > > "all is ok, we read 0 elements"). > > > > So from my point of view -ENOENT makes sense. However, comments say "Returns > > zero on success" which doesn't look true to me as I think that reading the > > whole map in one syscall is a success :) > > I get your point. The current implementation of BPF_MAP_LOOKUP_BATCH > does the following two things: > 1) returns 0 when the whole map has not been iterated but there is no > space for current bucket. The algorithm works per bucket. For a bucket number X it checks if there is enough space in the output buffer to store all bucket elements. If there is, ok, go to the next bucket. If not, then it checks if any elements were written already [from previous buckets]. If not, then it returns -ENOSPC, meaning, "you've asked to copy at most N elements, but I can only copy M > N, not less, please provide a bigger buffer." > 2) doesn't return 0 when the whole map has been iterated successfully > (and the requested count is fulfilled) > > For 1) I prefer to update the comments in uapi. If instead we fix the > implementation, we may break the existed users which need to check > ENOSPC to continue the batch op. > For 2) I don't have a preference. Both updating the comments and > implementation are fine to me. > > WDYT ? > I think that (1) is perfectly fine, -ENOSPC is returned only when we can't copy elements, which is an error. The (2) requires updating docs. The API is similar to get_next_key, and docs can be updated in the same way. By updating docs we're not changing any uapi, right? ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 5/6] selftests/bpf: test map percpu stats 2023-07-07 7:28 ` Anton Protopopov @ 2023-07-07 9:40 ` Hou Tao 2023-07-07 11:06 ` Anton Protopopov 0 siblings, 1 reply; 24+ messages in thread From: Hou Tao @ 2023-07-07 9:40 UTC (permalink / raw) To: Anton Protopopov Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf Hi, On 7/7/2023 3:28 PM, Anton Protopopov wrote: > On Fri, Jul 07, 2023 at 09:41:03AM +0800, Hou Tao wrote: >> Hi, >> >> On 7/6/2023 8:57 PM, Anton Protopopov wrote: >>> On Thu, Jul 06, 2023 at 08:21:17PM +0800, Hou Tao wrote: >>>> Hi, >>>> >>>> On 7/6/2023 7:54 PM, Anton Protopopov wrote: >>>> >> SNIP >>>>> +static void delete_and_lookup_batch(int map_fd, void *keys, __u32 count) >>>>> +{ >>>>> + static __u8 values[(8 << 10) * MAX_ENTRIES]; >>>>> + void *in_batch = NULL, *out_batch; >>>>> + __u32 save_count = count; >>>>> + int ret; >>>>> + >>>>> + ret = bpf_map_lookup_and_delete_batch(map_fd, >>>>> + &in_batch, &out_batch, >>>>> + keys, values, &count, >>>>> + NULL); >>>>> + >>>>> + /* >>>>> + * Despite what uapi header says, lookup_and_delete_batch will return >>>>> + * -ENOENT in case we successfully have deleted all elements, so check >>>>> + * this separately >>>>> + */ >>>>>> It seems it is a bug in __htab_map_lookup_and_delete_batch(). I could >>>>>> post a patch to fix it if you don't plan to do that by yourself. >>>>> This should be as simple as >>>>> >>>>> @@ -1876,7 +1876,8 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map, >>>>> total += bucket_cnt; >>>>> batch++; >>>>> if (batch >= htab->n_buckets) { >>>>> - ret = -ENOENT; >>>>> + if (!total) >>>>> + ret = -ENOENT; >>>>> goto after_loop; >>>>> } >>>>> goto again; >>>> No. I think changing it to "if (max_count > total) ret = -ENOENT;" will >>>> be more appropriate, because it means the requested count couldn't been >>>> fulfilled and it is also consistent with the comments in >>>> include/uapi/linux/bpf.h >>> Say, I have a map of size N and I don't know how many entries there are. >>> Then I will do >>> >>> count=N >>> lookup_and_delete(&count) >>> >>> In this case we will walk through the whole map, reach the 'batch >= >>> htab->n_buckets', and set the count to the number of elements we read. >>> >>> (If, in opposite, there's no space to read a whole bucket, then we check this >>> above and return -ENOSPC.) >>> >>>>> However, this might be already utilized by some apps to check that they've read >>>>> all entries. Two local examples are map_tests/map_in_map_batch_ops.c and >>>>> map_tests/htab_map_batch_ops.c. Another example I know is from BCC tools: >>>>> https://github.com/iovisor/bcc/blob/master/libbpf-tools/map_helpers.c#L58 >>>> I think these use cases will be fine. Because when the last element has >>>> been successfully iterated and returned, the out_batch is also updated, >>>> so if the batch op is called again, -ENOENT will be returned. >>>>> Can we update comments in include/uapi/linux/bpf.h? >>>> I think the comments are correct. >>> Currently we return -ENOENT as an indicator that (a) 'in_batch' is out of >>> bounds (b) we reached the end of map. So technically, this is an optimization, >>> as if we read elements in a loop by passing 'in_batch', 'out_batch', even if we >>> return 0 in case (b), the next syscall would return -ENOENT, because the new >>> 'in_batch' would point to out of bounds. >>> >>> This also makes sense for a map which is empty: we reached the end of map, >>> didn't find any elements, so we're returning -ENOENT (in contrast with saying >>> "all is ok, we read 0 elements"). >>> >>> So from my point of view -ENOENT makes sense. However, comments say "Returns >>> zero on success" which doesn't look true to me as I think that reading the >>> whole map in one syscall is a success :) >> I get your point. The current implementation of BPF_MAP_LOOKUP_BATCH >> does the following two things: >> 1) returns 0 when the whole map has not been iterated but there is no >> space for current bucket. > The algorithm works per bucket. For a bucket number X it checks if there is > enough space in the output buffer to store all bucket elements. If there is, > ok, go to the next bucket. If not, then it checks if any elements were written > already [from previous buckets]. If not, then it returns -ENOSPC, meaning, > "you've asked to copy at most N elements, but I can only copy M > N, not less, > please provide a bigger buffer." Yes. > >> 2) doesn't return 0 when the whole map has been iterated successfully >> (and the requested count is fulfilled) >> >> For 1) I prefer to update the comments in uapi. If instead we fix the >> implementation, we may break the existed users which need to check >> ENOSPC to continue the batch op. >> For 2) I don't have a preference. Both updating the comments and >> implementation are fine to me. >> >> WDYT ? >> > I think that (1) is perfectly fine, -ENOSPC is returned only when we can't copy > elements, which is an error. Maybe I misinterpreted the comments in bpf.h. As said in the comment: "On success, *count* elements from the map are copied into the user buffer", I think the count here means the value of count which is used as input instead of output. > > The (2) requires updating docs. The API is similar to get_next_key, and docs > can be updated in the same way. By updating docs we're not changing any uapi, > right? I think it is fine. ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 5/6] selftests/bpf: test map percpu stats 2023-07-07 9:40 ` Hou Tao @ 2023-07-07 11:06 ` Anton Protopopov 0 siblings, 0 replies; 24+ messages in thread From: Anton Protopopov @ 2023-07-07 11:06 UTC (permalink / raw) To: Hou Tao Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf On Fri, Jul 07, 2023 at 05:40:28PM +0800, Hou Tao wrote: > Hi, > > On 7/7/2023 3:28 PM, Anton Protopopov wrote: > > On Fri, Jul 07, 2023 at 09:41:03AM +0800, Hou Tao wrote: > >> Hi, > >> > >> On 7/6/2023 8:57 PM, Anton Protopopov wrote: > >>> On Thu, Jul 06, 2023 at 08:21:17PM +0800, Hou Tao wrote: > >>>> Hi, > >>>> > >>>> On 7/6/2023 7:54 PM, Anton Protopopov wrote: > >>>> > >> SNIP > >>>>> +static void delete_and_lookup_batch(int map_fd, void *keys, __u32 count) > >>>>> +{ > >>>>> + static __u8 values[(8 << 10) * MAX_ENTRIES]; > >>>>> + void *in_batch = NULL, *out_batch; > >>>>> + __u32 save_count = count; > >>>>> + int ret; > >>>>> + > >>>>> + ret = bpf_map_lookup_and_delete_batch(map_fd, > >>>>> + &in_batch, &out_batch, > >>>>> + keys, values, &count, > >>>>> + NULL); > >>>>> + > >>>>> + /* > >>>>> + * Despite what uapi header says, lookup_and_delete_batch will return > >>>>> + * -ENOENT in case we successfully have deleted all elements, so check > >>>>> + * this separately > >>>>> + */ > >>>>>> It seems it is a bug in __htab_map_lookup_and_delete_batch(). I could > >>>>>> post a patch to fix it if you don't plan to do that by yourself. > >>>>> This should be as simple as > >>>>> > >>>>> @@ -1876,7 +1876,8 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map, > >>>>> total += bucket_cnt; > >>>>> batch++; > >>>>> if (batch >= htab->n_buckets) { > >>>>> - ret = -ENOENT; > >>>>> + if (!total) > >>>>> + ret = -ENOENT; > >>>>> goto after_loop; > >>>>> } > >>>>> goto again; > >>>> No. I think changing it to "if (max_count > total) ret = -ENOENT;" will > >>>> be more appropriate, because it means the requested count couldn't been > >>>> fulfilled and it is also consistent with the comments in > >>>> include/uapi/linux/bpf.h > >>> Say, I have a map of size N and I don't know how many entries there are. > >>> Then I will do > >>> > >>> count=N > >>> lookup_and_delete(&count) > >>> > >>> In this case we will walk through the whole map, reach the 'batch >= > >>> htab->n_buckets', and set the count to the number of elements we read. > >>> > >>> (If, in opposite, there's no space to read a whole bucket, then we check this > >>> above and return -ENOSPC.) > >>> > >>>>> However, this might be already utilized by some apps to check that they've read > >>>>> all entries. Two local examples are map_tests/map_in_map_batch_ops.c and > >>>>> map_tests/htab_map_batch_ops.c. Another example I know is from BCC tools: > >>>>> https://github.com/iovisor/bcc/blob/master/libbpf-tools/map_helpers.c#L58 > >>>> I think these use cases will be fine. Because when the last element has > >>>> been successfully iterated and returned, the out_batch is also updated, > >>>> so if the batch op is called again, -ENOENT will be returned. > >>>>> Can we update comments in include/uapi/linux/bpf.h? > >>>> I think the comments are correct. > >>> Currently we return -ENOENT as an indicator that (a) 'in_batch' is out of > >>> bounds (b) we reached the end of map. So technically, this is an optimization, > >>> as if we read elements in a loop by passing 'in_batch', 'out_batch', even if we > >>> return 0 in case (b), the next syscall would return -ENOENT, because the new > >>> 'in_batch' would point to out of bounds. > >>> > >>> This also makes sense for a map which is empty: we reached the end of map, > >>> didn't find any elements, so we're returning -ENOENT (in contrast with saying > >>> "all is ok, we read 0 elements"). > >>> > >>> So from my point of view -ENOENT makes sense. However, comments say "Returns > >>> zero on success" which doesn't look true to me as I think that reading the > >>> whole map in one syscall is a success :) > >> I get your point. The current implementation of BPF_MAP_LOOKUP_BATCH > >> does the following two things: > >> 1) returns 0 when the whole map has not been iterated but there is no > >> space for current bucket. > > The algorithm works per bucket. For a bucket number X it checks if there is > > enough space in the output buffer to store all bucket elements. If there is, > > ok, go to the next bucket. If not, then it checks if any elements were written > > already [from previous buckets]. If not, then it returns -ENOSPC, meaning, > > "you've asked to copy at most N elements, but I can only copy M > N, not less, > > please provide a bigger buffer." > Yes. > > > >> 2) doesn't return 0 when the whole map has been iterated successfully > >> (and the requested count is fulfilled) > >> > >> For 1) I prefer to update the comments in uapi. If instead we fix the > >> implementation, we may break the existed users which need to check > >> ENOSPC to continue the batch op. > >> For 2) I don't have a preference. Both updating the comments and > >> implementation are fine to me. > >> > >> WDYT ? > >> > > I think that (1) is perfectly fine, -ENOSPC is returned only when we can't copy > > elements, which is an error. > > Maybe I misinterpreted the comments in bpf.h. As said in the comment: > "On success, *count* elements from the map are copied into the user > buffer", I think the count here means the value of count which is used > as input instead of output. Yes, also may be updated, as this is actually "up to *count*" (*count* is an output parameter which is not mentioned in the LOOKUP_BATCH description, only in LOOKUP_AND_DELETE_BATCH). On the other side, the LOOKUP_AND_DELETE_BATCH comment says "delete at least count" which is not true as well. > > The (2) requires updating docs. The API is similar to get_next_key, and docs > > can be updated in the same way. By updating docs we're not changing any uapi, > > right? > I think it is fine. ^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH v4 bpf-next 6/6] selftests/bpf: check that ->elem_count is non-zero for the hash map 2023-07-05 16:01 [PATCH v4 bpf-next 0/6] bpf: add percpu stats for bpf_map Anton Protopopov ` (4 preceding siblings ...) 2023-07-05 16:01 ` [PATCH v4 bpf-next 5/6] selftests/bpf: test map percpu stats Anton Protopopov @ 2023-07-05 16:01 ` Anton Protopopov 2023-07-06 1:26 ` Alexei Starovoitov 5 siblings, 1 reply; 24+ messages in thread From: Anton Protopopov @ 2023-07-05 16:01 UTC (permalink / raw) To: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf Cc: Anton Protopopov Previous commits populated the ->elem_count per-cpu pointer for hash maps. Check that this pointer is non-NULL in an existing map. Signed-off-by: Anton Protopopov <aspsk@isovalent.com> --- tools/testing/selftests/bpf/progs/map_ptr_kern.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tools/testing/selftests/bpf/progs/map_ptr_kern.c b/tools/testing/selftests/bpf/progs/map_ptr_kern.c index db388f593d0a..d6e234a37ccb 100644 --- a/tools/testing/selftests/bpf/progs/map_ptr_kern.c +++ b/tools/testing/selftests/bpf/progs/map_ptr_kern.c @@ -33,6 +33,7 @@ struct bpf_map { __u32 value_size; __u32 max_entries; __u32 id; + __s64 *elem_count; } __attribute__((preserve_access_index)); static inline int check_bpf_map_fields(struct bpf_map *map, __u32 key_size, @@ -111,6 +112,8 @@ static inline int check_hash(void) VERIFY(check_default_noinline(&hash->map, map)); + VERIFY(map->elem_count != NULL); + VERIFY(hash->n_buckets == MAX_ENTRIES); VERIFY(hash->elem_size == 64); -- 2.34.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 6/6] selftests/bpf: check that ->elem_count is non-zero for the hash map 2023-07-05 16:01 ` [PATCH v4 bpf-next 6/6] selftests/bpf: check that ->elem_count is non-zero for the hash map Anton Protopopov @ 2023-07-06 1:26 ` Alexei Starovoitov 2023-07-06 5:44 ` Anton Protopopov 0 siblings, 1 reply; 24+ messages in thread From: Alexei Starovoitov @ 2023-07-06 1:26 UTC (permalink / raw) To: Anton Protopopov Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf On Wed, Jul 5, 2023 at 9:00 AM Anton Protopopov <aspsk@isovalent.com> wrote: > > Previous commits populated the ->elem_count per-cpu pointer for hash maps. > Check that this pointer is non-NULL in an existing map. > > Signed-off-by: Anton Protopopov <aspsk@isovalent.com> > --- > tools/testing/selftests/bpf/progs/map_ptr_kern.c | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/tools/testing/selftests/bpf/progs/map_ptr_kern.c b/tools/testing/selftests/bpf/progs/map_ptr_kern.c > index db388f593d0a..d6e234a37ccb 100644 > --- a/tools/testing/selftests/bpf/progs/map_ptr_kern.c > +++ b/tools/testing/selftests/bpf/progs/map_ptr_kern.c > @@ -33,6 +33,7 @@ struct bpf_map { > __u32 value_size; > __u32 max_entries; > __u32 id; > + __s64 *elem_count; > } __attribute__((preserve_access_index)); > > static inline int check_bpf_map_fields(struct bpf_map *map, __u32 key_size, > @@ -111,6 +112,8 @@ static inline int check_hash(void) > > VERIFY(check_default_noinline(&hash->map, map)); > > + VERIFY(map->elem_count != NULL); > + imo that's worse than no test. Just use kfunc here and get the real count? ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 6/6] selftests/bpf: check that ->elem_count is non-zero for the hash map 2023-07-06 1:26 ` Alexei Starovoitov @ 2023-07-06 5:44 ` Anton Protopopov 2023-07-06 17:03 ` Alexei Starovoitov 0 siblings, 1 reply; 24+ messages in thread From: Anton Protopopov @ 2023-07-06 5:44 UTC (permalink / raw) To: Alexei Starovoitov Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf On Wed, Jul 05, 2023 at 06:26:25PM -0700, Alexei Starovoitov wrote: > On Wed, Jul 5, 2023 at 9:00 AM Anton Protopopov <aspsk@isovalent.com> wrote: > > > > Previous commits populated the ->elem_count per-cpu pointer for hash maps. > > Check that this pointer is non-NULL in an existing map. > > > > Signed-off-by: Anton Protopopov <aspsk@isovalent.com> > > --- > > tools/testing/selftests/bpf/progs/map_ptr_kern.c | 3 +++ > > 1 file changed, 3 insertions(+) > > > > diff --git a/tools/testing/selftests/bpf/progs/map_ptr_kern.c b/tools/testing/selftests/bpf/progs/map_ptr_kern.c > > index db388f593d0a..d6e234a37ccb 100644 > > --- a/tools/testing/selftests/bpf/progs/map_ptr_kern.c > > +++ b/tools/testing/selftests/bpf/progs/map_ptr_kern.c > > @@ -33,6 +33,7 @@ struct bpf_map { > > __u32 value_size; > > __u32 max_entries; > > __u32 id; > > + __s64 *elem_count; > > } __attribute__((preserve_access_index)); > > > > static inline int check_bpf_map_fields(struct bpf_map *map, __u32 key_size, > > @@ -111,6 +112,8 @@ static inline int check_hash(void) > > > > VERIFY(check_default_noinline(&hash->map, map)); > > > > + VERIFY(map->elem_count != NULL); > > + > > imo that's worse than no test. > Just use kfunc here and get the real count? Then, as I mentioned in the previous version, I will have to teach kfuncs to recognize const_ptr_to_map args just for the sake of this selftest, while we already testing all functionality in the new selftest for test_maps. So I would just omit this one. Or am I missing something? ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 6/6] selftests/bpf: check that ->elem_count is non-zero for the hash map 2023-07-06 5:44 ` Anton Protopopov @ 2023-07-06 17:03 ` Alexei Starovoitov 2023-07-06 17:43 ` Anton Protopopov 0 siblings, 1 reply; 24+ messages in thread From: Alexei Starovoitov @ 2023-07-06 17:03 UTC (permalink / raw) To: Anton Protopopov Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf On Wed, Jul 5, 2023 at 10:42 PM Anton Protopopov <aspsk@isovalent.com> wrote: > > On Wed, Jul 05, 2023 at 06:26:25PM -0700, Alexei Starovoitov wrote: > > On Wed, Jul 5, 2023 at 9:00 AM Anton Protopopov <aspsk@isovalent.com> wrote: > > > > > > Previous commits populated the ->elem_count per-cpu pointer for hash maps. > > > Check that this pointer is non-NULL in an existing map. > > > > > > Signed-off-by: Anton Protopopov <aspsk@isovalent.com> > > > --- > > > tools/testing/selftests/bpf/progs/map_ptr_kern.c | 3 +++ > > > 1 file changed, 3 insertions(+) > > > > > > diff --git a/tools/testing/selftests/bpf/progs/map_ptr_kern.c b/tools/testing/selftests/bpf/progs/map_ptr_kern.c > > > index db388f593d0a..d6e234a37ccb 100644 > > > --- a/tools/testing/selftests/bpf/progs/map_ptr_kern.c > > > +++ b/tools/testing/selftests/bpf/progs/map_ptr_kern.c > > > @@ -33,6 +33,7 @@ struct bpf_map { > > > __u32 value_size; > > > __u32 max_entries; > > > __u32 id; > > > + __s64 *elem_count; > > > } __attribute__((preserve_access_index)); > > > > > > static inline int check_bpf_map_fields(struct bpf_map *map, __u32 key_size, > > > @@ -111,6 +112,8 @@ static inline int check_hash(void) > > > > > > VERIFY(check_default_noinline(&hash->map, map)); > > > > > > + VERIFY(map->elem_count != NULL); > > > + > > > > imo that's worse than no test. > > Just use kfunc here and get the real count? > > Then, as I mentioned in the previous version, I will have to teach kfuncs to > recognize const_ptr_to_map args just for the sake of this selftest, while we > already testing all functionality in the new selftest for test_maps. So I would > just omit this one. Or am I missing something? Don't you want to do: val = bpf_map_lookup_elem(map, ...); cnt = bpf_map_sum_elem_count(map); and that's the main use case ? So teaching the verifier to understand that const_ptr_to_map matches BTF 'struct bpf_map *' is essential ? ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 6/6] selftests/bpf: check that ->elem_count is non-zero for the hash map 2023-07-06 17:03 ` Alexei Starovoitov @ 2023-07-06 17:43 ` Anton Protopopov 2023-07-06 17:48 ` Alexei Starovoitov 0 siblings, 1 reply; 24+ messages in thread From: Anton Protopopov @ 2023-07-06 17:43 UTC (permalink / raw) To: Alexei Starovoitov Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf On Thu, Jul 06, 2023 at 10:03:30AM -0700, Alexei Starovoitov wrote: > On Wed, Jul 5, 2023 at 10:42 PM Anton Protopopov <aspsk@isovalent.com> wrote: > > > > On Wed, Jul 05, 2023 at 06:26:25PM -0700, Alexei Starovoitov wrote: > > > On Wed, Jul 5, 2023 at 9:00 AM Anton Protopopov <aspsk@isovalent.com> wrote: > > > > > > > > Previous commits populated the ->elem_count per-cpu pointer for hash maps. > > > > Check that this pointer is non-NULL in an existing map. > > > > > > > > Signed-off-by: Anton Protopopov <aspsk@isovalent.com> > > > > --- > > > > tools/testing/selftests/bpf/progs/map_ptr_kern.c | 3 +++ > > > > 1 file changed, 3 insertions(+) > > > > > > > > diff --git a/tools/testing/selftests/bpf/progs/map_ptr_kern.c b/tools/testing/selftests/bpf/progs/map_ptr_kern.c > > > > index db388f593d0a..d6e234a37ccb 100644 > > > > --- a/tools/testing/selftests/bpf/progs/map_ptr_kern.c > > > > +++ b/tools/testing/selftests/bpf/progs/map_ptr_kern.c > > > > @@ -33,6 +33,7 @@ struct bpf_map { > > > > __u32 value_size; > > > > __u32 max_entries; > > > > __u32 id; > > > > + __s64 *elem_count; > > > > } __attribute__((preserve_access_index)); > > > > > > > > static inline int check_bpf_map_fields(struct bpf_map *map, __u32 key_size, > > > > @@ -111,6 +112,8 @@ static inline int check_hash(void) > > > > > > > > VERIFY(check_default_noinline(&hash->map, map)); > > > > > > > > + VERIFY(map->elem_count != NULL); > > > > + > > > > > > imo that's worse than no test. > > > Just use kfunc here and get the real count? > > > > Then, as I mentioned in the previous version, I will have to teach kfuncs to > > recognize const_ptr_to_map args just for the sake of this selftest, while we > > already testing all functionality in the new selftest for test_maps. So I would > > just omit this one. Or am I missing something? > > > Don't you want to do: > val = bpf_map_lookup_elem(map, ...); > cnt = bpf_map_sum_elem_count(map); > > and that's the main use case ? Not sure I understand what this ^ use case is... Our primary use case is to [periodically] get the number of elements from the user space. We can do this using an iterator as you've suggested and what is tested in the added selftest. > So teaching the verifier to understand that const_ptr_to_map matches > BTF 'struct bpf_map *' is essential ? ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 6/6] selftests/bpf: check that ->elem_count is non-zero for the hash map 2023-07-06 17:43 ` Anton Protopopov @ 2023-07-06 17:48 ` Alexei Starovoitov 2023-07-06 18:35 ` Anton Protopopov 0 siblings, 1 reply; 24+ messages in thread From: Alexei Starovoitov @ 2023-07-06 17:48 UTC (permalink / raw) To: Anton Protopopov Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf On Thu, Jul 6, 2023 at 10:42 AM Anton Protopopov <aspsk@isovalent.com> wrote: > > > > > Don't you want to do: > > val = bpf_map_lookup_elem(map, ...); > > cnt = bpf_map_sum_elem_count(map); > > > > and that's the main use case ? > > Not sure I understand what this ^ use case is... > > Our primary use case is to [periodically] get the number of elements from the > user space. We can do this using an iterator as you've suggested and what is > tested in the added selftest. Hmm. During the last office hours John explained that he needs to raise alarm to user space when the map is close to full capacity. Currently he's doing it with his own per-map counters implemented in a bpf prog. I'd expect the alarm to be done inline with updates to the counters. If you scan maps from user space every second or so there is a chance the spike in usage will be missed. If we're adding map counters they should be usable not only via iterators. John, did I describe your use case correctly? ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 6/6] selftests/bpf: check that ->elem_count is non-zero for the hash map 2023-07-06 17:48 ` Alexei Starovoitov @ 2023-07-06 18:35 ` Anton Protopopov 2023-07-06 20:33 ` Alexei Starovoitov 0 siblings, 1 reply; 24+ messages in thread From: Anton Protopopov @ 2023-07-06 18:35 UTC (permalink / raw) To: Alexei Starovoitov Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf On Thu, Jul 06, 2023 at 10:48:16AM -0700, Alexei Starovoitov wrote: > On Thu, Jul 6, 2023 at 10:42 AM Anton Protopopov <aspsk@isovalent.com> wrote: > > > > > > > > Don't you want to do: > > > val = bpf_map_lookup_elem(map, ...); > > > cnt = bpf_map_sum_elem_count(map); > > > > > > and that's the main use case ? > > > > Not sure I understand what this ^ use case is... > > > > Our primary use case is to [periodically] get the number of elements from the > > user space. We can do this using an iterator as you've suggested and what is > > tested in the added selftest. > > Hmm. During the last office hours John explained that he needs to raise > alarm to user space when the map is close to full capacity. > Currently he's doing it with his own per-map counters implemented > in a bpf prog. > I'd expect the alarm to be done inline with updates to the counters. > If you scan maps from user space every second or so there is a chance > the spike in usage will be missed. > > If we're adding map counters they should be usable not only via iterators. In some use cases this is ok to miss a spike in favour of not checking counters too often. But yes, for other use cases this makes sense to add support for const map ptr, so I will do this. > > John, > did I describe your use case correctly? ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v4 bpf-next 6/6] selftests/bpf: check that ->elem_count is non-zero for the hash map 2023-07-06 18:35 ` Anton Protopopov @ 2023-07-06 20:33 ` Alexei Starovoitov 0 siblings, 0 replies; 24+ messages in thread From: Alexei Starovoitov @ 2023-07-06 20:33 UTC (permalink / raw) To: Anton Protopopov Cc: Alexei Starovoitov, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf On Thu, Jul 6, 2023 at 11:34 AM Anton Protopopov <aspsk@isovalent.com> wrote: > > On Thu, Jul 06, 2023 at 10:48:16AM -0700, Alexei Starovoitov wrote: > > On Thu, Jul 6, 2023 at 10:42 AM Anton Protopopov <aspsk@isovalent.com> wrote: > > > > > > > > > > > Don't you want to do: > > > > val = bpf_map_lookup_elem(map, ...); > > > > cnt = bpf_map_sum_elem_count(map); > > > > > > > > and that's the main use case ? > > > > > > Not sure I understand what this ^ use case is... > > > > > > Our primary use case is to [periodically] get the number of elements from the > > > user space. We can do this using an iterator as you've suggested and what is > > > tested in the added selftest. > > > > Hmm. During the last office hours John explained that he needs to raise > > alarm to user space when the map is close to full capacity. > > Currently he's doing it with his own per-map counters implemented > > in a bpf prog. > > I'd expect the alarm to be done inline with updates to the counters. > > If you scan maps from user space every second or so there is a chance > > the spike in usage will be missed. > > > > If we're adding map counters they should be usable not only via iterators. > > In some use cases this is ok to miss a spike in favour of not checking counters > too often. But yes, for other use cases this makes sense to add support for > const map ptr, so I will do this. Great. Please send a follow up. I've applied the current set. ^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2023-07-07 11:06 UTC | newest] Thread overview: 24+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2023-07-05 16:01 [PATCH v4 bpf-next 0/6] bpf: add percpu stats for bpf_map Anton Protopopov 2023-07-05 16:01 ` [PATCH v4 bpf-next 1/6] bpf: add percpu stats for bpf_map elements insertions/deletions Anton Protopopov 2023-07-05 16:01 ` [PATCH v4 bpf-next 2/6] bpf: add a new kfunc to return current bpf_map elements count Anton Protopopov 2023-07-05 16:01 ` [PATCH v4 bpf-next 3/6] bpf: populate the per-cpu insertions/deletions counters for hashmaps Anton Protopopov 2023-07-06 1:24 ` Alexei Starovoitov 2023-07-06 5:47 ` Anton Protopopov 2023-07-05 16:01 ` [PATCH v4 bpf-next 4/6] bpf: make preloaded map iterators to display map elements count Anton Protopopov 2023-07-05 16:01 ` [PATCH v4 bpf-next 5/6] selftests/bpf: test map percpu stats Anton Protopopov 2023-07-06 10:49 ` Hou Tao 2023-07-06 11:54 ` Anton Protopopov 2023-07-06 12:21 ` Hou Tao 2023-07-06 12:57 ` Anton Protopopov 2023-07-07 1:41 ` Hou Tao 2023-07-07 7:28 ` Anton Protopopov 2023-07-07 9:40 ` Hou Tao 2023-07-07 11:06 ` Anton Protopopov 2023-07-05 16:01 ` [PATCH v4 bpf-next 6/6] selftests/bpf: check that ->elem_count is non-zero for the hash map Anton Protopopov 2023-07-06 1:26 ` Alexei Starovoitov 2023-07-06 5:44 ` Anton Protopopov 2023-07-06 17:03 ` Alexei Starovoitov 2023-07-06 17:43 ` Anton Protopopov 2023-07-06 17:48 ` Alexei Starovoitov 2023-07-06 18:35 ` Anton Protopopov 2023-07-06 20:33 ` Alexei Starovoitov
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox