[PATCH bpf] libbpf: clarify batch lookup semantics

[PATCH bpf] libbpf: clarify batch lookup semantics

[Martin Kelly]

The batch lookup APIs copy key memory into out_batch, which is then
supplied in later calls to in_batch. Thus both parameters need to point
to memory large enough to hold a single key (other than an initial NULL
in_batch). For many maps, keys are pointer sized or less, but for larger
maps, it's important to point to a larger block of memory to avoid
memory corruption.

Document these semantics to clarify the API.

Signed-off-by: Martin Kelly <martin.kelly@crowdstrike.com>
---
 include/uapi/linux/bpf.h |  5 ++++-
 tools/lib/bpf/bpf.h      | 15 ++++++++++-----
 2 files changed, 14 insertions(+), 6 deletions(-)

diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index d96708380e52..dae613b8778a 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -617,7 +617,10 @@ union bpf_iter_link_info {
  *		to NULL to begin the batched operation. After each subsequent
  *		**BPF_MAP_LOOKUP_BATCH**, the caller should pass the resultant
  *		*out_batch* as the *in_batch* for the next operation to
- *		continue iteration from the current point.
+ *		continue iteration from the current point. Both *in_batch* and
+ *		*out_batch* must point to memory large enough to hold a key,
+ *		except for maps of type **BPF_MAP_TYPE_HASH**, for which batch
+ *		parameters must be at least 4 bytes wide regardless of key size.
  *
  *		The *keys* and *values* are output parameters which must point
  *		to memory large enough to hold *count* items based on the key
diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h
index ab2570d28aec..c7e918ab0a60 100644
--- a/tools/lib/bpf/bpf.h
+++ b/tools/lib/bpf/bpf.h
@@ -190,10 +190,13 @@ LIBBPF_API int bpf_map_delete_batch(int fd, const void *keys,
 /**
  * @brief **bpf_map_lookup_batch()** allows for batch lookup of BPF map elements.
  *
- * The parameter *in_batch* is the address of the first element in the batch to read.
- * *out_batch* is an output parameter that should be passed as *in_batch* to subsequent
- * calls to **bpf_map_lookup_batch()**. NULL can be passed for *in_batch* to indicate
- * that the batched lookup starts from the beginning of the map.
+ * The parameter *in_batch* is the address of the first element in the batch to
+ * read. *out_batch* is an output parameter that should be passed as *in_batch*
+ * to subsequent calls to **bpf_map_lookup_batch()**. NULL can be passed for
+ * *in_batch* to indicate that the batched lookup starts from the beginning of
+ * the map. Both *in_batch* and *out_batch* must point to memory large enough to
+ * hold a single key, except for maps of type **BPF_MAP_TYPE_HASH**, for which
+ * the memory pointed to must be at least 4 bytes wide regardless of key size.
  *
  * The *keys* and *values* are output parameters which must point to memory large enough to
  * hold *count* items based on the key and value size of the map *map_fd*. The *keys*
@@ -226,7 +229,9 @@ LIBBPF_API int bpf_map_lookup_batch(int fd, void *in_batch, void *out_batch,
  *
  * @param fd BPF map file descriptor
  * @param in_batch address of the first element in batch to read, can pass NULL to
- * get address of the first element in *out_batch*
+ * get address of the first element in *out_batch*. If not NULL, must be large
+ * enough to hold a key. For **BPF_MAP_TYPE_HASH**, must be large enough to hold
+ * 4 bytes.
  * @param out_batch output parameter that should be passed to next call as *in_batch*
  * @param keys pointer to an array of *count* keys
  * @param values pointer to an array large enough for *count* values
-- 
2.34.1

Re: [PATCH bpf] libbpf: clarify batch lookup semantics

[Yonghong Song]

On 2/20/24 5:00 PM, Martin Kelly wrote:
> The batch lookup APIs copy key memory into out_batch, which is then

The above 'key memory' is not precise. The 'in_batch' and 'out_batch'
intends to be opaque and its size is map specific. So maybe we could
reword the description like below:

The batch lookup and lookup_and_delete APIs have two parameters,
in_batch and out_batch, to facilitate iterative lookup/lookup_and_deletion
operations for supported maps. Except NULL for in_batch at the start
of these two batch operations, both parameters need to point to memory
equal or larger than the respective map size, except for various hashmaps
(hash, percpu_hash, lru_hash, lru_percpu_hash) where the in_batch/out_batch
memory size should be at least 4 bytes.

Please also change your patch subject to
   [PATCH bpf-next] bpf: Clarify batch lookup/lookup_and_delete semantics

> supplied in later calls to in_batch. Thus both parameters need to point
> to memory large enough to hold a single key (other than an initial NULL
> in_batch). For many maps, keys are pointer sized or less, but for larger
> maps, it's important to point to a larger block of memory to avoid
> memory corruption.
>
> Document these semantics to clarify the API.
>
> Signed-off-by: Martin Kelly <martin.kelly@crowdstrike.com>
> ---
>   include/uapi/linux/bpf.h |  5 ++++-
>   tools/lib/bpf/bpf.h      | 15 ++++++++++-----
>   2 files changed, 14 insertions(+), 6 deletions(-)
>
> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> index d96708380e52..dae613b8778a 100644
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
> @@ -617,7 +617,10 @@ union bpf_iter_link_info {
>    *		to NULL to begin the batched operation. After each subsequent
>    *		**BPF_MAP_LOOKUP_BATCH**, the caller should pass the resultant
>    *		*out_batch* as the *in_batch* for the next operation to
> - *		continue iteration from the current point.
> + *		continue iteration from the current point. Both *in_batch* and
> + *		*out_batch* must point to memory large enough to hold a key,
> + *		except for maps of type **BPF_MAP_TYPE_HASH**, for which batch

Not just BPF_MAP_TYPE_HASH. It should be
BPF_MAP_TYPE_{HASH, PERCPU_HASH, LRU_HASH, LRU_PERCPU_HASH}.
Similar for some changes below.

> + *		parameters must be at least 4 bytes wide regardless of key size.
>    *
>    *		The *keys* and *values* are output parameters which must point
>    *		to memory large enough to hold *count* items based on the key

Please also sync updated include/uapi/linux/bpf.h to tools/include/uapi/linux/bpf.h.

> diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h
> index ab2570d28aec..c7e918ab0a60 100644
> --- a/tools/lib/bpf/bpf.h
> +++ b/tools/lib/bpf/bpf.h
> @@ -190,10 +190,13 @@ LIBBPF_API int bpf_map_delete_batch(int fd, const void *keys,
>   /**
>    * @brief **bpf_map_lookup_batch()** allows for batch lookup of BPF map elements.
>    *
> - * The parameter *in_batch* is the address of the first element in the batch to read.
> - * *out_batch* is an output parameter that should be passed as *in_batch* to subsequent
> - * calls to **bpf_map_lookup_batch()**. NULL can be passed for *in_batch* to indicate
> - * that the batched lookup starts from the beginning of the map.
> + * The parameter *in_batch* is the address of the first element in the batch to
> + * read. *out_batch* is an output parameter that should be passed as *in_batch*
> + * to subsequent calls to **bpf_map_lookup_batch()**. NULL can be passed for
> + * *in_batch* to indicate that the batched lookup starts from the beginning of
> + * the map. Both *in_batch* and *out_batch* must point to memory large enough to
> + * hold a single key, except for maps of type **BPF_MAP_TYPE_HASH**, for which
> + * the memory pointed to must be at least 4 bytes wide regardless of key size.
>    *
>    * The *keys* and *values* are output parameters which must point to memory large enough to
>    * hold *count* items based on the key and value size of the map *map_fd*. The *keys*
> @@ -226,7 +229,9 @@ LIBBPF_API int bpf_map_lookup_batch(int fd, void *in_batch, void *out_batch,
>    *
>    * @param fd BPF map file descriptor
>    * @param in_batch address of the first element in batch to read, can pass NULL to
> - * get address of the first element in *out_batch*
> + * get address of the first element in *out_batch*. If not NULL, must be large
> + * enough to hold a key. For **BPF_MAP_TYPE_HASH**, must be large enough to hold
> + * 4 bytes.
>    * @param out_batch output parameter that should be passed to next call as *in_batch*
>    * @param keys pointer to an array of *count* keys
>    * @param values pointer to an array large enough for *count* values