[PATCH v3 0/2] libbpf: API to partially consume items from ringbuffer

[PATCH v3 0/2] libbpf: API to partially consume items from ringbuffer

[Andrea Righi]

Introduce ring__consume_n() and ring_buffer__consume_n() API to
partially consume items from one (or more) ringbuffer(s).

This can be useful, for example, to consume just a single item or when
we need to copy multiple items to a limited user-space buffer from the
ringbuffer callback.

Practical example (where this API can be used):
https://github.com/sched-ext/scx/blob/b7c06b9ed9f72cad83c31e39e9c4e2cfd8683a55/rust/scx_rustland_core/src/bpf.rs#L217

See also:
https://lore.kernel.org/lkml/20240310154726.734289-1-andrea.righi@canonical.com/T/#u

[ Note: I haven't implemented the special case of n == 0 as "no limit",
because we still need to add a bunch of extra checks for INT_MAX, making
the code less readable and, from a performance perspective, it seems
that we may get more downsides than benefits, but I can try to look more
at this if you think it's worth it ]

v3:
 - rename ring__consume_max() -> ring__consume_n() and
   ring_buffer__consume_max() -> ring_buffer__consume_n()
 - add new API to a new 1.5.0 cycle
 - fixed minor nits / comments

v2:
 - introduce a new API instead of changing the callback's retcode
   behavior

Andrea Righi (2):
      libbpf: ringbuf: allow to consume up to a certain amount of items
      libbpf: Add ring__consume_n / ring_buffer__consume_n

 tools/lib/bpf/libbpf.h   | 12 ++++++++++
 tools/lib/bpf/libbpf.map |  6 +++++
 tools/lib/bpf/ringbuf.c  | 59 ++++++++++++++++++++++++++++++++++++++++--------
 3 files changed, 67 insertions(+), 10 deletions(-)

[PATCH v3 1/2] libbpf: ringbuf: allow to consume up to a certain amount of items

[Andrea Righi]

In some cases, instead of always consuming all items from ring buffers
in a greedy way, we may want to consume up to a certain amount of items,
for example when we need to copy items from the BPF ring buffer to a
limited user buffer.

This change allows to set an upper limit to the amount of items consumed
from one or more ring buffers.

Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
---
 tools/lib/bpf/ringbuf.c | 23 +++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git a/tools/lib/bpf/ringbuf.c b/tools/lib/bpf/ringbuf.c
index aacb64278a01..2c4031168413 100644
--- a/tools/lib/bpf/ringbuf.c
+++ b/tools/lib/bpf/ringbuf.c
@@ -231,7 +231,7 @@ static inline int roundup_len(__u32 len)
 	return (len + 7) / 8 * 8;
 }
 
-static int64_t ringbuf_process_ring(struct ring *r)
+static int64_t ringbuf_process_ring(struct ring *r, size_t n)
 {
 	int *len_ptr, len, err;
 	/* 64-bit to avoid overflow in case of extreme application behavior */
@@ -268,6 +268,9 @@ static int64_t ringbuf_process_ring(struct ring *r)
 			}
 
 			smp_store_release(r->consumer_pos, cons_pos);
+
+			if (cnt >= n)
+				goto done;
 		}
 	} while (got_new_data);
 done:
@@ -287,13 +290,15 @@ int ring_buffer__consume(struct ring_buffer *rb)
 	for (i = 0; i < rb->ring_cnt; i++) {
 		struct ring *ring = rb->rings[i];
 
-		err = ringbuf_process_ring(ring);
+		err = ringbuf_process_ring(ring, INT_MAX);
 		if (err < 0)
 			return libbpf_err(err);
 		res += err;
+		if (res > INT_MAX) {
+			res = INT_MAX;
+			break;
+		}
 	}
-	if (res > INT_MAX)
-		return INT_MAX;
 	return res;
 }
 
@@ -314,13 +319,15 @@ int ring_buffer__poll(struct ring_buffer *rb, int timeout_ms)
 		__u32 ring_id = rb->events[i].data.fd;
 		struct ring *ring = rb->rings[ring_id];
 
-		err = ringbuf_process_ring(ring);
+		err = ringbuf_process_ring(ring, INT_MAX);
 		if (err < 0)
 			return libbpf_err(err);
 		res += err;
+		if (res > INT_MAX) {
+			res = INT_MAX;
+			break;
+		}
 	}
-	if (res > INT_MAX)
-		return INT_MAX;
 	return res;
 }
 
@@ -375,7 +382,7 @@ int ring__consume(struct ring *r)
 {
 	int64_t res;
 
-	res = ringbuf_process_ring(r);
+	res = ringbuf_process_ring(r, INT_MAX);
 	if (res < 0)
 		return libbpf_err(res);
 
-- 
2.43.0

[PATCH v3 2/2] libbpf: Add ring__consume_n / ring_buffer__consume_n

[Andrea Righi]

Introduce a new API to consume items from a ring buffer, limited to a
specified amount, and return to the caller the actual number of items
consumed.

Link: https://lore.kernel.org/lkml/20240310154726.734289-1-andrea.righi@canonical.com/T
Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
---
 tools/lib/bpf/libbpf.h   | 12 ++++++++++++
 tools/lib/bpf/libbpf.map |  6 ++++++
 tools/lib/bpf/ringbuf.c  | 38 +++++++++++++++++++++++++++++++++++---
 3 files changed, 53 insertions(+), 3 deletions(-)

diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
index f88ab50c0229..4f775a6dcaa0 100644
--- a/tools/lib/bpf/libbpf.h
+++ b/tools/lib/bpf/libbpf.h
@@ -1293,6 +1293,7 @@ LIBBPF_API int ring_buffer__add(struct ring_buffer *rb, int map_fd,
 				ring_buffer_sample_fn sample_cb, void *ctx);
 LIBBPF_API int ring_buffer__poll(struct ring_buffer *rb, int timeout_ms);
 LIBBPF_API int ring_buffer__consume(struct ring_buffer *rb);
+LIBBPF_API int ring_buffer__consume_n(struct ring_buffer *rb, size_t n);
 LIBBPF_API int ring_buffer__epoll_fd(const struct ring_buffer *rb);
 
 /**
@@ -1367,6 +1368,17 @@ LIBBPF_API int ring__map_fd(const struct ring *r);
  */
 LIBBPF_API int ring__consume(struct ring *r);
 
+/**
+ * @brief **ring__consume_n()** consumes up to a requested amount of items from
+ * a ringbuffer without event polling.
+ *
+ * @param r A ringbuffer object.
+ * @param n Maximum amount of items to consume.
+ * @return The number of items consumed, or a negative number if any of the
+ * callbacks return an error.
+ */
+LIBBPF_API int ring__consume_n(struct ring *r, size_t n);
+
 struct user_ring_buffer_opts {
 	size_t sz; /* size of this struct, for forward/backward compatibility */
 };
diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
index 51732ecb1385..23d82bba021a 100644
--- a/tools/lib/bpf/libbpf.map
+++ b/tools/lib/bpf/libbpf.map
@@ -416,3 +416,9 @@ LIBBPF_1.4.0 {
 		btf__new_split;
 		btf_ext__raw_data;
 } LIBBPF_1.3.0;
+
+LIBBPF_1.5.0 {
+	global:
+		ring__consume_n;
+		ring_buffer__consume_n;
+} LIBBPF_1.4.0;
diff --git a/tools/lib/bpf/ringbuf.c b/tools/lib/bpf/ringbuf.c
index 2c4031168413..19cd34883011 100644
--- a/tools/lib/bpf/ringbuf.c
+++ b/tools/lib/bpf/ringbuf.c
@@ -277,6 +277,33 @@ static int64_t ringbuf_process_ring(struct ring *r, size_t n)
 	return cnt;
 }
 
+/* Consume available ring buffer(s) data without event polling, up to n
+ * records.
+ *
+ * Returns number of records consumed across all registered ring buffers (or
+ * n, whichever is less), or negative number if any of the callbacks return
+ * error.
+ */
+int ring_buffer__consume_n(struct ring_buffer *rb, size_t n)
+{
+	int64_t err, res = 0;
+	int i;
+
+	for (i = 0; i < rb->ring_cnt; i++) {
+		struct ring *ring = rb->rings[i];
+
+		err = ringbuf_process_ring(ring, n);
+		if (err < 0)
+			return libbpf_err(err);
+		res += err;
+		n -= err;
+
+		if (n == 0)
+			break;
+	}
+	return res;
+}
+
 /* Consume available ring buffer(s) data without event polling.
  * Returns number of records consumed across all registered ring buffers (or
  * INT_MAX, whichever is less), or negative number if any of the callbacks
@@ -378,17 +405,22 @@ int ring__map_fd(const struct ring *r)
 	return r->map_fd;
 }
 
-int ring__consume(struct ring *r)
+int ring__consume_n(struct ring *r, size_t n)
 {
-	int64_t res;
+	int res;
 
-	res = ringbuf_process_ring(r, INT_MAX);
+	res = ringbuf_process_ring(r, n);
 	if (res < 0)
 		return libbpf_err(res);
 
 	return res > INT_MAX ? INT_MAX : res;
 }
 
+int ring__consume(struct ring *r)
+{
+	return ring__consume_n(r, INT_MAX);
+}
+
 static void user_ringbuf_unmap_ring(struct user_ring_buffer *rb)
 {
 	if (rb->consumer_pos) {
-- 
2.43.0

Re: [PATCH v3 0/2] libbpf: API to partially consume items from ringbuffer

[Andrii Nakryiko]

On Fri, Apr 5, 2024 at 3:36 PM Andrea Righi <andrea.righi@canonical.com> wrote:
>
> Introduce ring__consume_n() and ring_buffer__consume_n() API to
> partially consume items from one (or more) ringbuffer(s).
>
> This can be useful, for example, to consume just a single item or when
> we need to copy multiple items to a limited user-space buffer from the
> ringbuffer callback.
>
> Practical example (where this API can be used):
> https://github.com/sched-ext/scx/blob/b7c06b9ed9f72cad83c31e39e9c4e2cfd8683a55/rust/scx_rustland_core/src/bpf.rs#L217
>
> See also:
> https://lore.kernel.org/lkml/20240310154726.734289-1-andrea.righi@canonical.com/T/#u
>
> [ Note: I haven't implemented the special case of n == 0 as "no limit",
> because we still need to add a bunch of extra checks for INT_MAX, making
> the code less readable and, from a performance perspective, it seems
> that we may get more downsides than benefits, but I can try to look more
> at this if you think it's worth it ]
>
> v3:
>  - rename ring__consume_max() -> ring__consume_n() and
>    ring_buffer__consume_max() -> ring_buffer__consume_n()
>  - add new API to a new 1.5.0 cycle

You also need to update Makefile, CI is failing right now ([0]).

Please also add a simple test to BPF selftests using the new API.
Other than that the changes look good.

  [0] https://github.com/kernel-patches/bpf/actions/runs/8576515495/job/23507610313#step:11:73

pw-bot: cr

>  - fixed minor nits / comments
>
> v2:
>  - introduce a new API instead of changing the callback's retcode
>    behavior
>
> Andrea Righi (2):
>       libbpf: ringbuf: allow to consume up to a certain amount of items
>       libbpf: Add ring__consume_n / ring_buffer__consume_n
>
>  tools/lib/bpf/libbpf.h   | 12 ++++++++++
>  tools/lib/bpf/libbpf.map |  6 +++++
>  tools/lib/bpf/ringbuf.c  | 59 ++++++++++++++++++++++++++++++++++++++++--------
>  3 files changed, 67 insertions(+), 10 deletions(-)

Re: [PATCH v3 0/2] libbpf: API to partially consume items from ringbuffer

[Andrea Righi]

On Fri, Apr 05, 2024 at 04:16:34PM -0700, Andrii Nakryiko wrote:
> On Fri, Apr 5, 2024 at 3:36 PM Andrea Righi <andrea.righi@canonical.com> wrote:
> >
> > Introduce ring__consume_n() and ring_buffer__consume_n() API to
> > partially consume items from one (or more) ringbuffer(s).
> >
> > This can be useful, for example, to consume just a single item or when
> > we need to copy multiple items to a limited user-space buffer from the
> > ringbuffer callback.
> >
> > Practical example (where this API can be used):
> > https://github.com/sched-ext/scx/blob/b7c06b9ed9f72cad83c31e39e9c4e2cfd8683a55/rust/scx_rustland_core/src/bpf.rs#L217
> >
> > See also:
> > https://lore.kernel.org/lkml/20240310154726.734289-1-andrea.righi@canonical.com/T/#u
> >
> > [ Note: I haven't implemented the special case of n == 0 as "no limit",
> > because we still need to add a bunch of extra checks for INT_MAX, making
> > the code less readable and, from a performance perspective, it seems
> > that we may get more downsides than benefits, but I can try to look more
> > at this if you think it's worth it ]
> >
> > v3:
> >  - rename ring__consume_max() -> ring__consume_n() and
> >    ring_buffer__consume_max() -> ring_buffer__consume_n()
> >  - add new API to a new 1.5.0 cycle
> 
> You also need to update Makefile, CI is failing right now ([0]).

Yep, I wasn't sure if should have bumped up the version as well
(in libbpf_version.h). I'll add that.

> 
> Please also add a simple test to BPF selftests using the new API.
> Other than that the changes look good.

Ok, will add a selftest.

Thanks,
-Andrea

> 
>   [0] https://github.com/kernel-patches/bpf/actions/runs/8576515495/job/23507610313#step:11:73
> 
> pw-bot: cr
> 
> >  - fixed minor nits / comments
> >
> > v2:
> >  - introduce a new API instead of changing the callback's retcode
> >    behavior
> >
> > Andrea Righi (2):
> >       libbpf: ringbuf: allow to consume up to a certain amount of items
> >       libbpf: Add ring__consume_n / ring_buffer__consume_n
> >
> >  tools/lib/bpf/libbpf.h   | 12 ++++++++++
> >  tools/lib/bpf/libbpf.map |  6 +++++
> >  tools/lib/bpf/ringbuf.c  | 59 ++++++++++++++++++++++++++++++++++++++++--------
> >  3 files changed, 67 insertions(+), 10 deletions(-)