BPF List
 help / color / mirror / Atom feed
From: Philo Lu <lulie@linux.alibaba.com>
To: bpf@vger.kernel.org
Cc: song@kernel.org, andrii@kernel.org, ast@kernel.org,
	Daniel Borkmann <daniel@iogearbox.net>,
	xuanzhuo@linux.alibaba.com, dust.li@linux.alibaba.com,
	guwen@linux.alibaba.com, alibuda@linux.alibaba.com,
	hengqi@linux.alibaba.com
Subject: Question about bpf perfbuf/ringbuf: pinned in backend with overwriting
Date: Thu, 7 Dec 2023 21:15:18 +0800	[thread overview]
Message-ID: <3dd9114c-599f-46b2-84b9-abcfd2dcbe33@linux.alibaba.com> (raw)

Hi all. I have a question when using perfbuf/ringbuf in bpf. I will 
appreciate it if you give me any advice.

Imagine a simple case: the bpf program output a log (some tcp 
statistics) to user every time a packet is received, and the user 
actively read the logs if he wants. I do not want to keep a user process 
alive, waiting for outputs of the buffer. User can read the buffer as 
need. BTW, the order does not matter.

To conclude, I hope the buffer performs like relayfs: (1) no need for 
user process to receive logs, and the user may read at any time (and no 
wakeup would be better); (2) old data can be overwritten by new ones.

Currently, it seems that perfbuf and ringbuf cannot satisfy both: (i) 
ringbuf: only satisfies (1). However, if data arrive when the buffer is 
full, the new data will be lost, until the buffer is consumed. (ii) 
perfbuf: only satisfies (2). But user cannot access the buffer after the 
process who creates it (including perf_event.rb via mmap) exits. 
Specifically, I can use BPF_F_PRESERVE_ELEMS flag to keep the 
perf_events, but I do not know how to get the buffer again in a new process.

In my opinion, this can be solved by either of the following: (a) add 
overwrite support in ringbuf (maybe a new flag for reserve), but we have 
to address synchronization between kernel and user, especially under 
variable data size, because when overwriting occurs, kernel has to 
update the consumer posi too; (b) implement map_fd_sys_lookup_elem for 
perfbuf to expose fds to user via map_lookup_elem syscall, and a 
mechanism is need to preserve perf_event->rb when process exits 
(otherwise the buffer will be freed by perf_mmap_close). I am not sure 
if they are feasible, and which is better. If not, perhaps we can 
develop another mechanism to achieve this?

             reply	other threads:[~2023-12-07 13:15 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-12-07 13:15 Philo Lu [this message]
2023-12-07 14:48 ` Question about bpf perfbuf/ringbuf: pinned in backend with overwriting Alan Maguire
2023-12-08 22:32   ` Andrii Nakryiko
2023-12-11 12:39     ` Philo Lu
2023-12-13 23:35       ` Andrii Nakryiko
2023-12-15 10:10         ` Philo Lu
2023-12-15 22:39           ` Andrii Nakryiko
2023-12-16  8:50             ` Dmitry Vyukov
2023-12-18 12:58               ` Philo Lu
2023-12-19 19:25               ` Andrii Nakryiko
2023-12-19  6:23         ` Shung-Hsi Yu
2023-12-19 13:38           ` Steven Rostedt
2023-12-19 17:01             ` Alexei Starovoitov
2023-12-19 17:28             ` Steven Rostedt
2023-12-21 13:00             ` Philo Lu
2023-12-21 14:49               ` Steven Rostedt
2023-12-22 12:25                 ` Philo Lu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3dd9114c-599f-46b2-84b9-abcfd2dcbe33@linux.alibaba.com \
    --to=lulie@linux.alibaba.com \
    --cc=alibuda@linux.alibaba.com \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=daniel@iogearbox.net \
    --cc=dust.li@linux.alibaba.com \
    --cc=guwen@linux.alibaba.com \
    --cc=hengqi@linux.alibaba.com \
    --cc=song@kernel.org \
    --cc=xuanzhuo@linux.alibaba.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox