* Re: Establishing /usr/lib/bpf as a convention for eBPF bytecode files? [not found] <87fthtlotk.fsf@toke.dk> @ 2019-12-10 1:40 ` Alexei Starovoitov 2019-12-10 9:10 ` Jesper Dangaard Brouer 0 siblings, 1 reply; 3+ messages in thread From: Alexei Starovoitov @ 2019-12-10 1:40 UTC (permalink / raw) To: Toke Høiland-Jørgensen Cc: bpf, Jiri Olsa, Jesper Dangaard Brouer, daniel, netdev On Mon, Dec 09, 2019 at 12:29:27PM +0100, Toke Høiland-Jørgensen wrote: > Hi everyone > > As you have no doubt noticed, we have started thinking about how to > package eBPF-related applications in distributions. As a part of this, > I've been thinking about what to recommend for applications that ship > pre-compiled BPF byte-code files. > > The obvious place to place those would be somewhere in the system > $LIBDIR (i.e., /usr/lib or /usr/lib64, depending on the distro). But > since BPF byte code is its own binary format, different from regular > executables, I think having a separate path to put those under makes > sense. So I'm proposing to establish a convention that pre-compiled BPF > programs be installed into /usr/lib{,64}/bpf. > > This would let users discover which BPF programs are shipped on their > system, and it could be used to discover which package loaded a > particular BPF program, by walking the directory to find the file a > loaded program came from. It would not work for dynamically-generated > bytecode, of course, but I think at least some applications will end up > shipping pre-compiled bytecode files (we're doing that for xdp-tools, > for instance). > > As I said, this would be a convention. We're already using it for > xdp-tools[0], so my plan is to use that as the "first mover", try to get > distributions to establish the path as a part of their filesystem > layout, and then just try to encourage packages to use it. Hopefully it > will catch on. > > Does anyone have any objections to this? Do you think it is a complete > waste of time, or is it worth giving it a shot? :) What will be the name of file/directory ? What is going to be the mechanism to clean it up? What will be stored in there? Just .o files ? libbcc stores original C and rewritten C in /var/tmp/bcc/bpf_prog_TAG/ It was useful for debugging. Since TAG is used as directory name reloading the same bcc script creates the same dir and /var/tmp periodically gets cleaned by reboot. Installing bpf .o into common location feels useful. Not sure though how you can convince folks to follow such convention. That was the main problem with libbcc. Not everything is using that lib. So teaching folks who debug bpf in production to look into /var/tmp/bcc wasn't enough. 'bpftool p s' is still the main mechanism. Some C++ services embed bpf .o as part of x86 binary and that binary is installed. They wouldn't want to split bpf .o into separate file since it will complicate dependency management, risk conflicts, etc. Just food for thought. ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Establishing /usr/lib/bpf as a convention for eBPF bytecode files? 2019-12-10 1:40 ` Establishing /usr/lib/bpf as a convention for eBPF bytecode files? Alexei Starovoitov @ 2019-12-10 9:10 ` Jesper Dangaard Brouer 2019-12-10 10:26 ` Toke Høiland-Jørgensen 0 siblings, 1 reply; 3+ messages in thread From: Jesper Dangaard Brouer @ 2019-12-10 9:10 UTC (permalink / raw) To: Alexei Starovoitov Cc: Toke Høiland-Jørgensen, bpf, Jiri Olsa, daniel, netdev, brouer On Mon, 9 Dec 2019 17:40:19 -0800 Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote: > On Mon, Dec 09, 2019 at 12:29:27PM +0100, Toke Høiland-Jørgensen wrote: > > Hi everyone > > > > As you have no doubt noticed, we have started thinking about how to > > package eBPF-related applications in distributions. As a part of this, > > I've been thinking about what to recommend for applications that ship > > pre-compiled BPF byte-code files. > > > > The obvious place to place those would be somewhere in the system > > $LIBDIR (i.e., /usr/lib or /usr/lib64, depending on the distro). But > > since BPF byte code is its own binary format, different from regular > > executables, I think having a separate path to put those under makes > > sense. So I'm proposing to establish a convention that pre-compiled BPF > > programs be installed into /usr/lib{,64}/bpf. > > > > This would let users discover which BPF programs are shipped on their > > system, and it could be used to discover which package loaded a > > particular BPF program, by walking the directory to find the file a > > loaded program came from. It would not work for dynamically-generated > > bytecode, of course, but I think at least some applications will end up > > shipping pre-compiled bytecode files (we're doing that for xdp-tools, > > for instance). > > > > As I said, this would be a convention. We're already using it for > > xdp-tools[0], so my plan is to use that as the "first mover", try to get > > distributions to establish the path as a part of their filesystem > > layout, and then just try to encourage packages to use it. Hopefully it > > will catch on. > > > > Does anyone have any objections to this? Do you think it is a complete > > waste of time, or is it worth giving it a shot? :) > > What will be the name of file/directory ? > What is going to be the mechanism to clean it up? > What will be stored in there? Just .o files ? > > libbcc stores original C and rewritten C in /var/tmp/bcc/bpf_prog_TAG/ > It was useful for debugging. Since TAG is used as directory name > reloading the same bcc script creates the same dir and /var/tmp > periodically gets cleaned by reboot. > > Installing bpf .o into common location feels useful. Not sure though > how you can convince folks to follow such convention. I imagine the files under /usr/lib{,64}/bpf/ will be pre-compiled binaries (fairly static file). These will be delivered together with the distro RPM file. The distro will detect/enforce that two packages cannot use the same name for bpf .o files. I see these files as part of the RPM software package binary files. In my opinion, this means/imply that the BPF application should not update these files runtime, because different consistency checksum tools should be able to verify that this files comes from the original RPM file. More below on dynamic files. > That was the main problem with libbcc. Not everything is using that lib. > So teaching folks who debug bpf in production to look into /var/tmp/bcc > wasn't enough. 'bpftool p s' is still the main mechanism. > Some C++ services embed bpf .o as part of x86 binary and that binary > is installed. They wouldn't want to split bpf .o into separate file > since it will complicate dependency management, risk conflicts, etc. > Just food for thought. I see three different types of BPF-object files, which belong in different places (suggestion in parentheses): 1. Pre-compiled binaries via RPM. (/usr/lib? [1]) 2. Application "startup" compiled "cached" BPF-object (/var/cache? [2]). 3. Runtime dynamic compiled BPF-objects short lived (/run? [3]) You can follow the links below, to see if match descriptions in the Filesystem Hierarchy Standard[4]. I think that filetype 1 + 2 makes sense to store in files. For filetype 3 (the highly dynamic runtime re-compiled files) I'm not sure it makes sense to store those in any central place. Applications could use /run/application-name/, but it will be a pain to deal with filename-clashes. As Alexei brings up cleanup; /run/ is cleared at the beginning of the boot process[3]. For fileytpe 2, I suggest /var/cache/bpf/, but with an additional application name as a subdir, this is to avoid name clashes (which then becomes the applications responsibility with in its own dir). Links: [1] /usr/lib: "Libraries for programming and packages" https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04s06.html [2] /var/cache: "Application cache data" https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s05.html [3] /run: "Run-time variable data" https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s15.html [4] https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html -- Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Establishing /usr/lib/bpf as a convention for eBPF bytecode files? 2019-12-10 9:10 ` Jesper Dangaard Brouer @ 2019-12-10 10:26 ` Toke Høiland-Jørgensen 0 siblings, 0 replies; 3+ messages in thread From: Toke Høiland-Jørgensen @ 2019-12-10 10:26 UTC (permalink / raw) To: Jesper Dangaard Brouer, Alexei Starovoitov Cc: bpf, Jiri Olsa, daniel, netdev, brouer Jesper Dangaard Brouer <brouer@redhat.com> writes: > On Mon, 9 Dec 2019 17:40:19 -0800 > Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote: > >> On Mon, Dec 09, 2019 at 12:29:27PM +0100, Toke Høiland-Jørgensen wrote: >> > Hi everyone >> > >> > As you have no doubt noticed, we have started thinking about how to >> > package eBPF-related applications in distributions. As a part of this, >> > I've been thinking about what to recommend for applications that ship >> > pre-compiled BPF byte-code files. >> > >> > The obvious place to place those would be somewhere in the system >> > $LIBDIR (i.e., /usr/lib or /usr/lib64, depending on the distro). But >> > since BPF byte code is its own binary format, different from regular >> > executables, I think having a separate path to put those under makes >> > sense. So I'm proposing to establish a convention that pre-compiled BPF >> > programs be installed into /usr/lib{,64}/bpf. >> > >> > This would let users discover which BPF programs are shipped on their >> > system, and it could be used to discover which package loaded a >> > particular BPF program, by walking the directory to find the file a >> > loaded program came from. It would not work for dynamically-generated >> > bytecode, of course, but I think at least some applications will end up >> > shipping pre-compiled bytecode files (we're doing that for xdp-tools, >> > for instance). >> > >> > As I said, this would be a convention. We're already using it for >> > xdp-tools[0], so my plan is to use that as the "first mover", try to get >> > distributions to establish the path as a part of their filesystem >> > layout, and then just try to encourage packages to use it. Hopefully it >> > will catch on. >> > >> > Does anyone have any objections to this? Do you think it is a complete >> > waste of time, or is it worth giving it a shot? :) >> >> What will be the name of file/directory ? >> What is going to be the mechanism to clean it up? >> What will be stored in there? Just .o files ? >> >> libbcc stores original C and rewritten C in /var/tmp/bcc/bpf_prog_TAG/ >> It was useful for debugging. Since TAG is used as directory name >> reloading the same bcc script creates the same dir and /var/tmp >> periodically gets cleaned by reboot. >> >> Installing bpf .o into common location feels useful. Not sure though >> how you can convince folks to follow such convention. > > I imagine the files under /usr/lib{,64}/bpf/ will be pre-compiled > binaries (fairly static file). These will be delivered together with > the distro RPM file. The distro will detect/enforce that two packages > cannot use the same name for bpf .o files. Yes, that was my intention. Packages can choose whether to create a subdirectory, or just dump files in /usr/lib{,64}/bpf (this is similar to /usr/lib). > I see three different types of BPF-object files, which belong in > different places (suggestion in parentheses): > > 1. Pre-compiled binaries via RPM. (/usr/lib? [1]) > 2. Application "startup" compiled "cached" BPF-object (/var/cache? [2]). > 3. Runtime dynamic compiled BPF-objects short lived (/run? [3]) > > You can follow the links below, to see if match descriptions in > the Filesystem Hierarchy Standard[4]. > > I think that filetype 1 + 2 makes sense to store in files. For > filetype 3 (the highly dynamic runtime re-compiled files) I'm not > sure it makes sense to store those in any central place. Applications > could use /run/application-name/, but it will be a pain to deal with > filename-clashes. As Alexei brings up cleanup; /run/ is cleared at the > beginning of the boot process[3]. > > For fileytpe 2, I suggest /var/cache/bpf/, but with an additional > application name as a subdir, this is to avoid name clashes (which then > becomes the applications responsibility with in its own dir). /var/cache/bpf seems reasonable, let's go with that. My plan is to try to get the directories established in distribution packaging ('filesystem' on Arch and Fedora, 'base-files' on Debian); this will mean the directories are already there on people's systems, which hopefully will encourage developers to use them. And then we can try to provide a bit more nudging through the distribution packaging. -Toke ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2019-12-10 10:26 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <87fthtlotk.fsf@toke.dk>
2019-12-10 1:40 ` Establishing /usr/lib/bpf as a convention for eBPF bytecode files? Alexei Starovoitov
2019-12-10 9:10 ` Jesper Dangaard Brouer
2019-12-10 10:26 ` Toke Høiland-Jørgensen
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).