Re: Packaging bpftool and libbpf: GitHub or kernel?

["Michal Suchánek" <msuchanek@suse.de>]

Hello,

On Fri, Apr 14, 2023 at 01:35:20AM +0100, Quentin Monnet wrote:
> Hi Shung-Hsi,
> 
> On Thu, 13 Apr 2023 at 10:23, Shung-Hsi Yu <shung-hsi.yu@suse.com> wrote:
> >
> > Hi,
> >
> > I'm considering switch to bpftool's mirror on GitHub for packaging (instead
> > of using the source found in kernel), but realize that it should goes
> > hand-in-hand with how libbpf is packaged, which eventually leads these
> > questions:
> >
> >   What is the suggested approach for packaging bpftool and libbpf?
> >   Which source is preferred, GitHub or kernel?
> 
> As you can see from the previous discussions, the suggested approach
> would be to package from the GitHub mirror, with libbpf and bpftool in
> sync.
> 
> My main argument for the mirror is that it keeps things simpler, and
> there's no need to deal with the rest of the kernel sources for these
> packages. Download from the mirrors, build, ship. But then I have
> limited experience at packaging for distros, and I can understand
> Toke's point of view, too. So ultimately, the call is yours.

Things get only ever more complex when submodules are involved.

> >   Does bpftool work on older kernel?
> 
> It should, although it's not perfect. Most features from current
> bpftool should work as expected on older kernels. However, if I
> remember correctly you would have trouble loading programs on pre-BTF
> kernels, because bpftool relies on libbpf >= 1.0 and only accepts map
> definitions with BTF info, and attempts to create these maps with BTF,
> which fails and blocks the load process.
> 
> But we're trying to keep backward-compatibility, so if we're only
> talking of kernels recent enough to support BTF, then I'd expect
> bpftool to work. If this is not the case, please report on this list.

It won't build:
https://lore.kernel.org/bpf/20220421003152.339542-3-alobakin@pm.me/

> > Our current approach is that we (openSUSE/SLES) essentially have two version
> > of libbpf: a public shared library that uses GitHub mirror as source, which
> > the general userspace sees and links to; and a private static library built
> > from kernel source used by bpftool, perf, resolve_btfids, selftests, etc.
> > A survey of other distros (Arch, Debian, Fedora, Ubuntu) suggest that they
> > took similar approach.
> 
> I would like them to reconsider this choice eventually. Sounds like
> for RHEL, this will be a tough sell :). At least, I'd love Ubuntu to
> have a real bpftool package instead of having to install
> linux-tools-common + linux-tools-generic, or to have distros in
> general (Ubuntu/Debian at least) stop compiling out the JIT
> disassembler, although this is not strictly related to the location of
> the sources. I've not found the time to reach out to package
> maintainers yet.
> 
> >
> > This approach means that the version of bpftool and libbpf are _not_ always
> > in sync[1], which I read may causes problem since libbpf and bpftool depends
> > on specific version of each other[2].
> 
> Whatever source you use, I would strongly recommend finding a way to
> keep both in sync. Libbpf has stabilised its API when reaching 1.0,
> but bpftool taps into some of the internals of the library. Features
> or new definitions are usually added at the same time to libbpf and
> bpftool, and if you get a mismatch between the two, you're taking
> risks to get build issues.

In other words no API exists.

> > Using the GitHub mirror of bpftool to package both libbpf and bpftool would
> > kept their version in sync, and was suggested[3]. Although the same could be
> > said if we switch back to packaging libbpf from kernel source, an additional
> > appeal for using GitHub mirrors is that it decouples bpftool from kernel,
> > making it easily upgradable and with a clearer changelog (the latter is
> > quite important for enterprise users) like libbpf.
> 
> Happy to read these changelogs I write are useful to someone :). Yes,
> this is my point.

Yes, publishing the changelog in a usable way relieves others of the
need to write thier own, usually with less understanding of the changes
in question. That's generally the idea of opensource - not endlessly
repeating what has already been done :)

> > The main concern with using GitHub mirror is that bpftool may be updated far
> > beyond the version that comes with the runtime kernel. AFAIK bpftool should
> > work on older kernel since CO-RE is used for built-in BPF iterators and the
> > underlying libbpf work on older kernel itself. Nonetheless, it would be nice
> > to get a confirmation from the maintainers.
> 
> As explained above - Mostly, it should work. Otherwise, we can look
> into fixing it.
> 
> As a side note, I'm open to suggestions/contributions to make life
> easier for packaging for the mirror. For example, Mahé and I recently
> added GitHub workflows to ship statically-built binaries for amd64 and
> arm64 on releases, as well as tarballs with both bpftool+libbpf
> sources. If there's something else to make packaging easier, I'm happy
> to talk about it.

Make it possible to build with system-installed libbpf. If it's released
it should have versioned dependency on a libbpf release, and libbpf from
that version on should be good enough to build it.

I tried copying those 'private' headers into a separate directory, and
link against static libbpf, and it seems to work. Of course, having
an actual API would be much better.

Thanks

Michal