Re: vmlinux.h overlap/conflict with network protocol definitions

[Stephen Hemminger <stephen@networkplumber.org>]

On Thu, 4 Apr 2024 14:45:04 -0700
Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote:

> > > This is a known issue as currently vmlinux.h does not support macros.
> > > There are some efforts by Edward Zingerman to support this but this has
> > > not done yet. At the same time, you could have a trivial header file
> > > like
> > > https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/progs/bpf_tracing_net.h
> > > to be used for bpf program and then your bpf program with vmlinux.h can
> > > have much easier CORE support.  
> >
> >
> > That is an example of header surgery which I would rather avoid having to carry
> > as long term technical debt baggage.
> >  
> 
> What's your ultimate goal? As Yonghong said, vmlinux.h is not
> compatible with other headers. So you have to pick either using
> vmlinux.h as a base + adding missing #define's (because those are not
> recorded in types, so can't be put into vmlinux.h), or not use
> vmlinux.h, use linux UAPI/internal headers and then use explicit CO-RE
> helpers/attributes to make your application CO-RE-relocatable.
> 
> It's not clear from your original email why exactly you wanted to
> switch to vmlinux.h in the first place.

Some backstory. There is not an existing TC filter for this, so the
original developer had the idea of using BPF to do it. 

The program is a small BPF program to implement a TC filter that looks at 
SKB and does mapping to queue based on L3 (or L3/L4) header. So not heavily dependent
on kernel data structure, but sk_buff is not necessarily stable; actual layout
depends on kernel config.

But the evolution of BPF has made the old code unusable. I am trying to get it working again,
and cleanup to modern BPF by using BPF skeleton code.

Luca was one who had the suggestion about vmlinux.

> Using bpftool to generate the header at build time is a bit icky,
> because it will look at sysfs on the build system, which is from the
> running kernel. But a build system's kernel might be some ancient LTS,
> and even be a completely different kconfig/build/distro from the actual
> runtime one.
> 
> We have ran in the same problem in systemd recently, and the solution
> is to have distros publish the vmlinux.h together with the kernel
> image/headers, that way we can rely on the fact that by build-depending
> on the right kernel package we get exactly the generated vmlinux.h that
> we want. This has already happened in Centos, Debian, Fedora and Arch,
> and I am trying to get Ubuntu onboard too.
> 
> The annoying thing is that every distro packages differently, so the
> path needs to be configurable with a meson option.
> 
> Feel free to pilfer the systemd meson glue:
> 
> https://github.com/systemd/systemd/pull/26826/commits/d917079e7e320aa281fc4ad6f8073b0814b9cb13
> 
> It's of course file to go to the runtime kernel if no vmlinux.h is
> specified, as a fallback, which is going to be useful for developers
> machines.
> 
> -- 
> Kind regards,
> Luca Boccassi