Re: [PATCH bpf-next 1/3] bpf: Add __bpf_kfunc tag for marking kernel functions as kfuncs

[David Vernet <void@manifault.com>]

On Mon, Jan 09, 2023 at 05:38:15PM +0530, Kumar Kartikeya Dwivedi wrote:
> On Mon, Jan 09, 2023 at 04:47:54AM IST, Alexei Starovoitov wrote:
> > On Fri, Jan 6, 2023 at 6:09 PM David Vernet <void@manifault.com> wrote:
> > >
> > > On Fri, Jan 06, 2023 at 05:04:02PM -0800, Alexei Starovoitov wrote:
> > > > On Fri, Jan 6, 2023 at 11:51 AM David Vernet <void@manifault.com> wrote:
> > > > >
> > > > > kfuncs are functions defined in the kernel, which may be invoked by BPF
> > > > > programs. They may or may not also be used as regular kernel functions,
> > > > > implying that they may be static (in which case the compiler could e.g.
> > > > > inline it away), or it could have external linkage, but potentially be
> > > > > elided in an LTO build if a function is observed to never be used, and
> > > > > is stripped from the final kernel binary.
> > > > >
> > > > > We therefore require some convenience macro that kfunc developers can
> > > > > use just add to their kfuncs, and which will prevent all of the above
> > > > > issues from happening. This is in contrast with what we have today,
> > > > > where some kfunc definitions have "noinline", some have "__used", and
> > > > > others are static and have neither.
> > > > >
> > > > > In addition to providing the obvious correctness benefits, having such a
> > > > > macro / tag also provides the following advantages:
> > > > >
> > > > > - Giving an easy and intuitive thing to query for if people are looking
> > > > >   for kfuncs, as Christoph suggested at the kernel maintainers summit
> > > > >   (https://lwn.net/Articles/908464/). This is currently possible by
> > > > >   grepping for BTF_ID_FLAGS(func, but having something more self
> > > > >   describing would be useful as well.
> > > > >
> > > > > - In the future, the tag can be expanded with other useful things such
> > > > >   as the ability to suppress -Wmissing-prototype for the kfuncs rather
> > > > >   than requiring developers to surround the kfunc with __diags to
> > > > >   suppress the warning (this requires compiler support that as far as I
> > > > >   know currently does not exist).
> > > >
> > > > Have you considered doing bpf_kfunc_start/bpf_kfunc_end ?
> > > > The former would include:
> > > > __diag_push(); __diag_ignore_all(); __used noinline
> > >
> > > Yeah that's certainly an option. The downside is that all functions
> > > within scope of the __diag_push() will be affected, and sometimes we mix
> > > kfuncs with non-kfuncs (including e.g. static helper functions that are
> > > used by the kfuncs themselves). -Wmissing-prototypes isn't a big deal,
> > > but __used and noinline are kind of unfortunate. Not a big deal though,
> > > it'll just result in a few extra __bpf_kfuncs_start() and
> > > __bpf_kfuncs_end() sprinkled throughout to avoid them being included.
> > > The upside is of course that we can get rid of the __diag_push()'es we
> > > currently have to prevent -Wmissing-prototypes.
> >
> > I meant to use bpf_kfunc_start/bpf_kfunc_end around every kfunc.
> > Ideally bpf_kfunc_start would be on the same line as func proto
> > for nice grepping.
> > Maybe it's an overkill.
> > Maybe 3 macroses then?
> > bpf_kfunc_start to hide __diag
> > bpf_kfunc on the proto line
> > bpf_kfunc_end to finish __diag_pop

Ah, I see. Hmm, I guess this is better than what we have now, but is
still a lot of macros and boilerplate which IMO is a sign we're not
going in quite the right direction. I don't really have a better
suggestion at this point, though I do like Kumar's suggestion below.

> There's also the option of doing this:
> 
> #define BPF_KFUNC(proto) proto; __used noinline proto
> 
> BPF_KFUNC(void kfunc(arg1, arg2)) {
> 	...
> }
> 
> No need to disable the warning with diag push/pop, just put a declaration before
> the definition to silence the compiler. The only awkward part is entire function
> prototype becoming a macro argument (unlike the common case void MACRO(...)) but
> it becomes less noisy and easy to grep as well.

If nobody would come after us with pitchforks for this, IMO this is the
most user-friendly option and what I would vote for. It doesn't seem
like this is violating anything in [0]?

[0]: https://www.kernel.org/doc/html/latest/process/coding-style.html#macros-enums-and-rtl