Re: [PATCH bpf-next 00/17] Improve BPF syscall command documentation

["Toke Høiland-Jørgensen" <toke@redhat.com>]

Joe Stringer <joe@wand.net.nz> writes:

> From: Joe Stringer <joe@cilium.io>
>
> The state of bpf(2) manual pages today is not exactly ideal. For the
> most part, it was written several years ago and has not kept up with the
> pace of development in the kernel tree. For instance, out of a total of
> ~35 commands to the BPF syscall available today, when I pull the
> kernel-man-pages tree today I find just 6 documented commands: The very
> basics of map interaction and program load.

Yes indeed! Thank you for tackling this! :)

> In contrast, looking at bpf-helpers(7), I am able today to run one
> command[0] to fetch API documentation of the very latest eBPF helpers
> that have been added to the kernel. This documentation is up to date
> because kernel maintainers enforce documenting the APIs as part of
> the feature submission process. As far as I can tell, we rely on manual
> synchronization from the kernel tree to the kernel-man-pages tree to
> distribute these more widely, so all locations may not be completely up
> to date. That said, the documentation does in fact exist in the first
> place which is a major initial hurdle to overcome.
>
> Given the relative success of the process around bpf-helpers(7) to
> encourage developers to document their user-facing changes, in this
> patch series I explore applying this technique to bpf(2) as well.
> Unfortunately, even with bpf(2) being so out-of-date, there is still a
> lot of content to convert over. In particular, I've identified at least
> the following aspects of the bpf syscall which could individually be
> generated from separate documentation in the header:
> * BPF syscall commands
> * BPF map types
> * BPF program types
> * BPF attachment points

Does this also include program subtypes (AKA expected_attach_type?)

> Rather than tackle everything at once, I have focused in this series on
> the syscall commands, "enum bpf_cmd". This series is structured to first
> import what useful descriptions there are from the kernel-man-pages
> tree, then piece-by-piece document a few of the syscalls in more detail
> in cases where I could find useful documentation from the git tree or
> from a casual read of the code. Not all documentation is comprehensive
> at this point, but a basis is provided with examples that can be further
> enhanced with subsequent follow-up patches. Note, the series in its
> current state only includes documentation around the syscall commands
> themselves, so in the short term it doesn't allow us to automate bpf(2)
> generation; Only one section of the man page could be replaced. Though
> if there is appetite for this approach, this should be trivial to
> improve on, even if just by importing the remaining static text from the
> kernel-man-pages tree.
>
> Following that, the series enhances the python scripting around parsing
> the descriptions from the header files and generating dedicated
> ReStructured Text and troff output. Finally, to expose the new text and
> reduce the likelihood of having it get out of date or break the docs
> parser, it is added to the selftests and exposed through the kernel
> documentation web pages.
>
> At this point I'd like to put this out for comments. In my mind, the
> ideal eventuation of this work would be to extend kernel UAPI headers
> such that each of the categories I had listed above (commands, maps,
> progs, hooks) have dedicated documentation in the kernel tree, and that
> developers must update the comments in the headers to document the APIs
> prior to patch acceptance, and that we could auto-generate the latest
> version of the bpf(2) manual pages based on a few static description
> sections combined with the dynamically-generated output from the header.

I like the approach, and I don't think it's too onerous to require
updates to the documentation everywhere like we (as you note) already do
for helpers.

So with that, please feel free to add my enthusiastic:

Acked-by: Toke Høiland-Jørgensen <toke@redhat.com>