Re: [PATCH 1/9] kernel/api: introduce kernel API specification framework

[Jonathan Corbet <corbet@lwn.net>]

Sasha Levin <sashal@kernel.org> writes:

> Add a framework for formally documenting kernel APIs with inline
> specifications. This framework provides:
>
> - Structured API documentation with parameter specifications, return
>   values, error conditions, and execution context requirements
> - Runtime validation capabilities for debugging (CONFIG_KAPI_RUNTIME_CHECKS)
> - Export of specifications via debugfs for tooling integration
> - Support for both internal kernel APIs and system calls

So I'll confess I have only scanned over the implementation, but I have
some thoughts on the earlier stuff.

[...]

> diff --git a/Documentation/dev-tools/kernel-api-spec.rst b/Documentation/dev-tools/kernel-api-spec.rst
> new file mode 100644
> index 0000000000000..7c0c1694f1f4a
> --- /dev/null
> +++ b/Documentation/dev-tools/kernel-api-spec.rst
> @@ -0,0 +1,482 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +======================================
> +Kernel API Specification Framework
> +======================================
> +
> +:Author: Sasha Levin <sashal@kernel.org>
> +:Date: June 2025

Has it not changed since then?

> +.. contents:: Table of Contents
> +   :depth: 3
> +   :local:
> +
> +Introduction
> +============

[...]

> +Usage Guide
> +===========
> +
> +Basic API Specification
> +-----------------------
> +
> +API specifications are written as KAPI-annotated kerneldoc comments directly in
> +the source file, immediately preceding the function implementation. The ``kapi``
> +tool extracts these annotations to produce structured specifications.
> +
> +.. code-block:: c
> +
> +    /**
> +     * kmalloc - allocate kernel memory
> +     * @size: Number of bytes to allocate
> +     * @flags: Allocation flags (GFP_*)

Given that the text thus far has talked about user-space API validation,
it's a bit surprising to see an internal function used as an example.

Also, maybe it should be kmalloc_obj()?  <runs away>

> +     * context-flags: KAPI_CTX_PROCESS | KAPI_CTX_SOFTIRQ | KAPI_CTX_HARDIRQ
> +     * param-count: 2

param-count is two, but you only document one of them?

> +     * param: size
> +     *   type: KAPI_TYPE_UINT
> +     *   flags: KAPI_PARAM_IN
> +     *   constraint-type: KAPI_CONSTRAINT_RANGE
> +     *   range: 0, KMALLOC_MAX_SIZE
> +     *
> +     * error: ENOMEM, Out of memory
> +     *   desc: Insufficient memory available for the requested allocation.
> +     */

Honest question: can this be made a bit easier for people to create,
with less shift-key wear?  My biggest worry with a system like this is
that people won't take the time to create and maintain the entries, so
anything that would ease the task would help.  Is there an impediment to
something like:

  contexts: process, softirq, hardirq

  param: size
    type: uint, input
    constraint: range(0, KMALLOC_MAX_SIZE)

See what I'm getting at?  ISTM that your DSL could be made a bit less
verbose and shouty while being just as well defined, but perhaps I'm
missing something?

Even better, of course, would be to add a "description" field for each
parameter, and allow that rather than the @param description that
kerneldoc currently uses.  That would keep all the information together,
at the minor cost of adding another significant complication to the
kernel-doc script.  Mauro won't mind :)

> +    void *kmalloc(size_t size, gfp_t flags)
> +    {
> +        /* Implementation */
> +    }
> +
> +Alternatively, specifications can be defined using the ``DEFINE_KERNEL_API_SPEC``
> +macro for compiled-in specs that are stored in the ``.kapi_specs`` ELF section:
> +
> +.. code-block:: c
> +
> +    #include <linux/kernel_api_spec.h>
> +
> +    DEFINE_KERNEL_API_SPEC(sys_open)
> +    KAPI_DESCRIPTION("Open or create a file")
> +    KAPI_CONTEXT(KAPI_CTX_PROCESS | KAPI_CTX_SLEEPABLE)
> +    /* ... parameter, error, constraint definitions ... */
> +    KAPI_END_SPEC

So the reason for two completely separate mechanisms is not entirely
clear to me.  The kerneldoc variant is essentially documentation, while
the macro stuff is to be built into the executable?  What if you want
both?

It would be nice to only have one way if at all possible; I'm sure that
crossed your mind at some point :)  If there have to be two, having both
examples describe the same function would make the parallels more clear.

> +System Call Specification
> +-------------------------
> +
> +System calls are documented inline in the implementation file (e.g., ``fs/open.c``)
> +using KAPI-annotated kerneldoc comments. When ``CONFIG_KAPI_RUNTIME_CHECKS`` is
> +enabled, the ``SYSCALL_DEFINEx`` macros automatically look up the specification
> +and validate parameters before and after the syscall executes.
> +
> +IOCTL Specification
> +-------------------
> +
> +IOCTLs use the same annotation approach with additional structure field
> +specifications

This might be a really good place for an example

[...]

> +Usage Examples
> +--------------
> +
> +Query specific API::
> +
> +    $ cat /sys/kernel/debug/kapi/apis/kmalloc/specification
> +    API: kmalloc
> +    Version: 3.0
> +    Description: Allocate kernel memory
> +
> +    Parameters:
> +      [0] size (size_t, in): Number of bytes to allocate
> +          Range: 0 - 4194304
> +      [1] flags (flags, in): Allocation flags (GFP_*)
> +          Mask: 0x1ffffff

Ah, you do document that second parameter somewhere :)

> +    Returns: pointer - Pointer to allocated memory or NULL
> +
> +    Errors:
> +      ENOMEM: Out of memory
> +
> +    Context: process, softirq, hardirq
> +
> +    Side Effects:
> +      - Allocates memory from kernel heap

That part wasn't in your example

> +Export all specifications::
> +
> +    $ cat /sys/kernel/debug/kapi/export/all.json > kernel-apis.json
> +
> +Enable validation for specific API::
> +
> +    $ echo 1 > /sys/kernel/debug/kapi/apis/kmalloc/validate
> +
> +Performance Considerations
> +==========================
> +
> +Memory Overhead
> +---------------
> +
> +Each API specification consumes approximately 400-450KB of memory due to the
> +fixed-size arrays in ``struct kernel_api_spec``. With the current 4 syscall
> +specifications, total memory usage is approximately 1.7MB. Consider:

Ouch.

> +Documentation Generation
> +------------------------
> +
> +The framework exports specifications via debugfs that can be used
> +to generate documentation. Tools for automatic documentation generation
> +from specifications are planned for future development.

Documentation always comes last :)

Interesting stuff.

jon