Re: [PATCH RFC 04/20] Documentation: ctlv: Add userspace ABI

[Jonathan Cameron <jic23@kernel.org>]

On Thu, 15 Oct 2026 16:50:37 +0200
Jiri Pirko <jiri@resnulli.us> wrote:

> From: Jiri Pirko <jiri@nvidia.com>
> 
> Describe the CTLV transport, discovery, wire format and userspace
> compatibility rules.
> 
> Assisted-by: Cursor: GPT-5.6 Sol
> Assisted-by: Cursor: Claude Opus 5.5
> Signed-off-by: Jiri Pirko <jiri@nvidia.com>
In general looks good to me - this let me get my head around
what was going on nicely (I think!).  A few trivial comments.
> ---
>  Documentation/userspace-api/ctlv.rst  | 507 ++++++++++++++++++++++++++
>  Documentation/userspace-api/index.rst |   1 +
>  MAINTAINERS                           |   1 +
>  3 files changed, 509 insertions(+)
>  create mode 100644 Documentation/userspace-api/ctlv.rst
> 
> diff --git a/Documentation/userspace-api/ctlv.rst b/Documentation/userspace-api/ctlv.rst
> new file mode 100644
> index 000000000000..5ebca3d4d94a
> --- /dev/null
> +++ b/Documentation/userspace-api/ctlv.rst

> +Discovery
> +---------
> +
> +Both uevents, addition and removal, carry the family chain as numbered
> +properties::
> +
> +    CTLV_FAMILY=ctlv-accelerator-foo
> +    CTLV_COMPATIBLE_0=ctlv
> +    CTLV_COMPATIBLE_1=ctlv-accelerator
> +    CTLV_COMPATIBLE_2=ctlv-accelerator-foo
> +    CTLV_COMPATIBLE_N=3
> +
> +``CTLV_FAMILY`` is the family the device answers as, the numbered properties
> +are its ancestry from the root to that family, and
> +``CTLV_COMPATIBLE_N`` is how many there are. An index never moves: changing a
> +family's parent is an ABI
> +break, adding a descendant only appends.

Odd line wrap.

> +
> +A program that wants every device, including ones appearing while it starts,
> +must listen before it looks, which closes the window between enumerating and
> +subscribing::

> +A whole message with every offset, for a request of one integer and one nest
> +carrying a string and a further nest::
> +
> +    offset  attribute                       len  flags   contents
> +    ------  ------------------------------  ---  ------  ----------------
> +         0  root                            120  NESTED  attr_id = 0
> +        16  .. chunks                        24  -       8-byte integer
> +        40  .. chunk                         80  NESTED
> +        56     .. label                      19  -       "ab" and its NUL
> +        80     .. item                       40  NESTED
> +        96        .. value                   24  -       8-byte integer
> +       120  end of message
> +
> +    +--------------------------------------------------------------+ 0
> +    | root: len = 120, flags = NESTED, attr_id = 0                 |
> +    +--------------------------------------------------------------+ 16

Diagram wise, should chunks and chunk be shown nested inside root like your
.. above is indicating?

> +    | chunks: len = 24 | payload: 8 bytes                          |
> +    +--------------------------------------------------------------+ 40
> +    | chunk: len = 80, flags = NESTED                              |
> +    |   +--------------------------------------------------------+ | 56
> +    |   | label: len = 19 | payload: 'a' 'b' 0 | 5 bytes padding | |
> +    |   +--------------------------------------------------------+ | 80
> +    |   | item: len = 40, flags = NESTED                         | |
> +    |   |   +--------------------------------------------------+ | | 96
> +    |   |   | value: len = 24 | payload: 8 bytes               | | |
> +    |   |   +--------------------------------------------------+ | | 120
> +    +--------------------------------------------------------------+
> +
> +``label.len`` is 19: its header and three payload bytes, not the five bytes of
> +padding after it. ``chunk`` counts those five, its ``len`` of 80 being its
> +header plus ``CTLV_ALIGN(19)`` plus ``CTLV_ALIGN(40)``. A walk over a nest's
> +children advances by ``CTLV_ALIGN(child.len)``, a walk of one payload by
> +``child.len``.

...

> +
> +Request occurrence order
> +------------------------
> +
> +Occurrences of one attribute ID must be contiguous within each request root or
> +nest. ``A A B`` is valid, while ``A B A`` is not. An ignored unknown attribute
> +is not an occurrence of any known attribute, but it ends the preceding run, so

An ignored unknown attribute, whilst not understood, ends the preceding run, so ..

(or something like that to avoid saying unknown is not known ;))

> +``A ignored-unknown A`` is also invalid. Different IDs otherwise have no
> +required numeric or schema order.
> +

> +
> +Executing an operation
> +======================
> +
> +One ioctl executes every operation::
> +
> +    struct ctlv_op_exec {
> +            __aligned_u64 op_id;
> +            __aligned_u64 request;      /* struct ctlv_attr __user * */
> +            __aligned_u64 reply;        /* struct ctlv_attr __user * */
> +            __u32 request_len;
> +            __u32 reply_len;
> +            __aligned_u64 generation;   /* queries only */
> +            __aligned_u64 cursor;       /* queries only */
> +            __aligned_u64 reserved[2];  /* must be zero */
> +    };
> +
> +    #define CTLV_IOC_OP_EXEC _IOWR(0xBB, 0x00, struct ctlv_op_exec)
> +
> +``request_len`` is how many bytes the kernel may read at ``request`` and
> +``reply_len`` how many it may write at ``reply``. They are copy boundaries, not
> +message lengths: a root's ``len`` may be shorter than ``request_len``, the
> +bytes after it being ignored, and may not be longer. 

I don't really mind this but why have this relaxed? Whatever sent this request
just filled in the root len so why wouldn't it use the same value for this structure?


>``reply_len`` is never
> +overwritten; the length written is the reply root's ``len``. The pointers are
> +fixed-width so a 32-bit and a 64-bit process pass the same structure, and in
> +place of a size field the ioctl number encodes the size and ``reserved``, which
> +must be zero, covers compatible growth.
> +
> +Every action and query carries a physical request root, including an empty
> +request. ``request_len`` is therefore at least one attr header and at most the
> +1 MiB transport limit.

Why this particular limit?  Any thought to making that discoverable rather
than fixed? (with a minimum to allow the operation for the length to be discovered).

Maybe it is better to just have fixed size.  I wonder if ultimately we
may end up with cases where the tlv is just passed on to firmware / device
(because it is nice and convenient!) and having to split or marshall
multiple messages would be a pain.

> The ``request-max-len`` reported by ``ops-dump`` is the
> +maximum request composed from the known schema; a newer caller may exceed it
> +with well-formed unknown attrs marked ``IGNORE_IF_UNKNOWN``, up to the
> +transport limit.

> +
> +Events
> +======
> +
> +An event reaches a file description that asked for it, and only that one:
> +
> +1. subscribe with the ``subscriptions-add`` action, naming operation IDs;
> +2. read records with ``read()``, one complete record per call;
> +3. unsubscribe with ``subscriptions-remove``.
> +
> +``poll()`` reports readability when a record is queued. Each record is a
> +header followed by exactly one root attribute::
> +
> +    struct ctlv_event_hdr {
> +            __u32 length;             /* header + aligned root */
> +            __u32 flags;              /* zero */
> +            __aligned_u64 op_id;      /* the event op that produced it */
> +            __aligned_u64 sequence;   /* per file description */
> +    };
> +
> +One ``read()`` returns one record, the header and the aligned root::
> +
> +    +---------------------------------------------------------------+ 0
> +    | length = 24 + CTLV_ALIGN(root.len)     | flags = 0            |
> +    +---------------------------------------------------------------+ 8
> +    | op_id: the event operation that produced this record          |
> +    +---------------------------------------------------------------+ 16
> +    | sequence: counted per file description                        |
> +    +---------------------------------------------------------------+ 24
> +    | root: len, flags = NESTED, attr_id = 0                        |
> +    |   the event's attributes, exactly as a reply carries them      |

stray space.

> +    +---------------------------------------------------------------+ length