From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 613283C061F for ; Thu, 15 Oct 2026 14:51:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.53 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792075884; cv=none; b=KSgyqmsOih+gkct/6ihR0vahY7lm6DJMDJggwmgqvTA0NgCoFPmSi0gpa32mG4rvKoMiCqnhB3xbD599eLbKwyQes8pAhL8AMMMwAr51WkD36RbOoJZ+4g9Oarm3gIl8ZL/qRndKa9S+6U5dxga/rLIjD0TUzU1G4PyQQqC+fvw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792075884; c=relaxed/simple; bh=dwWYyxevTdfeguv5WPvwvqF/DBlLqAs1tUw+DCELIVY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=m0Xv/mDpOWo++LGdZKIgEG3mrkWpH5FqeV/JDv9CCB2ftK0JiOn66BJL4mdAKqgHYIt/xe4mYsZ+i1aZrel99O2j8N5Q69HNnwlsD8JHnv2pZTkN2/L7Zquthbj4EL2ix4IE6CxdHCp64iouFzddsSJJo+h1FVrFBZvj1YUkkQI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=resnulli.us; spf=none smtp.mailfrom=resnulli.us; dkim=pass (2048-bit key) header.d=resnulli-us.20251104.gappssmtp.com header.i=@resnulli-us.20251104.gappssmtp.com header.b=nxkOUjR/; arc=none smtp.client-ip=209.85.218.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=resnulli.us Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=resnulli.us Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=resnulli-us.20251104.gappssmtp.com header.i=@resnulli-us.20251104.gappssmtp.com header.b="nxkOUjR/" Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-c20ce3c118aso11519866b.0 for ; Thu, 15 Oct 2026 07:51:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=resnulli-us.20251104.gappssmtp.com; s=20251104; t=1792075877; x=1792680677; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ZHoZuY34Xeq3GA+nZ2Q4muvn71l6McdVsJ+5HzjU5A0=; b=nxkOUjR/q62uwtc2KI3q2lYF063tzHwkOflMcXzigcd06tP2DrCd7RGIpfMS1P7+AR oKEbHzJ8MDAO0lSynaJVFSF/x11YpcdDj9bti9r5jUzyPumgFwdg/iH86l705RX1+I7L SHoAkOdGptf7xO0EGGDYGqdpXtB2OkXc3/y3V98pFjiqmDl8KFqwv3OWo0rhk00B5WtZ 3EypEvMHrAvbQQz7JXquB/1Th0kRAjpxNdu7yIpvsMECTA+QRVJirc7GsmNVm3x5p1JS TXS8yW2owXmk3zhncMg0+VyEfyqgIQYK9Gz74QX59Mi0k5yrpiNXDAWnBoxRgYJnmNdJ LpQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1792075877; x=1792680677; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=ZHoZuY34Xeq3GA+nZ2Q4muvn71l6McdVsJ+5HzjU5A0=; b=LE+tOIVID21xRmF1Czd1f9+IXb5nI/VDjADiI5nC0Uqryym/i6yLqXFgGbJoJoKurs E3fBO3FPUTwd7n+K53AUIr5WuRqqX82ajMe1imLt/KFVEE+D0uCpvClfTHvEj7hBLKhw hTMCHP4Nd36hSK4LEVPB//wjzXAvUoqLtlgYoO6bBB+s6sPmhMbtcVY68vdaAAP4hc0u V6qsjnxB+0yKnaERNyjXRZ6qmnkgx2DaHNAqSw4+aZUM8ahKEu5P2CKzQTphg0bsrbeH i2FJgYOKUl6yIkPgWnw4kmf06EqxZg4QKCtO0vqOMMLa/CHi/hlPTaaREm+Pd/OY+2k9 vXTg== X-Forwarded-Encrypted: i=1; AKwUvBxxzjXZewWIRpd+U4T9xm18/1+biYjFb22rBvk4IepDOWdYVRbhT7gFbRjFtsmYTRYVh19vr7x1HLg=@vger.kernel.org X-Gm-Message-State: AFq9FYLmq1fUX5Xlin/EYd5Q1UDQOMOm2DhWTfvO94CsrFhmJrp72KJU dEEBefMnic+Y8r0pzhSa7E3RoexbNbjY69A7zrjKc9bLwNdwVeJUul2ZJsbXXjUnow4= X-Gm-Gg: AYBFou0rz9SNuLg3cRA0yI1mpVOe2oD9y80XnS4QJl5tE+e9CEI8VCKig5q91835w2H GiJS+6vn2Fqeu7n69Hj6ejDVAkxFoXQntqnI79Dy707ceIoHJBQP+IvPP0BznEh78HvYxVOh+Di 5ZFlOIyRcJhys5QyIoEG5WV4DpzVNL55/ly5muLgqIgbuvQ8d/sfbYFBoxLIMkEpgBLhBh1a9Oz wrwThnx0e4y4iFfQzuqZ2/Skqf6rtPdnOiHc/cPAtJaXC2fLaYFvP+sRP60gc6VJfxyzYj1lHOO sqfNoKlHs6rXPwJuZWQMaTPpEBcUGVbFz1dtYZ3NoNCuZoL88dbuVthV5CgldqSba2JwNTVDaZA 08PvkU7Xpqs6Ow/UkPHJx3qi2IEs15YRXfaD/CZl/ybkKRg6TM0y0v/aZjkiNR8Yl4zrcFWH6S2 b53MTBbZr+Td9crXSakXcx9lVFXrFuaBnYUiLeE9BtRQIhBArU3+4fHaXUFW9s+0LIcuhkGu3re XAGzQDDzeaVtLaGrD3N X-Received: by 2002:a17:906:9c93:b0:c2e:204d:9490 with SMTP id a640c23a62f3a-c40217bfed6mr31106466b.17.1792075877222; Thu, 15 Oct 2026 07:51:17 -0700 (PDT) Received: from localhost ([140.209.217.212]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c3ed9640488sm136037066b.25.2026.10.15.07.51.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Oct 2026 07:51:16 -0700 (PDT) From: Jiri Pirko To: linux-coco@lists.linux.dev Cc: linux-kernel@vger.kernel.org, linux-api@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, jgg@ziepe.ca, david.quigley@intel.com, lukas@wunner.de, ksinyuk@kernel.org, rodrigo.vivi@intel.com, cedric.xing@intel.com, sameo@meta.com, drdeeglaze@gmail.com, James.Bottomley@HansenPartnership.com, aik@amd.com, jpiotrowski@nvidia.com, qinkun@google.com, pgonda@google.com, jic23@kernel.org, alistair.francis@wdc.com, ankita@nvidia.com, suzuki.poulose@arm.com, kas@kernel.org, rick.p.edgecombe@intel.com, elena.reshetova@intel.com, dave.hansen@linux.intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, peter.fang@intel.com Subject: [PATCH RFC 04/20] Documentation: ctlv: Add userspace ABI Date: Thu, 15 Oct 2026 16:50:37 +0200 Message-ID: <20261015145053.3664509-5-jiri@resnulli.us> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261015145053.3664509-1-jiri@resnulli.us> References: <20261015145053.3664509-1-jiri@resnulli.us> Precedence: bulk X-Mailing-List: linux-api@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Jiri Pirko Describe the CTLV transport, discovery, wire format and userspace compatibility rules. Assisted-by: Cursor: GPT-5.6 Sol Assisted-by: Cursor: Claude Opus 5.5 Signed-off-by: Jiri Pirko --- Documentation/userspace-api/ctlv.rst | 507 ++++++++++++++++++++++++++ Documentation/userspace-api/index.rst | 1 + MAINTAINERS | 1 + 3 files changed, 509 insertions(+) create mode 100644 Documentation/userspace-api/ctlv.rst diff --git a/Documentation/userspace-api/ctlv.rst b/Documentation/userspace-api/ctlv.rst new file mode 100644 index 000000000000..5ebca3d4d94a --- /dev/null +++ b/Documentation/userspace-api/ctlv.rst @@ -0,0 +1,507 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================== +CTLV userspace ABI +================== + +:Author: Jiri Pirko + +CTLV is a transport for device control interfaces. A family describes what its +devices accept in YAML, a generator turns that into kernel metadata and UAPI +headers, and CTLV validates every message before a family callback sees it. +Userspace sees one character device per instance, one ioctl, and ``read()`` for +events. + +This document is the ABI. What each family accepts is specified under +``Documentation/ctlv/specs``; writing a family is :doc:`/driver-api/ctlv`. + +Why a TLV transport +=================== + +CTLV puts the rules of every operation in one description and validates them +once: + +- One ioctl carries every operation of every family, so a new operation is a + new specification entry, not a new ioctl number. +- Attributes are optional or required by declaration, so a message can grow + without breaking a caller that does not know the new attribute. +- An unknown attribute is refused on a request and tolerated on a reply, so a + new kernel can say more to an old program. A request may mark an attribute an + old kernel is welcome to skip, so a new program can say more too. +- Bounds, lengths, occurrence counts and nesting depth are checked before a + family callback runs. + +Devices +======= + +Each registered instance appears as a character device in the ``ctlv`` class +with a dynamic minor:: + + /dev/ctlv0 + /sys/class/ctlv/ctlv0 + +Sysfs shows class membership, the ``dev`` number and the device-model link to +the parent device. There is no family sysfs attribute; ``families-dump`` on an +open descriptor is the authoritative answer to what a device is. + +Each open file description has its own event subscriptions and event queue. +``O_RDONLY`` opens an observation fd, ``O_RDWR`` opens a full-control fd, and +``O_WRONLY`` is rejected with ``-EACCES``. Every action and query declares the +authority it needs: ``read`` permits either accepted fd, while ``read-write`` +requires full control. Events are readable and subscribable through both. +``dup()`` and ``SCM_RIGHTS`` preserve that authority; attenuation requires a +separate observation open. + +Discovery +--------- + +Both uevents, addition and removal, carry the family chain as numbered +properties:: + + CTLV_FAMILY=ctlv-accelerator-foo + CTLV_COMPATIBLE_0=ctlv + CTLV_COMPATIBLE_1=ctlv-accelerator + CTLV_COMPATIBLE_2=ctlv-accelerator-foo + CTLV_COMPATIBLE_N=3 + +``CTLV_FAMILY`` is the family the device answers as, the numbered properties +are its ancestry from the root to that family, and +``CTLV_COMPATIBLE_N`` is how many there are. An index never moves: changing a +family's parent is an ABI +break, adding a descendant only appends. + +A program that wants every device, including ones appearing while it starts, +must listen before it looks, which closes the window between enumerating and +subscribing:: + + 1. create a udev monitor filtered to the ctlv subsystem, and enable it + 2. enumerate the existing ctlv class devices + 3. drain the events already queued on the monitor + 4. deduplicate by sysfs path + 5. open the device node + 6. ask families-dump on the open descriptor for the authoritative chain + +Steps 1 to 4 are pre-open discovery, all a udev rule can match on; step 6 does +not need a udev database. A rule matches a family through the compatibility +properties:: + + # /etc/udev/rules.d/70-ctlv.rules + # + # Anything of one family, whatever family implementation registered it. + SUBSYSTEM=="ctlv", ENV{CTLV_FAMILY}=="ctlv-accelerator-foo", \ + GROUP="accel", MODE="0660" + + # Anything descended from a family. + SUBSYSTEM=="ctlv", ENV{CTLV_COMPATIBLE_1}=="ctlv-accelerator", \ + TAG+="uaccess" + + # A stable name from the parent device, which is deployment policy. + SUBSYSTEM=="ctlv", ENV{CTLV_FAMILY}=="ctlv-accelerator-foo", \ + SYMLINK+="accel/by-path/$env{ID_PATH}" + +CTLV does not standardise such aliases and does not promise that a generic +identifier survives a reboot or a reprobe; a family with a meaningful identity +exposes it through an operation of its own. + +Wire format +=========== + +A message is a tree of attributes. Every attribute begins with the same +header, at an eight-byte boundary:: + + struct ctlv_attr { + __u32 len; /* header + payload, without padding */ + __u32 flags; /* CTLV_ATTR_F_* */ + __aligned_u64 attr_id; /* complete ID, or 0 for a root */ + }; + +One unsigned integer attribute with an eight-byte payload, on the wire:: + + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +---------------------------------------------------------------+ + | len = 24 | + +---------------------------------------------------------------+ + | flags = 0 | + +---------------------------------------------------------------+ + | | + + attr_id + + | | + +---------------------------------------------------------------+ + | | + + payload + + | | + +---------------------------------------------------------------+ + +The rules that make a tree out of that: + +- ``CTLV_ALIGNTO`` is 8 and ``CTLV_ALIGN()`` rounds up to it. Every attribute + starts at a multiple of eight. +- ``len`` counts this header plus this attribute's payload, not its own + trailing padding. +- A nested payload is the concatenation of ``CTLV_ALIGN(child.len)`` bytes per + child, so a child's padding is counted inside its parent's ``len`` and a + leaf's own padding by nobody. +- ``CTLV_ATTR_F_NESTED`` is set exactly on a root and on a nest. +- A message begins with one root attribute, whose ``attr_id`` is + ``CTLV_ATTR_ROOT`` (zero) and whose payload is the message. A root never + appears inside a nest, and a nest never carries the root ID. +- Scalar payloads are native-endian and their lengths exact: eight bytes for + an integer, one byte holding zero or one for a boolean, the string including + its NUL terminator, the declared length for binary data. +- Nesting is limited to ``CTLV_MAX_NEST_DEPTH`` levels, the root being level + zero. + +A whole message with every offset, for a request of one integer and one nest +carrying a string and a further nest:: + + offset attribute len flags contents + ------ ------------------------------ --- ------ ---------------- + 0 root 120 NESTED attr_id = 0 + 16 .. chunks 24 - 8-byte integer + 40 .. chunk 80 NESTED + 56 .. label 19 - "ab" and its NUL + 80 .. item 40 NESTED + 96 .. value 24 - 8-byte integer + 120 end of message + + +--------------------------------------------------------------+ 0 + | root: len = 120, flags = NESTED, attr_id = 0 | + +--------------------------------------------------------------+ 16 + | chunks: len = 24 | payload: 8 bytes | + +--------------------------------------------------------------+ 40 + | chunk: len = 80, flags = NESTED | + | +--------------------------------------------------------+ | 56 + | | label: len = 19 | payload: 'a' 'b' 0 | 5 bytes padding | | + | +--------------------------------------------------------+ | 80 + | | item: len = 40, flags = NESTED | | + | | +--------------------------------------------------+ | | 96 + | | | value: len = 24 | payload: 8 bytes | | | + | | +--------------------------------------------------+ | | 120 + +--------------------------------------------------------------+ + +``label.len`` is 19: its header and three payload bytes, not the five bytes of +padding after it. ``chunk`` counts those five, its ``len`` of 80 being its +header plus ``CTLV_ALIGN(19)`` plus ``CTLV_ALIGN(40)``. A walk over a nest's +children advances by ``CTLV_ALIGN(child.len)``, a walk of one payload by +``child.len``. + +An attribute ID names the family that declared it, a group within that family +and an attribute within that group:: + + 63 48 47 32 31 0 + +--------------+--------------+--------------------------------+ + | family | group | attr | + +--------------+--------------+--------------------------------+ + + CTLV_ATTR(family, group, attr) + CTLV_ATTR_FAMILY(attr) / CTLV_ATTR_GROUP(attr) / CTLV_ATTR_NUMBER(attr) + +An operation ID is the same shape with the group bits zero:: + + CTLV_OP(family, op) + CTLV_OP_FAMILY(op) / CTLV_OP_NUMBER(op) + +Use generated complete IDs. A family's header defines every ID it declares +and includes its parent's. + +Request occurrence order +------------------------ + +Occurrences of one attribute ID must be contiguous within each request root or +nest. ``A A B`` is valid, while ``A B A`` is not. An ignored unknown attribute +is not an occurrence of any known attribute, but it ends the preceding run, so +``A ignored-unknown A`` is also invalid. Different IDs otherwise have no +required numeric or schema order. + +Attributes a kernel may skip +---------------------------- + +A request attribute, the root aside, may carry +``CTLV_ATTR_F_IGNORE_IF_UNKNOWN``. If no family the device answers declares +that complete ID, the kernel steps over the whole attribute without looking +inside it; without the bit, an attribute the schema does not place where it +appears fails the request as it always has. + +The bit forgives an ID this kernel has never heard of and nothing else. An ID +the device's schema declares somewhere is held to that schema, so an attribute +sent where it does not belong fails whether the bit is set or not, and one sent +where it does belong is checked as usual. Every other rule applies too: the +length has to fit its nest and the header may carry no bit the format does not +define. Replies and events never carry it. + +This is how a program built against a newer version of a family talks to an +older kernel. What that kernel knows is validated, what it does not is +ignored, and a caller that has to know which is which asks ``op-schema-get`` +first. + +Executing an operation +====================== + +One ioctl executes every operation:: + + struct ctlv_op_exec { + __aligned_u64 op_id; + __aligned_u64 request; /* struct ctlv_attr __user * */ + __aligned_u64 reply; /* struct ctlv_attr __user * */ + __u32 request_len; + __u32 reply_len; + __aligned_u64 generation; /* queries only */ + __aligned_u64 cursor; /* queries only */ + __aligned_u64 reserved[2]; /* must be zero */ + }; + + #define CTLV_IOC_OP_EXEC _IOWR(0xBB, 0x00, struct ctlv_op_exec) + +``request_len`` is how many bytes the kernel may read at ``request`` and +``reply_len`` how many it may write at ``reply``. They are copy boundaries, not +message lengths: a root's ``len`` may be shorter than ``request_len``, the +bytes after it being ignored, and may not be longer. ``reply_len`` is never +overwritten; the length written is the reply root's ``len``. The pointers are +fixed-width so a 32-bit and a 64-bit process pass the same structure, and in +place of a size field the ioctl number encodes the size and ``reserved``, which +must be zero, covers compatible growth. + +Every action and query carries a physical request root, including an empty +request. ``request_len`` is therefore at least one attr header and at most the +1 MiB transport limit. The ``request-max-len`` reported by ``ops-dump`` is the +maximum request composed from the known schema; a newer caller may exceed it +with well-formed unknown attrs marked ``IGNORE_IF_UNKNOWN``, up to the +transport limit. + +The three kinds of operation differ in what they answer: + +action + Executed once and acknowledged. An action with no reply takes + ``reply_len`` zero; one with a bounded reply has its maximum in + ``ops-dump``. + +query + Executed once and answered in as many fragments as it needs, using + ``generation`` and ``cursor``. + +event + Not executed. A device produces records and a subscribed file description + reads them. + +Errors +------ + +A failed operation returns a negative errno. Where the failure is a rule the +message broke and the caller offered a reply buffer, CTLV also writes +error information there in TLV form: a ``validation-reason`` value, a +human-readable message, the offset of the offending attribute, the offset of +the nest it was in, the ID of a missing attribute, and for a stale query the +expected and the current generation. A buffer too small leaves it unwritten; +the errno is authoritative either way. + +What the transport answers with, as against a family's own callback: + +``ENOTTY`` + Not ``CTLV_IOC_OP_EXEC``; this transport has one ioctl. + +``EINVAL`` + A reserved field was not zero, ``op_id`` was zero, ``generation`` or + ``cursor`` was set on something that is not a query, or the message broke + a rule of the format or of the schema. + +``EFAULT`` + The structure, the request or the reply was not accessible. + +``EOPNOTSUPP`` + The device does not publish the operation, publishes it but has it + disabled, or the operation is an event, which is read rather than executed. + +``EACCES`` + The fd lacks the operation's declared authority, or an ``O_WRONLY`` open + was attempted. + +``EMSGSIZE`` + ``reply_len`` is smaller than the smallest fragment the operation can + answer with. Nothing was executed. + +``ESTALE`` + A continuation whose ``generation`` no longer matches. Start the query + again. + +``ENODEV`` + The device was unregistered. Descriptors stay open and every operation on + them fails this way. + +``EBADF``, ``EIO`` + A blob descriptor named something that is not a file, or the file refused + the transfer. + +``ENOMEM`` + The kernel could not allocate what the operation needed. + +``read()`` answers ``EAGAIN`` when no record is queued on a non-blocking +description, ``EINTR`` when a blocking read was interrupted, ``EMSGSIZE`` when +the buffer cannot hold the record at the head of the queue, and ``ENODEV`` +once a removed device's queue has been drained. + +Queries and fragments +===================== + +A query's answer may be larger than the buffer the caller wants to commit, so +the kernel fills the buffer with as much of it as fits where the tree can be +cut. The first call passes ``generation`` and ``cursor`` zero. On return: + +- the kernel puts the query's generation in ``generation``; +- ``cursor`` is nonzero when there is more to come, zero when the answer is + complete; +- the reply root carries ``CTLV_ATTR_F_FRAG_MORE`` when the answer continues, + and so does every nest still logically open at the cut. + +To continue, the caller repeats the call with the same ``op_id`` and request, +the returned ``cursor`` and the returned ``generation``, and reassembles by +concatenating the children of each still-open nest. + +One answer of three items, cut by a buffer that holds two:: + + call 1: cursor in = 0 call 2: cursor in = c + cursor out = c cursor out = 0 + + +----------------------------+ +----------------------------+ + | root: FRAG_MORE | | root: (no FRAG_MORE) | + | +-----------------------+ | | +-----------------------+ | + | | chunk: FRAG_MORE | | | | chunk: | | + | | +------------------+ | | | | +------------------+ | | + | | | item: value = 1 | | | | | | item: value = 3 | | | + | | +------------------+ | | | | +------------------+ | | + | | | item: value = 2 | | | | +-----------------------+ | + | | +------------------+ | | +----------------------------+ + | +-----------------------+ | + +----------------------------+ + + reassembled: root -> chunk -> item(1), item(2), item(3) + +``FRAG_MORE`` on the root says the answer continues; on ``chunk`` it says the +next fragment resumes inside that same chunk. A complete nest does not carry +it, so a reader knows whether to append or to open a new nest beside it. + +Every continuing fragment adds at least one complete logical attr beyond its +repeated path, and a cursor value cannot recur within one logical reply. +A reader must reject either violation. + +A continuation whose generation no longer matches fails with ``-ESTALE``, and +the error information carries the expected and the current generation. + +Events +====== + +An event reaches a file description that asked for it, and only that one: + +1. subscribe with the ``subscriptions-add`` action, naming operation IDs; +2. read records with ``read()``, one complete record per call; +3. unsubscribe with ``subscriptions-remove``. + +``poll()`` reports readability when a record is queued. Each record is a +header followed by exactly one root attribute:: + + struct ctlv_event_hdr { + __u32 length; /* header + aligned root */ + __u32 flags; /* zero */ + __aligned_u64 op_id; /* the event op that produced it */ + __aligned_u64 sequence; /* per file description */ + }; + +One ``read()`` returns one record, the header and the aligned root:: + + +---------------------------------------------------------------+ 0 + | length = 24 + CTLV_ALIGN(root.len) | flags = 0 | + +---------------------------------------------------------------+ 8 + | op_id: the event operation that produced this record | + +---------------------------------------------------------------+ 16 + | sequence: counted per file description | + +---------------------------------------------------------------+ 24 + | root: len, flags = NESTED, attr_id = 0 | + | the event's attributes, exactly as a reply carries them | + +---------------------------------------------------------------+ length + +A buffer shorter than ``length`` reads nothing and fails with ``-EMSGSIZE``. A +record is one message, bounded by ``CTLV_MAX_INLINE_MESSAGE_LEN``. A sequence +number is assigned to every eligible record, dropped ones included, so a gap in +the sequence is exactly what was lost. + +The queue is per file description: ``event-queue-get`` reports its length and +how much is in use, ``event-queue-set`` changes it up to +``CTLV_MAX_EVENT_QUEUE_LEN``. A record that does not fit is dropped, and the +next read returns an ``event-overflow`` record carrying the first and last +sequence numbers lost and how many records that was. + +Two events come from the root family: + +op-changed + An operation with dynamic availability was enabled or disabled. The record + carries the operation ID, the new state, and the state generation that + ``ops-dump`` also reports, so a client can order dump against event. + +device-removed + The device is gone. Operations fail from then on and the descriptor stays + valid until it is closed. + +Blobs +===== + +An operation that moves more data than a message should carry declares a blob +attribute, and the caller passes a descriptor instead of the data: a nest +carrying the backing type, the length and the identifier of the memory. One +backing is defined: + +``va`` + Anonymous userspace memory of the calling process, named by an address in + ``blob.address``. + +The backings an attribute accepts, the direction data may flow, the maximum +length and the required alignment are declared by the family and reported by +``op-schema-get``. Memory is acquired for the operation and released before it +returns; a descriptor the operation may not use is refused before the callback. +While the operation is in flight, userspace must not modify an ``input`` blob +or access an ``output`` or ``inout`` blob. Whether two descriptors overlap is +also the caller's responsibility. Keep every named VA mapping valid until the +operation returns. + +Introspection +============= + +Three queries describe a device to a program that was not compiled against it. +The CTLV core answers all three on every device. + +``families-dump`` + The device's family chain, root first, as a numeric ID and a name per + entry. The authoritative answer to what the device is. + +``ops-dump`` + One entry per published operation: its ID, kind, required fd access, + whether its availability is fixed or dynamic, whether it is currently + disabled, and the message lengths that apply on this device. An + unpublished operation has no entry and is indistinguishable from one no + family ever declared. Event entries have no fd-access field. + +``op-schema-get`` + The whole tree of one operation: every attribute of the request and the + reply, with its payload form, its bounds, its occurrence limit, and its + unit and scale where it has them. + +A schema that describes a schema does not end, so it is reported through +re-entry: a node whose members are those of a level already described carries +``schema.reenters``, saying how many levels up that level is. A consumer that +follows ``reenters`` handles any depth. + +The lengths in ``ops-dump`` include whatever a descendant family added to an +inherited operation, so ask rather than compute: a family's generated maximum +is right for its own devices and a lower bound on a descendant's. + +Limits +====== + +Implementation limits, not encoded maxima of the wire fields, so raising one +later is a compatible change:: + + CTLV_MAX_INLINE_MESSAGE_LEN 1 MiB request roots, action replies, + query fragments, event roots + CTLV_MAX_ERROR_LEN 16 KiB error information + CTLV_MAX_NEST_DEPTH 32 levels, root at zero + CTLV_DEFAULT_QUERY_REPLY_LEN 64 KiB suggested query buffer + CTLV_MAX_EVENT_QUEUE_LEN 16 MiB per file description diff --git a/Documentation/userspace-api/index.rst b/Documentation/userspace-api/index.rst index a68b1bea57a8..d1b53c4e16eb 100644 --- a/Documentation/userspace-api/index.rst +++ b/Documentation/userspace-api/index.rst @@ -45,6 +45,7 @@ Devices and I/O :maxdepth: 1 accelerators/ocxl + ctlv dma-buf-heaps dma-buf-alloc-exchange fwctl/index diff --git a/MAINTAINERS b/MAINTAINERS index 8ab0d6a7a2fd..882836d85125 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -7006,6 +7006,7 @@ M: Jiri Pirko L: linux-kernel@vger.kernel.org S: Maintained F: Documentation/ctlv/ +F: Documentation/userspace-api/ctlv.rst F: include/linux/ctlv*.h F: include/uapi/linux/ctlv*.h F: tools/ctlv/ -- 2.56.0