From: Mike Lothian <mike@fireburn.co.uk>
To: dri-devel@lists.freedesktop.org
Cc: Mike Lothian <mike@fireburn.co.uk>,
Maarten Lankhorst <maarten.lankhorst@linux.intel.com>,
Maxime Ripard <mripard@kernel.org>,
Thomas Zimmermann <tzimmermann@suse.de>,
David Airlie <airlied@gmail.com>, Simona Vetter <simona@ffwll.ch>,
Benno Lossin <lossin@kernel.org>, Gary Guo <gary@garyguo.net>,
linux-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org
Subject: [PATCH v3 8/13] drm/vino: add the KMS device and the atomic path
Date: Wed, 26 Aug 2026 17:37:33 +0100 [thread overview]
Message-ID: <20260826163913.7052-9-mike@fireburn.co.uk> (raw)
In-Reply-To: <20260826163913.7052-1-mike@fireburn.co.uk>
Add the KMS half of the sink: the DRM driver and its GEM and file
types, the CRTCs, planes and connectors the compositor drives over a
software vblank clock, the mode admission checks that keep a connector
from being handed a mode past the dock's budget, the settings a matched
profile installs, and the workers the atomic callbacks publish to.
An atomic callback may not sleep or touch USB, so it records what the dock
should be doing and wakes a worker; each operation class owns one slot,
which makes publication infallible and lets a stale cursor position be
overwritten rather than queued behind the state that replaced it.
Sample depth is decided here too, and from `max bpc` rather than from
the committed framebuffer's format: a compositor drives a ten-bit link
from an eight-bit surface, which is what the property means everywhere
else. The dock's pixel budget is shared and was measured with three
bytes stored per pixel, so a ten-bit connector costs a third more and
the whole dock is priced at its deepest one. Where a pair does not fit,
the depth gives way rather than the mode: a compositor handed EINVAL
disables the output instead of asking for a shallower link. The decision
is taken in the enable, beside the mode set that carries it to the dock,
because a check also runs for page flips and for TEST_ONLY commits that
are never applied.
The planes publish the single modifier scanout accepts, so userspace reads
IN_FORMATS rather than inferring what will be taken from the format list.
The dock-facing half of the sink -- activation, presence, the streams and
the encoder feed -- follows in the next commit.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Mike Lothian <mike@fireburn.co.uk>
---
drivers/gpu/drm/vino/drm_sink.rs | 1868 +++++++++++++++++
drivers/gpu/drm/vino/drm_sink/dispatch.rs | 446 ++++
drivers/gpu/drm/vino/drm_sink/driver.rs | 211 ++
drivers/gpu/drm/vino/drm_sink/limits.rs | 489 +++++
drivers/gpu/drm/vino/drm_sink/mode_objects.rs | 978 +++++++++
drivers/gpu/drm/vino/drm_sink/settings.rs | 574 +++++
drivers/gpu/drm/vino/drm_sink/worker.rs | 571 +++++
7 files changed, 5137 insertions(+)
create mode 100644 drivers/gpu/drm/vino/drm_sink.rs
create mode 100644 drivers/gpu/drm/vino/drm_sink/dispatch.rs
create mode 100644 drivers/gpu/drm/vino/drm_sink/driver.rs
create mode 100644 drivers/gpu/drm/vino/drm_sink/limits.rs
create mode 100644 drivers/gpu/drm/vino/drm_sink/mode_objects.rs
create mode 100644 drivers/gpu/drm/vino/drm_sink/settings.rs
create mode 100644 drivers/gpu/drm/vino/drm_sink/worker.rs
diff --git a/drivers/gpu/drm/vino/drm_sink.rs b/drivers/gpu/drm/vino/drm_sink.rs
new file mode 100644
index 000000000000..86708db89699
--- /dev/null
+++ b/drivers/gpu/drm/vino/drm_sink.rs
@@ -0,0 +1,1868 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! DRM/KMS integration for Vino.
+//!
+//! Each dock connector has a primary plane, cursor plane, CRTC, encoder and connector. Framebuffers
+//! are copied into driver-owned snapshots before atomic completion, then compressed and sent by
+//! per-connector workers. Connector modes come from downstream EDID tunneled over the dock's
+//! control protocol.
+
+use core::sync::atomic::{AtomicBool, AtomicI64, Ordering};
+use kernel::{
+ drm,
+ drm::kms::{
+ self,
+ connector::{self, Connector, ConnectorGuard, ConnectorModeValidation, ModeStatus, Status},
+ crtc::{self, CrtcAtomicCheck, CrtcAtomicCommit, RawCrtc as _, RawCrtcState as _},
+ encoder,
+ modes::DisplayMode,
+ plane::{self, PlaneAtomicCheck, PlaneAtomicCommit, RawPlaneState as _},
+ vblank::{
+ OwnedVblankRef, RawVblankCrtcState as _, VblankGuard, VblankSupport, VblankTimestamp,
+ },
+ KmsDriver, ModeConfigGuard, ModeConfigInfo, ModeObject as _, UnregisteredKmsDevice,
+ },
+ error::code::{EINVAL, ENODEV, ENOMEM, ENOTSUPP},
+ impl_has_hr_timer,
+ interrupt::LocalInterruptDisabled,
+ io::Io,
+ prelude::*,
+ sync::{
+ aref::ARef, new_mutex, new_spinlock, new_spinlock_irq, Arc, ArcBorrow, Completion, Mutex,
+ SpinLock, SpinLockIrq,
+ },
+ time::{
+ delay::{fsleep, udelay},
+ hrtimer::{
+ ArcHrTimerHandle, HrTimer, HrTimerCallback, HrTimerCallbackContext, HrTimerPointer,
+ HrTimerRestart, RelativeHardMode,
+ },
+ Delta, Instant, Monotonic,
+ },
+ workqueue::{
+ self, impl_has_delayed_work, impl_has_work, new_delayed_work, new_work, DelayedWork, Work,
+ WorkItem,
+ },
+ xxhash,
+};
+
+mod dispatch;
+mod driver;
+mod limits;
+mod mode_objects;
+mod settings;
+mod worker;
+
+pub(crate) use driver::VinoObject;
+use limits::{active_pixel_rate, timing_key, DEFAULT_MAX_HEAD_CLOCK_KHZ, DEFAULT_MAX_REFRESH_HZ};
+pub(super) use mode_objects::{
+ PlaneArgs, VblankTimer, VinoConnector, VinoCrtc, VinoEncoder, VinoPlane,
+};
+
+/// Connector mode used until a downstream EDID is available.
+const FALLBACK_W: i32 = 2560;
+const FALLBACK_H: i32 = 1440;
+
+/// Primary-plane format list (opaque 32bpp scanout).
+static PRIMARY_FORMATS: [u32; 1] = [drm::fourcc::XRGB8888];
+
+/// Primary-plane format list for a dock whose pipeline carries 10 bits per channel.
+///
+/// `XRGB8888` stays first: it is what every ordinary desktop commits, and a compositor choosing
+/// between them should not be pushed towards the deeper one by list order alone.
+static PRIMARY_FORMATS_HDR: [u32; 2] = [drm::fourcc::XRGB8888, drm::fourcc::XRGB2101010];
+
+/// The only framebuffer layout any of these planes accepts.
+///
+/// Publishing it gives userspace an `IN_FORMATS` property; without it the plane advertises formats
+/// with no modifier information at all.
+static LINEAR_MODIFIER: [u64; 1] = [drm::fourcc::FORMAT_MOD_LINEAR];
+
+/// Cursor-plane format list.
+static CURSOR_FORMATS: [u32; 1] = [drm::fourcc::ARGB8888];
+
+/// Stream-marker state which powers down a connector's downstream sink.
+///
+/// The resulting probe silence must be paired with [`VinoDrmData::self_blanked`] so it is not
+/// mistaken for a physical disconnect.
+///
+/// The vendor drives a DL-3x00 sink down with `0x2f` state 1 followed by `0x2e` state 3, and back
+/// up with `0x2e` state 0 then `0x2f` state 0. State 1 is what has been verified against a
+/// DL-6xxx dock, so the field is likely a bitmask rather than an enumeration; a platform that
+/// needs the vendor's exact value will have to carry it per profile.
+const BLANK_MARKER_STATE: u8 = 1;
+
+/// Delay before retrying a transient asynchronous control operation.
+const KMS_RETRY_MS: u32 = 50;
+
+/// Consecutive deferrals of a KMS command batch before it is dropped.
+///
+/// A command that fails because the dock has stopped answering will fail again for the same
+/// reason, and retrying it forever reprograms a dead link twenty times a second: it buries every
+/// other message in the log and keeps writing to a dock that has already abandoned its session.
+/// Anything genuinely transient clears well inside this, and a later commit or hotplug queues
+/// fresh work regardless. The bound is a little over the dock's own no-answer watchdog.
+const KMS_RETRY_LIMIT: u32 = 128;
+
+/// Maximum number of physical downstream connectors Vino exposes.
+///
+/// Ridge docks use the first two. Navarro has four physical DP sockets; connectors 0/2 share
+/// bulk endpoint 0x08 and connectors 1/3 share 0x0a. `DockProfile::connectors` selects the
+/// active prefix at runtime, while this constant keeps the DRM object layout fixed at registration.
+pub(crate) const MAX_CONNECTORS: usize = 4;
+
+/// Bulk transfer size on the video endpoints: a multiple of the 1024-byte maximum packet size, so
+/// only a frame's final transfer terminates short.
+const VIDEO_XFER: usize = 65536;
+
+/// Stream reports a connector sends after its stream is opened, on a dock that carries video on the
+/// control pipe.
+///
+/// The report restates the mode on a stream the dock has just been handed. DLM sends fourteen of
+/// them across 7115 frames and never two together, so one per stream is the whole of it; sending
+/// one per frame spends both the dock's bandwidth budget and its sealed block counter on a record
+/// it is not waiting for.
+const STREAM_REPORT_BURST: u32 = 1;
+
+/// Frame of a fresh stream that carries the report, counting the prologue as zero.
+///
+/// DLM restates the mode with the third frame, not the first: the two frames before it carry
+/// nothing but pixels and their closing record.
+const STREAM_REPORT_FRAME: u32 = 2;
+
+/// Whether a presentation named a ring slot, and so consumed a frame counter.
+///
+/// The counter belongs to whichever record names the slot: the frame opener on a dock that
+/// carries one, the frame trailer on a dock that carries the transition there. A presentation with
+/// neither says nothing about the ring, and counting it anyway puts every later record one slot
+/// ahead of the buffer the host actually wrote, so the dock scans out a buffer nothing has filled.
+pub(crate) fn names_ring_slot(opener: &[u8], trailer: &[u8]) -> bool {
+ !opener.is_empty() || !trailer.is_empty()
+}
+
+/// Maximum number of individual frame-damage rectangles re-converted per flip before they are
+/// collapsed into a single bounding box. Bounds the stack array used on the atomic-commit path
+/// (no per-flip allocation); a compositor that reports more clips than this just gets a coarser
+/// (still correct) repaint.
+const MAX_DAMAGE_CLIPS: usize = 16;
+/// Minimum interval between normal frames for one connector.
+const FRAME_PERIOD_MS: i64 = 5;
+/// The coalescing window in microseconds. Whole-millisecond arithmetic truncated the elapsed time
+/// and forced a 1 ms minimum sleep, so a frame could wait materially longer than the window.
+const FRAME_PERIOD_US: i64 = FRAME_PERIOD_MS * 1000;
+/// Interval between keepalive status queries on a dock whose profile does not state one.
+const STATUS_PERIOD_MS: i64 = 250;
+
+/// Activation timing relative to the mode-set submission.
+const PROMPT_VIDEO_MS: i64 = 110;
+const PROMPT_CLOSE_2F_MS: i64 = 123;
+const PROMPT_CLOSE_2E_MS: i64 = 125;
+/// Upper bound used to quiesce an already-running keepalive iteration.
+const PROMPT_KEEPALIVE_QUIESCE_MS: i64 = 40;
+/// Minimum interval between streaming status polls (`id=0x14 sub=0x0c`).
+///
+/// Issued per presentation this would be 200-600 CP round-trips a second across two connectors at
+/// ~100 fps, every one of them serialising on the single control link, and whichever connector
+/// looped fastest would re-acquire it immediately and starve the other. DLM issues ~3.8 per second,
+/// so a quarter-second floor matches the reference and leaves the link free for the other
+/// connector.
+const STATUS_POLL_MIN_MS: i64 = 250;
+const PROMPT_TRAINING_OPEN_MS: i64 = 0;
+const PROMPT_TRAINING_TAIL_MS: i64 = 400;
+/// How long [`VinoDrmData::blank_connector`] keeps presenting black when a CRTC is disabled.
+///
+/// It only has to outlast the dock's buffer rotation, which is at most three presentations on the
+/// current profiles; a black frame is ~200 KB and presents in a couple of milliseconds, so this is
+/// generous by an order of magnitude and still finishes well inside a DPMS transition. It is not a
+/// training window -- nothing downstream needs settling -- so it does not reuse
+/// [`PROMPT_TRAINING_TAIL_MS`].
+const BLANK_PRESENT_MS: i64 = 120;
+
+/// `edid_target` sentinel: nobody is waiting for an EDID.
+const NO_EDID_TARGET: u32 = u32::MAX;
+
+/// How much of a DL7400 frame's image data precedes its per-strip parameter map.
+///
+/// The map tells the dock how to read the records around it, and where it sits in the frame is
+/// load-bearing rather than cosmetic: the vendor's stream carries it this far into the image
+/// records of every frame, and a frame that carries the same valid records after all of its pixels
+/// instead is accepted twice and then leaves the endpoint permanently un-drained.
+pub(crate) const NAVARRO_PARAM_IMAGE_OFFSET: usize = 115168;
+
+/// How many leading record chunks precede the parameter map in a frame.
+///
+/// The map has to land on a record boundary, and an encoded frame's chunks are the boundaries this
+/// side knows: each holds whole records, while [`NAVARRO_PARAM_IMAGE_OFFSET`] itself falls wherever
+/// a frame's record lengths put it. So round the vendor's offset back to a chunk, and keep at least
+/// one chunk in front of the map -- what the dock will not take is the map arriving after every
+/// record it describes, not the exact byte it arrives at.
+pub(crate) fn param_map_chunk_split(frames: &[KVec<u8>]) -> usize {
+ let mut consumed = 0usize;
+ let mut chunks = 0usize;
+ for f in frames {
+ if consumed + f.len() > NAVARRO_PARAM_IMAGE_OFFSET {
+ break;
+ }
+ consumed += f.len();
+ chunks += 1;
+ }
+ chunks.max(1).min(frames.len())
+}
+type DamageRect = (usize, usize, usize, usize);
+type BoundInterface<'a> = super::UsbLink<'a>;
+
+/// A dock's unspent sustained-throughput credit.
+///
+/// Credit accrues at the profile's rate and is capped at one second of it, so a dock idle for a
+/// minute does not bank a minute of bytes and then hand them to the endpoint at once. Spending is
+/// allowed to overdraw: a frame's size is known only once it is encoded, and refusing to send a
+/// frame already committed to the wire would strand it. The debt is repaid before the next frame
+/// is selected, which is what turns the ledger into a rate.
+pub(crate) struct StreamCredit {
+ bytes: i64,
+ topped_up: Option<Instant<Monotonic>>,
+}
+
+impl StreamCredit {
+ fn new() -> Self {
+ Self {
+ bytes: 0,
+ topped_up: None,
+ }
+ }
+}
+
+/// Credit accrued over `elapsed_us` at `bps`, saturating rather than wrapping on a long idle.
+pub(crate) fn stream_credit_accrued(bps: u32, elapsed_us: i64) -> i64 {
+ let bps = i64::from(bps);
+ elapsed_us
+ .max(0)
+ .saturating_mul(bps)
+ .checked_div(1_000_000)
+ .unwrap_or(0)
+}
+
+/// Microseconds until an overdrawn ledger is back in credit.
+pub(crate) fn stream_credit_wait_us(bps: u32, bytes: i64) -> Option<i64> {
+ if bytes >= 0 {
+ return None;
+ }
+ let bps = i64::from(bps).max(1);
+ Some(
+ bytes
+ .saturating_neg()
+ .saturating_mul(1_000_000)
+ .checked_div(bps)
+ .unwrap_or(0)
+ .saturating_add(1),
+ )
+}
+
+/// Presentations made by one logical scanout submission.
+///
+/// Cold training remains a transport exception for docks with a dedicated video endpoint. Normal
+/// keyframe and delta counts are profile data because ring depth alone does not describe how a
+/// platform expects the host to populate that ring.
+pub(crate) fn frame_presentation_count(
+ policy: super::profile::FrameDelivery,
+ full: bool,
+ cold_training: bool,
+ video_on_ctrl_pipe: bool,
+) -> u32 {
+ if full && cold_training && !video_on_ctrl_pipe {
+ COLD_TRAINING_PRESENTATIONS
+ } else if full {
+ u32::from(policy.keyframe_presentations.max(1))
+ } else {
+ u32::from(policy.delta_presentations.max(1))
+ }
+}
+
+/// Account one accepted logical submission against per-strip delivery debt.
+pub(crate) fn pay_damage_debt(debt: &mut [u8], full: bool) {
+ if full {
+ debt.fill(0);
+ } else {
+ for remaining in debt {
+ *remaining = remaining.saturating_sub(1);
+ }
+ }
+}
+
+/// Content of the last frame successfully submitted for one connector, represented in the dock's
+/// native 64x16 strip grid. KWin frequently omits `FB_DAMAGE_CLIPS` when switching framebuffer
+/// objects; a raw-content shadow is therefore the authoritative way to distinguish an unchanged
+/// flip from a real repaint. `KVVec` permits vmalloc fallback for the roughly 29-KiB 1440p hash
+/// table.
+struct StripHashState {
+ padded_width: usize,
+ padded_height: usize,
+ hashes: KVVec<u64>,
+ /// Encoded strip bodies, parallel to `hashes`. Retransmit debt can reuse a body while its
+ /// pixels and encoding tag remain unchanged.
+ bodies: KVec<KVec<u8>>,
+ /// Everything other than the strip's own pixels that its encoded bytes depend on.
+ ///
+ /// The hash covers the raw framebuffer, so a change that alters the ENCODED output without
+ /// altering the source pixels would otherwise serve a stale body. Gamma is exactly that: a new
+ /// LUT re-maps every pixel on the way into the codec while leaving the framebuffer identical,
+ /// and although a gamma change owes a keyframe, a keyframe re-selects every strip rather than
+ /// invalidating anything, so the reuse test would still hit. Rotation is the same hazard, and
+ /// is handled by only ever caching under identity rotation.
+ tag: u64,
+}
+
+/// The DRM driver marker type.
+pub(super) struct VinoDrmDriver;
+
+/// Convenience alias for our concrete `drm::Device`.
+pub(super) type VinoDrmDevice = drm::Device<VinoDrmDriver>;
+
+/// Active control-protocol session.
+///
+/// `wire_seq` counts AES-CTR content blocks; the authentication tag does not consume keystream.
+/// `counter` is the inner protocol counter. The enclosing mutex advances both atomically with a
+/// complete request/reply transaction.
+pub(super) struct CpLink {
+ ks: kernel::crypto::Secret<16>,
+ riv: [u8; 8],
+ wire_seq: u32,
+ counter: u16,
+ ep84_q: Option<super::usb::BulkInQueue>,
+}
+
+/// How long the dock may answer nothing at all before the session is abandoned.
+///
+/// Measured in silence rather than in unanswered messages, because those are not the same thing. A
+/// connector whose sink is down leaves its own re-engage unanswered every few seconds for the life
+/// of the session while its sibling drives a lit panel and the status dialogue replies throughout;
+/// counting messages tears that session down. A dock with a video pipe of its own answers a lit,
+/// idle link continuously, so real silence there is unambiguous.
+const CP_SILENCE_LIMIT_MS: i64 = 5000;
+
+/// The same limit for a dock that shares its control pipe with video.
+///
+/// Silence is not evidence of anything on such a dock: the vendor's own capture has it say nothing
+/// for 79 s while a panel is lit, and the vendor sends nothing either. Applying the short limit
+/// there abandons a working session, and the reset that follows is what turns a stalled dock into
+/// one that has to be unplugged.
+const CP_SILENCE_LIMIT_SHARED_MS: i64 = 90_000;
+
+/// How often the watchdog checks the silence deadline.
+const CP_WATCHDOG_PERIOD_MS: u32 = 1000;
+
+/// A control-protocol operation deferred from the non-blocking atomic callbacks.
+enum KmsCmd {
+ ModeSet {
+ connector: u8,
+ timing: super::cp::Timing,
+ },
+ CursorCreate {
+ connector: u8,
+ w: u16,
+ h: u16,
+ },
+ CursorImage {
+ connector: u8,
+ w: u16,
+ h: u16,
+ bgra: KVec<u8>,
+ },
+ CursorMove {
+ connector: u8,
+ x: u16,
+ y: u16,
+ /// The dock's own visible flag. Hiding by parking the cursor at `u16::MAX` instead left a
+ /// ghost pointer at the top-left of both panels: the dock wraps an out-of-range origin
+ /// rather than clipping the cursor away.
+ visible: bool,
+ },
+ /// Drive the stream black and close its control-protocol bracket.
+ Blank {
+ connector: u8,
+ },
+}
+
+impl KmsCmd {
+ fn connector(&self) -> usize {
+ match self {
+ Self::ModeSet { connector, .. }
+ | Self::CursorCreate { connector, .. }
+ | Self::CursorImage { connector, .. }
+ | Self::CursorMove { connector, .. }
+ | Self::Blank { connector } => *connector as usize,
+ }
+ }
+}
+
+/// The cursor state one connector's dock connector is holding, as last acknowledged on the wire.
+///
+/// A mode set leaves the dock's cursor undefined -- `owe_keyframe` says so by bumping
+/// `cursor_epoch` -- but vino could only act on that when the compositor next committed the cursor
+/// plane. A pointer that is not moving produces no commit, so the cursor simply vanished until it
+/// was moved, and every mode set is now dock-wide, so it vanished on *both* panels for a change
+/// made to one. Keeping the last accepted bitmap and position lets the mode-set path put it back
+/// itself.
+struct CursorShot {
+ w: u16,
+ h: u16,
+ bgra: KVec<u8>,
+ x: u16,
+ y: u16,
+ visible: bool,
+}
+
+struct PendingKmsHead {
+ stream: Option<KmsCmd>,
+ cursor_create: Option<KmsCmd>,
+ cursor_image: Option<KmsCmd>,
+ cursor_move: Option<KmsCmd>,
+}
+
+impl PendingKmsHead {
+ const fn new() -> Self {
+ Self {
+ stream: None,
+ cursor_create: None,
+ cursor_image: None,
+ cursor_move: None,
+ }
+ }
+
+ fn slot(&mut self, cmd: &KmsCmd) -> &mut Option<KmsCmd> {
+ match cmd {
+ KmsCmd::ModeSet { .. } | KmsCmd::Blank { .. } => &mut self.stream,
+ KmsCmd::CursorCreate { .. } => &mut self.cursor_create,
+ KmsCmd::CursorImage { .. } => &mut self.cursor_image,
+ KmsCmd::CursorMove { .. } => &mut self.cursor_move,
+ }
+ }
+
+ fn update(&mut self, cmd: KmsCmd) {
+ let slot = self.slot(&cmd);
+ *slot = Some(cmd);
+ }
+
+ /// Restore a failed operation unless a newer desired operation already occupies its slot.
+ fn retry(&mut self, cmd: KmsCmd) {
+ let slot = self.slot(&cmd);
+ if slot.is_none() {
+ *slot = Some(cmd);
+ }
+ }
+
+ fn has_stream(&self) -> bool {
+ self.stream.is_some()
+ }
+}
+
+struct PendingKms {
+ connectors: [PendingKmsHead; MAX_CONNECTORS],
+}
+
+impl PendingKms {
+ const fn new() -> Self {
+ Self {
+ connectors: [const { PendingKmsHead::new() }; MAX_CONNECTORS],
+ }
+ }
+
+ fn is_empty(&self) -> bool {
+ self.connectors.iter().all(|connector| {
+ connector.stream.is_none()
+ && connector.cursor_create.is_none()
+ && connector.cursor_image.is_none()
+ && connector.cursor_move.is_none()
+ })
+ }
+
+ /// Discard every queued command. Used when the link they address is gone; see
+ /// `KMS_RETRY_LIMIT`.
+ fn clear(&mut self) {
+ self.connectors = [const { PendingKmsHead::new() }; MAX_CONNECTORS];
+ }
+
+ fn has_stream(&self) -> bool {
+ self.connectors.iter().any(PendingKmsHead::has_stream)
+ }
+
+ fn update(&mut self, cmd: KmsCmd) {
+ if let Some(pending) = self.connectors.get_mut(cmd.connector()) {
+ pending.update(cmd);
+ }
+ }
+
+ fn retry(&mut self, cmd: KmsCmd) {
+ if let Some(pending) = self.connectors.get_mut(cmd.connector()) {
+ pending.retry(cmd);
+ }
+ }
+
+ /// Restore a drained batch without replacing newer state published while it was executing.
+ fn retry_batch(&mut self, batch: Self) {
+ for connector in batch.connectors {
+ let PendingKmsHead {
+ stream,
+ cursor_create,
+ cursor_image,
+ cursor_move,
+ } = connector;
+ for cmd in [stream, cursor_create, cursor_image, cursor_move]
+ .into_iter()
+ .flatten()
+ {
+ self.retry(cmd);
+ }
+ }
+ }
+}
+
+fn kms_error_retryable(error: Error) -> bool {
+ error != EINVAL && error != ENOTSUPP
+}
+
+/// Latest primary-plane flip awaiting compression on the deferred worker. The framebuffer is
+/// refcounted, so it remains valid after the atomic commit callback returns. There is one slot per
+/// connector: a newer flip replaces an older unsent flip instead of building an unbounded queue
+/// behind a slow encoder. When replacement could lose accumulated damage, the newer flip is
+/// promoted to a full-output damage rectangle.
+struct PendingScanout {
+ connector: u8,
+ rotation: plane::Rotation,
+ clips: [DamageRect; MAX_DAMAGE_CLIPS],
+ nclips: usize,
+ w: usize,
+ h: usize,
+ /// Which private surface contains this commit's pixels.
+ shadow_idx: usize,
+ /// Generation of `shadow_idx`, used to reject a slot replaced before the worker claims it.
+ shadow_generation: u64,
+}
+
+impl Clone for PendingScanout {
+ fn clone(&self) -> Self {
+ Self {
+ connector: self.connector,
+ rotation: self.rotation,
+ clips: self.clips,
+ nclips: self.nclips,
+ w: self.w,
+ h: self.h,
+ shadow_idx: self.shadow_idx,
+ shadow_generation: self.shadow_generation,
+ }
+ }
+}
+
+/// Private snapshots let atomic commit copy into one while the worker encodes another. Copying
+/// before flip completion ensures that the compositor cannot reuse storage while Vino reads it.
+///
+/// Three, not two: a slot can be reserved by the encoder (`inflight`) *and* by a snapshot that has
+/// dropped the pool lock to copy (`writing`) at the same time, which with two slots left none free
+/// and silently dropped the commit. Three keeps one available in that state.
+const SHADOW_SLOTS: usize = 3;
+
+/// Maximum number of prepared compositor buffers retained per connector.
+///
+/// Compositors normally rotate through a small swapchain. Keeping four validated mappings moves
+/// vmap preparation out of repeated flips while bounding pinned memory when a client reallocates.
+const SOURCE_BINDINGS: usize = 4;
+
+struct SourceBinding {
+ framebuffer: ARef<kms::framebuffer::Framebuffer<VinoDrmDriver>>,
+ mapping: kms::framebuffer::FramebufferVMapOwned<VinoObject>,
+}
+
+struct SourceBindingCache {
+ entries: [Option<Arc<SourceBinding>>; SOURCE_BINDINGS],
+ next: usize,
+}
+
+impl SourceBindingCache {
+ const fn new() -> Self {
+ Self {
+ entries: [const { None }; SOURCE_BINDINGS],
+ next: 0,
+ }
+ }
+
+ fn get(
+ &mut self,
+ fb: &kms::framebuffer::Framebuffer<VinoDrmDriver>,
+ ) -> Result<Arc<SourceBinding>> {
+ if let Some(binding) = self
+ .entries
+ .iter()
+ .flatten()
+ .find(|binding| &*binding.framebuffer == fb)
+ {
+ return Ok(binding.clone());
+ }
+
+ let binding = Arc::new(
+ SourceBinding {
+ framebuffer: ARef::from(fb),
+ mapping: fb.owned_vmap::<VinoObject>()?,
+ },
+ GFP_KERNEL,
+ )?;
+ self.entries[self.next] = Some(binding.clone());
+ self.next = (self.next + 1) % SOURCE_BINDINGS;
+ Ok(binding)
+ }
+
+ fn discard(&mut self) {
+ self.entries = [const { None }; SOURCE_BINDINGS];
+ self.next = 0;
+ }
+}
+
+struct ShadowSurface {
+ w: usize,
+ h: usize,
+ pixels: KVVec<u8>,
+ /// Per-strip content hashes, computed while copying into this immutable snapshot.
+ hashes: KVVec<u64>,
+ /// Scratch holding one `STRIP_H`-row band of the source, in this surface's packed stride.
+ ///
+ /// The snapshot reads the source one full row at a time into this buffer and then hashes and
+ /// copies the band's strips out of it, so the source is read once per frame, sequentially, and
+ /// every strip's second pass hits a buffer small enough to stay in cache. At 2560 wide it is
+ /// 160 KiB against the 14.7 MB of `pixels`.
+ band: KVVec<u8>,
+}
+
+struct ShadowSlot {
+ generation: u64,
+ surface: Option<ShadowSurface>,
+}
+
+impl ShadowSlot {
+ const fn new() -> Self {
+ Self {
+ generation: 0,
+ surface: None,
+ }
+ }
+}
+
+/// One connector's shadow surfaces. Locked per connector: the snapshot copies ~14.7 MB while
+/// holding this, and it runs on the compositor's non-blocking commit tail, so a device-wide lock
+/// made one connector's commit stall the other's -- measured at up to 4.2 ms, half a 120 Hz frame
+/// budget.
+struct ShadowPool {
+ slots: [ShadowSlot; SHADOW_SLOTS],
+ inflight: Option<usize>,
+ /// Slot currently being written by a snapshot that has released the pool lock.
+ ///
+ /// The copy is far too long to hold the lock across, so the commit takes the slot's surface
+ /// out, drops the lock and copies into it unlocked. This marks the slot reserved for that
+ /// window, exactly as `inflight` does for the encoder's side.
+ writing: Option<usize>,
+ source_bindings: SourceBindingCache,
+}
+
+impl ShadowPool {
+ const fn new() -> Self {
+ Self {
+ slots: [const { ShadowSlot::new() }; SHADOW_SLOTS],
+ inflight: None,
+ writing: None,
+ source_bindings: SourceBindingCache::new(),
+ }
+ }
+
+ fn discard(&mut self) {
+ for slot in &mut self.slots {
+ slot.generation = slot.generation.wrapping_add(1);
+ slot.surface = None;
+ }
+ self.source_bindings.discard();
+ }
+}
+
+/// How long a cold downstream link is fed keyframes at frame cadence; see `sustain_window`.
+const SUSTAIN_MS: i64 = 3000;
+
+/// Delay before the one-shot post-keyframe repaint.
+const SETTLE_REPAINT_MS: i64 = 1200;
+
+/// Number of post-keyframe repaints. Cold-link training uses its separate bounded deadline.
+const SETTLE_REPAINTS: u32 = 1;
+
+/// Longest the DL7400 tolerates a silent video endpoint before it tears the link down.
+///
+/// Measured twice, with very different transfer shapes: a full 204 KB frame and a single 4 KB
+/// image record both ended with every outstanding URB completing `-ESHUTDOWN` 1.06 s and 1.10 s
+/// after the last video byte, the dock going deaf on the control plane at the same instant. DLM
+/// never gets near it -- it pairs a sealed report with every frame, a median 9-19 ms apart and at
+/// most 1.0 s apart even when the desktop is still.
+const NAVARRO_VIDEO_QUIET_MS: i64 = 1000;
+
+/// Period at which an idle DL7400 connector is re-fed, comfortably inside
+/// [`NAVARRO_VIDEO_QUIET_MS`].
+const NAVARRO_KEEPALIVE_MS: i64 = 250;
+
+/// Keep a missed repaint from being enough to trip the dock's teardown.
+const _: () = assert!(NAVARRO_KEEPALIVE_MS * 3 <= NAVARRO_VIDEO_QUIET_MS);
+
+/// DRM device-private data: the bound USB interface, engaged CP session, connector state, deferred
+/// scanout slots and per-connector transport state.
+#[pin_data]
+pub(super) struct VinoDrmData {
+ /// The USB I/O-permitted window for this device's interface, shared with the persistent
+ /// queues. `disconnect()` closes it, after which every transfer path here fails cleanly
+ /// instead of touching an unbound interface.
+ pub(super) io: Arc<super::usb::IoWindow>,
+ /// The dock's endpoints, resolved and direction/type-checked once during probe.
+ pub(super) endpoints: super::Endpoints,
+ /// Stops every producer before unplug drains the embedded work item. This is checked while
+ /// holding the producer's queue lock so a late atomic callback cannot enqueue a self-owning
+ /// `ARef<VinoDrmDevice>` after `cancel_sync()` has already returned.
+ shutting_down: AtomicBool,
+ #[pin]
+ cp_link: Mutex<Option<CpLink>>,
+ /// When the dock last answered anything, and whether a session exists at all.
+ ///
+ /// Deliberately outside `cp_link`. A dock that has stopped answering must be stopped talking
+ /// to, but the thread that discovers this is the one already stuck: `usb_bulk_msg` honours its
+ /// own timeout and then kills the URB, and *that* wait is unbounded, so a controller which
+ /// will not retire the transfer leaves the caller blocked uninterruptibly with `cp_link` held.
+ /// Everything that has to take the mutex to learn the link is stuck therefore blocks behind
+ /// the very transfer it is trying to diagnose -- including the keepalive's own liveness check.
+ /// A spinlock and an atomic are always available.
+ #[pin]
+ cp_last_reply: SpinLock<Instant<Monotonic>>,
+ cp_session_live: AtomicBool,
+ /// Set once this device has been asked to reset itself out of a wedged session.
+ ///
+ /// One attempt only. A reset that works re-probes into a fresh device with this cleared; a
+ /// reset that does not must not become a loop.
+ cp_reset_queued: AtomicBool,
+ /// Watchdog that enforces the silence deadline from off the control path.
+ ///
+ /// The keepalive cannot do this itself: it *is* the thread that wedges, so its own check at
+ /// the top of the loop is never reached again. Scheduled on the system queue rather than
+ /// vino's, which the stuck transaction owns.
+ #[pin]
+ cp_watchdog: DelayedWork<VinoDrmDevice, 5>,
+ /// Latest desired control/KMS state per connector.
+ #[pin]
+ pending_kms: Mutex<PendingKms>,
+ /// Coalescing per-connector scanout slots consumed by `cmd_work`.
+ ///
+ /// Compression and USB submission may sleep and therefore cannot run in
+ /// `atomic_update`.
+ #[pin]
+ pending_scanout: Mutex<[Option<PendingScanout>; MAX_CONNECTORS]>,
+ /// A one-shot repaint of the connector's newest known framebuffer. Cleared as soon as it is
+ /// taken, or whenever a real flip arrives (that flip already carries newer content, so the
+ /// redundant repaint is pointless). See [`SETTLE_REPAINT_MS`] for the hardware observation
+ /// behind it.
+ ///
+ /// The `bool` is "promote to a full keyframe". It is true for the post-keyframe settle repaint,
+ /// whose job is to replace a stale surface. It is false for a *debt* repaint, which carries
+ /// outstanding `dirty_ttl` retransmissions to the dock's second buffer without promoting them
+ /// to a keyframe.
+ #[pin]
+ settle_repaint: Mutex<[Option<(Instant<Monotonic>, PendingScanout, bool)>; MAX_CONNECTORS]>,
+ /// Private committed surfaces and their worker ownership state.
+ #[pin]
+ shadow: [Mutex<ShadowPool>; MAX_CONNECTORS],
+ /// Active software-vblank timers. The device owns their cancellation handles so shutdown does
+ /// not depend on atomic-disable callbacks running. A spinlock is required because
+ /// `enable_vblank` runs with local interrupts disabled.
+ #[pin]
+ vblank: SpinLock<[Option<(Arc<VblankTimer>, ArcHrTimerHandle<VblankTimer>)>; MAX_CONNECTORS]>,
+ /// Work item that drains control/KMS commands.
+ #[pin]
+ cmd_work: DelayedWork<VinoDrmDevice>,
+ /// Independent per-connector scanout workers. Their work IDs are const generics, so each
+ /// connector has an explicit field; transport state is taken from per-connector slots while a
+ /// frame is submitted.
+ #[pin]
+ scanout_work_h0: Work<VinoDrmDevice, 1>,
+ #[pin]
+ scanout_work_h1: Work<VinoDrmDevice, 2>,
+ #[pin]
+ scanout_work_h2: Work<VinoDrmDevice, 3>,
+ #[pin]
+ scanout_work_h3: Work<VinoDrmDevice, 4>,
+ /// Dedicated queue for initial authentication and the steady-state control session.
+ session_queue: workqueue::OwnedQueue,
+ /// Ordered queue for runtime KMS and cursor control transactions.
+ kms_queue: workqueue::OwnedQueue,
+ /// Per-device unbound queue for the two scanout workers.
+ scanout_queue: workqueue::OwnedQueue,
+ /// Downstream EDID per connector. Connector callbacks use their connector index to read this
+ /// owned state; publishing EDID therefore requires no raw pointer back into a DRM mode object.
+ #[pin]
+ cached_edids: Mutex<[Option<KVec<u8>>; MAX_CONNECTORS]>,
+ /// Bit N is set once CP confirms that a real downstream monitor is present on connector N.
+ connectors_present: core::sync::atomic::AtomicU32,
+ /// Each connector's gamma ramp cached from its CRTC atomic hook as three 256-entry 8-bit LUTs
+ /// (`[r; 256] ++ [g; 256] ++ [b; 256]`), or `None` for identity. Cached here (not read from the
+ /// CRTC state) because scanout runs in the plane path; each entry is `Copy`, so the scanout
+ /// snapshots its connector's entry under the lock and applies it without holding the lock in
+ /// the pixel loop. Per connector so a second display's gamma cannot clobber the first's.
+ #[pin]
+ color: Mutex<[Option<super::color::ColorPipeline>; MAX_CONNECTORS]>,
+ /// Per-connector strip hashes for the last frame accepted by the USB submission path. Updated
+ /// only after the complete frame has been queued, so a failed transfer can never advance the
+ /// shadow beyond what the dock may actually display.
+ #[pin]
+ strip_hashes: Mutex<[Option<StripHashState>; MAX_CONNECTORS]>,
+ /// The DL7400 per-strip size-class map most recently sent for each connector.
+ ///
+ /// The map describes the whole surface while a delta frame carries only its damaged strips, so
+ /// rebuilding it from zero each frame re-declares every untouched position as class 0. See
+ /// `video::haar::navarro_strip_params`.
+ #[pin]
+ strip_classes: Mutex<[KVec<u8>; MAX_CONNECTORS]>,
+ /// Per-strip retransmit debt. Spreading repeated updates across frames reaches both of the
+ /// dock's scanout buffers; consecutive presentations can target the same buffer.
+ #[pin]
+ dirty_ttl: Mutex<[Option<KVVec<u8>>; MAX_CONNECTORS]>,
+ /// Set once the dock engages the CP cipher (`wsub=0x45` acks > 0); EP08 scanout is gated on it.
+ /// Per device, so a second connected dock does not share one dock's engagement state.
+ cp_engaged: core::sync::atomic::AtomicBool,
+ /// Set once encrypted setup, initial sink discovery, and the platform's pre-mode-set readiness
+ /// interval have all completed. KMS producers may coalesce state before this, but no activation
+ /// may touch the dock until the bring-up worker publishes this one-way gate.
+ kms_activation_ready: core::sync::atomic::AtomicBool,
+ /// This device's codec geometry, packed; see [`VinoDrmData::geometry`] and
+ /// [`super::video::haar::Geometry`].
+ codec_geometry: core::sync::atomic::AtomicU32,
+ /// Keyframe, delta and damage-debt presentation counts, packed one byte each; see
+ /// [`super::profile::FrameDelivery`].
+ frame_delivery: core::sync::atomic::AtomicU32,
+ /// Whether a presence retry may reset a bracket beside a live connector; see
+ /// [`super::profile::ProbeBracket`].
+ probe_bracket: core::sync::atomic::AtomicU8,
+ /// Bit `h` set when connector `h`'s committed framebuffer is 10 bits per channel.
+ ///
+ /// Separate from `codec_geometry` because it is the one part of the codec's configuration that
+ /// is neither device-wide nor fixed: the DL7400 negotiates depth per connector, measured on
+ /// Windows holding one connector at 8 bits while the other ran at 10.
+ connector_ten_bit: core::sync::atomic::AtomicU32,
+ /// Bits per channel userspace asked the link to carry, one byte per connector, from `max bpc`.
+ ///
+ /// Deliberately separate from `connector_ten_bit`: that one says what the framebuffer holds and
+ /// decides how a pixel is decoded, this one says what the dock is told to carry.
+ connector_max_bpc: core::sync::atomic::AtomicU32,
+ /// Connectors whose requested ten-bit link does not fit the dock's shared bandwidth.
+ ///
+ /// Ten bits costs a third more per pixel, so a pair of modes that fits at eight may not fit at
+ /// ten. Refusing the mode is the wrong answer -- a compositor answers `EINVAL` by disabling the
+ /// output rather than choosing a shallower link -- so the depth gives way instead and both
+ /// connectors light at eight bits.
+ connector_deny_ten_bit: core::sync::atomic::AtomicU32,
+ /// Bit `h` set when connector `h`'s connector is being driven with the SMPTE ST 2084 (PQ)
+ /// transfer function, taken from the `HDR_OUTPUT_METADATA` blob userspace attached to it.
+ ///
+ /// Deliberately not folded into `connector_ten_bit`: depth and transfer function are two fields
+ /// of the dock's set-mode message and two independent decisions by the compositor.
+ head_st2084: core::sync::atomic::AtomicU32,
+ /// Which protocol generation this dock speaks; see `DockProfile::generation`. The two
+ /// platforms differ in their initialisation, per-connector HDCP framing, stream open and mode
+ /// description, so one flag drives all of them rather than three that can disagree.
+ dock_wide_modeset: core::sync::atomic::AtomicBool,
+ clear_mode_before_set: core::sync::atomic::AtomicBool,
+ blank_markers_held: core::sync::atomic::AtomicBool,
+ video_keepalive: core::sync::atomic::AtomicBool,
+ /// Whether the first frame after a mode set carries the cold ARM burst; see
+ /// `DockProfile::arm_burst`.
+ arm_burst: core::sync::atomic::AtomicBool,
+ /// How this dock states its framebuffer allocation; see [`profile::Allocation`].
+ allocation: kernel::sync::SetOnce<&'static super::profile::Allocation>,
+ /// Whether video records travel on the control bulk-OUT pipe; see
+ /// `DockProfile::video_on_ctrl_pipe`.
+ video_on_ctrl_pipe: core::sync::atomic::AtomicBool,
+ /// The `0x16/0x2e` state that takes a sink down; see `DockProfile::sink_down_state`.
+ sink_down_state: core::sync::atomic::AtomicU8,
+ post_mode_sink_states: core::sync::atomic::AtomicU16,
+ /// `DockProfile::pre_mode_sink_state`, with `u16::MAX` standing for `None`.
+ pre_mode_sink_state: core::sync::atomic::AtomicU16,
+ /// Heads whose sealed video stream has been opened, as a bitmask; see `set_video_keys`.
+ stream_opened: core::sync::atomic::AtomicU32,
+ /// Stream reports a connector still owes after its stream was opened; see
+ /// `arm_stream_prologue`.
+ stream_reports_owed: [core::sync::atomic::AtomicU32; MAX_CONNECTORS],
+ /// Consecutive deferrals of the asynchronous KMS batch; see `KMS_RETRY_LIMIT`.
+ kms_retries: core::sync::atomic::AtomicU32,
+ /// How this dock's video stream describes itself, packed: the layout word in the low sixteen
+ /// bits, the stream-marker kind above it, and the code-table form in bit 24. See
+ /// `set_video_stream_desc`.
+ video_stream_desc: core::sync::atomic::AtomicU32,
+ /// Shortest interval between frames on one connector; see `DockProfile::frame_period_ms`.
+ frame_period_us: core::sync::atomic::AtomicI64,
+ /// Interval between keepalive status queries; see `DockProfile::status_period_ms`.
+ status_period_ms: core::sync::atomic::AtomicI64,
+ /// Flat carrier frames a connector opens its stream with; see `DockProfile::carrier_frames`.
+ carrier_frames: core::sync::atomic::AtomicU32,
+ /// Sustained bytes per second this dock accepts; see `DockProfile::stream_pacing`.
+ stream_budget_bps: core::sync::atomic::AtomicU32,
+ /// Most that may leave back to back after an idle period; the credit ceiling.
+ stream_burst_bytes: core::sync::atomic::AtomicU32,
+ /// Unspent bytes of that budget, and when they were last topped up.
+ ///
+ /// One ledger for the whole dock rather than one per connector: what the budget describes is a
+ /// decoder behind a single endpoint, and two connectors sharing it spend from the same pool.
+ #[pin]
+ stream_credit: SpinLock<StreamCredit>,
+ /// How many downstream connectors this dock answers a presence probe for; see
+ /// `DockProfile::connectors`. Ridge: 2; Navarro: all four physical sockets.
+ connectors: core::sync::atomic::AtomicU8,
+ /// Excludes the independent keepalive loop while the mode worker emits the mode-relative
+ /// activation timeline. Without this, a keepalive poll can win `cp_link` between
+ /// two explicitly paced markers and stretch/reorder the sequence.
+ cp_timeline_exclusive: core::sync::atomic::AtomicBool,
+ /// Navarro's authenticated setup transcript continues directly into the first KMS
+ /// transaction: its first runtime message is a pipe clear, not a background status poll.
+ /// Hold the keepalive after publishing the session until that transaction has claimed the
+ /// control timeline.
+ initial_modeset_quiet: core::sync::atomic::AtomicBool,
+ /// Mode generation successfully programmed on each dock connector. Scanout must match it
+ /// because atomic plane updates can precede the deferred mode-set transaction.
+ modeset_active: [core::sync::atomic::AtomicU64; MAX_CONNECTORS],
+ /// Exact timing bytes most recently programmed on each connector.
+ ///
+ /// Kept apart from `modeset_active`: that atomic is the producer's request token, while
+ /// `dual_nivo` can be filled only after another connector on the endpoint publishes its own
+ /// request. No-op detection compares this exact dock-side state instead of pretending the
+ /// request token also describes a send-time topology correction.
+ #[pin]
+ programmed_timing: SpinLock<[Option<super::cp::Timing>; MAX_CONNECTORS]>,
+ /// Latest mode userspace currently requests per connector, encoded like `modeset_active`; zero
+ /// means the CRTC is disabled. The deferred worker uses this generation key to discard stale
+ /// mode-set commands and framebuffers left by a rapid disable/re-enable sequence.
+ modeset_requested: [core::sync::atomic::AtomicU64; MAX_CONNECTORS],
+ /// Whether a frame ending on a full packet is split; see
+ /// `DockProfile::split_full_packet_frame`.
+ split_full_packet_frame: AtomicBool,
+ /// Per-connector timestamp of the last accepted frame, used to bound scanout cadence.
+ #[pin]
+ last_frame: SpinLock<[Option<Instant<Monotonic>>; MAX_CONNECTORS]>,
+ /// When `queue_scanout` last ran for each connector, i.e. when KWin's commit tail last handed
+ /// us a framebuffer. Distinguishes "the compositor stopped committing" from "we dropped the
+ /// frame".
+ #[pin]
+ /// When the streaming status poll last went out, device-wide. The poll keeps the control
+ /// dialogue alive; it does not need to be per presentation.
+ #[pin]
+ last_status_poll: SpinLock<Option<Instant<Monotonic>>>,
+ /// When each connector's scanout work item last began executing.
+ #[pin]
+ /// Rate limiter for the stall diagnostic below.
+ #[pin]
+ /// Deadline for the sustained full-frame stream required to train a cold downstream link.
+ #[pin]
+ sustain_until: SpinLock<[Option<Instant<Monotonic>>; MAX_CONNECTORS]>,
+ /// Logical Haar frame sequence per connector.
+ #[pin]
+ scanout_seq: Mutex<[u32; MAX_CONNECTORS]>,
+ /// Persistent pipelined bulk-OUT queue per physical video endpoint. It remains live between
+ /// frames.
+ ///
+ /// The slot is the first connector whose endpoint address matches the caller's (see
+ /// [`UsbLink::video_pipe_index`](super::UsbLink::video_pipe_index)); duplicate slots remain
+ /// empty. Holding an individual slot mutex over a whole frame serializes connectors that share
+ /// a pipe without needlessly serializing independent endpoints.
+ #[pin]
+ video_q: [Mutex<Option<super::usb::BulkOutQueue>>; MAX_CONNECTORS],
+ /// Held by whoever is writing to a pipe that carries both planes; see [`Self::own_pipe`].
+ #[pin]
+ pipe_writer: Mutex<u8>,
+ /// One reusable 64-KiB coalescing window per connector. `frame_records` deliberately stores a
+ /// frame as small allocations so encoding never asks kmalloc for multi-megabyte physically
+ /// contiguous memory; scanout joins those fragments into this bounded window before
+ /// `BulkOutQueue::send` copies it into the persistent DMA ring. Internal record boundaries
+ /// remain invisible on USB.
+ #[pin]
+ video_staging: Mutex<[Option<KVec<u8>>; MAX_CONNECTORS]>,
+ /// Last requested timing, retained so scanout can retry a failed mode-set.
+ #[pin]
+ last_timing: SpinLock<[Option<super::cp::Timing>; MAX_CONNECTORS]>,
+ /// Heads whose next video stream must be prefixed with the pipe-arm records.
+ arm_prefix_pending: core::sync::atomic::AtomicU32,
+ /// Heads for which the read-only endpoint status at the first video stall was logged.
+ endpoint_status_logged: core::sync::atomic::AtomicU32,
+ /// Connectors still owed the short sealed open that names a stream vino does not drive.
+ ///
+ /// Held apart from `arm_prefix_pending` because it is the complement of it: a connector vino
+ /// is about to send pixels to opens its stream with the pipe descriptor instead, and both DLM
+ /// captures send this record only on the stream ids of the connectors left idle. The opens go
+ /// out before any connector's first frame, as DLM's do.
+ stream_open_pending: core::sync::atomic::AtomicU32,
+ /// Per-connector "owes a full keyframe" bitmask (bit `h` = connector `h`). Set (all connectors)
+ /// after a `KmsCmd::ModeSet` send: a new mode leaves the dock's framebuffer undefined, so the
+ /// first scanout after it must be a FULL frame ([`super::video::haar::colour_frame_ep08`]), not
+ /// a damage delta -- otherwise the un-redrawn strips stay garbage. Cleared for a connector once
+ /// its keyframe is sent; subsequent flips send only changed strips through
+ /// [`super::video::haar::colour_frame_ep08_damage`].
+ keyframe_pending: core::sync::atomic::AtomicU32,
+ /// Per-connector generation of the dock's cursor bitmap, bumped by [`Self::owe_keyframe`].
+ ///
+ /// The cursor plane re-uploads only when its bitmap differs from the last one sent, so it needs
+ /// to know when the dock stopped holding that bitmap. A mode-set discards it.
+ cursor_epoch: [core::sync::atomic::AtomicU32; MAX_CONNECTORS],
+ /// Rotates the shadow slot each commit so successive snapshots do not land in the same one.
+ shadow_rr: [core::sync::atomic::AtomicU32; MAX_CONNECTORS],
+ /// Geometry last announced with `cursor_create`, per connector. Whether the dock keeps one
+ /// shared cursor bitmap or one per connector is not established, so each connector announces
+ /// and uploads its own -- correct either way, at the cost of one extra upload per shape change.
+ #[pin]
+ cursor_geometry: Mutex<[Option<(u16, u16)>; MAX_CONNECTORS]>,
+ /// Heads whose next activation is a *repair* of a sink the dock dropped underneath us,
+ /// rather than a cold bring-up. A repair must not run the cold training window: the link
+ /// is already trained, and that window presents full keyframes at [`FRAME_PERIOD_MS`]
+ /// for three seconds -- measured at 1.07 GB over 12 seconds across two connectors, which is
+ /// the documented way to destabilise this dock.
+ repair_connectors: core::sync::atomic::AtomicU32,
+ /// The cursor each connector's connector is holding; see [`CursorShot`].
+ #[pin]
+ cursor_shot: Mutex<[Option<CursorShot>; MAX_CONNECTORS]>,
+ /// Dock-wide pixel-rate budget in pixels per second; zero means unknown.
+ dock_pixel_budget: core::sync::atomic::AtomicU32,
+ /// Highest refresh rate this dock is known to drive; see `DockProfile::max_refresh_hz`.
+ max_refresh_hz: core::sync::atomic::AtomicU32,
+ /// Highest per-mode pixel clock in kHz; see `DockProfile::max_connector_clock_khz`.
+ max_connector_clock_khz: core::sync::atomic::AtomicU32,
+ /// Excludes scanout while a mode-set batch can submit on a video endpoint. Paired with
+ /// `video_inflight` using sequentially consistent store-then-check handshakes.
+ cmd_busy: core::sync::atomic::AtomicBool,
+ /// Set around `run_pending_scanout`, allowing `cmd_work` to wait for a
+ /// frame already in flight when it set [`Self::cmd_busy`].
+ video_inflight: [core::sync::atomic::AtomicBool; MAX_CONNECTORS],
+ /// Consecutive failed live-scanout frames per connector, for log rate-limiting.
+ scanout_fails: [core::sync::atomic::AtomicU64; MAX_CONNECTORS],
+ /// Upcoming page flips to skip for per-connector transport backoff.
+ scanout_skip: [core::sync::atomic::AtomicU64; MAX_CONNECTORS],
+ /// Settle repaints this connector may still arm. See [`SETTLE_REPAINTS`].
+ settle_budget: [core::sync::atomic::AtomicU32; MAX_CONNECTORS],
+ /// Last inner status returned for each connector's presence probe.
+ presence_reply: [core::sync::atomic::AtomicU32; MAX_CONNECTORS],
+ /// Pending downstream-topology notification for this device's keepalive worker.
+ downstream_event: AtomicBool,
+ /// Head currently expecting an EDID from a re-engage, or [`NO_EDID_TARGET`].
+ ///
+ /// The EDID arrives as an `id=0x194` push, and during a re-engage it lands in `send_cp`'s
+ /// own lockstep drain rather than in `drain_cp_pushes`. This says "somebody is waiting for
+ /// one", so that drain can stash it instead of discarding it.
+ edid_target: core::sync::atomic::AtomicU32,
+ /// The blob that drain caught, handed back to [`VinoDrmData::reengage_connector`].
+ #[pin]
+ edid_caught: Mutex<Option<KVec<u8>>>,
+ /// Heads intentionally blanked by Vino. Their expected probe silence is not a hot-unplug.
+ self_blanked: core::sync::atomic::AtomicU32,
+ /// Heads whose blank bracket is still open on the dock, one bit each.
+ ///
+ /// Distinct from [`Self::self_blanked`], which `atomic_enable` clears on the commit thread
+ /// before the command worker runs; by then the wake choreography would no longer know a blank
+ /// was owed a close. A bracket left open keeps the sink dark through the next mode set.
+ blank_bracket_open: core::sync::atomic::AtomicU32,
+ /// Whether this dock's video pipeline can carry ten bits per channel, from its profile.
+ hdr_capable: AtomicBool,
+ /// Whether this dock composites a cursor bitmap of its own; see [`DockProfile::hw_cursor`].
+ hw_cursor: AtomicBool,
+ /// Whether the dock's presence probe describes a connector; see
+ /// [`DockProfile::reports_presence`].
+ reports_presence: AtomicBool,
+ /// Whether the connectors share one EDID handler; see [`DockProfile::shared_edid_handler`].
+ shared_edid_handler: AtomicBool,
+ /// Per-connector key and nonce used to seal pipe-arm records.
+ #[pin]
+ video_keys: Mutex<[kernel::crypto::Secret<32>; MAX_CONNECTORS]>,
+ /// When each connector last put a byte on its video endpoint.
+ ///
+ /// Drives the DL7400 keep-alive: see [`NAVARRO_VIDEO_QUIET_MS`] for why a connector that has
+ /// nothing to draw still has to say something.
+ #[pin]
+ last_video_at: SpinLock<[Option<Instant<Monotonic>>; MAX_CONNECTORS]>,
+ /// Per-connector AES-CTR block counter for the sealed records on that connector's video stream.
+ ///
+ /// Every sealed video record carries this counter in its wire `seq`, and `seal_livemac` uses
+ /// it both as the CTR block index and as the Dl3Cmac counter. It is stream state, not record
+ /// state: DLM advances it by `ceil(plaintext / 16)` for every sealed record it sends on a
+ /// stream and never rewinds it, so a re-arm continues the count rather than restarting. It is
+ /// reset only when new video keys arrive, because a fresh key is a fresh keystream.
+ video_seal_seq: [core::sync::atomic::AtomicU32; MAX_CONNECTORS],
+}
+
+impl VinoDrmData {
+ /// `hdr_capable`, `hw_cursor` and `connectors` come from the dock's profile and must be
+ /// supplied here rather than stored afterwards: `create_objects` runs inside
+ /// `drm::Registration::new_static`, which is *before* probe reaches the block that publishes
+ /// the rest of the profile. Set late, they were always false while the connectors and planes
+ /// were being built, so the ten-bit format and the three HDR properties were silently never
+ /// attached, and no dock could ever withhold its cursor plane.
+ ///
+ /// `connectors` decides how many connectors exist at all. A dock that advertises more
+ /// connectors than it has sockets offers userspace outputs that can never carry a monitor, and
+ /// a compositor that enables one makes the driver encode and transmit full frames to nothing.
+ pub(super) fn new(
+ io: Arc<super::usb::IoWindow>,
+ endpoints: super::Endpoints,
+ hdr_capable: bool,
+ hw_cursor: bool,
+ connectors: u8,
+ ) -> impl PinInit<Self, Error> {
+ try_pin_init!(Self {
+ io,
+ endpoints,
+ shutting_down: AtomicBool::new(false),
+ cp_link <- new_mutex!(Option::<CpLink>::None),
+ cp_last_reply <- new_spinlock!(Instant::<Monotonic>::now()),
+ cp_session_live: AtomicBool::new(false),
+ cp_reset_queued: AtomicBool::new(false),
+ cp_watchdog <- new_delayed_work!("vino::cp_watchdog"),
+ pending_kms <- new_mutex!(PendingKms::new()),
+ pending_scanout <- new_mutex!([const { None }; MAX_CONNECTORS]),
+ settle_repaint <- new_mutex!([const { None }; MAX_CONNECTORS]),
+ shadow <- pin_init::pin_init_array_from_fn(|_| new_mutex!(ShadowPool::new())),
+ vblank <- new_spinlock!([const { None }; MAX_CONNECTORS]),
+ cmd_work <- new_delayed_work!("vino::kms_cmd"),
+ scanout_work_h0 <- new_work!("vino::scanout_h0"),
+ scanout_work_h1 <- new_work!("vino::scanout_h1"),
+ scanout_work_h2 <- new_work!("vino::scanout_h2"),
+ scanout_work_h3 <- new_work!("vino::scanout_h3"),
+ session_queue: workqueue::Queue::new_ordered().build(kernel::c_str!("vino_session"))?,
+ // High priority: this queue carries cursor movement, and its work items are a few
+ // small control messages. Left at default priority they queue behind whatever else
+ // the machine is doing, and the pointer visibly stutters under load.
+ kms_queue: workqueue::Queue::new_ordered()
+ .highpri()
+ .build(kernel::c_str!("vino_kms"))?,
+ scanout_queue: workqueue::Queue::new_unbound()
+ .max_active(MAX_CONNECTORS as u32)
+ .build(kernel::c_str!("vino_scanout"))?,
+ cached_edids <- new_mutex!([const { None }; MAX_CONNECTORS]),
+ connectors_present: core::sync::atomic::AtomicU32::new(0),
+ color <- new_mutex!([None; MAX_CONNECTORS]),
+ strip_hashes <- new_mutex!([const { None }; MAX_CONNECTORS]),
+ strip_classes <- new_mutex!(core::array::from_fn(|_| KVec::new())),
+ dirty_ttl <- new_mutex!([const { None }; MAX_CONNECTORS]),
+ cp_engaged: core::sync::atomic::AtomicBool::new(false),
+ kms_activation_ready: core::sync::atomic::AtomicBool::new(false),
+ cp_timeline_exclusive: core::sync::atomic::AtomicBool::new(false),
+ initial_modeset_quiet: core::sync::atomic::AtomicBool::new(false),
+ modeset_active: core::array::from_fn(|_| core::sync::atomic::AtomicU64::new(0)),
+ programmed_timing <- new_spinlock!([None; MAX_CONNECTORS]),
+ modeset_requested: core::array::from_fn(|_| core::sync::atomic::AtomicU64::new(0)),
+ split_full_packet_frame: AtomicBool::new(false),
+ last_frame <- new_spinlock!([const { None }; MAX_CONNECTORS]),
+ last_status_poll <- new_spinlock!(None),
+ sustain_until <- new_spinlock!([const { None }; MAX_CONNECTORS]),
+ scanout_seq <- new_mutex!([0; MAX_CONNECTORS]),
+ video_q <- pin_init::pin_init_array_from_fn(|_| new_mutex!(None)),
+ pipe_writer <- new_mutex!(0u8),
+ video_staging <- new_mutex!([const { None }; MAX_CONNECTORS]),
+ last_timing <- new_spinlock!([None; MAX_CONNECTORS]),
+ arm_prefix_pending: core::sync::atomic::AtomicU32::new(0),
+ endpoint_status_logged: core::sync::atomic::AtomicU32::new(0),
+ stream_open_pending: core::sync::atomic::AtomicU32::new(0),
+ keyframe_pending: core::sync::atomic::AtomicU32::new(0),
+ cursor_epoch: core::array::from_fn(|_| core::sync::atomic::AtomicU32::new(0)),
+ shadow_rr: core::array::from_fn(|_| core::sync::atomic::AtomicU32::new(0)),
+ cursor_geometry <- new_mutex!([None; MAX_CONNECTORS]),
+ repair_connectors: core::sync::atomic::AtomicU32::new(0),
+ cursor_shot <- new_mutex!([const { None }; MAX_CONNECTORS]),
+ // D6000 default: 442,368,000 px/s (one 1440p@120) x2 compression headroom = dual
+ // 1440p@120. Replace it if a dock capability supplies a limit.
+ dock_pixel_budget: core::sync::atomic::AtomicU32::new(884_736_000),
+ max_refresh_hz: core::sync::atomic::AtomicU32::new(DEFAULT_MAX_REFRESH_HZ),
+ max_connector_clock_khz: core::sync::atomic::AtomicU32::new(DEFAULT_MAX_HEAD_CLOCK_KHZ),
+ cmd_busy: core::sync::atomic::AtomicBool::new(false),
+ video_inflight: core::array::from_fn(|_| core::sync::atomic::AtomicBool::new(false)),
+ scanout_fails: core::array::from_fn(|_| core::sync::atomic::AtomicU64::new(0)),
+ scanout_skip: core::array::from_fn(|_| core::sync::atomic::AtomicU64::new(0)),
+ settle_budget: core::array::from_fn(|_| core::sync::atomic::AtomicU32::new(0)),
+ presence_reply: core::array::from_fn(|_| core::sync::atomic::AtomicU32::new(0)),
+ downstream_event: AtomicBool::new(false),
+ edid_target: core::sync::atomic::AtomicU32::new(NO_EDID_TARGET),
+ edid_caught <- new_mutex!(None),
+ self_blanked: core::sync::atomic::AtomicU32::new(0),
+ blank_bracket_open: core::sync::atomic::AtomicU32::new(0),
+ hdr_capable: AtomicBool::new(hdr_capable),
+ hw_cursor: AtomicBool::new(hw_cursor),
+ reports_presence: AtomicBool::new(true),
+ shared_edid_handler: AtomicBool::new(false),
+ codec_geometry: core::sync::atomic::AtomicU32::new(0),
+ // Ridge-compatible defaults until probe publishes the matched profile.
+ frame_delivery: core::sync::atomic::AtomicU32::new(2 | (1 << 8) | (3 << 16)),
+ probe_bracket: core::sync::atomic::AtomicU8::new(
+ super::profile::ProbeBracket::Always as u8
+ ),
+ connector_ten_bit: core::sync::atomic::AtomicU32::new(0),
+ connector_max_bpc: core::sync::atomic::AtomicU32::new(0),
+ connector_deny_ten_bit: core::sync::atomic::AtomicU32::new(0),
+ head_st2084: core::sync::atomic::AtomicU32::new(0),
+ // A dock that names no connector count still has to expose something, so fall back to
+ // the maximum rather than building a card with no connectors at all.
+ connectors: core::sync::atomic::AtomicU8::new(if connectors == 0 {
+ MAX_CONNECTORS as u8
+ } else {
+ connectors.min(MAX_CONNECTORS as u8)
+ }),
+ dock_wide_modeset: core::sync::atomic::AtomicBool::new(false),
+ clear_mode_before_set: core::sync::atomic::AtomicBool::new(false),
+ blank_markers_held: core::sync::atomic::AtomicBool::new(false),
+ video_keepalive: core::sync::atomic::AtomicBool::new(false),
+ arm_burst: core::sync::atomic::AtomicBool::new(true),
+ allocation: kernel::sync::SetOnce::new(),
+ video_on_ctrl_pipe: core::sync::atomic::AtomicBool::new(false),
+ sink_down_state: core::sync::atomic::AtomicU8::new(BLANK_MARKER_STATE),
+ post_mode_sink_states: core::sync::atomic::AtomicU16::new(0x0303),
+ pre_mode_sink_state: core::sync::atomic::AtomicU16::new(u16::MAX),
+ stream_opened: core::sync::atomic::AtomicU32::new(0),
+ stream_reports_owed: core::array::from_fn(|_| core::sync::atomic::AtomicU32::new(0)),
+ kms_retries: core::sync::atomic::AtomicU32::new(0),
+ video_stream_desc: core::sync::atomic::AtomicU32::new(0),
+ frame_period_us: core::sync::atomic::AtomicI64::new(FRAME_PERIOD_US),
+ status_period_ms: core::sync::atomic::AtomicI64::new(STATUS_PERIOD_MS),
+ carrier_frames: core::sync::atomic::AtomicU32::new(u32::MAX),
+ stream_budget_bps: core::sync::atomic::AtomicU32::new(u32::MAX),
+ stream_burst_bytes: core::sync::atomic::AtomicU32::new(u32::MAX),
+ stream_credit <- new_spinlock!(StreamCredit::new()),
+ video_keys <- new_mutex!(core::array::from_fn(
+ |_| kernel::crypto::Secret::zeroed()
+ )),
+ last_video_at <- new_spinlock!([None; MAX_CONNECTORS]),
+ video_seal_seq: core::array::from_fn(
+ |_| core::sync::atomic::AtomicU32::new(0)
+ ),
+ })
+ }
+
+ /// Publish the producers' stop flag and nothing else.
+ ///
+ /// `disconnect()` calls this *before* `IoWindow::close()`. The scanout and command workers each
+ /// hold an `Io` token for as long as they loop and re-read `shutting_down` every iteration, so
+ /// setting it early is what keeps them from holding `close()`'s wait open. Everything in
+ /// [`shutdown`](Self::shutdown) proper must wait until USB I/O is quiesced; this must not.
+ pub(super) fn begin_shutdown(&self) {
+ self.shutting_down.store(true, Ordering::Release);
+ self.kms_activation_ready.store(false, Ordering::Release);
+ self.cp_timeline_exclusive.store(false, Ordering::Release);
+ self.initial_modeset_quiet.store(false, Ordering::Release);
+ }
+
+ /// Stand every producer down because the device is about to be reset.
+ ///
+ /// A reset takes the whole session with it: the dock forgets its content-protection keys, its
+ /// open streams and the sinks it was driving, and nothing this driver holds describes the
+ /// device on the other side of one. So the link is marked gone before the reset rather than
+ /// after, which is what stops a worker submitting a transfer across it.
+ pub(super) fn stop_for_reset(&self) {
+ self.cp_session_live.store(false, Ordering::Release);
+ self.begin_shutdown();
+ }
+
+ /// Whether the parent interface is being removed.
+ pub(super) fn is_shutting_down(&self) -> bool {
+ self.shutting_down.load(Ordering::Acquire)
+ }
+
+ /// Queue used by the session bring-up and keepalive work item.
+ pub(super) fn session_queue(&self) -> &workqueue::Queue {
+ &self.session_queue
+ }
+
+ /// Take this device's pending downstream-topology notification.
+ pub(super) fn take_downstream_event(&self) -> bool {
+ self.downstream_event.swap(false, Ordering::Acquire)
+ }
+
+ /// Stop deferred DRM work while the parent USB interface is still bound. `cmd_work` is
+ /// embedded in this DRM device and each successful enqueue temporarily owns an
+ /// `ARef<VinoDrmDevice>`; pending scanouts also retain compositor framebuffers. Quiesce both
+ /// producers, reclaim any queued work pointer, and drop those framebuffers before the final
+ /// device references disappear during devres teardown.
+ pub(super) fn shutdown(&self) {
+ // Idempotent, and `disconnect()` has normally already done this: see `begin_shutdown`.
+ self.begin_shutdown();
+ for mode in &self.modeset_requested {
+ mode.store(0, Ordering::Release);
+ }
+
+ // Stop the software vblank clocks before releasing their CRTC references. Take the
+ // registry out from under the spinlock before dropping the handles:
+ // Dropping `ArcHrTimerHandle` waits for a running callback and must
+ // not happen in atomic context.
+ let timers = {
+ let mut slots = self.vblank.lock();
+ core::mem::replace(&mut *slots, [const { None }; MAX_CONNECTORS])
+ };
+ // Clear `enabled` before cancelling so a callback racing the cancel returns `NoRestart`
+ // instead of re-arming behind it.
+ for (timer, _) in timers.iter().flatten() {
+ timer.enabled.store(false, Ordering::Relaxed);
+ }
+ // Split the registry: drop every `ArcHrTimerHandle` (each drop == `hrtimer_cancel`, which
+ // waits for a running callback), but keep the `Arc<VblankTimer>`s alive so the published
+ // CRTC handles can be released below. From here on no vblank callback can run or be
+ // re-armed, because `VinoCrtc::vblank` is only reachable through a CRTC of this device and
+ // every producer is already refusing work.
+ let timers = timers.map(|slot| {
+ slot.map(|(timer, handle)| {
+ drop(handle);
+ timer
+ })
+ });
+
+ // Break the two device-to-itself reference cycles. Both run through a `crtc::CrtcRef`,
+ // which owns an `ARef<VinoDrmDevice>`:
+ //
+ // 1. `VblankTimer::crtc`, published by the first `enable_vblank` and never released. The
+ // timer is owned by `VinoCrtc`, which lives inside the DRM device allocation.
+ // 2. `VinoCrtc::vblank_pinned`, the driver-held vblank reference. Teardown cannot rely on
+ // `atomic_disable` running before unplug.
+ //
+ // Safe to do here even though these were the last self-references: `shutdown()`'s only
+ // caller is `VinoDriver::disconnect`, which reaches it through the `drm::Registration`
+ // still held in the bound data -- and that owns an `ARef<VinoDrmDevice>` of its own -- so
+ // `&self` outlives this function regardless of what is dropped below. The taken values
+ // are dropped outside both locks: `drm_dev_put()` can end in `drm_dev_release()` and
+ // `drm_crtc_vblank_put()` takes the DRM vblank locks, neither of which may run under our
+ // spinlock.
+ for timer in timers.iter().flatten() {
+ let published = timer.crtc.lock().take();
+ if let Some(crtc_ref) = published {
+ // The software vblank clock has just stopped, and a page flip armed by
+ // `atomic_flush` is waiting on a tick that will never come. `drm_crtc_vblank_off()`
+ // both refuses further vblank references -- so `drm_atomic_helper_wait_for_vblanks`
+ // skips this CRTC instead of warning -- and sends every event still queued on the
+ // device's vblank list, which is exactly where `PendingVblankEvent::arm` put ours.
+ //
+ // Without it an unplug left the compositor's `commit_tail` blocked until DRM's own
+ // deadlines expired: this boot logged 73 `vblank wait timed out` warnings and 110
+ // pairs of `flip_done timed out` / `commit wait timed out`, ten seconds each, on
+ // top of every dock reset. That is most of the delay between a dock coming back and
+ // pixels reappearing.
+ crtc_ref.crtc().vblank_off();
+ let crtc: &VinoCrtc = crtc_ref.crtc();
+ drop(crtc.vblank_pinned.lock().take());
+ drop(crtc_ref);
+ }
+ }
+ drop(timers);
+
+ *self.pending_kms.lock() = PendingKms::new();
+ *self.pending_scanout.lock() = [const { None }; MAX_CONNECTORS];
+ *self.settle_repaint.lock() = [const { None }; MAX_CONNECTORS];
+ for h in 0..MAX_CONNECTORS {
+ self.shadow[h].lock().discard();
+ }
+ *self.strip_hashes.lock() = [const { None }; MAX_CONNECTORS];
+ *self.dirty_ttl.lock() = [const { None }; MAX_CONNECTORS];
+ // Cancel the queued drain and reclaim the `ARef<VinoDrmDevice>` the enqueue handed to
+ // the workqueue, if it was still pending. Dropping it here releases the self-reference
+ // that would otherwise keep this device alive until the work ran.
+ //
+ // Cancel `cmd_work` first because it can enqueue both scanout workers. `shutting_down` is
+ // already visible to all workers, so cancellation only waits for work already in flight.
+ drop(self.cmd_work.cancel_sync());
+ drop(self.cp_watchdog.cancel_sync());
+ drop(self.scanout_work_h0.cancel_sync());
+ drop(self.scanout_work_h1.cancel_sync());
+ drop(self.scanout_work_h2.cancel_sync());
+ drop(self.scanout_work_h3.cancel_sync());
+
+ // A running callback may have taken a batch just before shutdown was published. It has
+ // finished now; clear anything it left behind and tear the USB queues down while their
+ // parent interface is still in Bound context.
+ *self.pending_kms.lock() = PendingKms::new();
+ *self.pending_scanout.lock() = [const { None }; MAX_CONNECTORS];
+ *self.settle_repaint.lock() = [const { None }; MAX_CONNECTORS];
+ for h in 0..MAX_CONNECTORS {
+ self.shadow[h].lock().discard();
+ }
+ *self.strip_hashes.lock() = [const { None }; MAX_CONNECTORS];
+ *self.dirty_ttl.lock() = [const { None }; MAX_CONNECTORS];
+ for queue in &self.video_q {
+ *queue.lock() = None;
+ }
+ *self.video_staging.lock() = [const { None }; MAX_CONNECTORS];
+ self.cp_session_live.store(false, Ordering::Release);
+ *self.cp_link.lock() = None;
+ vino_debug!("vino: deferred KMS/video work drained for unplug\n");
+ }
+
+ /// Cache `connector`'s CRTC colour transform (from `RawCrtcState::gamma_lut` and
+ /// `RawCrtcState::ctm`) for the scanout to apply, or clear it to identity with two `None`s.
+ pub(super) fn update_color(
+ &self,
+ connector: usize,
+ lut: Option<&[crtc::ColorLut]>,
+ ctm: Option<&crtc::ColorCtm>,
+ ) {
+ let socket = connector + 1;
+ let cached = super::color::ColorPipeline::build(lut, ctm);
+ let changed = if let Some(slot) = self.color.lock().get_mut(connector) {
+ if *slot == cached {
+ false
+ } else {
+ *slot = cached;
+ true
+ }
+ } else {
+ false
+ };
+ if changed {
+ if cached.is_some() {
+ vino_debug!("vino: socket {socket} colour transform updated\n");
+ } else {
+ vino_debug!("vino: socket {socket} colour transform cleared\n");
+ }
+ // The encoded-strip cache keys on a strip's source pixels, so a transform change that
+ // leaves those pixels untouched would otherwise re-send stale bodies for the whole
+ // screen. Drop the cache and owe a keyframe.
+ self.strip_hashes.lock()[connector] = None;
+ self.dirty_ttl.lock()[connector] = None;
+ self.owe_keyframe(connector);
+ }
+ }
+
+ /// Snapshot `connector`'s cached colour transform for a scanout pass (`Copy`, so no lock is
+ /// held afterwards).
+ pub(super) fn color_snapshot(&self, connector: usize) -> Option<super::color::ColorPipeline> {
+ self.color.lock().get(connector).copied().flatten()
+ }
+
+ /// Number of physical connectors selected by the matched dock profile.
+ pub(super) fn connector_count(&self) -> usize {
+ usize::from(self.connectors.load(Ordering::Acquire)).min(MAX_CONNECTORS)
+ }
+
+ /// Whether `connector` represents a distinct runtime stream on this dock.
+ ///
+ /// Every physical connector does, including both halves of a shared endpoint: the DL7400 maps
+ /// its four connectors in pairs onto two video bulk endpoints, and treating the second of each
+ /// pair as an alias would make a monitor in socket 3 or 4 invisible. Sharing an endpoint is a
+ /// transport detail, handled where it belongs by [`UsbLink::video_pipe_index`], which gives
+ /// both connectors of a pair the same persistent queue.
+ ///
+ /// Empty sockets cost nothing here: the presence probe answers negative for them, and the
+ /// keepalive's re-engage retry stands down permanently once it has.
+ pub(super) fn runtime_connector(&self, connector: usize) -> bool {
+ connector < self.connector_count()
+ }
+
+ /// Whether bring-up has reached the point where KMS may touch the dock.
+ pub(super) fn kms_activation_ready(&self) -> bool {
+ self.kms_activation_ready.load(Ordering::Acquire)
+ }
+
+ /// Take every connector down once the control session has been abandoned.
+ ///
+ /// Userspace can move its windows off a connector that has disappeared, but not off one that
+ /// is merely frozen. Recovery is a replug, which rebinds and starts a fresh session.
+ pub(super) fn drop_connectors_with_session(&self, drm_dev: &VinoDrmDevice) {
+ let mut dropped = false;
+ for connector in 0..self.connector_count() {
+ if !self.runtime_connector(connector) || !self.connector_present(connector) {
+ continue;
+ }
+ self.set_disconnected(connector);
+ dropped = true;
+ pr_warn!(
+ "vino: socket {socket} dropped with the control session\n",
+ socket = connector + 1
+ );
+ }
+ if dropped {
+ drm_dev.hotplug_event();
+ }
+ }
+
+ /// Take the shared pipe for one indivisible sequence of writes.
+ ///
+ /// A record is never split: the vendor's control records sit between records, never inside
+ /// one. On a dock with a video pipe of its own that is free -- the two planes cannot collide.
+ /// Here they share an endpoint, and a control write submitted between two of a frame's URBs
+ /// lands in the middle of an image record, where it desynchronises the dock's parser for the
+ /// rest of the frame. The dock accepts every byte and shows nothing, which is indistinguishable
+ /// from a dead sink.
+ ///
+ /// Returns `None` on a dock whose planes have separate endpoints, where the exclusion would
+ /// only cost the control plane latency.
+ ///
+ /// Lock order is `cp_link` then this then `video_q`. Nothing may send a control message while
+ /// holding it.
+ pub(super) fn own_pipe(
+ &self,
+ ) -> Option<kernel::sync::lock::Guard<'_, u8, kernel::sync::lock::mutex::MutexBackend>> {
+ self.video_on_ctrl_pipe().then(|| self.pipe_writer.lock())
+ }
+
+ /// Retire every outstanding URB after a physical video queue reports an error.
+ ///
+ /// The caller must still own this endpoint's `own_pipe()` guard (when present) and its
+ /// canonical `video_q` slot. Taking and explicitly dropping the complete queue synchronously
+ /// kills every submitted URB before a stalled endpoint is cleared. In particular, no queued
+ /// frame tail may resume after `usb_clear_halt()` without the prefix that made it parseable.
+ /// Keeping the canonical slot empty also makes the next writer create a fresh queue on a
+ /// dedicated video endpoint.
+ ///
+ /// A shared control/video pipe cannot resume locally. Submission advances the host's frame
+ /// and ring counters before asynchronous URBs complete, so cancellation can leave the dock
+ /// expecting an earlier frame than Vino. There is no independent endpoint to re-arm and no
+ /// safe counter-only rewind; abandon the complete session and let USB reset establish a new
+ /// one instead. Dedicated video endpoints retain their local drain/clear recovery.
+ pub(super) fn retire_failed_video_queue(
+ &self,
+ dev: &BoundInterface<'_>,
+ connector: usize,
+ queue_slot: &mut Option<super::usb::BulkOutQueue>,
+ cause: Error,
+ clear_halt: bool,
+ ) -> Result {
+ let doomed = queue_slot.take();
+ drop(doomed);
+
+ vino_debug!(
+ "vino: connector={} retired failed physical video queue ({:?})\n",
+ connector,
+ cause
+ );
+ if self.video_on_ctrl_pipe()
+ && (cause == kernel::error::code::EPIPE
+ || cause == kernel::error::code::EPROTO
+ || cause == kernel::error::code::ETIMEDOUT)
+ {
+ // Publish the terminal state before invalidating connectors or requesting reset. A CP
+ // writer already queued behind `own_pipe()` rechecks this flag after it acquires the
+ // pipe, and a scanout writer may not recreate the canonical queue once it is false.
+ if self
+ .cp_session_live
+ .compare_exchange(true, false, Ordering::AcqRel, Ordering::Acquire)
+ .is_ok()
+ {
+ pr_warn!(
+ "vino: shared video/control pipe failed ({:?}); abandoning the session\n",
+ cause
+ );
+ }
+ // This device instance is terminal even though USB reset is asynchronous. Close the
+ // producer gate as well as the transport gate so KMS callbacks may coalesce their
+ // latest state, but cannot enqueue another activation in the reset window. The fresh
+ // probe owns a new VinoDrmData and publishes its own readiness after setup completes.
+ self.kms_activation_ready.store(false, Ordering::Release);
+ let mut programmed = self.programmed_timing.lock();
+ for h in 0..self.connector_count() {
+ self.modeset_active[h].store(0, Ordering::Release);
+ programmed[h] = None;
+ }
+ drop(programmed);
+ self.reset_after_wedge();
+ return Ok(());
+ }
+ if clear_halt
+ && (cause == kernel::error::code::EPIPE || cause == kernel::error::code::EPROTO)
+ {
+ dev.clear_video_halt(connector)?;
+ pr_info!(
+ "vino: connector {} video queue drained and endpoint halt cleared\n",
+ connector
+ );
+ }
+ Ok(())
+ }
+
+ /// Keep Navarro's setup-to-first-mode-set control stream free of background traffic.
+ pub(super) fn hold_cp_for_initial_modeset(&self) {
+ self.initial_modeset_quiet.store(true, Ordering::Release);
+ }
+
+ /// Whether the initial Navarro mode set still owns the next control message.
+ pub(super) fn initial_modeset_quiet(&self) -> bool {
+ self.initial_modeset_quiet.load(Ordering::Acquire)
+ }
+
+ /// Release the initial hold if userspace never submits a mode set.
+ pub(super) fn release_initial_modeset_quiet(&self) {
+ self.initial_modeset_quiet.store(false, Ordering::Release);
+ }
+
+ /// Store the per-connector video keys produced by the `id=0x32` exchange.
+ ///
+ /// Called with [`publish_session`](Self::publish_session) when CP engages. `opened` is the
+ /// connector bitmask whose streams the setup burst opened; each of those consumed block zero of
+ /// its stream, so its chain continues at block one. A connector not in the mask had no sink at
+ /// setup time and still owes its open, so its chain must start at zero -- sealing at one leaves
+ /// a gap the dock accounts for as a keystream it never received, and it discards the record
+ /// with nothing on the wire to say so.
+ pub(super) fn set_video_keys(
+ &self,
+ keys: [kernel::crypto::Secret<32>; MAX_CONNECTORS],
+ opened: u32,
+ ) {
+ *self.video_keys.lock() = keys;
+ self.stream_opened.store(opened, Ordering::Release);
+ // A new key is a new keystream, so the block counters start over with it.
+ for (connector, seq) in self.video_seal_seq.iter().enumerate() {
+ let used = u32::from(opened & (1u32 << connector) != 0);
+ seq.store(used, Ordering::Release);
+ }
+ }
+
+ /// Make a connector owe the records that open its stream, ahead of its next frame.
+ ///
+ /// On a dock that shares its control pipe the prologue restarts the stream, and the dock's
+ /// frame counter with it: DLM's next opener names ring slot 0 and frame 1 whatever the
+ /// connector had reached before. Carrying the old count over hands the dock a slot it is still
+ /// scanning out. A dock with a video pipe of its own has no such restart, and keeps counting.
+ fn arm_stream_prologue(&self, connector: usize) {
+ self.arm_prefix_pending
+ .fetch_or(1u32 << connector, Ordering::Release);
+ if self.video_on_ctrl_pipe() && connector < MAX_CONNECTORS {
+ self.scanout_seq.lock()[connector] = 0;
+ self.stream_reports_owed[connector].store(STREAM_REPORT_BURST, Ordering::Release);
+ }
+ }
+
+ /// Reserve `blocks` AES-CTR blocks on a connector's video stream and return the counter to seal
+ /// at.
+ ///
+ /// Sealed video records must tile the stream's keystream without gaps or overlaps: the dock
+ /// tracks the same counter, and a record that repeats a block a previous record already used is
+ /// a replay of that keystream. Reserving before sealing keeps that true no matter how the
+ /// records are grouped into transfers.
+ fn take_seal_seq(&self, connector: usize, blocks: u32) -> u32 {
+ self.video_seal_seq[connector].fetch_add(blocks, Ordering::AcqRel)
+ }
+
+ /// Status polls issued immediately before the first video presentation of a mode generation.
+ ///
+ /// Captured sequences interleave two status messages here and begin video while the stream
+ /// bracket is still active. The longer downstream training interval follows the bracket.
+ const PREWRITE_POLLS: u32 = 2;
+ const PREWRITE_POLL_MS: u64 = 1;
+ /// Send one `id=0x14 sub=0x000c` device-status poll.
+ fn poll_status(&self, dev: &BoundInterface<'_>) -> Result {
+ self.send_cp(dev, 0x14, 0, |ctr| super::cp::device_query_req(ctr, 0x000c))
+ }
+
+ /// One `id=0x16 sub=0x2e|0x2f` stream/display marker. State lives in byte 23, not byte 22
+ /// (byte 22 is constantly `1` -- reading it makes every marker look like state=1).
+ fn stream_marker(&self, dev: &BoundInterface<'_>, connector: u8, sub: u16, st: u8) -> Result {
+ self.send_cp(dev, 0x16, 0, |ctr| {
+ super::cp::stream_marker(ctr, connector, sub, st)
+ })
+ }
+
+ /// Send one captured Navarro sink-reset operation.
+ fn navarro_cold_op(&self, dev: &BoundInterface<'_>, op: NavarroColdOp) -> Result {
+ match op {
+ NavarroColdOp::Poll => self.poll_status(dev),
+ NavarroColdOp::EdidState(connector, state) => self.send_cp(dev, 0x16, 0, |ctr| {
+ super::cp::edid_readiness_state(ctr, connector, state)
+ }),
+ NavarroColdOp::Probe(connector) => self.send_cp(dev, 0x15, 0, |ctr| {
+ super::cp::get_edid_req_sub(ctr, 0x20, connector)
+ }),
+ NavarroColdOp::Fetch(connector) => {
+ self.send_cp(dev, 0x15, 0, |ctr| super::cp::get_edid_req(ctr, connector))
+ }
+ NavarroColdOp::SinkTeardown(connector) => self.send_cp(dev, 0x16, 0, |ctr| {
+ super::cp::edid_sink_state(ctr, connector, 0xff)
+ }),
+ NavarroColdOp::Engage(connector) => self.send_cp(dev, 0x16, 0, |ctr| {
+ super::cp::edid_engage_req(ctr, connector)
+ }),
+ NavarroColdOp::PostEdid(connector) => self.send_cp(dev, 0x15, 0, |ctr| {
+ super::cp::post_edid_query(ctr, connector)
+ }),
+ NavarroColdOp::Clear(connector) => {
+ self.send_cp(dev, 0x48, 0, |ctr| super::cp::clear_mode(ctr, connector))
+ }
+ }
+ }
+
+ /// Whether another connector on `connector`'s video endpoint is also being driven.
+ ///
+ /// `0x08` owns connectors {0, 2} and `0x0a` owns {1, 3}, so a connector's partner is the one
+ /// two away. Such a pair must declare `Dual NIVO` in both mode sets or the dock drives only one
+ /// of the two streams it is sent, however correctly they are tagged.
+ pub(super) fn endpoint_is_shared(&self, connector: usize) -> bool {
+ self.endpoint_is_shared_in_mask(connector, self.requested_connector_mask())
+ }
+
+ /// Requested connectors as one topology snapshot.
+ fn requested_connector_mask(&self) -> u32 {
+ self.modeset_requested
+ .iter()
+ .enumerate()
+ .fold(0u32, |mask, (connector, requested)| {
+ mask | (u32::from(requested.load(Ordering::Acquire) != 0) << connector)
+ })
+ }
+
+ /// Whether `connector` shares its endpoint with another requested connector in `mask`.
+ fn endpoint_is_shared_in_mask(&self, connector: usize, mask: u32) -> bool {
+ if connector >= MAX_CONNECTORS || self.connector_count() <= 2 {
+ return false;
+ }
+ let partner = connector ^ 2;
+ partner < MAX_CONNECTORS && mask & (1u32 << partner) != 0
+ }
+}
+
+#[cfg(CONFIG_DRM_VINO_KUNIT_TEST)]
+#[kunit_tests(vino_sink)]
+mod tests {
+ use super::*;
+ use crate::*;
+
+ #[test]
+ fn only_a_presentation_that_names_a_ring_slot_advances_the_frame_counter() -> Result {
+ // The frame counter belongs to the ring, and every generation names the ring in the
+ // record that closes a frame. A presentation carrying neither an opener nor a trailer says
+ // nothing about the ring and must not consume a slot, or every later record names a buffer
+ // one ahead of the one the host filled.
+ assert!(!names_ring_slot(&[], &video::haar::FrameTrailer::none()));
+ let ella = profile::PROFILE_ELLA.geometry();
+ assert!(names_ring_slot(
+ &[],
+ &video::haar::FrameTrailer::one(&video::haar::ella_frame_close(ella, 0, 0))
+ ));
+
+ // Both other generations close every frame, so every presentation advances the counter and
+ // this rule leaves them exactly as they were.
+ let ridge = profile::PROFILE_RIDGE.geometry();
+ assert!(names_ring_slot(
+ &[],
+ &video::haar::frame_trailer(ridge, 0, 0)
+ ));
+ let navarro = profile::PROFILE_NAVARRO.geometry();
+ assert!(names_ring_slot(
+ &[],
+ &video::haar::navarro_frame_trailer(navarro, 0, 0)
+ ));
+ Ok(())
+ }
+
+ /// The DL7400 parameter map goes among a frame's records, not after all of them.
+ ///
+ /// The dock reads the records around the map with what the map says, and takes a frame that
+ /// carries it after every record it describes twice before it stops draining the endpoint
+ /// altogether. The split lands on a chunk because that is a record boundary; the vendor's byte
+ /// offset on its own is wherever a frame's record lengths put it.
+ #[test]
+ fn param_map_lands_among_a_frame_s_records() -> Result {
+ let chunk = |len: usize| -> Result<KVec<u8>> {
+ let mut c = KVec::new();
+ c.resize(len, 0, GFP_KERNEL)?;
+ Ok(c)
+ };
+
+ // A frame of even chunks: the split is the last chunk that fits under the vendor's offset,
+ // and leaves the rest of the frame behind the map.
+ let mut even: KVec<KVec<u8>> = KVec::new();
+ for _ in 0..27 {
+ even.push(chunk(16_000)?, GFP_KERNEL)?;
+ }
+ let split = param_map_chunk_split(&even);
+ assert_eq!(split, 7);
+ assert!(split * 16_000 <= NAVARRO_PARAM_IMAGE_OFFSET);
+ assert!((split + 1) * 16_000 > NAVARRO_PARAM_IMAGE_OFFSET);
+
+ // A frame smaller than the offset still puts records in front of the map, and never names
+ // a chunk it does not have.
+ let mut small: KVec<KVec<u8>> = KVec::new();
+ small.push(chunk(4_000)?, GFP_KERNEL)?;
+ small.push(chunk(4_000)?, GFP_KERNEL)?;
+ assert_eq!(param_map_chunk_split(&small), 2);
+
+ // A single chunk larger than the offset cannot be split, and the map goes behind it rather
+ // than in front of every record in the frame.
+ let mut one: KVec<KVec<u8>> = KVec::new();
+ one.push(chunk(NAVARRO_PARAM_IMAGE_OFFSET * 2)?, GFP_KERNEL)?;
+ assert_eq!(param_map_chunk_split(&one), 1);
+ Ok(())
+ }
+
+ #[test]
+ fn frame_delivery_is_profile_data_not_ring_geometry() {
+ let ridge = profile::PROFILE_RIDGE.protocol.frame_delivery;
+ let navarro = profile::PROFILE_NAVARRO.protocol.frame_delivery;
+ let ella = profile::PROFILE_ELLA.protocol.frame_delivery;
+
+ // Preserve both established dedicated-pipe families exactly.
+ assert_eq!(ridge.keyframe_presentations, 2);
+ assert_eq!(ridge.delta_presentations, 1);
+ assert_eq!(ridge.damage_frames, 3);
+ assert_eq!(navarro.keyframe_presentations, 3);
+ assert_eq!(navarro.delta_presentations, 1);
+ assert_eq!(navarro.damage_frames, 4);
+ for (policy, keys, deltas) in [(ridge, 2, 1), (navarro, 3, 1)] {
+ assert_eq!(frame_presentation_count(policy, true, false, false), keys);
+ assert_eq!(
+ frame_presentation_count(policy, false, false, false),
+ deltas
+ );
+ }
+
+ // Ella still initialises all three buffers, but DLM carries one ordinary presentation per
+ // logical frame. Later debt frames walk the ring without multiplying each update in place.
+ assert_eq!(ella.keyframe_presentations, 3);
+ assert_eq!(ella.delta_presentations, 1);
+ assert_eq!(ella.damage_frames, 3);
+ assert_eq!(frame_presentation_count(ella, true, false, true), 3);
+ assert_eq!(frame_presentation_count(ella, false, false, true), 1);
+
+ // Dedicated endpoints retain their bounded cold-training burst. A shared control pipe
+ // uses its profile keyframe count instead, so it never receives eight multi-megabyte
+ // copies back to back.
+ assert_eq!(
+ frame_presentation_count(ridge, true, true, false),
+ drm_sink::COLD_TRAINING_PRESENTATIONS
+ );
+ assert_eq!(frame_presentation_count(ridge, false, true, false), 1);
+ assert_eq!(frame_presentation_count(ella, true, true, true), 3);
+
+ // `damage_frames` includes the first accepted submission. Ella therefore leaves exactly
+ // two scheduled debt submissions after the changed frame, covering slots 0, 1 and 2 once.
+ let mut debt = [ella.damage_frames, 1, 0];
+ pay_damage_debt(&mut debt, false);
+ assert_eq!(debt, [2, 0, 0]);
+ pay_damage_debt(&mut debt, false);
+ assert_eq!(debt, [1, 0, 0]);
+ pay_damage_debt(&mut debt, false);
+ assert_eq!(debt, [0, 0, 0]);
+ let mut full = [3, 2, 1];
+ pay_damage_debt(&mut full, true);
+ assert_eq!(full, [0, 0, 0]);
+ }
+
+ /// The pacing envelope is the vendor's, and only a shared-pipe dock declares one.
+ #[test]
+ fn stream_pacing_is_the_vendors_envelope_on_the_shared_pipe_dock_only() {
+ assert!(!profile::PROFILE_RIDGE.protocol.stream_pacing.is_metered());
+ assert!(!profile::PROFILE_NAVARRO.protocol.stream_pacing.is_metered());
+ let pacing = profile::PROFILE_ELLA.protocol.stream_pacing;
+ assert!(pacing.is_metered());
+ assert_eq!(pacing.bytes_per_sec, 8_000_000);
+ assert_eq!(pacing.burst_bytes, 24_000_000);
+ // Room for the dock-wide activation keyframe, which is 9.56 MB inside half a second and
+ // is accepted every time.
+ assert!(pacing.burst_bytes > 9_560_000);
+ // And within reach of the vendor's own worst second, rather than a fraction of it.
+ assert!(i64::from(pacing.burst_bytes) + i64::from(pacing.bytes_per_sec) >= 30_000_000);
+
+ let bps = pacing.bytes_per_sec;
+ // A second of idle accrues exactly a second of budget, before the burst cap applies.
+ assert_eq!(stream_credit_accrued(bps, 1_000_000), 8_000_000);
+ assert_eq!(stream_credit_accrued(bps, 1_000), 8_000);
+ // A long idle must not wrap into a negative windfall.
+ assert!(stream_credit_accrued(bps, i64::MAX) > 0);
+ assert_eq!(stream_credit_accrued(bps, -5), 0);
+
+ // In credit, a frame goes now. Overdrawn, it waits for exactly the debt.
+ assert_eq!(stream_credit_wait_us(bps, 1), None);
+ assert_eq!(stream_credit_wait_us(bps, 0), None);
+ // Overdrawn by a second's refill, a frame waits exactly a second.
+ assert_eq!(stream_credit_wait_us(bps, -8_000_000), Some(1_000_001));
+ assert!(stream_credit_wait_us(bps, i64::MIN).is_some());
+ }
+}
diff --git a/drivers/gpu/drm/vino/drm_sink/dispatch.rs b/drivers/gpu/drm/vino/drm_sink/dispatch.rs
new file mode 100644
index 000000000000..bd759ad4bc99
--- /dev/null
+++ b/drivers/gpu/drm/vino/drm_sink/dispatch.rs
@@ -0,0 +1,446 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Publishing desired state to the asynchronous workers.
+//!
+//! Atomic callbacks run in contexts that may not sleep or touch USB, so they record what the
+//! dock should be doing and wake a worker. Each operation class owns one slot, which makes an
+//! update infallible and lets a stale cursor position or stream state be overwritten rather
+//! than queued behind the state that replaced it.
+
+use super::*;
+
+impl VinoDrmData {
+ /// Publish the latest desired operation for a connector and wake the async worker.
+ ///
+ /// Each operation class has one fixed slot, so updates cannot fail allocation and obsolete
+ /// cursor positions or stream states do not build a backlog.
+ pub(super) fn queue_cmd(&self, dev: &VinoDrmDevice, cmd: KmsCmd) {
+ let mut pending = self.pending_kms.lock();
+ if self.shutting_down.load(Ordering::Acquire) {
+ return;
+ }
+ pending.update(cmd);
+ // Registration precedes the blocking encrypted setup and platform readiness interval.
+ // Retain and coalesce commands that arrive there, but do not let them touch the dock.
+ if !self.kms_activation_ready() {
+ return;
+ }
+ // Enqueue while the queue lock still serializes us with `shutdown()`. Otherwise shutdown
+ // could cancel an idle work item between this unlock and enqueue, leaving a late work-owned
+ // device reference behind after teardown.
+ //
+ // `::<_, 0>` names `cmd_work`. The ID is only inferrable while a single `WorkItem` impl
+ // exists; adding the per-connector scanout items made every bare `enqueue` ambiguous, which
+ // is exactly the failure mode you want here -- an unannotated enqueue would otherwise be
+ // free to pick the wrong worker.
+ let _ = self.kms_queue.enqueue::<_, 0>(ARef::from(dev));
+ drop(pending);
+ }
+
+ /// Publish the end of bring-up and wake transport state retained while it ran.
+ pub(crate) fn publish_kms_activation_ready(&self, dev: &VinoDrmDevice) {
+ let pending = self.pending_kms.lock();
+ if self.shutting_down.load(Ordering::Acquire) {
+ return;
+ }
+ // Share `pending_kms` with `queue_cmd` as the readiness/wakeup handshake: either this sees
+ // a retained command, or a later producer sees readiness and enqueues the work itself.
+ self.kms_activation_ready.store(true, Ordering::Release);
+ if !pending.is_empty() {
+ let _ = self.kms_queue.enqueue::<_, 0>(ARef::from(dev));
+ }
+ drop(pending);
+ // Plane state may also have arrived after CP engagement but before activation readiness.
+ // Its worker gates on the same flag and leaves the coalesced frame in place until this
+ // wake.
+ self.enqueue_scanout_all(dev);
+ }
+
+ /// Publish the latest framebuffer for one connector and wake the same deferred worker used by
+ /// the blocking runtime CP commands. Replacing an unsent flip is deliberate backpressure: the
+ /// dock needs the newest desktop, not every historical compositor buffer. If damaged flips are
+ /// coalesced, carry the unsent damage into the newest framebuffer so no intermediate update is
+ /// lost without needlessly promoting every busy compositor interval to a full-screen refresh.
+ pub(super) fn queue_scanout(
+ &self,
+ dev: &VinoDrmDevice,
+ fb: &kms::framebuffer::Framebuffer<VinoDrmDriver>,
+ mut frame: PendingScanout,
+ ) {
+ let connector = frame.connector as usize;
+ let socket = connector + 1;
+ if connector >= MAX_CONNECTORS || self.shutting_down.load(Ordering::Acquire) {
+ return;
+ }
+ // Do not snapshot faster than the encoder consumes. An unclaimed frame in the coalescing
+ // slot means the worker has not caught up, so this snapshot would be overwritten before it
+ // was ever read -- and it is not free: it runs on the compositor's atomic-commit thread and
+ // reads the whole source to hash it. On a busy machine the encoder falls behind by many
+ // commits, and paying that read for every one of them stalls the compositor itself, on
+ // every output it drives rather than only this one.
+ //
+ // Nothing is lost by dropping this flip. Damage is decided by comparing strip hashes
+ // against the last *encoded* baseline, never by the compositor's damage clips, so whatever
+ // this commit changed is still described by the next snapshot that gets through. The frame
+ // the worker eventually takes is at most one encode period old, which is the pacing the
+ // hardware imposes anyway.
+ //
+ // Geometry changes and owed keyframes are never dropped: the first makes the pending
+ // frame's damage coordinates meaningless, and the second is a mode set waiting on current
+ // content rather than an ordinary repaint.
+ let coalesce = {
+ let pending = self.pending_scanout.lock();
+ pending[connector].as_ref().is_some_and(|queued| {
+ queued.w == frame.w
+ && queued.h == frame.h
+ && queued.rotation == frame.rotation
+ && self.keyframe_pending.load(Ordering::Acquire) & (1u32 << connector) == 0
+ })
+ };
+ if coalesce {
+ vino_debug!("vino: socket {socket} flip coalesced before snapshot\n");
+ return;
+ }
+ // The snapshot below is format-agnostic -- both layouts are four bytes per pixel -- so the
+ // depth only has to be recorded, not acted on, before the copy.
+ if let Some(depth) = crate::video::haar::Depth::from_fourcc(fb.format()) {
+ self.set_connector_depth(frame.connector, depth);
+ }
+ let (source_w, source_h) = src_dims(frame.rotation, frame.w, frame.h);
+ // Reserve a slot and lend its surface out, so the ~14.7 MB copy below runs with the pool
+ // lock dropped. Holding it across the copy put this connector's scanout worker into
+ // `mutex_spin_on_owner` for the whole snapshot -- 5.4% of the machine, burnt spinning.
+ let (mut surface, binding, idx) = {
+ let mut pool = self.shadow[connector].lock();
+ // Rotate, rather than always taking the first free slot. `find` returned slot 0 on
+ // every commit whenever nothing was inflight, so consecutive snapshots overwrote the
+ // same slot and bumped its generation -- invalidating any frame the worker had already
+ // selected from it, which showed up as a third of all frames being dropped at the
+ // generation check. Alternating means a fresh snapshot lands clear of the frame the
+ // worker is about to pick up.
+ let start = self.shadow_rr[connector].fetch_add(1, Ordering::Relaxed) as usize;
+ let Some(idx) = (0..SHADOW_SLOTS)
+ .map(|i| (start + i) % SHADOW_SLOTS)
+ .find(|&idx| pool.inflight != Some(idx) && pool.writing != Some(idx))
+ else {
+ return;
+ };
+ let binding = match pool.source_bindings.get(fb) {
+ Ok(binding) => binding,
+ Err(e) => {
+ pr_warn!("vino: socket {socket} framebuffer binding failed ({e:?})\n");
+ return;
+ }
+ };
+ pool.writing = Some(idx);
+ (pool.slots[idx].surface.take(), binding, idx)
+ };
+
+ let r = snapshot_to_shadow(
+ self.geometry(),
+ &mut surface,
+ &binding.mapping,
+ source_w,
+ source_h,
+ );
+
+ let snapshot = {
+ let mut pool = self.shadow[connector].lock();
+ pool.writing = None;
+ let slot = &mut pool.slots[idx];
+ slot.surface = surface;
+ // Bump unconditionally: the slot's contents have been rewritten either way, so any
+ // frame still pointing at the old generation must not be encoded from it.
+ slot.generation = slot.generation.wrapping_add(1);
+ r.map(|()| (idx, slot.generation))
+ };
+ let (idx, generation) = match snapshot {
+ Ok(snapshot) => snapshot,
+ Err(e) => {
+ pr_warn!("vino: socket {socket} framebuffer snapshot failed ({e:?})\n");
+ return;
+ }
+ };
+ frame.shadow_idx = idx;
+ frame.shadow_generation = generation;
+
+ // A real flip carries newer content than an armed repaint.
+ self.settle_repaint.lock()[connector] = None;
+
+ let mut pending = self.pending_scanout.lock();
+ if self.shutting_down.load(Ordering::Acquire) {
+ return;
+ }
+ if let Some(old) = pending[connector].take() {
+ if old.w != frame.w || old.h != frame.h || old.rotation != frame.rotation {
+ // Damage coordinates are not comparable across a geometry transform. A mode-set
+ // already owes a keyframe, but keep this conservative for a rotation-only commit.
+ frame.clips[0] = (0, 0, frame.w, frame.h);
+ frame.nclips = 1;
+ } else if old.nclips + frame.nclips <= MAX_DAMAGE_CLIPS {
+ // `frame` names the newest complete framebuffer. Repainting the union of its own
+ // damage and every unsent older clip reproduces all intermediate changes directly
+ // from that newest image.
+ for &clip in &old.clips[..old.nclips] {
+ frame.clips[frame.nclips] = clip;
+ frame.nclips += 1;
+ }
+ } else {
+ // Too many rectangles for the bounded atomic-state payload: collapse their union
+ // to one bounding box. This may repaint extra strips, but unlike the previous
+ // full-output fallback it remains small for typical pointer/window motion.
+ let mut bb = (frame.w, frame.h, 0usize, 0usize);
+ for &r in &frame.clips[..frame.nclips] {
+ bb = (bb.0.min(r.0), bb.1.min(r.1), bb.2.max(r.2), bb.3.max(r.3));
+ }
+ for &r in &old.clips[..old.nclips] {
+ bb = (bb.0.min(r.0), bb.1.min(r.1), bb.2.max(r.2), bb.3.max(r.3));
+ }
+ if bb.0 < bb.2 && bb.1 < bb.3 {
+ frame.clips[0] = bb;
+ frame.nclips = 1;
+ } else {
+ frame.nclips = 0;
+ }
+ }
+ }
+ pending[connector] = Some(frame);
+ self.enqueue_scanout(dev, connector);
+ drop(pending);
+ }
+
+ /// Wake `connector`'s scanout worker. The work ID is a const generic, so the runtime connector
+ /// index has to be matched into it here. Enqueueing an already-pending item is a no-op, and
+ /// enqueueing one that is currently running re-arms it, preserving a flip that arrives during
+ /// encoding for the worker's next pass.
+ pub(super) fn enqueue_scanout(&self, dev: &VinoDrmDevice, connector: usize) {
+ match connector {
+ 0 => {
+ let _ = self.scanout_queue.enqueue::<_, 1>(ARef::from(dev));
+ }
+ 1 => {
+ let _ = self.scanout_queue.enqueue::<_, 2>(ARef::from(dev));
+ }
+ 2 => {
+ let _ = self.scanout_queue.enqueue::<_, 3>(ARef::from(dev));
+ }
+ 3 => {
+ let _ = self.scanout_queue.enqueue::<_, 4>(ARef::from(dev));
+ }
+ _ => {}
+ }
+ }
+
+ /// Wait for any frame already in flight on a scanout worker to finish, after [`Self::cmd_busy`]
+ /// has been published. A worker that has not yet started re-checks `cmd_busy` and backs off on
+ /// its own; this only covers one that got past that check before the flag was set.
+ ///
+ /// Bounded, and it proceeds anyway on timeout: a mode-set that never reaches the dock is worse
+ /// than one that races a frame, and this is the path cold activation depends on. The bound is
+ /// generous against a worst-case frame (a ~3.19 MB keyframe: ~21 ms to encode plus its wire
+ /// time), so exceeding it means something is genuinely wedged and the log line is the point.
+ pub(super) fn wait_for_video_idle(&self) {
+ use core::sync::atomic::Ordering::SeqCst;
+ for _ in 0..500 {
+ if !self.video_inflight.iter().any(|f| f.load(SeqCst)) {
+ return;
+ }
+ fsleep(Delta::from_millis(1));
+ }
+ pr_warn!("vino: timed out waiting for in-flight scanout before a mode-set; proceeding\n");
+ }
+
+ /// Wake every connector's scanout worker. Used by `cmd_work` once its batch is done, since a
+ /// command batch is exactly what makes the scanout workers bail (see [`run_scanout_worker`]).
+ pub(crate) fn enqueue_scanout_all(&self, dev: &VinoDrmDevice) {
+ for connector in 0..MAX_CONNECTORS {
+ self.enqueue_scanout(dev, connector);
+ }
+ }
+
+ /// Record that `connector` owes a full keyframe, and refill its settle-repaint budget.
+ ///
+ /// Mode sets, output enables, and gamma changes use this path. Training
+ /// and settle repaints may re-raise the keyframe bit without refilling
+ /// the budget, which bounds idle keyframe generation.
+ pub(super) fn owe_keyframe(&self, connector: usize) {
+ self.keyframe_pending
+ .fetch_or(1u32 << connector, Ordering::Release);
+ self.settle_budget[connector].store(SETTLE_REPAINTS, Ordering::Relaxed);
+ // Whatever left the dock's framebuffer undefined left its cursor bitmap undefined too, so
+ // the two invalidations are raised together. Keeping them in one place is deliberate:
+ // their being separate is exactly how the cursor came to be dropped on a mode-set.
+ self.cursor_epoch[connector].fetch_add(1, Ordering::Release);
+ self.cursor_geometry.lock()[connector] = None;
+ }
+
+ /// Note a cursor command the dock has just accepted, so [`Self::rearm_cursor`] can replay it.
+ pub(super) fn record_cursor(&self, cmd: &KmsCmd) {
+ let connector = cmd.connector();
+ if connector >= MAX_CONNECTORS {
+ return;
+ }
+ let mut slots = self.cursor_shot.lock();
+ match cmd {
+ KmsCmd::CursorImage { w, h, bgra, .. } => {
+ let mut copy = KVec::new();
+ if copy.extend_from_slice(bgra, GFP_KERNEL).is_err() {
+ // A cursor that cannot be cached is still on the dock; it just will not be
+ // restored across the next mode set. Nothing else depends on this.
+ return;
+ }
+ match &mut slots[connector] {
+ Some(shot) => {
+ shot.w = *w;
+ shot.h = *h;
+ shot.bgra = copy;
+ }
+ slot @ None => {
+ *slot = Some(CursorShot {
+ w: *w,
+ h: *h,
+ bgra: copy,
+ x: 0,
+ y: 0,
+ visible: false,
+ })
+ }
+ }
+ }
+ KmsCmd::CursorMove { x, y, visible, .. } => {
+ if let Some(shot) = &mut slots[connector] {
+ shot.x = *x;
+ shot.y = *y;
+ shot.visible = *visible;
+ }
+ }
+ _ => {}
+ }
+ }
+
+ /// Re-upload the cursor on every connector in `connectors` after a mode set discarded it.
+ ///
+ /// `owe_keyframe` marks the dock's cursor stale, but only a compositor commit on the cursor
+ /// plane acted on that mark -- and a pointer that is not moving never produces one, so the
+ /// cursor stayed missing until it was moved. Replaying the cached shot closes that window
+ /// without waiting for userspace.
+ pub(super) fn rearm_cursor(&self, dev: &VinoDrmDevice, connectors: u32) {
+ for connector in 0..MAX_CONNECTORS {
+ if connectors & (1u32 << connector) == 0 {
+ continue;
+ }
+ let (w, h, bgra, x, y, visible) = {
+ let slots = self.cursor_shot.lock();
+ let Some(shot) = &slots[connector] else {
+ continue;
+ };
+ let mut copy = KVec::new();
+ if copy.extend_from_slice(&shot.bgra, GFP_KERNEL).is_err() {
+ continue;
+ }
+ (shot.w, shot.h, copy, shot.x, shot.y, shot.visible)
+ };
+ let connector = connector as u8;
+ // The same order the plane callback uses, and the same order `cmd_work` drains them
+ // in: geometry, then bitmap, then position.
+ self.queue_cmd(dev, KmsCmd::CursorCreate { connector, w, h });
+ self.queue_cmd(
+ dev,
+ KmsCmd::CursorImage {
+ connector,
+ w,
+ h,
+ bgra,
+ },
+ );
+ self.queue_cmd(
+ dev,
+ KmsCmd::CursorMove {
+ connector,
+ x,
+ y,
+ visible,
+ },
+ );
+ }
+ }
+
+ /// Choose the next frame or delay for `connector`.
+ ///
+ /// Neither means this connector is idle and its worker can exit.
+ pub(super) fn select_scanout(&self, connector: usize) -> (Option<PendingScanout>, Option<i64>) {
+ let socket = connector + 1;
+ // Keep a frame that arrived before the cadence deadline in the
+ // coalescing slot. Userspace may stop committing after that flip, so
+ // discarding it could leave the newest image unsent.
+ // A dock with a sustained budget is held to it ahead of everything else, including an owed
+ // keyframe: the frame that overruns it costs the session, not a repaint. The coalescing
+ // slot keeps the newest image meanwhile, so waiting here drops intermediate frames rather
+ // than delaying the desktop.
+ if let Some(us) = self.stream_budget_wait_us() {
+ return (None, Some(us));
+ }
+ let mut pending = self.pending_scanout.lock();
+ let mut selected = None;
+ let mut wait_us: Option<i64> = None;
+ if self.modeset_requested[connector].load(Ordering::Acquire) != 0
+ && pending[connector].is_some()
+ {
+ let owes_keyframe =
+ self.keyframe_pending.load(Ordering::Acquire) & (1u32 << connector) != 0;
+ let elapsed_us = self.last_frame.lock()[connector]
+ .map_or(self.frame_period_us(), |t| t.elapsed().as_micros_ceil());
+ // An owed keyframe normally jumps the cadence queue, because it is the frame that makes
+ // the output correct and a compositor may not send another. A dock sharing the control
+ // pipe cannot grant that: a keyframe is its largest frame, and anything that re-raises
+ // the keyframe bit would let it bypass the interval repeatedly and hold the endpoint
+ // for as long as it keeps being raised. There the interval binds every frame.
+ let urgent = owes_keyframe && !self.video_on_ctrl_pipe();
+ if urgent || elapsed_us >= self.frame_period_us() {
+ selected = pending[connector].take();
+ // A busy compositor continuously replaces `settle_repaint`.
+ // Force cadence-selected frames to be keyframes while training;
+ // the elapsed check above still applies the cadence limit.
+ let sustaining = self.sustain_until.lock()[connector]
+ .is_some_and(|until| (until - Instant::<Monotonic>::now()).as_millis() > 0);
+ if sustaining {
+ self.keyframe_pending
+ .fetch_or(1u32 << connector, Ordering::Release);
+ }
+ } else {
+ wait_us = Some(self.frame_period_us() - elapsed_us);
+ }
+ }
+ // Nothing flipped in. Fall back to the one-shot settle repaint if one is due, so a
+ // compositor that went idle straight after enabling the output still ends up with its real
+ // desktop on the panel rather than the buffer that happened to be current when the
+ // mode-set's keyframe went out.
+ if selected.is_none() {
+ let mut settle = self.settle_repaint.lock();
+ if self.modeset_requested[connector].load(Ordering::Acquire) == 0 {
+ settle[connector] = None;
+ } else if let Some((due, _, _)) = settle[connector].as_ref() {
+ let remaining = *due - Instant::<Monotonic>::now();
+ if remaining.as_millis() <= 0 {
+ let taken = settle[connector].take();
+ let as_keyframe = taken.as_ref().is_some_and(|(_, _, kf)| *kf);
+ selected = taken.map(|(_, f, _)| f);
+ if as_keyframe {
+ self.keyframe_pending
+ .fetch_or(1u32 << connector, Ordering::Release);
+ }
+ let kind = if as_keyframe {
+ "settle repaint (compositor idle after mode-set)"
+ } else {
+ "debt repaint (retransmissions owed, compositor idle)"
+ };
+ vino_debug!("vino: socket {socket} {kind}\n");
+ } else {
+ let remaining = remaining.as_micros_ceil().max(1);
+ wait_us = Some(wait_us.map_or(remaining, |old| old.min(remaining)));
+ }
+ }
+ }
+ (selected, wait_us)
+ }
+}
diff --git a/drivers/gpu/drm/vino/drm_sink/driver.rs b/drivers/gpu/drm/vino/drm_sink/driver.rs
new file mode 100644
index 000000000000..aed8339f6336
--- /dev/null
+++ b/drivers/gpu/drm/vino/drm_sink/driver.rs
@@ -0,0 +1,211 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Registration: the DRM driver description, its GEM object and file types, and the KMS
+//! entry points the core calls into.
+
+use super::*;
+
+/// GEM object inner data. Empty: the shmem-backed `drm::gem::shmem::Object` (which
+/// wires `drm_gem_shmem_dumb_create`, so userspace `DRM_IOCTL_MODE_CREATE_DUMB`
+/// works) is enough until the EP08 scanout path consumes the framebuffers.
+#[pin_data]
+pub(crate) struct VinoObject {}
+
+impl drm::gem::DriverObject for VinoObject {
+ type Driver = VinoDrmDriver;
+ type Args = ();
+
+ fn new(
+ _dev: &drm::Device<VinoDrmDriver>,
+ _size: usize,
+ _args: (),
+ ) -> impl PinInit<Self, Error> {
+ try_pin_init!(VinoObject {})
+ }
+}
+
+/// Per-open DRM client state. The generic DRM fops pin the owning module for the file lifetime.
+#[pin_data]
+pub(crate) struct VinoDrmFile {}
+
+impl drm::file::DriverFile for VinoDrmFile {
+ type Driver = VinoDrmDriver;
+
+ fn open(_dev: &drm::Device<Self::Driver>) -> Result<Pin<KBox<Self>>> {
+ KBox::try_pin_init(try_pin_init!(Self {}), GFP_KERNEL)
+ }
+}
+
+pub(super) const INFO: drm::DriverInfo = drm::DriverInfo {
+ major: 0,
+ minor: 1,
+ patchlevel: 0,
+ name: c"vino",
+ desc: c"DisplayLink DL3 (Dell D6000) DRM driver",
+};
+
+#[vtable]
+impl drm::Driver for VinoDrmDriver {
+ type Data = VinoDrmData;
+ type File = VinoDrmFile;
+ type Object = drm::gem::shmem::Object<VinoObject>;
+ type ParentDevice<Ctx: kernel::device::DeviceContext> = crate::usb::Interface<Ctx>;
+ type RegistrationData<'a> = ();
+ type Kms = Self;
+
+ const INFO: drm::DriverInfo = INFO;
+
+ // No driver-private ioctls (GEM/dumb + KMS handled by the DRM core).
+ kernel::declare_drm_ioctls! {}
+}
+
+#[vtable]
+impl KmsDriver for VinoDrmDriver {
+ type Connector = VinoConnector;
+ type Plane = VinoPlane;
+ type Crtc = VinoCrtc;
+ type Encoder = VinoEncoder;
+
+ fn mode_config_info(
+ _dev: &kernel::device::Device,
+ _drm_data: &Self::Data,
+ ) -> Result<ModeConfigInfo> {
+ Ok(ModeConfigInfo {
+ min_resolution: (0, 0),
+ max_resolution: (4096, 4096),
+ max_cursor: (64, 64),
+ preferred_depth: 32,
+ preferred_fourcc: Some(drm::fourcc::XRGB8888),
+ })
+ }
+
+ fn create_objects(dev: &UnregisteredKmsDevice<'_, Self>) -> Result {
+ let data: &VinoDrmData = dev;
+ // Build one independent connector (CRTC + primary/cursor plane + encoder + connector) per
+ // wired display, each pinned to its own video endpoint via its connector index.
+ //
+ // Only as many as the dock has sockets. `MAX_CONNECTORS` is the largest any supported dock
+ // has, so it sizes the per-connector arrays, but building that many objects on a
+ // two-connector dock publishes outputs with nothing behind them: they never gain an EDID, a
+ // compositor is free to enable one anyway, and the driver then encodes and transmits whole
+ // frames to a socket that cannot display them -- onto the same endpoint the real connector
+ // is using.
+ for connector in 0..data.connector_count() {
+ // `possible_crtcs` for the plane/encoder is a bitmask of CRTC *indices*, which only
+ // exist once `UnregisteredCrtc::new` runs -- but planes must exist before the CRTC that
+ // references them. CRTCs are created here one per connector in order, so this
+ // connector's CRTC index is `connector` and its mask is `1 << connector`.
+ let crtc_mask = 1u32 << connector;
+ let primary = plane::UnregisteredPlane::<VinoPlane>::new(
+ dev,
+ crtc_mask,
+ if data.hdr_capable() {
+ &PRIMARY_FORMATS_HDR[..]
+ } else {
+ &PRIMARY_FORMATS[..]
+ },
+ // Scanout is linear and nothing else is accepted: `Framebuffer` rejects any other
+ // modifier outright. Saying so publishes IN_FORMATS, so a compositor picks a format
+ // knowing what the plane takes rather than inferring it from the bare format list.
+ Some(&LINEAR_MODIFIER[..]),
+ plane::Type::Primary,
+ None,
+ PlaneArgs {
+ connector: connector as u8,
+ is_cursor: false,
+ },
+ )?;
+ // Tell compositors that this primary plane accepts the standard FB_DAMAGE_CLIPS
+ // property. The scanout path already consumes those clips and emits only intersecting
+ // 64x16 Haar strips, but without attaching the property KWin cannot provide them:
+ // unchanged commits arrive with an empty clip list while real updates fall back to
+ // ambiguous framebuffer swaps. That left the first keyframe frozen when empty damage
+ // was correctly treated as a no-op, or forced multi-megabyte full frames when it was
+ // treated as a repaint. EVDI exposes the same property before plane registration.
+ primary.enable_fb_damage_clips();
+ // Advertise every rotation vino's re-encode can produce by remapping source pixels
+ // (`rot_src`): the four 90-degree rotations plus the two reflections.
+ primary.create_rotation_property(
+ plane::Rotation::ROTATE_0,
+ plane::Rotation::ROTATE_0
+ | plane::Rotation::ROTATE_90
+ | plane::Rotation::ROTATE_180
+ | plane::Rotation::ROTATE_270
+ | plane::Rotation::REFLECT_X
+ | plane::Rotation::REFLECT_Y,
+ )?;
+ // A dock that composites no cursor of its own gets no cursor plane, rather than a
+ // plane whose messages are then withheld: a cursor plane whose atomic commit succeeds
+ // makes the compositor hand the pointer over and stop drawing its own, so starving one
+ // loses the pointer entirely instead of falling back to software. A CRTC with no
+ // cursor plane is how a driver says "draw it yourself".
+ let cursor = if data.hw_cursor() {
+ let cursor = plane::UnregisteredPlane::<VinoPlane>::new(
+ dev,
+ crtc_mask,
+ &CURSOR_FORMATS,
+ Some(&LINEAR_MODIFIER[..]),
+ plane::Type::Cursor,
+ None,
+ PlaneArgs {
+ connector: connector as u8,
+ is_cursor: true,
+ },
+ )?;
+ // An alpha framebuffer requires a blend-mode property. The dock composites the
+ // cursor from a premultiplied bitmap, so premultiplied is the only supported mode.
+ cursor.create_blend_mode_property(plane::BlendModes::PREMULTIPLIED)?;
+ Some(cursor)
+ } else {
+ None
+ };
+ let crtc_obj = crtc::UnregisteredCrtc::<VinoCrtc>::new(
+ dev,
+ primary,
+ cursor,
+ None,
+ connector as u8,
+ )?;
+ // Advertise CTM and a 256-entry GAMMA_LUT; the scanout applies both (cached via the
+ // CRTC hooks). The dock has no colour hardware, so software application here is the
+ // only place a compositor's correction can land -- KDE's Night Colour and GNOME's
+ // Night Light drive these properties rather than rewriting the framebuffer.
+ crtc_obj.enable_color_mgmt(0, true, crate::color::LUT_LEN as u32);
+ let enc = encoder::UnregisteredEncoder::<VinoEncoder>::new(
+ dev,
+ encoder::Type::Virtual,
+ crtc_obj.mask(),
+ 0,
+ None,
+ (),
+ )?;
+ let conn = connector::UnregisteredConnector::<VinoConnector>::new(
+ dev,
+ // DisplayPort connectors receive DRM's standard EDID property. A virtual connector
+ // would not, and therefore could not publish the downstream monitor's modes.
+ connector::Type::DisplayPort,
+ connector as u8,
+ )?;
+ conn.attach_encoder(&*enc)?;
+ // HDR is a property of the dock's pipeline, not of the monitor: a sink that declares
+ // ST 2084 is useless if the dock cannot be told to carry ten bits. `hdr_capable`
+ // keeps a Ridge connector from advertising an output it has no set-mode encoding for.
+ //
+ // The whole path exists now: the transfer function is offset-42 bit 6
+ // (`ST2084 colorspace used (HDR)`, read out of DLM's own `setupVideo` decode),
+ // `atomic_enable` takes it from this connector's HDR_OUTPUT_METADATA EOTF, and the
+ // depth (offset 69) and DMA format (offset 23, `NM30`) go with it.
+ //
+ // Attaching these is what makes a compositor re-encode the desktop in PQ/BT.2020. If
+ // the sink does not follow, the failure is a washed-out grey desktop that is invisible
+ // on the wire: a capture shows correct PQ code words in both the working and the
+ // broken case.
+ if data.hdr_capable() {
+ conn.attach_max_bpc_property(8, 10)?;
+ conn.attach_colorspace_property()?;
+ conn.attach_hdr_output_metadata_property();
+ }
+ }
+ Ok(())
+ }
+}
diff --git a/drivers/gpu/drm/vino/drm_sink/limits.rs b/drivers/gpu/drm/vino/drm_sink/limits.rs
new file mode 100644
index 000000000000..96d2d8dcf8c2
--- /dev/null
+++ b/drivers/gpu/drm/vino/drm_sink/limits.rs
@@ -0,0 +1,489 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! What a dock will accept: pixel-clock ceilings, refresh ceilings and the shared bandwidth
+//! budget a multi-connector commit has to fit inside.
+//!
+//! A dock silently refuses to light a mode past its budget rather than reporting anything, so
+//! these are the checks that keep a connector from being handed one.
+
+use super::*;
+
+/// Per-mode pixel-clock ceiling in kHz for a dock whose profile has not been applied yet.
+///
+/// Ridge's DLM never programs above 497.75 MHz, so no Ridge capture fills the high half of the
+/// offset-70 `u32`; this keeps the value that half can express on its own.
+pub(super) const DEFAULT_MAX_HEAD_CLOCK_KHZ: u32 = 655_350;
+
+/// Refresh ceiling for a dock whose profile has not been applied yet.
+///
+/// This is Ridge's limit, which is also DLM's: asked for 2560x1440@180 it puts 119.998 Hz on the
+/// wire, and asked for @85 it programs the 59.95 Hz CVT-RB timing.
+pub(super) const DEFAULT_MAX_REFRESH_HZ: u32 = 120;
+
+/// Return the active pixel rate, saturating on invalidly large modes.
+pub(crate) fn active_pixel_rate(hdisplay: u16, vdisplay: u16, vrefresh: i32) -> u32 {
+ u32::from(hdisplay)
+ .saturating_mul(u32::from(vdisplay))
+ .saturating_mul(vrefresh.max(0) as u32)
+}
+
+/// Nonzero generation key for every deterministic field of a set-mode timing.
+///
+/// Zero means "disabled" in the atomics that carry this key. This is a fingerprint rather than
+/// a packed subset: porches, sync widths, allocation, VIC, depth and dual-pipe state all change
+/// bytes the dock consumes and therefore all have to invalidate a previously active mode.
+pub(crate) fn timing_key(t: &crate::cp::Timing) -> u64 {
+ let mut hash = 0x7669_6e6f_6d6f_6465u64;
+ for field in [
+ u64::from(t.hactive),
+ u64::from(t.hblank),
+ u64::from(t.hsync_front),
+ u64::from(t.hsync_width),
+ u64::from(t.vactive),
+ u64::from(t.vblank),
+ u64::from(t.vsync_front),
+ u64::from(t.vsync_width),
+ u64::from(t.refresh_hz),
+ u64::from(t.pixel_clock_10khz),
+ u64::from(t.sync_flags),
+ u64::from(t.stride),
+ u64::from(t.total_rows),
+ u64::from(t.vic_word),
+ u64::from(t.ten_bit),
+ u64::from(t.st2084),
+ u64::from(t.dual_nivo),
+ ] {
+ hash = xxhash::xxh64(&field.to_le_bytes(), hash);
+ }
+ if hash == 0 {
+ 1
+ } else {
+ hash
+ }
+}
+
+/// Whether a live connector already holds the exact effective Timing a command would send.
+pub(crate) fn programmed_mode_matches(
+ active_generation: u64,
+ programmed: Option<crate::cp::Timing>,
+ effective: crate::cp::Timing,
+) -> bool {
+ active_generation != 0 && programmed == Some(effective)
+}
+
+impl VinoDrmData {
+ /// Exact Timing that would be put on the wire for the current requested topology.
+ pub(super) fn effective_timing(
+ &self,
+ connector: usize,
+ timing: &crate::cp::Timing,
+ ) -> crate::cp::Timing {
+ self.effective_timing_in_mask(connector, timing, self.requested_connector_mask())
+ }
+
+ /// Exact Timing for a stable requested-connector snapshot shared by a multi-connector
+ /// transaction.
+ pub(super) fn effective_timing_in_mask(
+ &self,
+ connector: usize,
+ timing: &crate::cp::Timing,
+ requested_heads: u32,
+ ) -> crate::cp::Timing {
+ crate::cp::Timing {
+ dual_nivo: self.endpoint_is_shared_in_mask(connector, requested_heads),
+ ..*timing
+ }
+ }
+
+ /// Adopt an already-programmed exact mode under the caller's current request generation.
+ ///
+ /// A dynamic `dual_nivo` correction can make two request tokens differ while the exact timing
+ /// the dock holds is unchanged. Conversely, equal tokens are not enough if endpoint topology
+ /// changed. Compare the separately recorded wire state and change only `modeset_active`, whose
+ /// job is to gate scanout against the current producer request. An explicit repair clears
+ /// `modeset_active`, so it can never be optimized away here.
+ pub(super) fn adopt_programmed_mode(
+ &self,
+ connector: usize,
+ timing: &crate::cp::Timing,
+ want: u64,
+ ) -> bool {
+ if connector >= MAX_CONNECTORS
+ || self.modeset_requested[connector].load(Ordering::Acquire) != want
+ {
+ return false;
+ }
+ let active = self.modeset_active[connector].load(Ordering::Acquire);
+ if !programmed_mode_matches(
+ active,
+ self.programmed_timing.lock()[connector],
+ self.effective_timing(connector, timing),
+ ) {
+ return false;
+ }
+ // A disable that races this compare either clears `active` first (CAS fails) or clears it
+ // after (the disable wins). A newer nonzero request may leave the old active token in
+ // place, but scanout remains gated until its own command adopts or programs that request.
+ self.modeset_active[connector]
+ .compare_exchange(active, want, Ordering::AcqRel, Ordering::Acquire)
+ .is_ok()
+ }
+
+ /// The dock's total pixel-rate budget shared across all connectors, unpriced by depth.
+ ///
+ /// Zero means unknown and disables limiting.
+ pub(super) fn dock_budget(&self) -> u32 {
+ self.dock_pixel_budget
+ .load(core::sync::atomic::Ordering::Relaxed)
+ }
+
+ /// Whether a commit totalling `combined` pixels per second can be driven at ten bits.
+ ///
+ /// The answer moves the depth rather than refusing the commit: a compositor handed `EINVAL`
+ /// disables the output instead of asking for a shallower link.
+ pub(super) fn ten_bit_fits(&self, combined: u32) -> bool {
+ let raw = self.dock_budget();
+ raw == 0 || combined <= self.budget_at_depth(raw, true)
+ }
+
+ /// A budget priced for a connector driven at ten bits per channel.
+ ///
+ /// The budget was measured with the dock storing three bytes per pixel; ten bits stores four,
+ /// so the same pixel costs a third more. The whole dock is priced at its deepest connector,
+ /// because the bandwidth is shared.
+ ///
+ /// `budget` must be the unpriced [`Self::dock_budget`]. Pricing an already-priced budget leaves
+ /// nine sixteenths of the dock, which no pair of ten-bit connectors fits inside.
+ pub(super) fn budget_at_depth(&self, budget: u32, ten_bit: bool) -> u32 {
+ if budget != 0 && ten_bit {
+ budget / 4 * 3
+ } else {
+ budget
+ }
+ }
+
+ /// Record this dock's pixel-rate budget, refresh ceiling and pixel-clock ceiling.
+ pub(crate) fn set_mode_limits(
+ &self,
+ pixel_budget: u32,
+ max_refresh_hz: u32,
+ max_connector_clock_khz: u32,
+ ) {
+ self.dock_pixel_budget
+ .store(pixel_budget, core::sync::atomic::Ordering::Relaxed);
+ self.max_refresh_hz.store(
+ if max_refresh_hz == 0 {
+ DEFAULT_MAX_REFRESH_HZ
+ } else {
+ max_refresh_hz
+ },
+ core::sync::atomic::Ordering::Relaxed,
+ );
+ self.max_connector_clock_khz.store(
+ if max_connector_clock_khz == 0 {
+ DEFAULT_MAX_HEAD_CLOCK_KHZ
+ } else {
+ max_connector_clock_khz
+ },
+ core::sync::atomic::Ordering::Relaxed,
+ );
+ }
+
+ /// Highest per-mode pixel clock in kHz this dock is known to accept.
+ pub(crate) fn max_connector_clock_khz(&self) -> u32 {
+ self.max_connector_clock_khz
+ .load(core::sync::atomic::Ordering::Relaxed)
+ }
+
+ /// Highest refresh rate this dock is known to drive.
+ pub(crate) fn max_refresh_hz(&self) -> u32 {
+ self.max_refresh_hz
+ .load(core::sync::atomic::Ordering::Relaxed)
+ }
+
+ /// Whether DRM's rounded refresh rate is within this dock's limit.
+ pub(super) fn refresh_within_limit(&self, vrefresh: i32) -> bool {
+ vrefresh <= 0 || (vrefresh as u32) <= self.max_refresh_hz()
+ }
+
+ /// Combined pixel rate of every connector *except* `connector` that currently has a mode driven
+ /// onto it.
+ ///
+ /// A connector the commit carries is taken from the commit; every other connector is taken from
+ /// what is programmed. Both halves are needed: a commit reconfiguring several connectors at
+ /// once must be weighed at the rates it is asking for, and a connector standing outside it
+ /// still spends what it was last given.
+ ///
+ /// Only active connectors consume the shared limit. `last_timing` survives `atomic_disable`, so
+ /// activity is read from `modeset_requested`, which is cleared on disable.
+ ///
+ /// A connector whose monitor has gone is not spending anything either, whatever mode it was
+ /// last asked for: its downstream sink is already torn down. That has to be read from presence
+ /// rather than from the mode state, because a removal and the arrival that replaces it reach
+ /// userspace as two events -- moving a monitor from one socket to another is checked at its new
+ /// socket while the disable of the old one has not been committed yet, and charging the dock
+ /// for the dark connector is what refuses the new one its mode.
+ pub(super) fn other_connectors_rate(
+ &self,
+ state: &kernel::drm::kms::atomic::AtomicStateMutator<VinoDrmDriver>,
+ connector: usize,
+ ) -> u32 {
+ // A connector this commit describes is charged what the commit gives it. Reading its
+ // programmed rate instead would price every connector in a multi-connector commit at what
+ // it is leaving, so a pair that rises together would be admitted at the sum of the rates it
+ // is abandoning.
+ let mut proposed: [Option<u32>; MAX_CONNECTORS] = [None; MAX_CONNECTORS];
+ state.for_each_new_crtc_state(|crtc, crtc_state| {
+ let Some(slot) = proposed.get_mut(crtc.connector as usize) else {
+ return;
+ };
+ *slot = Some(if crtc_state.active() {
+ let m = crtc_state.mode();
+ active_pixel_rate(m.hdisplay(), m.vdisplay(), m.vrefresh())
+ } else {
+ 0
+ });
+ });
+
+ let timings = *self.last_timing.lock();
+ let mut total: u32 = 0;
+ for (i, t) in timings.iter().enumerate() {
+ if i == connector {
+ continue;
+ }
+ if let Some(rate) = proposed[i] {
+ total = total.saturating_add(rate);
+ continue;
+ }
+ if self.modeset_requested[i].load(Ordering::Acquire) == 0 || !self.connector_present(i)
+ {
+ continue;
+ }
+ if let Some(t) = t {
+ total = total.saturating_add(
+ u32::from(t.hactive)
+ .saturating_mul(u32::from(t.vactive))
+ .saturating_mul(u32::from(t.refresh_hz)),
+ );
+ }
+ }
+ total
+ }
+}
+
+#[cfg(CONFIG_DRM_VINO_KUNIT_TEST)]
+#[kunit_tests(vino_mode_limits)]
+mod tests {
+ use super::*;
+ use crate::*;
+
+ #[test]
+ fn timing_key_covers_every_set_mode_field() {
+ let base = cp::Timing {
+ hactive: 1920,
+ hblank: 280,
+ hsync_front: 88,
+ hsync_width: 44,
+ vactive: 1080,
+ vblank: 45,
+ vsync_front: 4,
+ vsync_width: 5,
+ refresh_hz: 60,
+ pixel_clock_10khz: 14_850,
+ sync_flags: 0x0400,
+ stride: 0x0800,
+ total_rows: 0x2000,
+ vic_word: 0x2810,
+ ten_bit: false,
+ st2084: false,
+ dual_nivo: false,
+ };
+ let timings = [
+ base,
+ cp::Timing {
+ hactive: 1921,
+ ..base
+ },
+ cp::Timing {
+ hblank: 281,
+ ..base
+ },
+ cp::Timing {
+ hsync_front: 89,
+ ..base
+ },
+ cp::Timing {
+ hsync_width: 45,
+ ..base
+ },
+ cp::Timing {
+ vactive: 1081,
+ ..base
+ },
+ cp::Timing { vblank: 46, ..base },
+ cp::Timing {
+ vsync_front: 5,
+ ..base
+ },
+ cp::Timing {
+ vsync_width: 6,
+ ..base
+ },
+ // Exercise the high byte that the former packed key discarded.
+ cp::Timing {
+ refresh_hz: 0x013c,
+ ..base
+ },
+ // Exercise bits above the former 22-bit pixel-clock mask.
+ cp::Timing {
+ pixel_clock_10khz: 0x0140_3a02,
+ ..base
+ },
+ cp::Timing {
+ sync_flags: 0x0401,
+ ..base
+ },
+ cp::Timing {
+ stride: 0x0880,
+ ..base
+ },
+ cp::Timing {
+ total_rows: 0x2001,
+ ..base
+ },
+ cp::Timing {
+ vic_word: 0x281f,
+ ..base
+ },
+ cp::Timing {
+ ten_bit: true,
+ ..base
+ },
+ cp::Timing {
+ st2084: true,
+ ..base
+ },
+ cp::Timing {
+ dual_nivo: true,
+ ..base
+ },
+ ];
+ let mut keys = [0u64; 18];
+ for (i, timing) in timings.iter().enumerate() {
+ keys[i] = timing_key(timing);
+ assert_ne!(keys[i], 0);
+ for previous in &keys[..i] {
+ assert_ne!(keys[i], *previous);
+ }
+ }
+ }
+
+ #[test]
+ fn no_op_mode_set_compares_the_exact_programmed_timing() {
+ let raw = cp::Timing {
+ hactive: 2560,
+ hblank: 160,
+ hsync_front: 48,
+ hsync_width: 32,
+ vactive: 1440,
+ vblank: 41,
+ vsync_front: 3,
+ vsync_width: 5,
+ refresh_hz: 60,
+ pixel_clock_10khz: 24_150,
+ sync_flags: 0x0600,
+ stride: 0x0a80,
+ total_rows: 0x66db,
+ vic_word: 0x0800,
+ ten_bit: false,
+ st2084: false,
+ dual_nivo: false,
+ };
+ let effective = cp::Timing {
+ dual_nivo: true,
+ ..raw
+ };
+
+ // The request can have been queued before an endpoint partner appeared, so its raw token
+ // and the exact Timing corrected at send time legitimately differ.
+ assert_ne!(timing_key(&raw), timing_key(&effective));
+ assert!(programmed_mode_matches(
+ timing_key(&raw),
+ Some(effective),
+ effective
+ ));
+ assert!(!programmed_mode_matches(
+ timing_key(&raw),
+ Some(raw),
+ effective
+ ));
+ // Clearing the active generation is an explicit request to touch hardware, even if an old
+ // programmed-state snapshot remains available for diagnostics.
+ assert!(!programmed_mode_matches(0, Some(effective), effective));
+ }
+
+ /// The boundary cases matter most: each must pass by equality. A `<` would prune a dock's
+ /// working configuration and dark its panels.
+ #[test]
+ fn mode_ceilings_bound_bandwidth_not_refresh() {
+ let refresh_ok =
+ |p: &DockProfile, hz: i32| hz <= 0 || (hz as u32) <= p.capabilities.max_refresh_hz;
+ let clock_ok = |p: &DockProfile, khz: u32| khz <= p.capabilities.max_connector_clock_khz;
+ let rate = active_pixel_rate;
+
+ // Ridge carries 2560x1440p144, so no refresh cap may hide it: 597.29 MHz of clock and
+ // 530,841,600 pixels per second both sit inside its ceilings. The 180 Hz request DLM
+ // answers with 119.998 Hz is 746.64 MHz, which the clock ceiling already refuses -- that
+ // is the whole of what a refresh cap here would have bought.
+ assert!(refresh_ok(&profile::PROFILE_RIDGE, 144));
+ assert!(clock_ok(&profile::PROFILE_RIDGE, 597_290));
+ assert!(rate(2560, 1440, 144) <= profile::PROFILE_RIDGE.capabilities.pixel_budget);
+ assert!(!clock_ok(&profile::PROFILE_RIDGE, 746_640));
+
+ // The DL7400 is bounded by link rate alone too -- DLM drives it at 2560x1440@164.96.
+ assert!(
+ refresh_ok(&profile::PROFILE_NAVARRO, 180)
+ && refresh_ok(&profile::PROFILE_NAVARRO, 240)
+ );
+
+ // 2560x1440: p165 is 699.50 MHz and carried; p180 is 714.81 MHz and is the mode the dock
+ // accepts and then fails to deliver.
+ assert!(clock_ok(&profile::PROFILE_NAVARRO, 699_500));
+ assert!(!clock_ok(&profile::PROFILE_NAVARRO, 714_810));
+ // Ridge carries 2560x1440p144 at 597.29 MHz and blanks the sink at p165's 699.50 MHz.
+ assert!(
+ clock_ok(&profile::PROFILE_RIDGE, 597_290)
+ && !clock_ok(&profile::PROFILE_RIDGE, 699_500)
+ );
+
+ // A degenerate mode reports 0 Hz and carries no rate information; a signed refresh must
+ // never be read as a huge unsigned one.
+ assert!(
+ refresh_ok(&profile::PROFILE_NAVARRO, 0) && refresh_ok(&profile::PROFILE_NAVARRO, -1)
+ );
+
+ // Each budget admits its own dual-connector configuration and nothing beyond it.
+ assert_eq!(rate(2560, 1440, 120), 442_368_000);
+ // Ridge sustains 2560x1440p144 beside 2560x1440p120, so its budget must admit that pair.
+ assert_eq!(
+ profile::PROFILE_RIDGE.capabilities.pixel_budget,
+ rate(2560, 1440, 144) + rate(2560, 1440, 120)
+ );
+ assert_eq!(
+ profile::PROFILE_NAVARRO.capabilities.pixel_budget,
+ 2 * rate(2560, 1440, 165)
+ );
+ // That pair is admitted at 24 bpp and refused at 30, which is what the hardware does: a
+ // budget large enough to admit it deep leaves both sinks powered off with nothing logged.
+ let price = |budget: u32| budget / 4 * 3;
+ let deep = price(profile::PROFILE_NAVARRO.capabilities.pixel_budget);
+ assert!(2 * rate(2560, 1440, 165) > deep);
+ assert!(2 * rate(2560, 1440, 120) <= deep);
+ // Two 1440p120 connectors fit deep, and stop fitting the moment the price is charged
+ // twice, so a budget priced at the depth being decided withdraws the ten bits that same
+ // pair was just admitted at.
+ assert!(2 * rate(2560, 1440, 120) > price(deep));
+ assert_eq!(rate(65535, 65535, 65535), u32::MAX); // saturates, never wraps small
+ assert_eq!(rate(2560, 1440, -1), 0);
+ }
+}
diff --git a/drivers/gpu/drm/vino/drm_sink/mode_objects.rs b/drivers/gpu/drm/vino/drm_sink/mode_objects.rs
new file mode 100644
index 000000000000..d3ec818ec1da
--- /dev/null
+++ b/drivers/gpu/drm/vino/drm_sink/mode_objects.rs
@@ -0,0 +1,978 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! The KMS objects the compositor drives: CRTC, primary and cursor planes, encoder and connector,
+//! plus the software vblank timer that paces them.
+//!
+//! These callbacks run under the DRM atomic lock and must not block, so anything that talks to the
+//! dock is queued for [`super::VinoDrmData`]'s workers rather than done here.
+
+use super::*;
+// `hdr_output_eotf` reads the connector's HDR metadata from the CRTC enable path; the trait is
+// implemented for every connector state but has to be in scope to be called on an opaque one.
+use kernel::drm::kms::connector::RawConnectorState;
+
+/// A software vblank source: an hrtimer that fires once per frame and drives
+/// `drm_crtc_handle_vblank()`. It stops when vblank is disabled and is also cancelled
+/// unconditionally by [`VinoDrmData::shutdown`].
+#[pin_data]
+pub(crate) struct VblankTimer {
+ #[pin]
+ timer: HrTimer<Self>,
+ /// Owned CRTC reference used by the hard-timer callback.
+ ///
+ /// This reference forms a cycle through the DRM device, so shutdown clears it after cancelling
+ /// the timer. The IRQ-aware lock permits access from both process and hard-timer context.
+ #[pin]
+ pub(super) crtc: SpinLockIrq<Option<crtc::CrtcRef<VinoCrtc>>>,
+ /// One scanout frame in nanoseconds (from the mode's `framedur_ns`).
+ interval_ns: AtomicI64,
+ /// Whether vblanks should currently be delivered (toggled by enable/disable_vblank).
+ pub(super) enabled: AtomicBool,
+}
+
+impl VblankTimer {
+ fn new() -> impl PinInit<Self> {
+ pin_init!(VblankTimer {
+ timer <- HrTimer::new(),
+ crtc <- new_spinlock_irq!(None, "vino::vblank_crtc"),
+ interval_ns: AtomicI64::new(16_666_666), // ~60 Hz until a mode sets it
+ enabled: AtomicBool::new(false),
+ })
+ }
+}
+
+impl HrTimerCallback for VblankTimer {
+ type Pointer<'a> = Arc<Self>;
+
+ fn run(this: ArcBorrow<'_, Self>, mut ctx: HrTimerCallbackContext<'_, Self>) -> HrTimerRestart {
+ // Vblank is off: let the timer die instead of ticking uselessly; `enable_vblank` re-arms
+ // it. A concurrent re-arm racing this return is safe -- hrtimer keeps a timer that was
+ // re-queued during its callback enqueued even on NORESTART.
+ if !this.enabled.load(Ordering::Relaxed) {
+ return HrTimerRestart::NoRestart;
+ }
+ // Take an owned copy of the published handle and release the lock *before* delivering the
+ // vblank. `drm_crtc_handle_vblank()` takes `dev->vblank_time_lock`, and `enable_vblank`
+ // runs the other way round -- it is called with the DRM vblank locks already held and
+ // acquires this one -- so holding this lock across the delivery would be a lock inversion.
+ // Cloning is just a `drm_dev_get()`, and the clone cannot drop the last reference: the
+ // handle we cloned from stays published for the whole callback, because the only code that
+ // clears it (`VinoDrmData::shutdown`) does so after `hrtimer_cancel` has waited for this
+ // callback to return.
+ let crtc = this.crtc.lock_with(ctx.local_interrupt_disabled()).clone();
+ if let Some(crtc) = crtc {
+ crtc.crtc().handle_vblank();
+ }
+ let interval = this.interval_ns.load(Ordering::Relaxed).max(1_000_000);
+ ctx.forward_now(Delta::from_nanos(interval));
+ HrTimerRestart::Restart
+ }
+}
+
+impl_has_hr_timer! {
+ impl HasHrTimer<Self> for VblankTimer {
+ mode: RelativeHardMode<Monotonic>, field: self.timer
+ }
+}
+
+#[pin_data]
+pub(crate) struct VinoCrtc {
+ /// Which display connector (0-based) this CRTC drives. Names the connector in diagnostics, and
+ /// maps a CRTC in an atomic commit onto the connector whose rate it spends from the dock-wide
+ /// budget.
+ pub(super) connector: u8,
+ /// The software vblank source for this CRTC.
+ vblank: Arc<VblankTimer>,
+ /// One driver-owned DRM vblank reference held for the whole active interval. A USB display has
+ /// no hardware interrupt to bootstrap the compositor's first post-modeset presentation; if no
+ /// initial page-flip event is attached, the DRM core never calls `enable_vblank`, the software
+ /// timer never starts, and KWin leaves the first framebuffer frozen forever. Pinning one ref
+ /// while active starts the clock deterministically; `atomic_disable` balances it before off.
+ /// The vblank reference held for the whole time this CRTC is active. Taken in
+ /// `atomic_enable` and released in `atomic_disable`, which is longer than a borrowed
+ /// `VblankRef` can live, so an owned one is stored here.
+ #[pin]
+ pub(super) vblank_pinned: Mutex<Option<OwnedVblankRef<VinoCrtc>>>,
+}
+
+#[derive(Clone, Default)]
+pub(crate) struct VinoCrtcState;
+
+impl crtc::DriverCrtcState for VinoCrtcState {
+ type Crtc = VinoCrtc;
+}
+
+/// Whether a connector state asks for a ten-bit link.
+///
+/// A compositor sets `max bpc` to ten on every connector it can, so the request alone is not the
+/// condition: the vendor moves a link to ten bits when the connector is driven in PQ. A commit
+/// carrying no connector state, which is every page flip, asks for nothing.
+fn asks_for_ten_bits(
+ conn: Option<&kernel::drm::kms::connector::OpaqueConnectorState<VinoDrmDriver>>,
+) -> bool {
+ conn.is_some_and(|conn| {
+ conn.max_requested_bpc() >= 10
+ && conn.hdr_output_eotf() == Some(connector::Eotf::SmpteSt2084)
+ })
+}
+
+#[vtable]
+impl crtc::DriverCrtc for VinoCrtc {
+ type Args = u8;
+ type Driver = VinoDrmDriver;
+ type State = VinoCrtcState;
+ type VblankImpl = Self;
+
+ fn new(_device: &drm::Device<Self::Driver>, connector: &u8) -> impl PinInit<Self, Error> {
+ try_pin_init!(VinoCrtc {
+ connector: *connector,
+ vblank: Arc::pin_init(VblankTimer::new(), GFP_KERNEL)?,
+ vblank_pinned <- new_mutex!(None),
+ })
+ }
+
+ /// The display is turning on (scanout begins). Enables vblank pacing, pushes a live mode-set CP
+ /// message for the negotiated mode. The command is queued and is a no-op until CP engages.
+ const HAS_ATOMIC_CHECK: bool = true;
+
+ /// Reject a commit that exceeds the dock's combined active-connector budget.
+ ///
+ /// `mode_valid` checks each connector against the complete budget because the
+ /// advertised modes must not depend on another connector's current state.
+ /// A commit that does not increase this connector's rate is always allowed (it can only hold or
+ /// reduce the combined total); no limiting when the budget is 0 (unknown).
+ fn atomic_check(check: CrtcAtomicCheck<'_, Self>) -> Result {
+ let crtc = check.crtc();
+ let connector = crtc.connector as usize;
+ let data: &VinoDrmData = crtc.drm_dev();
+ let budget = data.dock_budget();
+ let (state, old, mut new) = check.take_all();
+ let wants_deep =
+ data.hdr_capable() && asks_for_ten_bits(state.new_connector_state_for_crtc(crtc));
+ // The colour description reaches the dock in the mode-set message, which is only sent from
+ // `atomic_enable`. Toggling HDR on a live output changes nothing the core considers a mode
+ // change, so ask for one: otherwise the compositor starts encoding PQ into a sink that was
+ // never told, and the picture washes out.
+ let colour_changed = {
+ let eotf = |c: Option<&_>| -> (u32, Option<connector::Eotf>) {
+ c.map_or(
+ (0, None),
+ |c: &kernel::drm::kms::connector::OpaqueConnectorState<VinoDrmDriver>| {
+ (c.colorspace(), c.hdr_output_eotf())
+ },
+ )
+ };
+ eotf(state.old_connector_state_for_crtc(crtc))
+ != eotf(state.new_connector_state_for_crtc(crtc))
+ };
+ if colour_changed {
+ new.set_mode_changed(true);
+ }
+ let old_rate = if old.active() {
+ let m = old.mode();
+ active_pixel_rate(m.hdisplay(), m.vdisplay(), m.vrefresh())
+ } else {
+ 0
+ };
+ let new_rate = if new.active() {
+ let m = new.mode();
+ if (!old.active() || new.mode_changed()) && !crate::cp::mode_supported(m) {
+ pr_warn!(
+ "vino: socket {socket} mode {}x{}@{} has no dock profile\n",
+ m.hdisplay(),
+ m.vdisplay(),
+ m.vrefresh(),
+ socket = connector + 1
+ );
+ return Err(EINVAL);
+ }
+ active_pixel_rate(m.hdisplay(), m.vdisplay(), m.vrefresh())
+ } else {
+ 0
+ };
+ // Refresh ceiling, enforced here as well as in `mode_valid`, because pruning the mode list
+ // is not a limit: a client can commit a user-defined mode that was never advertised
+ // (`xrandr --newmode`, a modeline in a compositor config, `drm_mode_setcrtc` with its own
+ // timing). This is the check that actually stops the dock being driven at a rate it goes
+ // dark on, and it costs one comparison on the commit path.
+ //
+ // Only a commit that raises the refresh is rejected, exactly as the budget check below only
+ // examines a commit that raises the rate. Every page flip carries the CRTC state through
+ // here, so revalidating an unchanged rate would only add work to the hot path.
+ let old_refresh = if old.active() {
+ old.mode().vrefresh()
+ } else {
+ 0
+ };
+ let new_refresh = if new.active() {
+ new.mode().vrefresh()
+ } else {
+ 0
+ };
+ if new_refresh > old_refresh && !data.refresh_within_limit(new_refresh) {
+ let limit = data.max_refresh_hz();
+ pr_warn!(
+ "vino: socket {socket} refresh {new_refresh} exceeds {limit} Hz\n",
+ socket = connector + 1
+ );
+ return Err(EINVAL);
+ }
+ if budget == 0 || new_rate <= old_rate {
+ return Ok(());
+ }
+ let others = data.other_connectors_rate(&state, connector);
+ let combined = new_rate.saturating_add(others);
+ // Price the dock at the depth this commit drives, and at what its other connectors already
+ // hold, because the bandwidth is shared. Nothing is recorded here: a check also runs for
+ // page flips and for `TEST_ONLY` commits that are never applied, so a depth recorded from
+ // one would move the codec with no mode set to re-state it to the dock.
+ let deep = wants_deep && data.ten_bit_fits(combined);
+ let budget = data.budget_at_depth(
+ budget,
+ deep || data.other_connector_programmed_ten_bit(crtc.connector),
+ );
+ if combined > budget {
+ pr_warn!(
+ "vino: socket {socket} combined rate {combined} exceeds {budget}\n",
+ socket = connector + 1
+ );
+ return Err(EINVAL);
+ }
+ Ok(())
+ }
+
+ fn atomic_enable(commit: CrtcAtomicCommit<'_, Self>) {
+ let crtc = commit.crtc();
+ crtc.vblank_on();
+ // Keep the software presentation clock running for the complete active interval. The
+ // reference is stored as an owned one because it must outlive this callback and is only
+ // released in `atomic_disable`. Page-flip events take their own additional refs.
+ let mut pinned = crtc.vblank_pinned.lock();
+ if pinned.is_none() {
+ match crtc.vblank_get() {
+ Ok(vblank_ref) => *pinned = Some(vblank_ref.into_owned()),
+ Err(e) => pr_warn!(
+ "vino: failed to start connector {} software vblank clock ({e:?})\n",
+ crtc.connector
+ ),
+ }
+ }
+ drop(pinned);
+ let connector = crtc.connector;
+ let dev: &VinoDrmDevice = crtc.drm_dev();
+ let data: &VinoDrmData = dev;
+ let (state, new) = commit.take_state_new_state();
+ // The transfer function is a connector property, but the mode set that carries it to the
+ // dock is built here, so read it across from the connector routed to this CRTC. Only PQ
+ // is distinguished: the dock's flags word has exactly one HDR bit, and every other EOTF
+ // (including HLG, which it cannot express) is carried as SDR rather than mislabelled.
+ let st2084 = state
+ .new_connector_state_for_crtc(crtc)
+ .and_then(|conn| conn.hdr_output_eotf())
+ == Some(connector::Eotf::SmpteSt2084);
+ data.set_connector_st2084(connector, st2084);
+ // The link depth userspace asked for, which is a separate question from the framebuffer's
+ // format: an eight-bit surface over a ten-bit link is the ordinary case, and reading the
+ // format alone silently ignores the request.
+ let requested_bpc = state
+ .new_connector_state_for_crtc(crtc)
+ .map_or(0, |conn| conn.max_requested_bpc());
+ data.set_connector_max_bpc(connector, requested_bpc);
+ // Decide the depth rather than refuse the mode: a pair that fits at eight bits may not fit
+ // at ten, and a compositor answers a rejected mode by disabling the output. Recorded here
+ // and not in the check, so that the decision and the set-mode carrying it are one commit.
+ let combined = if new.active() {
+ let m = new.mode();
+ active_pixel_rate(m.hdisplay(), m.vdisplay(), m.vrefresh())
+ } else {
+ 0
+ }
+ .saturating_add(data.other_connectors_rate(&state, connector as usize));
+ let wants_deep =
+ data.hdr_capable() && asks_for_ten_bits(state.new_connector_state_for_crtc(crtc));
+ data.set_connector_ten_bit_denied(connector, wants_deep && !data.ten_bit_fits(combined));
+ // Cache this connector's colour transform for the scanout to apply.
+ data.update_color(connector as usize, new.gamma_lut(), new.ctm());
+ // Whatever this connector's sink state was, the enable path re-runs the full bracket and
+ // mode-set, so any silence from here is the dock's news, not vino's.
+ data.set_self_blanked(connector as usize, false);
+ // The depth the dock is told must match the depth the plane will actually send.
+ // `connector_ten_bit` is set from the committed framebuffer's fourcc, so a connector that
+ // never gets a 10-bit buffer is never announced as 30 bpp. It goes in rather than being
+ // patched on afterwards because the framebuffer allocation the set-mode states is derived
+ // from it.
+ let ten_bit = data.connector_is_ten_bit(connector as usize);
+ let timing = match crate::cp::timing_from_drm_mode(new.mode(), data.allocation(), ten_bit) {
+ Ok(mut timing) => {
+ // Read back rather than reusing the local, so both halves of the colour
+ // description come from the same per-connector state every other path consults.
+ timing.st2084 = data.connector_is_st2084(connector as usize);
+ // Declare the shared video endpoint. Four connectors are multiplexed onto two bulk
+ // endpoints, and DLM names bit 2 of the flags word `Dual NIVO`; a connector whose
+ // partner connector is also live has to say so, or the dock drives only one of the
+ // two streams it is being sent.
+ timing.dual_nivo = data.endpoint_is_shared(connector as usize);
+ timing
+ }
+ Err(e) => {
+ pr_err!(
+ "vino: connector {} reached atomic enable with an unsupported mode ({e:?})\n",
+ connector
+ );
+ return;
+ }
+ };
+ vino_debug!(
+ "vino: KMS CRTC enable -- connector {} display ON, mode {}x{}@{} {} bpc{} (scanout begins)\n",
+ connector,
+ timing.hactive,
+ timing.vactive,
+ timing.refresh_hz,
+ if timing.ten_bit { 10 } else { 8 },
+ if timing.st2084 { " PQ" } else { "" }
+ );
+ // Publish the desired timing; atomic callbacks must not block on USB.
+ let mode_key = timing_key(&timing);
+ data.last_timing.lock()[connector as usize] = Some(timing);
+ data.modeset_requested[connector as usize].store(mode_key, Ordering::Release);
+ data.queue_cmd(dev, KmsCmd::ModeSet { connector, timing });
+ }
+
+ /// The display is turning off (DPMS-off/blank/suspend all land here in atomic KMS).
+ /// Resets the scanout state so a later re-enable sends a full keyframe rather than diffing
+ /// against a shadow the dock may have dropped. Do not send the monitor's DDC/CI VCP 0xd6
+ /// here: hard standby is separate from stopping the DisplayLink stream and can leave a panel
+ /// asleep across a dock power cycle.
+ fn atomic_disable(commit: CrtcAtomicCommit<'_, Self>) {
+ let crtc = commit.crtc();
+ // Dropping the stored reference releases the vblank reference `atomic_enable` took.
+ drop(crtc.vblank_pinned.lock().take());
+ crtc.vblank_off();
+ let connector = crtc.connector;
+ let dev: &VinoDrmDevice = crtc.drm_dev();
+ let data: &VinoDrmData = dev;
+ data.update_color(connector as usize, None, None);
+ // The stream is torn down; a later re-enable must re-send the mode-set before any video
+ // write (the dock EPIPEs a write onto an unconfigured stream). Forget the active mode so
+ // the scanout gate defers until the re-enable's mode-set lands.
+ data.modeset_requested[connector as usize].store(0, Ordering::Release);
+ data.modeset_active[connector as usize].store(0, core::sync::atomic::Ordering::Release);
+ data.programmed_timing.lock()[connector as usize] = None;
+ // Drop a framebuffer queued while this CRTC was active. Otherwise the deferred worker can
+ // retry its old mode and paint after DPMS-off.
+ data.pending_scanout.lock()[connector as usize] = None;
+ data.settle_repaint.lock()[connector as usize] = None;
+ // Spend nothing on a connector that is off; the re-enable's mode-set refills it.
+ data.settle_budget[connector as usize].store(0, Ordering::Relaxed);
+ // Release the ~14.7 MB private copy; a re-enable owes a keyframe and re-snapshots.
+ data.shadow[connector as usize].lock().discard();
+ data.sustain_until.lock()[connector as usize] = None;
+ data.strip_hashes.lock()[connector as usize] = None;
+ data.dirty_ttl.lock()[connector as usize] = None;
+ vino_debug!(
+ "vino: KMS CRTC disable -- socket {socket} display OFF (scanout stopped)\n",
+ socket = connector + 1
+ );
+ // Stopping locally is not enough: the dock goes on scanning out whatever it last received,
+ // so a DPMS-off left the panel lit on a frozen desktop. Queue the dock-side take-down for
+ // the command worker -- this callback must not block on USB (see `KmsCmd`). It is queued
+ // last, after the mode generation has been zeroed, because `blank_connector` keys its write
+ // on exactly that zero.
+ data.queue_cmd(dev, KmsCmd::Blank { connector });
+ }
+
+ /// Arm the page-flip completion event to be sent by the next vblank tick, so userspace is paced
+ /// to the refresh rate rather than signalled immediately.
+ fn atomic_flush(commit: CrtcAtomicCommit<'_, Self>) {
+ let crtc = commit.crtc();
+ let data: &VinoDrmData = crtc.drm_dev();
+ let mut new = commit.take_new_state();
+ // Re-cache the colour transform on every commit that touches this CRTC, so a dynamic
+ // GAMMA_LUT or CTM change on an already-enabled connector (which does not re-run
+ // atomic_enable) is picked up rather than deferred to the next full modeset. A night-light
+ // corrector ramps its CTM continuously, so this is the path that carries it, not
+ // atomic_enable.
+ data.update_color(crtc.connector as usize, new.gamma_lut(), new.ctm());
+ if let Some(pending) = new.get_pending_vblank_event() {
+ match crtc.vblank_get() {
+ Ok(vbl_ref) => pending.arm(vbl_ref),
+ // Vblank couldn't be enabled (e.g. mid-teardown): fall back to sending now.
+ Err(_) => pending.send(),
+ }
+ }
+ }
+}
+
+impl VblankSupport for VinoCrtc {
+ type Crtc = VinoCrtc;
+
+ fn enable_vblank(
+ crtc: &crtc::Crtc<Self::Crtc>,
+ vblank_guard: &VblankGuard<'_, Self::Crtc>,
+ irq: &LocalInterruptDisabled,
+ ) -> Result {
+ let data: &VinoCrtc = crtc;
+ // Track the mode's real frame duration so the tick matches the negotiated refresh rate.
+ let fd = vblank_guard.frame_duration();
+ if fd > 0 {
+ data.vblank.interval_ns.store(fd as i64, Ordering::Relaxed);
+ }
+ // Publish the CRTC for the timer callback. Only the first enable stores it; the CRTC a
+ // given timer serves never changes, and re-taking the reference on every enable would just
+ // leak one `drm_dev_get()` per DPMS cycle. `lock_with` because the DRM core already called
+ // us with local interrupts disabled -- proven by the `irq` token.
+ {
+ let mut published = data.vblank.crtc.lock_with(irq);
+ if published.is_none() {
+ *published = Some(crtc.to_owned_ref());
+ }
+ }
+ data.vblank.enabled.store(true, Ordering::Relaxed);
+ let interval = data.vblank.interval_ns.load(Ordering::Relaxed);
+ // The started timer is registered on the DEVICE, so teardown can cancel it without
+ // depending on the DRM core calling `disable_vblank` -- see `VinoDrmData::vblank`.
+ let drm_data: &VinoDrmData = crtc.drm_dev();
+ let connector = usize::from(data.connector);
+ if connector >= MAX_CONNECTORS {
+ return Ok(());
+ }
+ let mut slots = drm_data.vblank.lock();
+ match &slots[connector] {
+ None => {
+ // First enable: start the timer and keep the handle as its sole owner.
+ slots[connector] = Some((
+ data.vblank.clone(),
+ data.vblank.clone().start(Delta::from_nanos(interval)),
+ ));
+ }
+ Some((_, h)) => {
+ // Re-enable after `disable_vblank` let the timer die (NoRestart): re-queue it in
+ // place. `restart` removes and re-inserts a still-pending timer, so this is
+ // correct whether the final disabled tick has already fired or not, and it never
+ // blocks on the callback -- which matters because we are called under the vblank
+ // locks with interrupts disabled.
+ h.restart(Delta::from_nanos(interval));
+ }
+ }
+ Ok(())
+ }
+
+ fn disable_vblank(
+ crtc: &crtc::Crtc<Self::Crtc>,
+ _vblank_guard: &VblankGuard<'_, Self::Crtc>,
+ _irq: &LocalInterruptDisabled,
+ ) {
+ let data: &VinoCrtc = crtc;
+ data.vblank.enabled.store(false, Ordering::Relaxed);
+ }
+
+ fn get_vblank_timestamp(
+ _crtc: &crtc::Crtc<Self::Crtc>,
+ _in_vblank_irq: bool,
+ ) -> Option<VblankTimestamp> {
+ // Let DRM estimate the timestamp from the mode timings.
+ None
+ }
+}
+
+// ---- Planes: primary (scanout) + cursor -------------------------------------
+//
+// The safe KMS layer allows one `DriverPlane` type per driver, so `VinoPlane` serves both the
+// primary and cursor planes, told apart by `is_cursor` (from the plane's `Args`).
+
+/// Constructor arguments for a [`VinoPlane`]: which connector it belongs to and whether it is that
+/// connector's cursor plane (vs. its primary scanout plane).
+#[derive(Clone, Copy)]
+pub(crate) struct PlaneArgs {
+ pub(super) connector: u8,
+ pub(super) is_cursor: bool,
+}
+
+#[pin_data]
+pub(crate) struct VinoPlane {
+ /// Which display connector (0-based) this plane belongs to. Selects the scanout video endpoint
+ /// (see `DockProfile::video_endpoints`) and the cursor CP `connector` field.
+ connector: u8,
+ /// Whether this is the cursor plane (vs. the primary scanout plane).
+ is_cursor: bool,
+ /// The framebuffer region last uploaded as the cursor bitmap.
+ #[pin]
+ cursor_last: Mutex<Option<CursorUpload>>,
+}
+
+struct CursorUpload {
+ framebuffer: ARef<kms::framebuffer::Framebuffer<VinoDrmDriver>>,
+ /// Value of this connector's `cursor_epoch` when the bitmap was sent. A newer epoch means the
+ /// dock has since been reconfigured and is no longer holding it.
+ epoch: u32,
+}
+
+#[derive(Clone, Default)]
+pub(crate) struct VinoPlaneState;
+
+impl plane::DriverPlaneState for VinoPlaneState {
+ type Plane = VinoPlane;
+}
+
+#[vtable]
+impl plane::DriverPlane for VinoPlane {
+ type Args = PlaneArgs;
+ type Driver = VinoDrmDriver;
+ type State = VinoPlaneState;
+
+ fn new(_device: &drm::Device<Self::Driver>, args: PlaneArgs) -> impl PinInit<Self, Error> {
+ try_pin_init!(VinoPlane {
+ connector: args.connector,
+ is_cursor: args.is_cursor,
+ cursor_last <- new_mutex!(None),
+ })
+ }
+
+ /// Validate plane geometry and populate `drm_plane_state.visible`, which the damage iterator
+ /// requires before it can report changed rectangles.
+ fn atomic_check(check: PlaneAtomicCheck<'_, Self>) -> Result {
+ let plane = check.plane();
+ let (state, _old, mut new) = check.take_all();
+ let Some(crtc) = new.crtc::<VinoDrmDriver>() else {
+ // A disabled plane is not visible and needs no geometry validation.
+ return Ok(());
+ };
+ let crtc_state = match state.get_new_crtc_state(crtc) {
+ Some(s) => s,
+ None => state.add_crtc_state(crtc)?,
+ };
+ // Vino supports 1:1 scanout only. Primary planes must cover the CRTC; cursor planes may be
+ // positioned and clipped by the helper. Updates on a disabled CRTC remain disallowed.
+ new.atomic_helper_check::<_, VinoDrmDriver>(&crtc_state, plane.is_cursor, false)?;
+
+ // The transfer paths currently consume a full framebuffer, not an arbitrary source crop.
+ // Require exactly that at the UAPI boundary. Cursor clipping performed by the helper above
+ // is represented separately in its derived source rectangle.
+ if let Some(fb) = new.framebuffer::<VinoDrmDriver>() {
+ let full_width = fb.width().checked_shl(16).ok_or(EINVAL)?;
+ let full_height = fb.height().checked_shl(16).ok_or(EINVAL)?;
+ if new.source_x_16_16() != 0
+ || new.source_y_16_16() != 0
+ || new.source_width_16_16() != full_width
+ || new.source_height_16_16() != full_height
+ {
+ return Err(EINVAL);
+ }
+ }
+
+ Ok(())
+ }
+
+ /// A new framebuffer was flipped in. Maps it, converts XRGB8888 -> RGB565 (or feeds the
+ /// Haar colour codec directly for an aligned mode), and bulk-writes the resulting EP08
+ /// frame(s).
+ ///
+ /// EP08 writes happen only after CP engagement and a matching mode-set has landed.
+ fn atomic_update(commit: PlaneAtomicCommit<'_, Self>) {
+ let plane = commit.plane();
+ let connector = plane.connector;
+ let dev: &VinoDrmDevice = plane.drm_dev();
+ let data: &VinoDrmData = dev;
+ if !data.cp_engaged.load(core::sync::atomic::Ordering::SeqCst) {
+ return;
+ }
+
+ // Cursor plane: publish bitmap and position commands for the asynchronous control worker.
+ // The protocol uses id=0x1b for create, id=0x1c with the inner bitmap flag set for image,
+ // and id=0x1a for movement.
+ if plane.is_cursor {
+ let new = commit.take_new_state();
+ match new.framebuffer::<VinoDrmDriver>() {
+ Some(fb) => {
+ let Some(source) = new.visible_source().ok().flatten() else {
+ *plane.cursor_last.lock() = None;
+ data.queue_cmd(
+ dev,
+ KmsCmd::CursorMove {
+ connector,
+ x: 0,
+ y: 0,
+ visible: false,
+ },
+ );
+ return;
+ };
+ let Some(destination) = new.visible_destination() else {
+ return;
+ };
+ // The complete framebuffer, not the helper's clipped rectangle: the dock
+ // expects a fixed-size cursor and clips at the panel edge itself.
+ let Ok(w) = u16::try_from(fb.width()) else {
+ return;
+ };
+ let Ok(h) = u16::try_from(fb.height()) else {
+ return;
+ };
+ let epoch = data.cursor_epoch[usize::from(connector)].load(Ordering::Acquire);
+ let mut last = plane.cursor_last.lock();
+ // Re-send when the bitmap changed, or when a reconfigure means the dock is no
+ // longer holding it: `CursorMove` succeeds against a cursor that no longer
+ // exists, so a stale match is silent on the wire.
+ let unchanged = last.as_ref().is_some_and(|last| {
+ core::ptr::eq(&*last.framebuffer, fb) && last.epoch == epoch
+ });
+ if !unchanged {
+ if let Ok(bgra) = read_cursor_bgra(fb, usize::from(w), usize::from(h)) {
+ // One shared bitmap per device: announce geometry only when it
+ // changes, not on every shape change.
+ let hi = usize::from(connector);
+ if data.cursor_geometry.lock()[hi].replace((w, h)) != Some((w, h)) {
+ data.queue_cmd(dev, KmsCmd::CursorCreate { connector, w, h });
+ }
+ data.queue_cmd(
+ dev,
+ KmsCmd::CursorImage {
+ connector,
+ w,
+ h,
+ bgra,
+ },
+ );
+ *last = Some(CursorUpload {
+ framebuffer: ARef::from(fb),
+ epoch,
+ });
+ }
+ }
+ // The dock positions the whole bitmap by its top-left, so this is the
+ // unclipped origin: scanout is 1:1, so how far into the source the helper
+ // started is how far off-screen the origin is. Clamped because the wire
+ // coordinates are unsigned -- the pointer stops at the edge.
+ let Ok(x) = u16::try_from((destination.x1 - source.x1).max(0)) else {
+ return;
+ };
+ let Ok(y) = u16::try_from((destination.y1 - source.y1).max(0)) else {
+ return;
+ };
+ data.queue_cmd(
+ dev,
+ KmsCmd::CursorMove {
+ connector,
+ x,
+ y,
+ visible: true,
+ },
+ );
+ }
+ // Cursor disabled: clear the dock's visible flag and forget the bitmap so a later
+ // enable uploads it again.
+ None => {
+ *plane.cursor_last.lock() = None;
+ data.queue_cmd(
+ dev,
+ KmsCmd::CursorMove {
+ connector,
+ x: 0,
+ y: 0,
+ visible: false,
+ },
+ );
+ }
+ }
+ return;
+ }
+
+ // Primary plane: take both old and new state so the frame-damage clips can be merged.
+ let (old, new) = commit.take_old_new_state();
+ let Some(fb) = new.framebuffer::<VinoDrmDriver>() else {
+ return;
+ };
+ // Plane rotation/reflection (identity unless the compositor set the rotation property).
+ let rotation = new.rotation();
+ // atomic_check has already rejected scaling, positioning, and partial source rectangles,
+ // so these destination dimensions describe the complete output and cannot overrun the
+ // framebuffer under any advertised rotation.
+ let (w, h) = (new.crtc_w() as usize, new.crtc_h() as usize);
+ // Collect the client's individual frame-damage clips (the rectangles that
+ // `damage_merged()` would collapse into one bounding box), each clamped to the output, so
+ // only the genuinely changed rectangles are re-converted from the source rather than their
+ // whole enclosing box. Only for identity rotation (the clips are in un-rotated source
+ // space; mapping them through 90/270 is not worth it for the throttled fallback path), and
+ // never on the Haar keyframe path -- see `encode_and_send`. A fixed stack array keeps the
+ // atomic-commit path allocation-free; on overflow the clips collapse into one bounding box.
+ // An empty list means the client reported no changed pixels. Rotation/reflection still
+ // promotes it to a full frame in `encode_and_send_haar`, because source-space clips cannot
+ // yet be transformed safely for those cases.
+ let mut clips = [(0usize, 0usize, 0usize, 0usize); MAX_DAMAGE_CLIPS];
+ let mut nclips = 0usize;
+ if rotation.angle() == plane::Rotation::ROTATE_0
+ && !rotation.contains(plane::Rotation::REFLECT_X | plane::Rotation::REFLECT_Y)
+ {
+ new.for_each_damage_clip(old, |r| {
+ let c = (
+ (r.x1.max(0) as usize).min(w),
+ (r.y1.max(0) as usize).min(h),
+ (r.x2.max(0) as usize).min(w),
+ (r.y2.max(0) as usize).min(h),
+ );
+ if nclips < MAX_DAMAGE_CLIPS {
+ clips[nclips] = c;
+ nclips += 1;
+ } else {
+ // Overflow: collapse ALL accumulated clips plus `c` into a single bounding box
+ // in clips[0]. Folding only clips[0] with `c` here would silently drop the
+ // damage in clips[1..], leaving those regions stale on screen; union every
+ // pending clip so the whole changed area is still repainted.
+ let mut bb = c;
+ for &r in &clips[..nclips] {
+ bb = (bb.0.min(r.0), bb.1.min(r.1), bb.2.max(r.2), bb.3.max(r.3));
+ }
+ clips[0] = bb;
+ nclips = 1;
+ }
+ });
+ }
+
+ use core::sync::atomic::Ordering::Relaxed;
+ // Throttle: while scanout is failing (dock NAKing because CP isn't engaged), skip the
+ // upcoming pageflips set by the backoff below instead of converting+encoding+sending a
+ // frame the dock will just drop.
+ let skip = data.scanout_skip[connector as usize].load(Relaxed);
+ if skip > 0 {
+ data.scanout_skip[connector as usize].store(skip - 1, Relaxed);
+ return;
+ }
+ data.queue_scanout(
+ dev,
+ fb,
+ PendingScanout {
+ connector,
+ rotation,
+ clips,
+ nclips,
+ w,
+ h,
+ shadow_idx: 0,
+ shadow_generation: 0,
+ },
+ );
+ }
+}
+
+/// The connector's fixed encoder. The dock has no encoder configuration of its own.
+#[pin_data]
+pub(crate) struct VinoEncoder;
+
+#[vtable]
+impl encoder::DriverEncoder for VinoEncoder {
+ type Driver = VinoDrmDriver;
+ type Args = ();
+
+ fn new(_device: &drm::Device<Self::Driver>, _args: ()) -> impl PinInit<Self, Error> {
+ try_pin_init!(VinoEncoder {})
+ }
+}
+
+// ---- Connector --------------------------------------------------------------
+
+#[pin_data]
+pub(crate) struct VinoConnector {
+ /// Index into the owning device's per-connector EDID/presence arrays.
+ connector: u8,
+}
+
+#[derive(Clone, Default)]
+pub(crate) struct VinoConnectorState;
+
+impl connector::DriverConnectorState for VinoConnectorState {
+ type Connector = VinoConnector;
+}
+
+#[vtable]
+impl connector::DriverConnector for VinoConnector {
+ type Args = u8;
+ type Driver = VinoDrmDriver;
+ type State = VinoConnectorState;
+
+ fn new(_device: &drm::Device<Self::Driver>, connector: u8) -> impl PinInit<Self, Error> {
+ try_pin_init!(VinoConnector { connector })
+ }
+
+ /// Install the dock's real EDID (read during probe) when available; otherwise fall back
+ /// to a single 1920x1080@60 CVT mode. Reading the real EDID gives the true monitor
+ /// name/size and its native mode list; the fallback keeps the connector usable when
+ /// nothing is plugged into the dock or the CP channel has not yet delivered the EDID.
+ fn get_modes<'a>(
+ connector: ConnectorGuard<'a, Self>,
+ guard: &ModeConfigGuard<'a, Self::Driver>,
+ ) -> i32 {
+ let data: &VinoDrmData = connector.drm_dev();
+ let edids = data.cached_edids.lock();
+ if let Some(blob) = edids
+ .get(connector.connector as usize)
+ .and_then(Option::as_ref)
+ {
+ // A failed EDID update adds no modes; fall through to the built-in list rather than
+ // reporting a count the core did not actually get.
+ match connector.add_edid_modes(blob) {
+ Ok(n) if n > 0 => return n,
+ other => {
+ // A cached EDID that yields no modes is indistinguishable downstream from a
+ // socket with nothing in it: both land on the synthesised list. Say which.
+ vino_debug!(
+ "vino: connector {} EDID ({} B) produced no modes ({:?}); using the built-in list\n",
+ connector.connector,
+ blob.len(),
+ other.map(|n| n)
+ );
+ }
+ }
+ }
+ drop(edids);
+ let _ = guard;
+ // A connector handed to DRM's EDID override must report NO modes here: the core applies the
+ // override (`drm_kms_helper.edid_firmware=`, or a debugfs `edid_override` write) purely as
+ // a fallback for a connector that is connected and produced none. Synthesising the
+ // built-in list below would satisfy the probe helper and the override would never be
+ // consulted.
+ if data.edid_from_userspace(connector.connector as usize) {
+ return 0;
+ }
+ // No downstream EDID yet: advertise the standard mode list up to the fallback resolution
+ // and prefer it, keeping the connector usable until the dock delivers a real EDID.
+ let n = connector.add_modes_noedid((FALLBACK_W as u32, FALLBACK_H as u32));
+ connector.set_preferred_mode((FALLBACK_W as u32, FALLBACK_H as u32));
+ n
+ }
+
+ /// Report the connector connected once the dock has delivered this connector's downstream EDID
+ /// (a real monitor is attached and described) OR the bring-up work item confirmed CP engagement
+ /// + this connector's DISPLAY-CAP push (`connectors_present`: on 3.4.26 the raw-EDID path can
+ /// fail, so cached EDID alone would leave every connector permanently disconnected despite a
+ /// fully-engaged dock). A connector with neither stays disconnected rather than advertising a
+ /// phantom output.
+ fn detect(connector: &Connector<Self>, _force: bool) -> Status {
+ let data: &VinoDrmData = connector.drm_dev();
+ let connector = connector.connector as usize;
+ if connector >= data.connector_count() {
+ return Status::Disconnected;
+ }
+ // On a dock that reports no downstream presence, both signals are absent for a connector
+ // that has a monitor on it, and the whole measured choreography -- the framebuffer
+ // allocation the set-mode states, the mode-set bracket, the sink states -- is the vendor
+ // driving every connector it has. Offering a subset of that leaves the dock configured for
+ // a display arrangement nobody described; `get_modes` supplies the fallback list until an
+ // EDID arrives.
+ if !data.reports_presence() {
+ // The dock recovers an EDID for the socket a monitor is in and nothing for the one
+ // that is empty, so the EDID is the whole presence signal here. The socket with
+ // nothing in it is still configured -- it joins the dock-wide transaction at its
+ // sibling's mode -- so it no longer has to be advertised to hold its place.
+ //
+ // Bring-up owns the endpoint until encrypted setup publishes the runtime CP link, so
+ // a connector must not be offered before then whatever its EDID says.
+ return if data.connector_has_edid(connector) && data.kms_activation_ready() {
+ Status::Connected
+ } else {
+ Status::Disconnected
+ };
+ }
+ let has_edid = data
+ .cached_edids
+ .lock()
+ .get(connector)
+ .is_some_and(Option::is_some);
+ let present = data
+ .connectors_present
+ .load(core::sync::atomic::Ordering::Acquire)
+ & (1 << connector)
+ != 0;
+ if has_edid || present {
+ Status::Connected
+ } else {
+ Status::Disconnected
+ }
+ }
+
+ /// Prune modes whose pixel clock exceeds a single connector's bandwidth ceiling
+ /// ([`MAX_HEAD_CLOCK_KHZ`], ~4K@60), whose refresh rate exceeds what the dock has ever been
+ /// shown to display ([`DOCK_MAX_REFRESH_HZ`]), or whose pixel rate exceeds the dock's budget.
+ fn mode_valid(connector: ConnectorModeValidation<'_, Self>, mode: &DisplayMode) -> ModeStatus {
+ let data: &VinoDrmData = connector.drm_dev();
+ if mode.clock() < 0 || mode.clock() as u32 > data.max_connector_clock_khz() {
+ return ModeStatus::ClockHigh;
+ }
+ if !data.refresh_within_limit(mode.vrefresh()) {
+ return ModeStatus::ClockHigh;
+ }
+ if !crate::cp::mode_supported(mode) {
+ return ModeStatus::Bad;
+ }
+ // Reject a mode only when that connector exceeds the dock's whole pixel budget. The atomic
+ // CRTC check enforces the combined rate of simultaneously active connectors.
+ let budget = data.dock_budget();
+ let connector_rate = active_pixel_rate(mode.hdisplay(), mode.vdisplay(), mode.vrefresh());
+ if budget != 0 && connector_rate > budget {
+ return ModeStatus::Bad;
+ }
+ ModeStatus::Ok
+ }
+}
+
+/// Map an output pixel `(dx, dy)` back to its source-framebuffer pixel `(sx, sy)` under a DRM
+/// plane `rotation` bitmask (`DRM_MODE_ROTATE_*` | `DRM_MODE_REFLECT_*`, the values the
+/// standard `drm_plane_create_rotation_property` exposes). `sw`/`sh` are the source
+/// (framebuffer) dimensions. Rotation is clockwise; reflection is applied in source space
+/// after rotation. Pure and total (saturating), so it is unit-tested directly. Applied per source
+/// pixel in [`encode_and_send`]/[`encode_and_send_haar`] for the plane's rotation property.
+#[inline]
+pub(crate) fn rot_src(
+ rotation: plane::Rotation,
+ dx: usize,
+ dy: usize,
+ sw: usize,
+ sh: usize,
+) -> (usize, usize) {
+ let xmax = sw.saturating_sub(1);
+ let ymax = sh.saturating_sub(1);
+ let rot = rotation.angle();
+ let (mut sx, mut sy) = if rot == plane::Rotation::ROTATE_90 {
+ (dy, ymax.saturating_sub(dx))
+ } else if rot == plane::Rotation::ROTATE_180 {
+ (xmax.saturating_sub(dx), ymax.saturating_sub(dy))
+ } else if rot == plane::Rotation::ROTATE_270 {
+ (xmax.saturating_sub(dy), dx)
+ } else {
+ (dx, dy) // ROTATE_0 / unset
+ };
+ if rotation.contains(plane::Rotation::REFLECT_X) {
+ sx = xmax.saturating_sub(sx);
+ }
+ if rotation.contains(plane::Rotation::REFLECT_Y) {
+ sy = ymax.saturating_sub(sy);
+ }
+ (sx, sy)
+}
+
+#[cfg(CONFIG_DRM_VINO_KUNIT_TEST)]
+#[kunit_tests(vino_mode_objects)]
+mod tests {
+ use super::*;
+
+ #[test]
+ fn rotation_pixel_mapping() {
+ use drm::kms::plane::Rotation;
+
+ // Source 2x3 (sw=2, sh=3). 0deg is identity; 180deg mirrors both axes.
+ assert_eq!(rot_src(Rotation::ROTATE_0, 0, 0, 2, 3), (0, 0));
+ assert_eq!(rot_src(Rotation::ROTATE_0, 1, 2, 2, 3), (1, 2));
+ assert_eq!(rot_src(Rotation::ROTATE_180, 0, 0, 2, 3), (1, 2));
+ assert_eq!(rot_src(Rotation::ROTATE_180, 1, 2, 2, 3), (0, 0));
+ // 90deg: output dims are (sh,sw)=(3,2); (dx,dy) -> (dy, sh-1-dx).
+ assert_eq!(rot_src(Rotation::ROTATE_90, 0, 0, 2, 3), (0, 2));
+ assert_eq!(rot_src(Rotation::ROTATE_90, 2, 1, 2, 3), (1, 0));
+ // 270deg: (dx,dy) -> (sw-1-dy, dx).
+ assert_eq!(rot_src(Rotation::ROTATE_270, 0, 0, 2, 3), (1, 0));
+ assert_eq!(rot_src(Rotation::ROTATE_270, 2, 1, 2, 3), (0, 2));
+ // Reflect-X composes on top of the rotation (here identity): sx -> sw-1-sx.
+ assert_eq!(
+ rot_src(Rotation::ROTATE_0 | Rotation::REFLECT_X, 0, 0, 2, 3),
+ (1, 0)
+ );
+ }
+}
diff --git a/drivers/gpu/drm/vino/drm_sink/settings.rs b/drivers/gpu/drm/vino/drm_sink/settings.rs
new file mode 100644
index 000000000000..2142b5f20230
--- /dev/null
+++ b/drivers/gpu/drm/vino/drm_sink/settings.rs
@@ -0,0 +1,574 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! The runtime knobs a matched dock profile installs on the KMS device.
+//!
+//! The KMS side has one code path per operation and no per-dock branches; every difference
+//! between generations arrives here as data, is stored in an atomic, and is read back by the
+//! workers. Nothing in this module decides anything -- [`crate::profile`] does.
+
+use super::*;
+
+impl VinoDrmData {
+ /// Record this device's codec geometry; see [`Self::geometry`].
+ pub(crate) fn set_codec_geometry(
+ &self,
+ strip_blocks_x: usize,
+ interlaced: bool,
+ band_parity: bool,
+ connector_selector_shift: u8,
+ stream_id_mask: u8,
+ dock_buffers: u8,
+ coding: crate::video_arm::CodeTables,
+ steady_sub_bit: u8,
+ ) {
+ let narrow = coding == crate::video_arm::CodeTables::Narrow;
+ let packed = (strip_blocks_x as u32 & 0xff)
+ | ((interlaced as u32) << 8)
+ | ((band_parity as u32) << 9)
+ | ((narrow as u32) << 10)
+ | (((steady_sub_bit != 0) as u32) << 11)
+ | ((connector_selector_shift as u32) << 16)
+ | ((stream_id_mask as u32) << 20)
+ | ((dock_buffers as u32) << 28);
+ self.codec_geometry
+ .store(packed | 0x8000, core::sync::atomic::Ordering::Release);
+ }
+
+ /// This device's codec geometry, to be passed into every codec call made on its behalf.
+ ///
+ /// Stored packed because the DRM device allocation is pin-initialised before `probe` knows
+ /// which dock it matched. A device with no profile applied reads the Ridge layout.
+ pub(crate) fn geometry(&self) -> crate::video::haar::Geometry {
+ let p = self
+ .codec_geometry
+ .load(core::sync::atomic::Ordering::Acquire);
+ if p & 0x8000 == 0 {
+ return crate::video::haar::RIDGE_GEOMETRY;
+ }
+ crate::video::haar::Geometry::new(
+ ((p & 0xff) as usize).max(1),
+ p & (1 << 8) != 0,
+ p & (1 << 9) != 0,
+ ((p >> 16) & 0xf) as u8,
+ ((p >> 20) & 0xff) as u8,
+ ((p >> 28) & 0xf) as u8,
+ )
+ .with_coding(if p & (1 << 10) != 0 {
+ crate::video_arm::CodeTables::Narrow
+ } else {
+ crate::video_arm::CodeTables::Wide
+ })
+ .with_steady_sub_bit(if p & (1 << 11) != 0 { 0x20 } else { 0 })
+ }
+
+ /// Record how this dock delivers logical framebuffer updates to its ring buffers.
+ pub(crate) fn set_frame_delivery(&self, policy: crate::profile::FrameDelivery) {
+ let packed = u32::from(policy.keyframe_presentations.max(1))
+ | (u32::from(policy.delta_presentations.max(1)) << 8)
+ | (u32::from(policy.damage_frames.max(1)) << 16);
+ self.frame_delivery.store(packed, Ordering::Release);
+ }
+
+ /// Snapshot this dock's frame-delivery policy.
+ pub(crate) fn frame_delivery(&self) -> crate::profile::FrameDelivery {
+ let packed = self.frame_delivery.load(Ordering::Acquire);
+ crate::profile::FrameDelivery::new(
+ ((packed & 0xff) as u8).max(1),
+ (((packed >> 8) & 0xff) as u8).max(1),
+ (((packed >> 16) & 0xff) as u8).max(1),
+ )
+ }
+
+ /// Record whether timed presence probes may reset a bracket beside a live connector.
+ pub(crate) fn set_probe_bracket(&self, policy: crate::profile::ProbeBracket) {
+ self.probe_bracket.store(policy as u8, Ordering::Release);
+ }
+
+ /// This dock's bracket-reset policy.
+ pub(crate) fn probe_bracket(&self) -> crate::profile::ProbeBracket {
+ match self.probe_bracket.load(Ordering::Acquire) {
+ x if x == crate::profile::ProbeBracket::DeferWithActiveSibling as u8 => {
+ crate::profile::ProbeBracket::DeferWithActiveSibling
+ }
+ _ => crate::profile::ProbeBracket::Always,
+ }
+ }
+
+ /// Whether this dock can be driven at ten bits per channel; see [`DockProfile::hdr_capable`].
+ pub(crate) fn hdr_capable(&self) -> bool {
+ self.hdr_capable.load(Ordering::Acquire)
+ }
+
+ /// Whether this dock composites a cursor bitmap of its own; see [`DockProfile::hw_cursor`].
+ pub(crate) fn hw_cursor(&self) -> bool {
+ self.hw_cursor.load(Ordering::Acquire)
+ }
+
+ /// Record whether the dock's presence probe describes a connector; see
+ /// [`DockProfile::reports_presence`].
+ pub(crate) fn set_reports_presence(&self, on: bool) {
+ self.reports_presence.store(on, Ordering::Release);
+ }
+
+ /// Whether the dock's presence probe describes a connector; see
+ /// [`DockProfile::reports_presence`].
+ pub(crate) fn reports_presence(&self) -> bool {
+ self.reports_presence.load(Ordering::Acquire)
+ }
+
+ /// Record whether the connectors share one EDID handler; see
+ /// [`DockProfile::shared_edid_handler`].
+ pub(crate) fn set_shared_edid_handler(&self, on: bool) {
+ self.shared_edid_handler.store(on, Ordering::Release);
+ }
+
+ /// Whether the connectors share one EDID handler; see [`DockProfile::shared_edid_handler`].
+ pub(crate) fn shared_edid_handler(&self) -> bool {
+ self.shared_edid_handler.load(Ordering::Acquire)
+ }
+
+ /// Record whether a frame ending on a full packet is split; see
+ /// [`DockProfile::split_full_packet_frame`].
+ pub(crate) fn set_split_full_packet_frame(&self, on: bool) {
+ self.split_full_packet_frame.store(on, Ordering::Release);
+ }
+
+ /// Whether a frame ending on a full packet is split.
+ pub(crate) fn split_full_packet_frame(&self) -> bool {
+ self.split_full_packet_frame.load(Ordering::Acquire)
+ }
+
+ /// Whether `connector`'s committed framebuffer is 10 bits per channel.
+ ///
+ /// Read by the mode-set path, which must announce a depth to the dock that matches the one the
+ /// plane will actually send: the dock sizes its buffer from the pair and mis-sizes it if they
+ /// disagree.
+ pub(super) fn connector_is_ten_bit(&self, connector: usize) -> bool {
+ self.connector_wire_ten_bit(connector as u8)
+ }
+
+ /// Whether `connector`'s connector is being driven in PQ; see [`Self::set_connector_st2084`].
+ pub(super) fn connector_is_st2084(&self, connector: usize) -> bool {
+ self.head_st2084.load(Ordering::Acquire) & (1u32 << connector) != 0
+ }
+
+ /// Record the transfer function userspace has asked for on `connector`.
+ ///
+ /// Driven by the connector's `HDR_OUTPUT_METADATA` blob rather than by anything vino decides,
+ /// for the same reason [`Self::set_connector_depth`] follows the framebuffer's fourcc: the dock
+ /// must be told what the pixels actually are, and any state of our own could drift from them.
+ pub(super) fn set_connector_st2084(&self, connector: u8, on: bool) {
+ let bit = 1u32 << u32::from(connector);
+ if on {
+ self.head_st2084.fetch_or(bit, Ordering::Release);
+ } else {
+ self.head_st2084.fetch_and(!bit, Ordering::Release);
+ }
+ }
+
+ /// Record the link depth userspace asked this connector to carry.
+ ///
+ /// Four bits per connector is enough for every value `max bpc` takes, and keeping them in one
+ /// word means the scanout path reads the whole set with a single load.
+ pub(super) fn set_connector_max_bpc(&self, connector: u8, bpc: u32) {
+ let shift = u32::from(connector) * 4;
+ let field = bpc.min(15) << shift;
+ let mask = !(0xfu32 << shift);
+ let mut current = self.connector_max_bpc.load(Ordering::Acquire);
+ loop {
+ let next = (current & mask) | field;
+ match self.connector_max_bpc.compare_exchange_weak(
+ current,
+ next,
+ Ordering::AcqRel,
+ Ordering::Acquire,
+ ) {
+ Ok(_) => break,
+ Err(seen) => current = seen,
+ }
+ }
+ }
+
+ /// The sample depth of the framebuffer this connector is scanning out.
+ ///
+ /// This is how a pixel must be *decoded*; [`Self::geometry_for_connector`] says what the dock
+ /// is told to carry, and the two differ whenever userspace asks for a deeper link than the
+ /// surface it hands over.
+ pub(super) fn connector_buffer_depth(&self, connector: u8) -> crate::video::haar::Depth {
+ if self.connector_ten_bit.load(Ordering::Acquire) & (1u32 << u32::from(connector)) != 0 {
+ crate::video::haar::Depth::Ten
+ } else {
+ crate::video::haar::Depth::Eight
+ }
+ }
+
+ /// Whether this connector's link is driven at ten bits per channel.
+ ///
+ /// True when the framebuffer is already ten-bit, and when userspace asks for a ten-bit link
+ /// through `max bpc` on a dock that can carry one and is driving the connector in PQ. An
+ /// eight-bit surface over a ten-bit link is the ordinary case on every other driver.
+ ///
+ /// This is the decision, not the depth in force. Read it only where a mode set carries the
+ /// answer to the dock; everything else wants [`Self::connector_programmed_ten_bit`].
+ pub(super) fn connector_wire_ten_bit(&self, connector: u8) -> bool {
+ if self.connector_ten_bit.load(Ordering::Acquire) & (1u32 << u32::from(connector)) != 0 {
+ return true;
+ }
+ if self.connector_deny_ten_bit.load(Ordering::Acquire) & (1u32 << u32::from(connector)) != 0
+ {
+ return false;
+ }
+ let shift = u32::from(connector) * 4;
+ let requested = (self.connector_max_bpc.load(Ordering::Acquire) >> shift) & 0xf;
+ self.hdr_capable() && requested >= 10 && self.connector_is_st2084(connector as usize)
+ }
+
+ /// Record whether this connector's ten-bit link fits the dock's shared bandwidth.
+ pub(super) fn set_connector_ten_bit_denied(&self, connector: u8, denied: bool) {
+ let bit = 1u32 << u32::from(connector);
+ if denied {
+ self.connector_deny_ten_bit.fetch_or(bit, Ordering::Release);
+ } else {
+ self.connector_deny_ten_bit
+ .fetch_and(!bit, Ordering::Release);
+ }
+ }
+
+ /// Whether any connector other than `connector` is programmed at ten bits per channel.
+ ///
+ /// The dock's bandwidth is shared, so a commit is priced at the dock's deepest connector.
+ pub(super) fn other_connector_programmed_ten_bit(&self, connector: u8) -> bool {
+ (0..self.connector_count())
+ .any(|c| c as u8 != connector && self.connector_programmed_ten_bit(c as u8))
+ }
+
+ /// Whether the mode this connector was last programmed with drives ten bits per channel.
+ ///
+ /// The depth the dock is decoding at, so the only one the codec may encode at and the only one
+ /// the set-mode and the decoder configuration may state. Deliberately not
+ /// [`Self::connector_wire_ten_bit`], which moves between mode sets.
+ pub(super) fn connector_programmed_ten_bit(&self, connector: u8) -> bool {
+ self.last_timing
+ .lock()
+ .get(connector as usize)
+ .copied()
+ .flatten()
+ .is_some_and(|t| t.ten_bit)
+ }
+
+ /// This device's codec geometry at one connector's programmed sample depth.
+ ///
+ /// Every path that touches pixels wants this rather than [`Self::geometry`]: the depth decides
+ /// the entropy coder's escape ceiling, and getting it wrong desynchronises the dock's decoder
+ /// rather than merely degrading the picture. See [`crate::video::haar::Depth`].
+ pub(super) fn geometry_for_connector(&self, connector: u8) -> crate::video::haar::Geometry {
+ let ten = self.connector_programmed_ten_bit(connector);
+ self.geometry().with_depth(if ten {
+ crate::video::haar::Depth::Ten
+ } else {
+ crate::video::haar::Depth::Eight
+ })
+ }
+
+ /// Record the sample depth of the framebuffer a connector is scanning out.
+ ///
+ /// Driven by the committed framebuffer's fourcc rather than by any state of our own, so it
+ /// cannot drift from the pixels actually in hand. A format the codec does not know leaves the
+ /// connector where it was; `atomic_check` is what rejects those, and a plane list that only
+ /// offers `XRGB8888` means this never sees one.
+ pub(super) fn set_connector_depth(&self, connector: u8, depth: crate::video::haar::Depth) {
+ let bit = 1u32 << u32::from(connector);
+ let previous = match depth {
+ crate::video::haar::Depth::Ten => {
+ self.connector_ten_bit.fetch_or(bit, Ordering::Release)
+ }
+ crate::video::haar::Depth::Eight => {
+ self.connector_ten_bit.fetch_and(!bit, Ordering::Release)
+ }
+ };
+ // Report the change, because this is the only place the sample depth is decided and
+ // nothing else on the wire says what was chosen. A connector that advertises ten bits and
+ // is driven in ST2084 still goes out at eight if the compositor never hands over a
+ // ten-bit framebuffer, and that difference is otherwise visible only on the panel.
+ let was_ten = previous & bit != 0;
+ let now_ten = matches!(depth, crate::video::haar::Depth::Ten);
+ if was_ten != now_ten {
+ let socket = connector + 1;
+ let bits = if now_ten { 10 } else { 8 };
+ vino_debug!("vino: socket {socket} scanout depth is now {bits} bits per channel\n");
+ }
+ }
+
+ /// Record the mode-programming and blanking behaviour this dock wants.
+ pub(crate) fn set_mode_behaviour(&self, profile: &'static crate::profile::DockProfile) {
+ self.dock_wide_modeset
+ .store(profile.protocol.dock_wide_modeset, Ordering::Release);
+ self.clear_mode_before_set
+ .store(profile.protocol.clear_mode_before_set, Ordering::Release);
+ self.video_keepalive
+ .store(profile.protocol.video_keepalive, Ordering::Release);
+ self.blank_markers_held.store(
+ matches!(
+ profile.protocol.blank_bracket,
+ crate::profile::BlankBracket::MarkersHeld
+ ),
+ Ordering::Release,
+ );
+ }
+
+ /// Whether a connector must keep being fed while its content is unchanged.
+ pub(crate) fn video_keepalive(&self) -> bool {
+ self.video_keepalive.load(Ordering::Acquire)
+ }
+
+ /// Whether programming any connector reconfigures the whole dock.
+ pub(crate) fn dock_wide_modeset(&self) -> bool {
+ self.dock_wide_modeset.load(Ordering::Acquire)
+ }
+
+ /// Whether a connector's pipe is torn down before a timing is programmed onto it.
+ pub(crate) fn clear_mode_before_set(&self) -> bool {
+ self.clear_mode_before_set.load(Ordering::Acquire)
+ }
+
+ /// How a connector blanks; see [`crate::profile::BlankBracket`].
+ pub(crate) fn blank_bracket(&self) -> crate::profile::BlankBracket {
+ if self.blank_markers_held.load(Ordering::Acquire) {
+ crate::profile::BlankBracket::MarkersHeld
+ } else {
+ crate::profile::BlankBracket::BlackThenClose
+ }
+ }
+
+ /// Record how this dock states its framebuffer allocation in a set-mode.
+ pub(crate) fn set_allocation(&self, allocation: &'static crate::profile::Allocation) {
+ let _ = self.allocation.populate(allocation);
+ }
+
+ /// How this dock states its framebuffer allocation; Ridge's device override until probe has
+ /// matched a profile, as with every other value published there.
+ pub(crate) fn allocation(&self) -> &'static crate::profile::Allocation {
+ self.allocation
+ .as_ref()
+ .copied()
+ .unwrap_or(&crate::profile::PROFILE_RIDGE.protocol.allocation)
+ }
+
+ /// Record whether this dock opens a stream with the ARM burst; see `DockProfile::arm_burst`.
+ pub(crate) fn set_arm_burst(&self, on: bool) {
+ self.arm_burst.store(on, Ordering::Release);
+ }
+
+ /// Whether the first frame after a mode set carries the cold ARM burst.
+ pub(super) fn uses_arm_burst(&self) -> bool {
+ self.arm_burst.load(Ordering::Acquire)
+ }
+
+ /// Length of a continuous-presentation window, for this dock.
+ ///
+ /// The activation carrier and the blank presentation both work by presenting one encoded frame
+ /// back to back for a fixed wall-clock window, with no control transaction in between. That
+ /// trains a downstream link on a dock with a video pipe of its own. On a dock that carries
+ /// video on the control pipe it instead holds the endpoint for the whole window, and the dock
+ /// is silenced at exactly the moment the mode set needs it to answer. Such a dock gets a
+ /// single presentation instead, which is what `submit_prompt_training` does at zero.
+ pub(super) fn carrier_ms(&self, base: i64) -> i64 {
+ if self.video_on_ctrl_pipe() {
+ 0
+ } else {
+ base
+ }
+ }
+
+ /// How many carrier frames a connector presents before its first content frame; see
+ /// `DockProfile::carrier_frames`.
+ pub(super) fn carrier_presentations(&self) -> u32 {
+ self.carrier_frames
+ .load(core::sync::atomic::Ordering::Relaxed)
+ }
+
+ /// Record how this dock's video stream describes itself.
+ ///
+ /// The three values travel together because they are read together, by the one builder that
+ /// states a stream's mode and decoder tables.
+ pub(crate) fn set_video_stream_desc(
+ &self,
+ layout_word: u16,
+ marker_kind: u8,
+ tables: crate::video_arm::CodeTables,
+ ) {
+ let narrow = matches!(tables, crate::video_arm::CodeTables::Narrow);
+ let packed =
+ u32::from(layout_word) | (u32::from(marker_kind) << 16) | ((narrow as u32) << 24);
+ self.video_stream_desc.store(packed, Ordering::Release);
+ }
+
+ /// The word repeated beside the surface size in this dock's stream mode header.
+ pub(crate) fn layout_word(&self) -> u16 {
+ self.video_stream_desc.load(Ordering::Acquire) as u16
+ }
+
+ /// The byte naming this dock in a sealed stream's opening marker.
+ pub(crate) fn stream_marker_kind(&self) -> u8 {
+ (self.video_stream_desc.load(Ordering::Acquire) >> 16) as u8
+ }
+
+ /// Which form of decoder code tables this dock's stream configuration states.
+ pub(crate) fn code_tables(&self) -> crate::video_arm::CodeTables {
+ if self.video_stream_desc.load(Ordering::Acquire) & (1 << 24) != 0 {
+ crate::video_arm::CodeTables::Narrow
+ } else {
+ crate::video_arm::CodeTables::Wide
+ }
+ }
+
+ /// Record this dock's minimum frame interval; see `DockProfile::frame_period_ms`.
+ pub(crate) fn set_frame_period_ms(&self, ms: i64) {
+ let ms = if ms <= 0 { FRAME_PERIOD_MS } else { ms };
+ self.frame_period_us
+ .store(ms * 1000, core::sync::atomic::Ordering::Relaxed);
+ }
+
+ /// Record how many carrier frames open a stream; see `DockProfile::carrier_frames`.
+ pub(crate) fn set_carrier_frames(&self, frames: u32) {
+ self.carrier_frames
+ .store(frames.max(1), core::sync::atomic::Ordering::Relaxed);
+ }
+
+ /// Record this dock's keepalive status interval; see `DockProfile::status_period_ms`.
+ pub(crate) fn set_status_period_ms(&self, ms: i64) {
+ let ms = if ms <= 0 { STATUS_PERIOD_MS } else { ms };
+ self.status_period_ms
+ .store(ms, core::sync::atomic::Ordering::Relaxed);
+ }
+
+ /// This dock's interval between keepalive status queries, in milliseconds.
+ pub(crate) fn status_period_ms(&self) -> i64 {
+ self.status_period_ms
+ .load(core::sync::atomic::Ordering::Relaxed)
+ }
+
+ /// This dock's minimum interval between frames on one connector, in milliseconds.
+ pub(crate) fn frame_period_ms(&self) -> i64 {
+ self.frame_period_us() / 1000
+ }
+
+ /// This dock's minimum interval between frames on one connector, in microseconds.
+ pub(super) fn frame_period_us(&self) -> i64 {
+ self.frame_period_us
+ .load(core::sync::atomic::Ordering::Relaxed)
+ }
+
+ /// Record how much of this dock's endpoint may be occupied; see `DockProfile::stream_pacing`.
+ pub(crate) fn set_stream_pacing(&self, pacing: crate::profile::StreamPacing) {
+ self.stream_budget_bps.store(
+ pacing.bytes_per_sec.max(1),
+ core::sync::atomic::Ordering::Relaxed,
+ );
+ self.stream_burst_bytes.store(
+ pacing.burst_bytes.max(1),
+ core::sync::atomic::Ordering::Relaxed,
+ );
+ let mut credit = self.stream_credit.lock();
+ *credit = StreamCredit::new();
+ }
+
+ /// How long a frame must wait for this dock's sustained budget, or `None` to send it now.
+ ///
+ /// Tops the ledger up for the time since it was last read, so an idle dock is always in credit
+ /// and this costs a busy dock one spinlock per frame.
+ pub(super) fn stream_budget_wait_us(&self) -> Option<i64> {
+ let bps = self
+ .stream_budget_bps
+ .load(core::sync::atomic::Ordering::Relaxed);
+ if bps == u32::MAX {
+ return None;
+ }
+ let now = Instant::<Monotonic>::now();
+ let mut credit = self.stream_credit.lock();
+ let elapsed_us = credit
+ .topped_up
+ .map_or(1_000_000, |last| (now - last).as_micros_ceil());
+ credit.topped_up = Some(now);
+ // Cap the ledger at the burst allowance, not at a second of throughput. A dock idle for a
+ // minute must not bank a minute of bytes; nor may it bank a whole second's worth, which is
+ // more than this dock survives in one go.
+ let ceiling = i64::from(
+ self.stream_burst_bytes
+ .load(core::sync::atomic::Ordering::Relaxed),
+ );
+ credit.bytes = credit
+ .bytes
+ .saturating_add(stream_credit_accrued(bps, elapsed_us))
+ .min(ceiling);
+ stream_credit_wait_us(bps, credit.bytes)
+ }
+
+ /// Charge a frame that reached the dock against the sustained budget.
+ pub(super) fn charge_stream_budget(&self, bytes: usize) {
+ if self
+ .stream_budget_bps
+ .load(core::sync::atomic::Ordering::Relaxed)
+ == u32::MAX
+ {
+ return;
+ }
+ let mut credit = self.stream_credit.lock();
+ credit.bytes = credit.bytes.saturating_sub(bytes as i64);
+ }
+
+ /// Record the state that takes this dock's sinks down; see `DockProfile::sink_down_state`.
+ /// Record the `0x2e` state re-sent mid-bracket; see `DockProfile::bracket_reopen_state`.
+ pub(crate) fn set_post_mode_sink_states(&self, states: [u8; 2]) {
+ let packed = u16::from(states[0]) | (u16::from(states[1]) << 8);
+ self.post_mode_sink_states
+ .store(packed, core::sync::atomic::Ordering::Release);
+ }
+
+ /// Record the state this dock wants before a mode set; see
+ /// `DockProfile::pre_mode_sink_state`.
+ pub(crate) fn set_pre_mode_sink_state(&self, state: Option<u8>) {
+ self.pre_mode_sink_state.store(
+ state.map_or(u16::MAX, u16::from),
+ core::sync::atomic::Ordering::Release,
+ );
+ }
+
+ pub(crate) fn pre_mode_sink_state(&self) -> Option<u8> {
+ match self
+ .pre_mode_sink_state
+ .load(core::sync::atomic::Ordering::Acquire)
+ {
+ u16::MAX => None,
+ state => Some(state as u8),
+ }
+ }
+
+ pub(super) fn post_mode_sink_state(&self, index: usize) -> u8 {
+ let packed = self
+ .post_mode_sink_states
+ .load(core::sync::atomic::Ordering::Acquire);
+ (packed >> (8 * index)) as u8
+ }
+
+ pub(crate) fn set_sink_down_state(&self, state: u8) {
+ self.sink_down_state
+ .store(state, core::sync::atomic::Ordering::Release);
+ }
+
+ /// The `0x16/0x2e` state that takes a downstream sink down on this dock.
+ pub(crate) fn sink_down_state(&self) -> u8 {
+ self.sink_down_state
+ .load(core::sync::atomic::Ordering::Acquire)
+ }
+
+ /// Record whether video shares the control pipe; see `DockProfile::video_on_ctrl_pipe`.
+ pub(crate) fn set_video_on_ctrl_pipe(&self, on: bool) {
+ self.video_on_ctrl_pipe.store(on, Ordering::Release);
+ }
+
+ /// Whether video records travel on the control bulk-OUT pipe.
+ pub(crate) fn video_on_ctrl_pipe(&self) -> bool {
+ self.video_on_ctrl_pipe.load(Ordering::Acquire)
+ }
+}
diff --git a/drivers/gpu/drm/vino/drm_sink/worker.rs b/drivers/gpu/drm/vino/drm_sink/worker.rs
new file mode 100644
index 000000000000..f06f469d1715
--- /dev/null
+++ b/drivers/gpu/drm/vino/drm_sink/worker.rs
@@ -0,0 +1,571 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! The workqueue items that carry out what the atomic callbacks asked for.
+//!
+//! One item reconciles KMS state, one per connector drives scanout, and one watches the
+//! control plane for silence. They are the only contexts in the driver allowed to block on
+//! USB.
+
+use super::*;
+
+impl_has_delayed_work! {
+ impl HasDelayedWork<VinoDrmDevice> for VinoDrmData { self.cmd_work }
+ impl HasDelayedWork<VinoDrmDevice, 5> for VinoDrmData { self.cp_watchdog }
+}
+
+impl WorkItem<5> for VinoDrmData {
+ type Pointer = ARef<VinoDrmDevice>;
+ fn run(this: ARef<VinoDrmDevice>) {
+ run_cp_watchdog(this);
+ }
+}
+
+/// Enforce the control session's silence deadline from off the control path.
+///
+/// Checking the deadline when a caller arrives to start a transaction is too late by
+/// construction: the thread that would arrive is the keepalive, and the keepalive is the thread
+/// that gets stuck -- a wedge then runs to 15 s against a 5 s limit, and ends only because the
+/// dock re-enumerates. This runs on the system queue, so it is scheduled whatever vino's own
+/// queues are doing, and it touches nothing a stuck transfer can be holding.
+pub(super) fn run_cp_watchdog(this: ARef<VinoDrmDevice>) {
+ let data: &VinoDrmData = &this;
+ if data.shutting_down.load(Ordering::Acquire) {
+ return;
+ }
+ // Vino must not call a dock silent while vino has chosen not to speak to it. Navarro's
+ // setup-to-first-mode-set hold runs as long as the deadline itself, so holding the deadline
+ // off here -- rather than at each of the two places the hold ends -- is what stops a healthy
+ // cold bring-up from abandoning its own session at the boundary.
+ if data.initial_modeset_quiet() {
+ data.note_cp_reply();
+ } else if data.cp_link_alive() {
+ let silent_ms = data.cp_silent_for_ms();
+ if silent_ms >= data.cp_silence_limit_ms() {
+ data.abandon_cp_session(silent_ms);
+ data.drop_connectors_with_session(&this);
+ data.reset_after_wedge();
+ }
+ }
+ data.start_cp_watchdog(&this);
+}
+
+impl_has_work! {
+ impl HasWork<VinoDrmDevice, 1> for VinoDrmData { self.scanout_work_h0 }
+ impl HasWork<VinoDrmDevice, 2> for VinoDrmData { self.scanout_work_h1 }
+ impl HasWork<VinoDrmDevice, 3> for VinoDrmData { self.scanout_work_h2 }
+ impl HasWork<VinoDrmDevice, 4> for VinoDrmData { self.scanout_work_h3 }
+}
+
+/// One scanout work item exists per connector, and its work ID is a const generic. Keep this
+/// assertion adjacent to the explicit fields/arms below: adding another connector without all
+/// three would silently leave its frames in `pending_scanout`.
+const _: () = assert!(
+ MAX_CONNECTORS == 4,
+ "add a scanout_work_hN work item per connector (see VinoDrmData::enqueue_scanout)"
+);
+
+impl WorkItem<1> for VinoDrmData {
+ type Pointer = ARef<VinoDrmDevice>;
+ fn run(this: ARef<VinoDrmDevice>) {
+ run_scanout_worker(this, 0);
+ }
+}
+
+impl WorkItem<2> for VinoDrmData {
+ type Pointer = ARef<VinoDrmDevice>;
+ fn run(this: ARef<VinoDrmDevice>) {
+ run_scanout_worker(this, 1);
+ }
+}
+
+impl WorkItem<3> for VinoDrmData {
+ type Pointer = ARef<VinoDrmDevice>;
+ fn run(this: ARef<VinoDrmDevice>) {
+ run_scanout_worker(this, 2);
+ }
+}
+
+impl WorkItem<4> for VinoDrmData {
+ type Pointer = ARef<VinoDrmDevice>;
+ fn run(this: ARef<VinoDrmDevice>) {
+ run_scanout_worker(this, 3);
+ }
+}
+
+/// One connector's deferred scanout loop: pick this connector's due frame, encode it, transmit it,
+/// repeat until the connector has nothing left to do. Both connectors run concurrently on the
+/// per-device scanout queue.
+///
+/// A queued `ModeSet` must reach the dock before video for that connector, so this worker returns
+/// while a stream command is pending or executing. `cmd_work` re-enqueues the scanout workers when
+/// the command batch completes, and the pending framebuffer remains in its coalescing slot.
+///
+/// Two conditions, and both are needed: a stream operation in `pending_kms` (not yet drained), and
+/// [`VinoDrmData::cmd_busy`] (drained and executing -- the window in which `pending_kms` is
+/// misleadingly empty). The `video_inflight` store must be published *before* reading `cmd_busy`,
+/// and both use `SeqCst`, so this and `wait_for_video_idle` cannot both conclude the other is idle.
+pub(super) fn run_scanout_worker(this: ARef<VinoDrmDevice>, connector: usize) {
+ use core::sync::atomic::Ordering::SeqCst;
+ let data: &VinoDrmData = &this;
+ // Plane updates can be accepted once CP engages, before the later platform readiness interval
+ // finishes. Leave their coalesced frames untouched until bring-up publishes activation safety;
+ // the publisher wakes every scanout worker after opening this gate.
+ if !data.kms_activation_ready() {
+ return;
+ }
+ // As in `cmd_work`: once the I/O window refuses a token, unplug has begun and there is no USB
+ // left to do. `drm_dev_enter()` holds the parent interface Bound for the duration.
+ let Ok(link) = crate::UsbLink::open(&data.io, data.endpoints) else {
+ return;
+ };
+ let dev = &link;
+ loop {
+ if data.shutting_down.load(Ordering::Acquire) {
+ return;
+ }
+ // Claim the connector's video endpoint first, then look for a reason not to use it.
+ data.video_inflight[connector].store(true, SeqCst);
+ let blocked = data.cmd_busy.load(SeqCst) || data.pending_kms.lock().has_stream();
+ if blocked {
+ data.video_inflight[connector].store(false, SeqCst);
+ return;
+ }
+ let (frame, cadence_wait_us) = data.select_scanout(connector);
+ if let Some(frame) = frame {
+ run_pending_scanout(dev, data, frame);
+ data.video_inflight[connector].store(false, SeqCst);
+ continue;
+ }
+ data.video_inflight[connector].store(false, SeqCst);
+ if let Some(us) = cadence_wait_us {
+ // Bound the sleep. The settle-repaint arm can ask for its full deadline
+ // ([`SETTLE_REPAINT_MS`], 1.2 s); sleeping that long inside the work item makes the
+ // connector unreachable, because a flip arriving meanwhile finds the item already
+ // running and its enqueue is dropped. Waking at the cadence window instead costs a few
+ // extra wakeups while idle and keeps the connector responsive to real frames.
+ let us = us.min(data.frame_period_us());
+ fsleep(Delta::from_micros(us));
+ continue;
+ }
+ // Re-check before exiting. A frame published between `select_scanout` above and this point
+ // finds the work item still running, so its `enqueue_scanout` is dropped and the frame
+ // waits for some *later* flip to enqueue successfully -- a lost wakeup that showed up as
+ // multi-second stalls on whichever connector lost the race. The condition mirrors
+ // `select_scanout`'s own guard so a connector with no mode-set cannot spin here.
+ if data.modeset_requested[connector].load(Ordering::Acquire) != 0
+ && data.pending_scanout.lock()[connector].is_some()
+ {
+ continue;
+ }
+ return;
+ }
+}
+
+impl WorkItem for VinoDrmData {
+ type Pointer = ARef<VinoDrmDevice>;
+
+ /// Reconcile the latest desired stream and cursor state from the atomic callbacks.
+ fn run(this: ARef<VinoDrmDevice>) {
+ let data: &VinoDrmData = &this;
+ // Registration deliberately precedes the blocking CP/EDID setup, so userspace can publish a
+ // complete atomic mode-set batch before the runtime session exists. Do not drain that batch
+ // yet: attempting its dock-wide activation returns ENODEV, then the ordinary command loop
+ // can split it into per-connector activations as setup becomes ready between those two
+ // attempts. A readiness deferral is not a failed transport operation, so it neither
+ // consumes `kms_retries` nor rewrites any pending slot; newer atomic state may continue to
+ // coalesce there. The common worker gate applies to every dock generation.
+ if !data.kms_activation_ready() {
+ return;
+ }
+ // `drm_dev_enter()` holds the parent USB interface in Bound typestate until this worker
+ // finishes. If unplug has begun, discard queued transport work without touching USB.
+ // The I/O window is closed by `disconnect()` before it returns, so once it refuses a token
+ // there is no USB left to do: discard the queued transport work.
+ let Ok(link) = crate::UsbLink::open(&data.io, data.endpoints) else {
+ return;
+ };
+ let dev = &link;
+ loop {
+ if data.shutting_down.load(Ordering::Acquire) {
+ return;
+ }
+ // A dual-connector atomic commit runs `atomic_enable` once per connector, and each of
+ // those queues its own `ModeSet` and wakes this worker -- microseconds apart, but far
+ // less than it takes to get scheduled. Taking the first one alone turns one dock-wide
+ // wake into two single-connector activations and skips the cold choreography that arms
+ // the video endpoints, so wait, briefly and boundedly, for the siblings the compositor
+ // has already published a timing for.
+ {
+ let started = Instant::<Monotonic>::now();
+ let present = data.connectors_present.load(Ordering::Acquire);
+ loop {
+ let queued = data.pending_kms.lock().connectors.iter().enumerate().fold(
+ 0u32,
+ |acc, (h, p)| {
+ if matches!(p.stream, Some(KmsCmd::ModeSet { .. })) {
+ acc | (1u32 << h)
+ } else {
+ acc
+ }
+ },
+ );
+ // Nothing to wait for until at least one mode set has landed, and nothing
+ // left to wait for once every connector with a monitor is either already active
+ // or represented in this batch.
+ let outstanding = (0..MAX_CONNECTORS).any(|h| {
+ present & (1u32 << h) != 0
+ && queued & (1u32 << h) == 0
+ && data.modeset_active[h].load(Ordering::Acquire) == 0
+ });
+ if queued == 0
+ || !outstanding
+ || (Instant::<Monotonic>::now() - started).as_millis()
+ >= MODESET_BATCH_SETTLE_MS
+ {
+ break;
+ }
+ fsleep(Delta::from_millis(1));
+ }
+ }
+ let pending = core::mem::replace(&mut *data.pending_kms.lock(), PendingKms::new());
+ // A cold dual-connector atomic commit is one dock-wide wake: both mode-sets precede
+ // either connector's video. Detect that shape before consuming the owned state.
+ let mut dual_timings: [Option<crate::cp::Timing>; MAX_CONNECTORS] =
+ [None; MAX_CONNECTORS];
+ let mut cmd_connectors = 0u32;
+ for connector in &pending.connectors {
+ if let Some(KmsCmd::ModeSet {
+ connector: cmd_head,
+ timing,
+ }) = &connector.stream
+ {
+ let connector_index = *cmd_head as usize;
+ if connector_index < MAX_CONNECTORS {
+ cmd_connectors |= 1u32 << connector_index;
+ if data.modeset_active[connector_index].load(Ordering::Acquire) == 0
+ && data.modeset_requested[connector_index].load(Ordering::Acquire)
+ == timing_key(timing)
+ {
+ dual_timings[connector_index] = Some(*timing);
+ }
+ }
+ }
+ }
+ // A dock that comes up as one transaction over every connector it has needs a timing
+ // for each of them, and the compositor only describes the sockets it can see. A socket
+ // with nothing plugged into it still has to be configured -- the scanout path already
+ // declines to paint a connector with no EDID -- so let it join at its sibling's mode.
+ //
+ // The generation has to be published with the timing. A connector whose requested mode
+ // is zero is a connector the activation waits on and never gets, so it defers on every
+ // commit for as long as the dock is up, and that retry churn is what takes the shared
+ // pipe down. Publishing it is also what makes this happen once: the connector then has
+ // a request of its own and no longer looks unasked-for.
+ if data.video_on_ctrl_pipe() && dual_timings.iter().flatten().count() == 1 {
+ let sibling = dual_timings.iter().flatten().copied().next();
+ if let Some(timing) = sibling {
+ for connector in 0..data.connector_count().min(MAX_CONNECTORS) {
+ if dual_timings[connector].is_some()
+ || data.modeset_requested[connector].load(Ordering::Acquire) != 0
+ || data.modeset_active[connector].load(Ordering::Acquire) != 0
+ {
+ continue;
+ }
+ data.last_timing.lock()[connector] = Some(timing);
+ data.modeset_requested[connector]
+ .store(timing_key(&timing), Ordering::Release);
+ dual_timings[connector] = Some(timing);
+ cmd_connectors |= 1u32 << connector;
+ vino_debug!(
+ "vino: socket {} has no monitor and is configured at its sibling's mode\n",
+ connector + 1
+ );
+ }
+ }
+ }
+ // Exclude the scanout workers for exactly as long as this batch can touch a video
+ // endpoint. `activate_dual_wake` and the `ModeSet` arm both run
+ // `submit_prompt_training`, which writes the activation carrier to the connector's
+ // endpoint; a concurrent scanout frame there would interleave records on the wire and
+ // would have its `video_q` slot double-opened. Cursor-only batches deliberately skip
+ // this: they never touch video, and a mouse in motion produces a continuous stream of
+ // them. `Blank` writes the connector's video endpoint for the same reason `ModeSet`
+ // does, so it needs the same exclusion against the scanout workers -- otherwise a frame
+ // already in flight interleaves its records with the blanking frames on the wire.
+ let has_modeset = pending.has_stream();
+ // On the DL7400 a mode set is dock-wide, not per connector. Configuring one connector
+ // while any other is lit makes the dock re-enumerate about 100 ms after the next video
+ // write, on every shape of the change: 120 -> 165, 165 -> 120 and 120 -> 60 on a live
+ // connector, waking a second connector a second after the first, and reconfiguring a
+ // connector whose sibling has been lit and idle for minutes. The same changes with the
+ // sibling disabled are clean, and so is the simultaneous `activate_dual_wake` path. DLM
+ // behaves the same way: it logs `[Profile change] Recreating device` and re-runs a
+ // bring-up-shaped burst rather than reconfiguring one connector in place.
+ //
+ // So fold every already-active connector into this batch. Zeroing its mode generation
+ // makes `activate_dual_wake` treat it as a fresh wake, and the whole dock is then taken
+ // through the one choreography the hardware accepts. The cost is that the sibling
+ // blinks through a mode change on its neighbour; the alternative is a dock reset and
+ // tens of seconds of dark panels on both. `cmd_connectors` rather than `has_modeset`: a
+ // `Blank`-only batch also counts as a stream command, and it must not drag every lit
+ // connector through a re-activation. Folding the lit connectors in makes
+ // `activate_dual_wake` name every connector at once, which is the only shape of mode
+ // set this dock accepts while more than one connector is lit. This replays the cold
+ // choreography -- a dock-wide sink reset and pipe clears -- on a dock that is already
+ // driving its sinks. That is the cost of the only mode set this dock will take.
+ if cmd_connectors != 0 && data.dock_wide_modeset() {
+ // Gather the whole dock's desired state first, and only commit to it if at least
+ // two connectors end up in it. Below two there is nothing for the dual path to do
+ // and the per-connector schedule is the proven one, so nothing is disturbed.
+ let mut fold: [Option<crate::cp::Timing>; MAX_CONNECTORS] = [None; MAX_CONNECTORS];
+ for connector in 0..MAX_CONNECTORS {
+ // A connector this batch is already mode-setting: take the requested timing,
+ // even if the connector is currently lit. A live reconfigure is exactly the
+ // case that must not go down the per-connector path.
+ if cmd_connectors & (1u32 << connector) != 0 {
+ if let Some(timing) = data.last_timing.lock()[connector] {
+ if data.modeset_requested[connector].load(Ordering::Acquire)
+ == timing_key(&timing)
+ {
+ fold[connector] = Some(timing);
+ }
+ }
+ continue;
+ }
+ // A connector this batch does not name, but which is lit and still wants the
+ // mode it is showing. One whose request has already moved on has its own
+ // `ModeSet` queued behind this batch and is left to it.
+ let active = data.modeset_active[connector].load(Ordering::Acquire);
+ if active != 0
+ && data.modeset_requested[connector].load(Ordering::Acquire) == active
+ {
+ fold[connector] = data.last_timing.lock()[connector];
+ }
+ }
+ if fold.iter().flatten().count() >= 2 {
+ for connector in 0..MAX_CONNECTORS {
+ let socket = connector + 1;
+ let Some(timing) = fold[connector] else {
+ continue;
+ };
+ // `activate_dual_wake` only accepts a connector whose generation is zero; a
+ // dock-wide transaction re-establishes every connector from scratch, so say
+ // so.
+ data.modeset_active[connector].store(0, Ordering::Release);
+ dual_timings[connector] = Some(timing);
+ cmd_connectors |= 1u32 << connector;
+ vino_debug!(
+ "vino: socket {socket} joins a dock-wide mode set ({}x{}@{})\n",
+ timing.hactive,
+ timing.vactive,
+ timing.refresh_hz
+ );
+ }
+ }
+ }
+ if has_modeset {
+ data.cmd_busy
+ .store(true, core::sync::atomic::Ordering::SeqCst);
+ data.wait_for_video_idle();
+ }
+ // Both dock-wide schedules need two connectors coming up together; below that the
+ // per-connector path is the proven one. Which schedule applies is a property of the
+ // dock: the Ridge and DL7400 cold timeline consists of operations a dock carrying video
+ // on its control pipe does not take, and that dock has its own measured choreography in
+ // `ELLA_DOCK_WIDE`. Driving either one from the other's table fails every pass.
+ let both_connectors = dual_timings.iter().flatten().count() >= 2;
+ let dual_wake = both_connectors && !data.video_on_ctrl_pipe();
+ let dock_wide = both_connectors && data.video_on_ctrl_pipe();
+ if has_modeset {
+ vino_debug!(
+ "vino: KMS batch -- stream cmds {}, dual timings {}, dual_wake {}, requested [{} {} {} {}], active [{} {} {} {}]\n",
+ (0..MAX_CONNECTORS).filter(|&h| cmd_connectors & (1u32 << h) != 0).count(),
+ dual_timings.iter().flatten().count(),
+ dual_wake || dock_wide,
+ data.modeset_requested[0].load(Ordering::Acquire),
+ data.modeset_requested[1].load(Ordering::Acquire),
+ data.modeset_requested[2].load(Ordering::Acquire),
+ data.modeset_requested[3].load(Ordering::Acquire),
+ data.modeset_active[0].load(Ordering::Acquire),
+ data.modeset_active[1].load(Ordering::Acquire),
+ data.modeset_active[2].load(Ordering::Acquire),
+ data.modeset_active[3].load(Ordering::Acquire),
+ );
+ }
+ let multihead_attempted = dock_wide || dual_wake;
+ let dual_complete = if dock_wide {
+ match data.activate_dock_wide(dev, ELLA_DOCK_WIDE, dual_timings) {
+ Ok(done) => done,
+ Err(e) => {
+ pr_warn!("vino: dock-wide activation failed ({e:?})\n");
+ false
+ }
+ }
+ } else {
+ dual_wake
+ && match data.activate_dual_wake(dev, dual_timings) {
+ Ok(done) => done,
+ Err(e) => {
+ pr_warn!("vino: dual-connector activation failed ({e:?})\n");
+ false
+ }
+ }
+ };
+ if multihead_attempted && !dual_complete {
+ // A multihead activation is one indivisible transport transaction. In particular,
+ // never let a failed Ella cold table fall through to the ordinary per-connector
+ // loop: connector 0 can then light just as setup becomes ready and force connector
+ // 1 down the live runtime table, so the cold two-connector choreography never
+ // lands. Restore the entire owned batch; newer producer state already occupying a
+ // slot still wins.
+ if has_modeset {
+ data.cmd_busy
+ .store(false, core::sync::atomic::Ordering::SeqCst);
+ }
+ let mut retry_pending = data.pending_kms.lock();
+ retry_pending.retry_batch(pending);
+ let attempts = data.kms_retries.fetch_add(1, Ordering::Relaxed) + 1;
+ if attempts >= KMS_RETRY_LIMIT {
+ pr_warn!(
+ "vino: dropping atomic multihead KMS batch after {} deferrals; the link is not coming back on its own\n",
+ KMS_RETRY_LIMIT
+ );
+ retry_pending.clear();
+ data.kms_retries.store(0, Ordering::Relaxed);
+ return;
+ }
+ drop(retry_pending);
+ vino_debug!("vino: atomic multihead KMS batch deferred\n");
+ if !data.shutting_down.load(Ordering::Acquire) {
+ let delay = kernel::time::msecs_to_jiffies(KMS_RETRY_MS);
+ let _ = workqueue::system().enqueue_delayed::<_, 0>(ARef::from(&*this), delay);
+ }
+ return;
+ }
+ // Heads whose mode this batch actually re-programmed, and whose dock-side cursor is
+ // therefore gone. See `rearm_cursor`.
+ let mut relit = if dual_complete { cmd_connectors } else { 0 };
+ let mut cmds: [Option<KmsCmd>; MAX_CONNECTORS * 4] =
+ [const { None }; MAX_CONNECTORS * 4];
+ for (connector, pending) in pending.connectors.into_iter().enumerate() {
+ cmds[connector] = pending.stream;
+ cmds[MAX_CONNECTORS + connector] = pending.cursor_create;
+ cmds[MAX_CONNECTORS * 2 + connector] = pending.cursor_image;
+ cmds[MAX_CONNECTORS * 3 + connector] = pending.cursor_move;
+ }
+ // Control-plane ordering comes first. An enabling atomic commit queues the plane flip
+ // before its CRTC mode-set. Finish the mode transaction before
+ // selecting a pending framebuffer.
+ let mut cmds = cmds.into_iter().flatten();
+ let mut retry = false;
+ while let Some(cmd) = cmds.next() {
+ let mut mode_programmed = 0u32;
+ let res = match &cmd {
+ KmsCmd::ModeSet { connector, timing } => {
+ if dual_complete {
+ // `activate_dual_wake` consumed the current generation for both
+ // connectors. A superseding generation published while it ran remains
+ // in `pending_kms` for the next outer iteration.
+ continue;
+ }
+ let connector_index = *connector as usize;
+ let key = timing_key(timing);
+ if connector_index >= MAX_CONNECTORS
+ || data.modeset_requested[connector_index].load(Ordering::Acquire)
+ != key
+ {
+ Ok(()) // superseded or disabled while queued
+ } else {
+ data.activate_head(dev, *connector, timing, key)
+ .map(|activated| {
+ if activated {
+ mode_programmed = 1u32 << connector;
+ }
+ })
+ }
+ }
+ KmsCmd::CursorCreate { connector, w, h } => data.send_cp(dev, 0x1b, 0, |ctr| {
+ crate::cp::cursor_create(ctr, *connector, *w, *h)
+ }),
+ KmsCmd::CursorImage {
+ connector,
+ w,
+ h,
+ bgra,
+ } => data.send_cp(dev, 0x1c, 0, |ctr| {
+ crate::cp::cursor_image(ctr, *connector, *w, *h, bgra)
+ }),
+ KmsCmd::CursorMove {
+ connector,
+ x,
+ y,
+ visible,
+ } => data.send_cp(dev, 0x1a, 0, |ctr| {
+ crate::cp::cursor_move(ctr, *connector, *x, *y, *visible)
+ }),
+ KmsCmd::Blank { connector } => data.blank_connector(dev, *connector),
+ };
+ // Remember what the dock accepted, so a later mode set can put it back. Recorded
+ // here rather than where the atomic callback queues it, because only a command
+ // that actually went out describes the dock's state.
+ if res.is_ok() {
+ data.record_cursor(&cmd);
+ relit |= mode_programmed;
+ }
+ if let Err(e) = res {
+ if !kms_error_retryable(e) {
+ pr_warn!("vino: dropping invalid asynchronous KMS command ({e:?})\n");
+ continue;
+ }
+
+ // Preserve the failed command and everything ordered behind it. Concurrent
+ // atomic callbacks may already have published newer state into these slots;
+ // `retry` never replaces that newer state with this drained batch.
+ let mut pending = data.pending_kms.lock();
+ pending.retry(cmd);
+ for cmd in cmds {
+ pending.retry(cmd);
+ }
+ retry = true;
+ vino_debug!("vino: asynchronous KMS command deferred after {e:?}\n");
+ if data.kms_retries.fetch_add(1, Ordering::Relaxed) + 1 >= KMS_RETRY_LIMIT {
+ pr_warn!(
+ "vino: dropping asynchronous KMS work after {} deferrals ({e:?}); the link is not coming back on its own\n",
+ KMS_RETRY_LIMIT
+ );
+ pending.clear();
+ data.kms_retries.store(0, Ordering::Relaxed);
+ retry = false;
+ }
+ break;
+ }
+ }
+ if has_modeset {
+ data.cmd_busy
+ .store(false, core::sync::atomic::Ordering::SeqCst);
+ }
+ // Put each re-programmed connector's cursor back. Queued rather than sent inline so it
+ // drains through the ordinary path on the next turn of this loop, behind anything the
+ // compositor has published in the meantime -- a real cursor commit always wins.
+ if relit != 0 && !retry {
+ data.rearm_cursor(&this, relit);
+ }
+
+ if retry {
+ if !data.shutting_down.load(Ordering::Acquire) {
+ let delay = kernel::time::msecs_to_jiffies(KMS_RETRY_MS);
+ let _ = workqueue::system().enqueue_delayed::<_, 0>(ARef::from(&*this), delay);
+ }
+ return;
+ }
+ // A batch that got through means whatever was wrong has cleared.
+ data.kms_retries.store(0, Ordering::Relaxed);
+ if data.pending_kms.lock().is_empty() {
+ break;
+ }
+ }
+ // Wake both scanout workers after the command batch. They stop while
+ // a queued mode set must reach the dock before video and resume here.
+ data.enqueue_scanout_all(&this);
+ }
+}
next prev parent reply other threads:[~2026-08-26 16:40 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-26 16:37 [PATCH v3 0/13] drm/vino: a Rust driver for DisplayLink DL3 docks Mike Lothian
2026-08-26 16:37 ` Mike Lothian [this message]
2026-08-26 16:37 ` [PATCH v3 9/13] drm/vino: add the dock activation and scanout path Mike Lothian
2026-08-26 16:37 ` [PATCH v3 11/13] drm/vino: add the USB driver frontend Mike Lothian
2026-08-26 16:37 ` [PATCH v3 12/13] drm/vino: allow the driver to be built Mike Lothian
2026-08-26 16:37 ` [PATCH v3 13/13] Documentation/gpu: document the Vino driver Mike Lothian
2026-08-26 17:28 ` Randy Dunlap
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260826163913.7052-9-mike@fireburn.co.uk \
--to=mike@fireburn.co.uk \
--cc=airlied@gmail.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=gary@garyguo.net \
--cc=linux-kernel@vger.kernel.org \
--cc=lossin@kernel.org \
--cc=maarten.lankhorst@linux.intel.com \
--cc=mripard@kernel.org \
--cc=rust-for-linux@vger.kernel.org \
--cc=simona@ffwll.ch \
--cc=tzimmermann@suse.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox