From: Mike Lothian <mike@fireburn.co.uk>
To: dri-devel@lists.freedesktop.org
Cc: Mike Lothian <mike@fireburn.co.uk>,
Maarten Lankhorst <maarten.lankhorst@linux.intel.com>,
Maxime Ripard <mripard@kernel.org>,
Thomas Zimmermann <tzimmermann@suse.de>,
David Airlie <airlied@gmail.com>, Simona Vetter <simona@ffwll.ch>,
Benno Lossin <lossin@kernel.org>, Gary Guo <gary@garyguo.net>,
linux-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org
Subject: [PATCH v3 9/13] drm/vino: add the dock activation and scanout path
Date: Wed, 26 Aug 2026 17:37:34 +0100 [thread overview]
Message-ID: <20260826163913.7052-10-mike@fireburn.co.uk> (raw)
In-Reply-To: <20260826163913.7052-1-mike@fireburn.co.uk>
Add the dock-facing half of the sink. A cold dock will not light from a
correct message sequence alone: it needs a measured bring-up choreography,
and one driven faster than its vendor drives it stays dark or resets. Those
timelines are data here, read off captures of each vendor stack waking the
same hardware, and they are expressed in transcript slots rather than
connector numbers because they were recorded from docks whose panels were
in the first two sockets.
With that come the pieces the choreography needs: connector presence and
its EDID, the brackets a mode program has to sit inside, stream opening,
the control-plane keepalive, and the scanout path itself -- a shadow
surface snapshotted inside the atomic commit so the encoder never races the
compositor's buffer, damage selection over the dock's own strip grid, and a
per-strip retransmit debt, because a dock holds more than one buffer and a
delta that reaches only one of them ghosts.
The encoder only ever sees the link's depth. Where userspace asked for a
deeper link than the surface it handed over, a sample is widened after
decoding by replicating its top bits into the low ones, so both
endpoints stay exact where a plain shift would leave full white three
codes short and tint every highlight. The strip cache is keyed on that
depth alongside the colour transform, since changing it re-maps every
sample on the way into the codec while leaving the framebuffer byte for
byte identical.
Waking a connector whose blank held its markers is a re-engage rather than
just a bracket close: the blank powered its downstream sink down, and the
closing markers do not bring it back. The vendor follows them with a probe,
EDID fetch and sink engage before it programs a timing, and so does this. A
dock woken without them comes back slowly, or not at all once the sink has
been down long enough for the dock to have let go of it.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Mike Lothian <mike@fireburn.co.uk>
---
drivers/gpu/drm/vino/drm_sink.rs | 11 +
drivers/gpu/drm/vino/drm_sink/activation.rs | 1196 +++++++++++
drivers/gpu/drm/vino/drm_sink/bracket.rs | 369 ++++
drivers/gpu/drm/vino/drm_sink/cp_session.rs | 414 ++++
drivers/gpu/drm/vino/drm_sink/presence.rs | 435 ++++
drivers/gpu/drm/vino/drm_sink/scanout.rs | 1958 +++++++++++++++++++
drivers/gpu/drm/vino/drm_sink/stream.rs | 470 +++++
drivers/gpu/drm/vino/drm_sink/timeline.rs | 571 ++++++
8 files changed, 5424 insertions(+)
create mode 100644 drivers/gpu/drm/vino/drm_sink/activation.rs
create mode 100644 drivers/gpu/drm/vino/drm_sink/bracket.rs
create mode 100644 drivers/gpu/drm/vino/drm_sink/cp_session.rs
create mode 100644 drivers/gpu/drm/vino/drm_sink/presence.rs
create mode 100644 drivers/gpu/drm/vino/drm_sink/scanout.rs
create mode 100644 drivers/gpu/drm/vino/drm_sink/stream.rs
create mode 100644 drivers/gpu/drm/vino/drm_sink/timeline.rs
diff --git a/drivers/gpu/drm/vino/drm_sink.rs b/drivers/gpu/drm/vino/drm_sink.rs
index 86708db89699..535e7d18ff50 100644
--- a/drivers/gpu/drm/vino/drm_sink.rs
+++ b/drivers/gpu/drm/vino/drm_sink.rs
@@ -46,11 +46,18 @@
xxhash,
};
+mod activation;
+mod bracket;
+mod cp_session;
mod dispatch;
mod driver;
mod limits;
mod mode_objects;
+mod presence;
+mod scanout;
mod settings;
+mod stream;
+mod timeline;
mod worker;
pub(crate) use driver::VinoObject;
@@ -58,6 +65,10 @@
pub(super) use mode_objects::{
PlaneArgs, VblankTimer, VinoConnector, VinoCrtc, VinoEncoder, VinoPlane,
};
+use scanout::{read_cursor_bgra, run_pending_scanout, snapshot_to_shadow, src_dims};
+// Almost every item in `timeline` is read by the activation path; naming them individually
+// would list the module.
+pub(crate) use timeline::*;
/// Connector mode used until a downstream EDID is available.
const FALLBACK_W: i32 = 2560;
diff --git a/drivers/gpu/drm/vino/drm_sink/activation.rs b/drivers/gpu/drm/vino/drm_sink/activation.rs
new file mode 100644
index 000000000000..e505d80ed80d
--- /dev/null
+++ b/drivers/gpu/drm/vino/drm_sink/activation.rs
@@ -0,0 +1,1196 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Bringing a connector's downstream sink up.
+//!
+//! A sink that is merely programmed with a timing stays dark. The dock has to be walked through a
+//! sequence of bracket states, carrier frames and mode sets, in an order and at intervals its own
+//! vendor was measured using, so the constants and the ordering here are load-bearing.
+
+use super::*;
+
+impl VinoDrmData {
+ /// Continuously present one already-encoded activation carrier for at least `duration_ms`.
+ ///
+ /// This deliberately performs no CP transaction between presentations. `BringUp` runs the
+ /// status/heartbeat dialogue concurrently on another work item; doing it here would put a
+ /// 10--15 ms control round-trip between tiny black frames and recreate the endpoint starvation
+ /// this path exists to remove. A persistent eight-URB queue bounds how far submission can run
+ /// ahead, so elapsed wall time closely follows actual endpoint progress rather than merely
+ /// copying an arbitrary number of frames into unbounded memory.
+ pub(super) fn submit_prompt_training(
+ &self,
+ dev: &BoundInterface<'_>,
+ connector: u8,
+ want: u64,
+ frames: &[KVec<u8>],
+ ordinary_frames: &[KVec<u8>],
+ duration_ms: i64,
+ with_arm: bool,
+ ) -> Result<u32> {
+ if frames.is_empty() {
+ return Err(kernel::error::code::EINVAL);
+ }
+ let geometry = self.geometry();
+ let xfer: usize = VIDEO_XFER;
+ let connector_index = connector as usize;
+ let pipe_i = dev.video_pipe_index(connector_index)?;
+ let connector_bit = 1u32 << connector;
+ // The DL7400's per-strip parameter map, ahead of the pixels it describes. This path --
+ // the startup/prompt-training submit -- is the one the dock's first frames go out on, so
+ // wiring the map only into the steady-state scanout left it absent from every frame that
+ // matters. Ridge has no equivalent record and gets an empty slice.
+ let params: KVec<u8> = if geometry.connector_selector_shift == 0 {
+ KVec::new()
+ } else {
+ let t = self
+ .last_timing
+ .lock()
+ .get(connector_index)
+ .copied()
+ .flatten();
+ match t {
+ Some(t) => {
+ let (sw, sh) = (geometry.strip_w(), geometry.strip_h());
+ let padded_width = (t.hactive as usize).div_ceil(sw) * sw;
+ let padded_height = (t.vactive as usize).div_ceil(sh) * sh;
+ crate::video::haar::navarro_strip_params(
+ geometry,
+ connector,
+ padded_width,
+ padded_height,
+ &frames,
+ &mut self.strip_classes.lock()[connector_index],
+ )?
+ }
+ None => KVec::new(),
+ }
+ };
+ vino_debug!(
+ "vino: connector={connector} prompt-training parameter map {} B\n",
+ params.len()
+ );
+ let arm = if with_arm {
+ // The bit is cleared only once this connector's carrier has gone out, so finding it
+ // clear means the stream is already open and the carrier already presented. That is
+ // this call's whole purpose, so report it done. Reporting it as a failure instead makes
+ // the caller re-arm and present a second carrier, and a retried activation then walks
+ // the dock's ring one slot further on every pass -- six flat frames where the vendor
+ // sends one, with the picture in a slot the dock is not showing.
+ if self.arm_prefix_pending.load(Ordering::Acquire) & connector_bit == 0 {
+ return Ok(0);
+ }
+ Some(self.build_stream_prefix_buf(connector_index)?)
+ } else {
+ None
+ };
+ if arm.is_some() {
+ self.send_stream_open(dev, connector_index)?;
+ // A stream that is being opened here starts its ring and its frame counter from the
+ // beginning, whatever the connector reached before. Arming the prologue already resets
+ // this, but an activation that is retried arms once and presents several times, so a
+ // connector could open a stream and immediately tell the dock it was filling a later
+ // slot with a later frame number -- and the dock scans out a slot nothing wrote.
+ self.scanout_seq.lock()[connector_index] = 0;
+ }
+ let startup = arm.is_some();
+ let seq0 = self.scanout_seq.lock()[connector_index];
+ let started = Instant::<Monotonic>::now();
+ let mut repeat = 0u32;
+ // Presentations that named a ring slot, which is what the frame counter counts. See
+ // `names_ring_slot`.
+ let mut named = 0u32;
+
+ loop {
+ if self.shutting_down.load(Ordering::Acquire)
+ || self.modeset_requested[connector_index].load(Ordering::Acquire) != want
+ || self.modeset_active[connector_index].load(Ordering::Acquire) != want
+ {
+ return Err(kernel::error::code::ENODEV);
+ }
+
+ let seq = seq0.wrapping_add(named);
+ let trailer = self.build_frame_trailer(connector, seq);
+ let arm_slice: &[u8] = if repeat == 0 {
+ arm.as_ref().map_or(&[], |a| &a[..])
+ } else {
+ &[]
+ };
+ let prologue_frame = startup && repeat == 0;
+ let opener = self.build_frame_opener(connector, seq, prologue_frame);
+ let opener_slice: &[u8] = opener.as_ref().map_or(&[], |o| &o[..]);
+ // Counted once this presentation has actually gone out; see the loop's tail. A pass
+ // that defers because the endpoint is full sends nothing, and a ring slot the dock
+ // never saw must not be spent.
+ let names_slot = names_ring_slot(opener_slice, &trailer);
+ // The report leads the frame it rides with, ahead of any pixels; see
+ // `build_stream_report_buf` for which frames carry one. The frame bearing the prologue
+ // never does: every generation goes from its decoder configuration into image records.
+ //
+ // The parameter map is not the report and has no such exception. The same-day DLM cold
+ // capture carries the map part-way through frame zero, and Windows carries it after
+ // the image records. Both put it before frame close. Omitting it made vino's first
+ // frame exactly 5,984 bytes short and left the dock briefly scanning a partially
+ // described framebuffer.
+ let report = if prologue_frame {
+ None
+ } else {
+ self.build_stream_report_buf(connector_index, seq)?
+ };
+ let report_slice: &[u8] = report.as_ref().map_or(&[], |r| &r[..]);
+ let params_slice: &[u8] = ¶ms[..];
+ // The prologue and ordinary DLM carriers contain identical strips but different
+ // producer record boundaries. Select by the presence of the one-shot arm rather than
+ // by this helper's local `repeat`: the cold timeline invokes the helper once per
+ // measured presentation, so every invocation starts with repeat zero.
+ let frame_parts = if prologue_frame {
+ frames
+ } else {
+ ordinary_frames
+ };
+ let image_len: usize = frame_parts.iter().map(|f| f.len()).sum();
+ let image_parts = frame_parts.len();
+ let wire_len = arm_slice.len()
+ + opener_slice.len()
+ + report_slice.len()
+ + params_slice.len()
+ + image_len
+ + trailer.len();
+ {
+ // One writer owns a shared pipe for the whole frame; see `own_pipe`.
+ let _pipe = self.own_pipe();
+ let mut staging_slots = self.video_staging.lock();
+ let staging_slot = &mut staging_slots[connector_index];
+ if staging_slot.is_none() {
+ let mut staging = KVec::new();
+ staging.resize(xfer, 0, GFP_KERNEL)?;
+ *staging_slot = Some(staging);
+ }
+ let staging = staging_slot.as_mut().ok_or(kernel::error::code::ENOMEM)?;
+
+ let mut queue_slot = self.video_q[pipe_i].lock();
+ if queue_slot.is_none() {
+ // Navarro's required EP08/EP0a clears are sent together at their captured
+ // pre-commit point in `send_cp_setup`. Nothing clears here: doing so after
+ // stream setup has begun changes SuperSpeed endpoint sequence state at a
+ // point DLM does not.
+ *queue_slot = Some(dev.video_queue(connector_index, 8, xfer)?);
+ vino_debug!(
+ "vino: connector={} endpoint={:#04x} persistent video queue opened by prompt training\n",
+ connector,
+ dev.endpoints.video[connector_index].address()
+ );
+ }
+ let queue = queue_slot
+ .as_mut()
+ .get_mut()
+ .as_mut()
+ .ok_or(kernel::error::code::ENODEV)?;
+
+ // A carrier is one protocol frame split over several URBs. Never block after
+ // submitting only its prefix: DLM's video producer and control workers are
+ // independent, while this cold-timeline worker also owes the marker burst that
+ // surrounds the first video bytes. On a non-draining endpoint the old path filled
+ // the eight slots with two complete frames plus one URB of frame three, then
+ // waited a second for frame-three URB two. The due +14-ms marker consequently
+ // never reached EP02 even though the dock had authenticated the preceding marker.
+ // Defer the whole presentation when it cannot fit; the next scheduled carrier can
+ // retry after the control messages have advanced the dock state.
+ let frame_urbs = wire_len.div_ceil(xfer);
+ if !queue.can_send_n(dev.io(), frame_urbs)? {
+ if self
+ .endpoint_status_logged
+ .fetch_or(connector_bit, Ordering::AcqRel)
+ & connector_bit
+ == 0
+ {
+ match dev.video_endpoint_status(connector_index) {
+ Ok(status) => vino_debug!(
+ "vino: connector={} endpoint={:#04x} stopped accepting video: GET_STATUS={:#06x} halt={}\n",
+ connector,
+ dev.endpoints.video[connector_index].address(),
+ status,
+ status & 1
+ ),
+ Err(e) => pr_warn!(
+ "vino: connector={} endpoint={:#04x} stopped accepting video: GET_STATUS failed ({e:?})\n",
+ connector,
+ dev.endpoints.video[connector_index].address()
+ ),
+ }
+ }
+ if duration_ms <= 0 {
+ return Ok(repeat);
+ }
+ if (Instant::<Monotonic>::now() - started).as_millis() >= duration_ms {
+ break;
+ }
+ fsleep(Delta::from_millis(1));
+ continue;
+ }
+
+ // Navarro's captured DLM stream flushes its two parameter records after exactly
+ // NAVARRO_PARAM_IMAGE_OFFSET bytes of image records in both the prologue and
+ // ordinary carriers. Build a borrowed scatter list in the captured order. A
+ // carrier's chunks all hold the same records, so the exact offset is usable here:
+ // the insertion may fall inside one allocation chunk, but is always on a
+ // wire-record boundary. A content frame's are not uniform and round to a chunk.
+ let mut wire_parts: KVec<&[u8]> = KVec::with_capacity(image_parts + 6, GFP_KERNEL)?;
+ if !arm_slice.is_empty() {
+ wire_parts.push(arm_slice, GFP_KERNEL)?;
+ }
+ if !opener_slice.is_empty() {
+ wire_parts.push(opener_slice, GFP_KERNEL)?;
+ }
+ if !report_slice.is_empty() {
+ wire_parts.push(report_slice, GFP_KERNEL)?;
+ }
+ let param_at = NAVARRO_PARAM_IMAGE_OFFSET.min(image_len);
+ let mut image_off = 0usize;
+ let mut param_inserted = params_slice.is_empty();
+ for f in frame_parts.iter() {
+ if image_off >= image_len {
+ break;
+ }
+ let n = f.len().min(image_len - image_off);
+ let split = param_at.saturating_sub(image_off).min(n);
+ if !param_inserted && image_off + n >= param_at {
+ if split != 0 {
+ wire_parts.push(&f[..split], GFP_KERNEL)?;
+ }
+ wire_parts.push(params_slice, GFP_KERNEL)?;
+ param_inserted = true;
+ if split != n {
+ wire_parts.push(&f[split..n], GFP_KERNEL)?;
+ }
+ } else if n != 0 {
+ wire_parts.push(&f[..n], GFP_KERNEL)?;
+ }
+ image_off += n;
+ }
+ if !param_inserted {
+ wire_parts.push(params_slice, GFP_KERNEL)?;
+ }
+ wire_parts.push(&trailer[..], GFP_KERNEL)?;
+
+ let part_count = wire_parts.len();
+ let mut part_i = 0usize;
+ let mut part_off = 0usize;
+ let mut wire_off = 0usize;
+ // DLM's authenticated first connector-0 prologue does not put all four URBs on the
+ // xHCI ring at once. It submits at +0/+806/+851/+873 us and receives completion
+ // of the first at +104 us, so the dock gets a ~700-us ready interval before the
+ // second transfer and then a three-URB pipeline. Submitting chunk two immediately
+ // leaves Navarro NRDY forever after exactly one completed URB. Preserve this
+ // producer boundary only for the one-shot, full-size prologue; ordinary frames
+ // use the normal eight-deep queue, just as DLM does.
+ const NAVARRO_PROLOGUE_SUBMIT_US: [i64; 4] = [0, 806, 851, 873];
+ let pace_prologue = !arm_slice.is_empty()
+ && xfer == VIDEO_XFER
+ && geometry.connector_selector_shift != 0;
+ let mut prologue_anchor: Option<Instant<Monotonic>> = None;
+ while wire_off < wire_len {
+ let data_len = (wire_len - wire_off).min(xfer);
+ let dst = &mut staging[..data_len];
+ let mut dst_off = 0usize;
+ while dst_off < dst.len() && part_i < part_count {
+ let part = wire_parts[part_i];
+ let n = (part.len() - part_off).min(dst.len() - dst_off);
+ dst[dst_off..dst_off + n].copy_from_slice(&part[part_off..part_off + n]);
+ dst_off += n;
+ part_off += n;
+ if part_off == part.len() {
+ part_i += 1;
+ part_off = 0;
+ }
+ }
+ if pace_prologue {
+ let chunk = wire_off / xfer;
+ if let Some(anchor) = prologue_anchor {
+ if let Some(&target_us) = NAVARRO_PROLOGUE_SUBMIT_US.get(chunk) {
+ Self::wait_video_offset(anchor, target_us);
+ }
+ }
+ }
+ // DLM's mixed transport: prologue chunk zero is reaped below, then the rest
+ // and all ordinary frames are pipelined.
+ queue.send(dev.io(), dst, crate::timeout())?;
+ if pace_prologue && wire_off == 0 {
+ let anchor = Instant::<Monotonic>::now();
+ prologue_anchor = Some(anchor);
+ // Do not expose chunk two to xHCI before chunk zero completes. The capture
+ // has the first completion at +104 us and the next submit at +806 us.
+ queue.flush(dev.io(), crate::timeout())?;
+ }
+ self.last_video_at.lock()[connector_index] = Some(Instant::<Monotonic>::now());
+ wire_off += data_len;
+ }
+ }
+
+ if repeat == 0 && startup {
+ self.arm_prefix_pending
+ .fetch_and(!connector_bit, Ordering::Release);
+ // The window runs from the first frame the dock actually receives, not from the
+ // mode set, so it is re-armed here. Whether there is a window at all is
+ // `sustain_window`'s decision and must not be second-guessed: a dock that shares
+ // its control pipe is granted none, and re-arming one unconditionally spent three
+ // seconds of full keyframes on the endpoint its control plane needs.
+ let mut sustain = self.sustain_until.lock();
+ if sustain[connector_index].is_some() {
+ sustain[connector_index] =
+ Some(Instant::<Monotonic>::now() + Delta::from_millis(SUSTAIN_MS));
+ }
+ drop(sustain);
+ vino_debug!(
+ "vino: connector {} startup frame submitted after {} ms ({} bytes)\n",
+ connector,
+ (Instant::<Monotonic>::now() - started).as_millis(),
+ wire_len
+ );
+ }
+ repeat = repeat.wrapping_add(1);
+ if names_slot {
+ named = named.wrapping_add(1);
+ }
+ self.scanout_seq.lock()[connector_index] = seq0.wrapping_add(named);
+
+ if repeat >= self.carrier_presentations() {
+ break;
+ }
+ // A zero window means this dock is bounded by the count above, not by wall clock.
+ // Testing the elapsed time against it regardless ends the carrier after one frame,
+ // whatever the count says.
+ if duration_ms > 0 && (Instant::<Monotonic>::now() - started).as_millis() >= duration_ms
+ {
+ break;
+ }
+ // Leave the endpoint between carrier frames on a dock that shares it, at the same
+ // interval its ordinary frames are paced to. A back-to-back carrier is what silenced
+ // this dock before, and it is the reason its window was reduced to a single frame.
+ if self.video_on_ctrl_pipe() {
+ fsleep(Delta::from_millis(self.frame_period_ms()));
+ }
+ }
+
+ vino_debug!(
+ "vino: connector={} training complete ({} presentations, {} ms)\n",
+ connector,
+ repeat,
+ (Instant::<Monotonic>::now() - started).as_millis()
+ );
+ Ok(repeat)
+ }
+ /// Apply one desired mode generation and its activation carrier.
+ ///
+ /// The control timeline is always released before returning. Any failed bracket, mode-set, arm,
+ /// or carrier transfer clears `modeset_active`, allowing the desired generation to be retried.
+ pub(super) fn activate_head(
+ &self,
+ dev: &BoundInterface<'_>,
+ connector: u8,
+ timing: &crate::cp::Timing,
+ want: u64,
+ ) -> Result<bool> {
+ let connector_index = connector as usize;
+ if connector_index >= MAX_CONNECTORS
+ || self.modeset_requested[connector_index].load(Ordering::Acquire) != want
+ {
+ return Ok(false);
+ }
+ // Keep no-op adoption at the activation boundary rather than in one caller. The command
+ // worker and the scanout worker can both arrive here, and either must avoid reopening an
+ // exact mode that is already usable under a different raw request token.
+ if self.adopt_programmed_mode(connector_index, timing, want) {
+ vino_debug!(
+ "vino: connector {connector_index} already active at this mode; no re-activation\n"
+ );
+ return Ok(false);
+ }
+ // Reconfiguring one connector of a dock that shares its control pipe, while another
+ // connector is lit, is a sequence of its own: see `ELLA_RUNTIME_MODE`. The schedule below
+ // is the cold one, and a dock already driving a sink stops answering partway through it,
+ // taking the lit connector down with it.
+ if self.video_on_ctrl_pipe()
+ && (0..MAX_CONNECTORS).any(|h| {
+ h != connector_index && self.modeset_active[h].load(Ordering::Acquire) != 0
+ })
+ {
+ let mut timings: [Option<crate::cp::Timing>; MAX_CONNECTORS] = [None; MAX_CONNECTORS];
+ timings[connector_index] = Some(*timing);
+ return self.activate_dock_wide(dev, ELLA_RUNTIME_MODE, timings);
+ }
+ // `wake` describes the state on entry and therefore has to be captured before invalidating
+ // it. From this point onward the transaction is going to touch the dock; do not leave the
+ // old token adoptable while a bracket, clear-mode, or set-mode is only partly complete.
+ let wake = self.modeset_active[connector_index].swap(0, Ordering::AcqRel) == 0;
+ self.programmed_timing.lock()[connector_index] = None;
+ // A superseding callback can land between the initial/adoption checks and the swap.
+ // Clearing the old active token is conservative, but sending its stale transaction is not:
+ // leave the newer request to its queued command or inline retry.
+ if self.modeset_requested[connector_index].load(Ordering::Acquire) != want {
+ return Ok(false);
+ }
+ // Same as the dual path: a connector coming back from a blank has a bracket owed before
+ // anything else is sent to it.
+ self.close_blank_bracket(dev, connector)?;
+ // The caller's timing was built when this connector was enabled, possibly before its
+ // endpoint partner existed. The request token remains the caller's generation; the exact
+ // corrected timing sent below is recorded separately in `programmed_timing`.
+ let timing = &self.effective_timing(connector_index, timing);
+
+ let geometry = self.geometry();
+ let padded_width =
+ (timing.hactive as usize + geometry.strip_w() - 1) & !(geometry.strip_w() - 1);
+ let padded_height =
+ (timing.vactive as usize + geometry.strip_h() - 1) & !(geometry.strip_h() - 1);
+ let prompt =
+ crate::video::haar::black_frame_ep08(geometry, padded_width, padded_height, connector)?;
+ let prompt_ordinary = crate::video::haar::black_frame_ep08_ordinary(
+ geometry,
+ padded_width,
+ padded_height,
+ connector,
+ )?;
+
+ self.begin_cp_timeline();
+ let transaction = (|| -> Result<bool> {
+ // The timing is what the sink has to be retrained onto, so the bracket belongs to the
+ // mode program and not to how the connector got here. A connector arriving from a blank
+ // needs it exactly as much as one being configured cold.
+ self.modeset_bracket_pre(dev, connector)?;
+ let mode_anchor = Instant::<Monotonic>::now();
+ // Tear the connector's pipe down before configuring it, as DLM does. See
+ // `cp::clear_mode`: the dock expects a connector to be torn down before it is
+ // configured.
+ if self.clear_mode_before_set() {
+ self.send_cp(dev, 0x48, 0, |ctr| crate::cp::clear_mode(ctr, connector))?;
+ }
+ self.send_cp(dev, 0x48, 0, |ctr| {
+ crate::cp::set_mode(ctr, connector, timing)
+ })?;
+ self.programmed_timing.lock()[connector_index] = Some(*timing);
+ if self.modeset_requested[connector_index].load(Ordering::Acquire) != want {
+ return Ok(false);
+ }
+
+ self.modeset_active[connector_index].store(want, Ordering::Release);
+ self.sustain_until.lock()[connector_index] = self.sustain_window(connector_index);
+ let bit = 1u32 << connector;
+ self.arm_stream_prologue(connector_index);
+ // A driven connector's stream opens with its pipe descriptor, not the idle open.
+ self.stream_open_pending.fetch_and(!bit, Ordering::Release);
+ self.owe_keyframe(connector_index);
+ self.strip_hashes.lock()[connector_index] = None;
+ self.dirty_ttl.lock()[connector_index] = None;
+
+ self.modeset_bracket_post_open(dev, connector, mode_anchor)?;
+ let opening = self.submit_prompt_training(
+ dev,
+ connector,
+ want,
+ &prompt,
+ &prompt_ordinary,
+ self.carrier_ms(PROMPT_TRAINING_OPEN_MS),
+ true,
+ );
+ let closing = self.modeset_bracket_post_close(dev, connector, mode_anchor);
+ opening?;
+ closing?;
+ Ok(true)
+ })();
+ self.end_cp_timeline();
+
+ let activated = match transaction {
+ Ok(activated) => activated,
+ Err(e) => {
+ self.modeset_active[connector_index].store(0, Ordering::Release);
+ self.programmed_timing.lock()[connector_index] = None;
+ self.unwind_bracket(dev, connector);
+ return Err(e);
+ }
+ };
+ if !activated {
+ self.programmed_timing.lock()[connector_index] = None;
+ self.unwind_bracket(dev, connector);
+ return Ok(false);
+ }
+ // The tail continues a carrier that is bounded by wall clock, so a dock bounded by a frame
+ // count has already presented all of it and this would add one more. The count is what the
+ // vendor's stream opens with, and the frames it names walk the dock's ring: an extra one
+ // puts every later frame a slot further on than the vendor puts it, and the dock presents
+ // a slot holding the flat carrier rather than the one holding the picture.
+ if self.carrier_ms(PROMPT_TRAINING_TAIL_MS) > 0 {
+ if let Err(e) = self.submit_prompt_training(
+ dev,
+ connector,
+ want,
+ &prompt,
+ &prompt_ordinary,
+ self.carrier_ms(PROMPT_TRAINING_TAIL_MS),
+ false,
+ ) {
+ self.modeset_active[connector_index].store(0, Ordering::Release);
+ self.programmed_timing.lock()[connector_index] = None;
+ self.unwind_bracket(dev, connector);
+ return Err(e);
+ }
+ }
+
+ vino_debug!(
+ "vino: applied {} stream-enable sequence for connector {}\n",
+ if wake { "wake" } else { "mode-change" },
+ connector
+ );
+ Ok(true)
+ }
+ /// Activate both downstream connectors using the dock-wide cold-link schedule.
+ ///
+ /// Both mode sets precede either connector's video. Single-connector activation and live mode
+ /// changes use the per-connector schedule.
+ ///
+ /// Every connector number in a [`ColdTimeline`] is a transcript slot, not a connector: slot 0
+ /// is the lowest-numbered activating connector and slot 1 the next, and they are resolved to
+ /// real connectors at the point of send. Taken literally they address connectors 0 and 1, whose
+ /// bits are absent from `sent` for any other pair of sockets, so no marker and no video would
+ /// go out at all.
+ pub(super) fn activate_dual_wake(
+ &self,
+ dev: &BoundInterface<'_>,
+ mut timings: [Option<crate::cp::Timing>; MAX_CONNECTORS],
+ ) -> Result<bool> {
+ let geometry = self.geometry();
+ let mut prompts: [Option<KVec<KVec<u8>>>; MAX_CONNECTORS] = core::array::from_fn(|_| None);
+ let mut ordinary_prompts: [Option<KVec<KVec<u8>>>; MAX_CONNECTORS] =
+ core::array::from_fn(|_| None);
+ let mut keys = [0u64; MAX_CONNECTORS];
+ let mut valid = 0u32;
+ // Snapshot topology once for the whole dock-wide transaction, so two partner modes cannot
+ // disagree about whether they share an endpoint if another callback lands mid-loop.
+ let requested_heads = self.requested_connector_mask();
+
+ // Pre-encode both tiny carriers before excluding the keepalive or starting either
+ // mode-set. Encoding work must not serialize the dock's back-to-back mode pair.
+ for connector in 0..MAX_CONNECTORS {
+ let Some(timing) = timings[connector] else {
+ continue;
+ };
+ let key = timing_key(&timing);
+ if self.modeset_requested[connector].load(Ordering::Acquire) != key
+ || self.modeset_active[connector].load(Ordering::Acquire) != 0
+ {
+ continue;
+ }
+ timings[connector] =
+ Some(self.effective_timing_in_mask(connector, &timing, requested_heads));
+ vino_debug!(
+ "vino: dual activation connector={} mode={}x{}@{}\n",
+ connector,
+ timing.hactive,
+ timing.vactive,
+ timing.refresh_hz
+ );
+ let padded_width =
+ (timing.hactive as usize + geometry.strip_w() - 1) & !(geometry.strip_w() - 1);
+ let padded_height =
+ (timing.vactive as usize + geometry.strip_h() - 1) & !(geometry.strip_h() - 1);
+ prompts[connector] = Some(crate::video::haar::black_frame_ep08(
+ geometry,
+ padded_width,
+ padded_height,
+ connector as u8,
+ )?);
+ ordinary_prompts[connector] = Some(crate::video::haar::black_frame_ep08_ordinary(
+ geometry,
+ padded_width,
+ padded_height,
+ connector as u8,
+ )?);
+ keys[connector] = key;
+ valid |= 1u32 << connector;
+ }
+ if valid.count_ones() < 2 {
+ return Ok(false);
+ }
+
+ // The two connectors this activation is about, in the order the timeline brings them up.
+ // Both cold timelines describe exactly two connectors, so a third is reported rather than
+ // silently left out of the choreography.
+ let mut slots = [0u8; 2];
+ let mut n = 0;
+ for connector in 0..MAX_CONNECTORS {
+ if valid & (1u32 << connector) != 0 {
+ if n < slots.len() {
+ slots[n] = connector as u8;
+ }
+ n += 1;
+ }
+ }
+ if n > slots.len() {
+ pr_warn!(
+ "vino: {n} connectors activating but the cold timeline describes {}; choreographing {} and {}\n",
+ slots.len(),
+ slots[0],
+ slots[1]
+ );
+ }
+ // Slot -> connector. Out-of-range slots cannot occur (both timelines only name 0 and 1) but
+ // clamp rather than panic, because a timeline is data and this runs under the CP lock.
+ let connector_of = |slot: u8| usize::from(slots[usize::from(slot).min(slots.len() - 1)]);
+
+ // Keep the clear/settle phase and the real mode-set choreography in one exclusive control
+ // transaction. Their deadlines have separate anchors because the Navarro cold timeline was
+ // measured from the real connector-0 mode set, 1,156 ms after its pipe clear.
+ // A blank is closed before the choreography, not during it: the bracket must be shut
+ // before anything re-probes or re-sets the mode.
+ for connector in 0..MAX_CONNECTORS {
+ if valid & (1u32 << connector) != 0 {
+ self.close_blank_bracket(dev, connector as u8)?;
+ }
+ }
+ self.begin_cp_timeline();
+ let activation_started = Instant::<Monotonic>::now();
+ let mut anchor = activation_started;
+ let mut sent = 0u32;
+ let mut started = 0u32;
+ let timeline = (|| -> Result<(u32, u32)> {
+ if self.dock_wide_modeset() {
+ let remap = |slot: u8| connector_of(slot) as u8;
+
+ // DLM's first clear pair begins a dock-wide sink reset. The authenticated
+ // transcript then stops/restarts each EDID reader, disengages/re-engages the
+ // downstream sinks, and clears each pipe a second time before any real mode.
+ for (slot, &connector) in slots.iter().enumerate() {
+ if slot == 1 {
+ Self::wait_mode_offset(activation_started, NAVARRO_PRIME_CLEAR_H1_MS);
+ }
+ self.send_cp(dev, 0x48, 0, |ctr| crate::cp::clear_mode(ctr, connector))?;
+ }
+ for &(at, op) in NAVARRO_COLD_PRELUDE {
+ Self::wait_mode_offset(activation_started, at);
+ self.navarro_cold_op(dev, op.remap_head(&remap))?;
+ }
+ Self::wait_mode_offset(activation_started, NAVARRO_REAL_MODE_H0_MS);
+ anchor = Instant::<Monotonic>::now();
+ }
+
+ let dock_wide_counters = if self.dock_wide_modeset() {
+ Some(self.reserve_cp_counters::<NAVARRO_COLD_COUNTERS>()?)
+ } else {
+ None
+ };
+
+ // Three cursors walk the sorted schedules; `cp_until` drains everything due at or
+ // before a given offset, preserving the ordering between markers, polls, and EDID
+ // reads.
+ let mut mi = 0usize;
+ let mut pi = 0usize;
+ let mut ei = 0usize;
+ // Replaying Ridge's choreography at Navarro leaves its video endpoint unarmed, so the
+ // timeline follows the dock, not the driver.
+ let timeline: &ColdTimeline = if self.uses_arm_burst() {
+ &COLD_RIDGE
+ } else {
+ &COLD_NAVARRO
+ };
+ let mut remoded = 0u32;
+
+ macro_rules! cp_until {
+ ($limit:expr) => {{
+ let limit: i64 = $limit;
+ loop {
+ let nm = timeline.markers.get(mi).map(|m| m.0);
+ let np = timeline.polls.get(pi).copied();
+ let ne = timeline.edid.get(ei).map(|e| e.0);
+ let next = [nm, np, ne]
+ .into_iter()
+ .flatten()
+ .filter(|&o| o <= limit)
+ .min();
+ let Some(off) = next else { break };
+ Self::wait_mode_offset(anchor, off);
+ if nm == Some(off) {
+ let (_, slot, sub, state) = timeline.markers[mi];
+ let connector = connector_of(slot) as u8;
+ if sent & (1u32 << connector) != 0 {
+ if let Some(counters) = dock_wide_counters.as_ref() {
+ let slot =
+ *NAVARRO_MARKER_COUNTER_SLOTS.get(mi).ok_or(EINVAL)?;
+ let ctr = *counters.get(slot).ok_or(EINVAL)?;
+ self.send_cp_reserved(dev, 0x16, ctr, |ctr| {
+ crate::cp::stream_marker(ctr, connector, sub, state)
+ })?;
+ } else {
+ self.stream_marker(dev, connector, sub, state)?;
+ }
+ }
+ mi += 1;
+ } else if np == Some(off) {
+ if let Some(counters) = dock_wide_counters.as_ref() {
+ let slot = *NAVARRO_POLL_COUNTER_SLOTS.get(pi).ok_or(EINVAL)?;
+ let ctr = *counters.get(slot).ok_or(EINVAL)?;
+ self.send_cp_reserved(dev, 0x14, ctr, |ctr| {
+ crate::cp::device_query_req(ctr, 0x000c)
+ })?;
+ } else {
+ self.poll_status(dev)?;
+ }
+ pi += 1;
+ } else {
+ let (_, slot, fetch) = timeline.edid[ei];
+ let connector = connector_of(slot) as u8;
+ // Re-read the sink's EDID at its required place in the transaction.
+ // This dock-side DDC operation is not a source of new modes, so discard
+ // its reply rather than publishing a hotplug during a mode set.
+ self.send_cp(dev, 0x15, 0, |ctr| {
+ if fetch {
+ crate::cp::get_edid_req(ctr, connector)
+ } else {
+ crate::cp::get_edid_req_sub(ctr, 0x0020, connector)
+ }
+ })?;
+ ei += 1;
+ }
+ }
+ }};
+ }
+
+ // Both real mode sets go out before any video, spaced according to this dock's
+ // measured cold timeline. Navarro's pipe clears were sent during the settling phase
+ // above; do not collapse them back into this loop.
+ for slot in 0..slots.len() {
+ let connector = connector_of(slot as u8);
+ let bit = 1u32 << connector;
+ if valid & bit == 0 {
+ continue;
+ }
+ let Some(timing) = timings[connector] else {
+ continue;
+ };
+ // The second connector's mode set is spaced from the first by this dock's measured
+ // interval -- 757 ms on Navarro, 29 ms on Ridge. Gate on the slot, so the spacing
+ // survives whichever sockets the monitors are in.
+ if slot == 1 {
+ cp_until!(timeline.h1_mode - 1);
+ Self::wait_mode_offset(anchor, timeline.h1_mode);
+ }
+ // Retrain the downstream link onto the new timing. The sink goes down
+ // immediately ahead of the timing and the post-mode bracket brings it back up; a
+ // dock whose vendor does not bracket this way carries no state here.
+ if let Some(state) = self.pre_mode_sink_state() {
+ self.stream_marker(dev, connector as u8, 0x2f, 1)?;
+ self.stream_marker(dev, connector as u8, 0x2e, state)?;
+ }
+ if let Some(counters) = dock_wide_counters.as_ref() {
+ // Reservation-token slots for the two mode sets, by activation order. Keyed on
+ // the connector number these collided for any pair but (0, 1), and Navarro NAKs
+ // from the first flattened counter onward.
+ let ctr_slot = if slot == 0 { 0 } else { 3 };
+ let ctr = *counters.get(ctr_slot).ok_or(EINVAL)?;
+ self.send_cp_reserved(dev, 0x48, ctr, |ctr| {
+ crate::cp::set_mode(ctr, connector as u8, &timing)
+ })?;
+ } else {
+ self.send_cp(dev, 0x48, 0, |ctr| {
+ crate::cp::set_mode(ctr, connector as u8, &timing)
+ })?;
+ }
+ self.programmed_timing.lock()[connector] = Some(timing);
+ if self.modeset_requested[connector].load(Ordering::Acquire) != keys[connector] {
+ continue;
+ }
+ self.modeset_active[connector].store(keys[connector], Ordering::Release);
+ self.sustain_until.lock()[connector] = self.sustain_window(connector);
+ self.arm_stream_prologue(connector);
+ // A driven connector's stream opens with its pipe descriptor, not the idle open.
+ self.stream_open_pending.fetch_and(!bit, Ordering::Release);
+ self.owe_keyframe(connector);
+ self.strip_hashes.lock()[connector] = None;
+ self.dirty_ttl.lock()[connector] = None;
+ sent |= bit;
+ }
+
+ // Preserve the required silent window on EP02 between the connector-1 mode set and
+ // `cold::QUIET_END`. The exclusive control timeline already excludes keepalives.
+ Self::wait_mode_offset(anchor, timeline.quiet_end);
+
+ // Bracket, status polls and the mid-bracket EDID re-read, up to the first video.
+ cp_until!(timeline.video[0].1 - 1);
+
+ // DLM opens a short sealed stream on every connector without a monitor at this point.
+ // Vino does not: this dock re-enumerates when a stream is driven at an empty connector,
+ // and an empty socket's index is not stable across bring-ups.
+
+ for &(vslot, at) in timeline.video {
+ let connector = connector_of(vslot as u8);
+ cp_until!(at - 1);
+ // Some docks set a connector's mode a second time shortly before its video.
+ for &(off, reslot) in timeline.remode {
+ let replay_connector = connector_of(reslot as u8);
+ if off >= at
+ || remoded & (1u32 << replay_connector) != 0
+ || sent & (1u32 << replay_connector) == 0
+ {
+ continue;
+ }
+ let Some(timing) = timings[replay_connector] else {
+ continue;
+ };
+ cp_until!(off - 1);
+ Self::wait_mode_offset(anchor, off);
+ self.send_cp(dev, 0x48, 0, |ctr| {
+ crate::cp::set_mode(ctr, replay_connector as u8, &timing)
+ })?;
+ self.programmed_timing.lock()[replay_connector] = Some(timing);
+ remoded |= 1u32 << replay_connector;
+ }
+ cp_until!(at - 1);
+ Self::wait_mode_offset(anchor, at);
+ let bit = 1u32 << connector;
+ if sent & bit == 0 {
+ continue;
+ }
+ // Exactly one ARM+carrier presentation keeps the closing markers from being
+ // delayed behind a blocking multi-frame submission.
+ let frames = prompts[connector].as_ref().ok_or(EINVAL)?;
+ let ordinary_frames = ordinary_prompts[connector].as_ref().ok_or(EINVAL)?;
+ let t_sub = Instant::<Monotonic>::now();
+ let first_for_head = started & bit == 0;
+ self.submit_prompt_training(
+ dev,
+ connector as u8,
+ keys[connector],
+ frames,
+ ordinary_frames,
+ self.carrier_ms(PROMPT_TRAINING_OPEN_MS),
+ first_for_head,
+ )?;
+ vino_debug!(
+ "vino: connector {} video submit took {} ms (timeline offset {} ms, {} ms since anchor)\n",
+ connector,
+ (Instant::<Monotonic>::now() - t_sub).as_millis(),
+ at,
+ (Instant::<Monotonic>::now() - anchor).as_millis()
+ );
+ started |= bit;
+ }
+
+ // Remaining polls and the closing markers.
+ cp_until!(i64::MAX);
+ Ok((sent, started))
+ })();
+ self.end_cp_timeline();
+ let (sent, started) = match timeline {
+ Ok(state) => state,
+ Err(e) => {
+ for connector in 0..MAX_CONNECTORS {
+ if sent & (1u32 << connector) != 0
+ && self.modeset_active[connector].load(Ordering::Acquire) == keys[connector]
+ {
+ self.modeset_active[connector].store(0, Ordering::Release);
+ }
+ }
+ // The choreography opens every activating connector's bracket well before it sets a
+ // mode, so unwind on `valid` rather than `sent`: a connector that failed before its
+ // mode set is still open on the dock.
+ for connector in 0..MAX_CONNECTORS {
+ if valid & (1u32 << connector) != 0 {
+ self.unwind_bracket(dev, connector as u8);
+ }
+ }
+ return Err(e);
+ }
+ };
+ if sent.count_ones() < 2 {
+ for connector in 0..MAX_CONNECTORS {
+ if sent & (1u32 << connector) != 0
+ && self.modeset_active[connector].load(Ordering::Acquire) == keys[connector]
+ {
+ self.modeset_active[connector].store(0, Ordering::Release);
+ }
+ }
+ return Ok(false);
+ }
+
+ // Keep both endpoints busy through downstream clock training, so the carrier outlives the
+ // bracket rather than stopping with it.
+ let tail_started = Instant::<Monotonic>::now();
+ while (Instant::<Monotonic>::now() - tail_started).as_millis()
+ < self.carrier_ms(cold::CARRIER_TAIL_MS)
+ {
+ for connector in 0..MAX_CONNECTORS {
+ if started & (1u32 << connector) == 0 {
+ continue;
+ }
+ let frames = prompts[connector].as_ref().ok_or(EINVAL)?;
+ let ordinary_frames = ordinary_prompts[connector].as_ref().ok_or(EINVAL)?;
+ if let Err(e) = self.submit_prompt_training(
+ dev,
+ connector as u8,
+ keys[connector],
+ frames,
+ ordinary_frames,
+ self.carrier_ms(PROMPT_TRAINING_OPEN_MS),
+ false,
+ ) {
+ for reset in 0..MAX_CONNECTORS {
+ if sent & (1u32 << reset) != 0
+ && self.modeset_active[reset].load(Ordering::Acquire) == keys[reset]
+ {
+ self.modeset_active[reset].store(0, Ordering::Release);
+ }
+ }
+ for reset in 0..MAX_CONNECTORS {
+ if valid & (1u32 << reset) != 0 {
+ self.unwind_bracket(dev, reset as u8);
+ }
+ }
+ return Err(e);
+ }
+ }
+ }
+ vino_debug!(
+ "vino: dual-connector activation complete after {} ms (mode/started masks 0x{:x}/0x{:x})\n",
+ (Instant::<Monotonic>::now() - anchor).as_millis(),
+ sent,
+ started
+ );
+ Ok(true)
+ }
+ /// Activate both connectors of a dock that carries its video on the control pipe.
+ ///
+ /// Replays [`ELLA_DOCK_WIDE`]: one transaction configures both connectors, the first streams,
+ /// and the second's sink comes up behind it. The per-connector schedule cannot express this,
+ /// because its second pass sets a mode the dock has already been given and opens a bracket the
+ /// dock stops answering.
+ pub(super) fn activate_dock_wide(
+ &self,
+ dev: &BoundInterface<'_>,
+ steps: &[DockWideStep],
+ mut timings: [Option<crate::cp::Timing>; MAX_CONNECTORS],
+ ) -> Result<bool> {
+ // Connectors the table drives, which is what decides how many the caller has to supply.
+ let wanted = steps
+ .iter()
+ .filter_map(|step| match *step {
+ DockWideStep::SetMode(slot) => Some(u32::from(slot) + 1),
+ _ => None,
+ })
+ .max()
+ .unwrap_or(0);
+ let geometry = self.geometry();
+ let mut prompts: [Option<KVec<KVec<u8>>>; MAX_CONNECTORS] = core::array::from_fn(|_| None);
+ let mut ordinary_prompts: [Option<KVec<KVec<u8>>>; MAX_CONNECTORS] =
+ core::array::from_fn(|_| None);
+ let mut keys = [0u64; MAX_CONNECTORS];
+ let mut valid = 0u32;
+ let requested_heads = self.requested_connector_mask();
+
+ // Encode every carrier before the transaction opens. Where a table sets two modes they are
+ // adjacent on the wire and must not be separated by this dock's encoder.
+ for connector in 0..MAX_CONNECTORS {
+ let Some(timing) = timings[connector] else {
+ continue;
+ };
+ let key = timing_key(&timing);
+ // Only that the connector still wants this mode. Whether it is already lit is the
+ // caller's to decide: the cold table takes connectors whose generation is zero, and the
+ // runtime table exists precisely to reconfigure one that is streaming.
+ if self.modeset_requested[connector].load(Ordering::Acquire) != key {
+ continue;
+ }
+ timings[connector] =
+ Some(self.effective_timing_in_mask(connector, &timing, requested_heads));
+ let padded_width =
+ (timing.hactive as usize + geometry.strip_w() - 1) & !(geometry.strip_w() - 1);
+ let padded_height =
+ (timing.vactive as usize + geometry.strip_h() - 1) & !(geometry.strip_h() - 1);
+ prompts[connector] = Some(crate::video::haar::black_frame_ep08(
+ geometry,
+ padded_width,
+ padded_height,
+ connector as u8,
+ )?);
+ ordinary_prompts[connector] = Some(crate::video::haar::black_frame_ep08_ordinary(
+ geometry,
+ padded_width,
+ padded_height,
+ connector as u8,
+ )?);
+ keys[connector] = key;
+ valid |= 1u32 << connector;
+ }
+ if valid.count_ones() < wanted {
+ return Ok(false);
+ }
+
+ // Slot -> connector, in activation order. Two is this dock's whole complement of
+ // connectors.
+ let mut slots = [0u8; 2];
+ let mut n = 0;
+ for connector in 0..MAX_CONNECTORS {
+ if valid & (1u32 << connector) != 0 {
+ if n < slots.len() {
+ slots[n] = connector as u8;
+ }
+ n += 1;
+ }
+ }
+ if n > slots.len() {
+ pr_warn!(
+ "vino: {n} connectors activating but this dock has {}; choreographing {} and {}\n",
+ slots.len(),
+ slots[0],
+ slots[1]
+ );
+ }
+ let connector_of = |slot: u8| usize::from(slots[usize::from(slot).min(slots.len() - 1)]);
+
+ // The runtime table can enter with a live target. Once any table step runs, its old stream
+ // state is no longer safe to adopt: Ella's live table changes sink markers before SetMode.
+ // Invalidate every participant before the first bracket/control write. A request that moved
+ // while carriers were encoded is then left at zero for its newer command to establish.
+ for connector in 0..MAX_CONNECTORS {
+ if valid & (1u32 << connector) != 0 {
+ self.modeset_active[connector].store(0, Ordering::Release);
+ self.programmed_timing.lock()[connector] = None;
+ }
+ }
+ if (0..MAX_CONNECTORS).any(|connector| {
+ valid & (1u32 << connector) != 0
+ && self.modeset_requested[connector].load(Ordering::Acquire) != keys[connector]
+ }) {
+ return Ok(false);
+ }
+
+ for connector in 0..MAX_CONNECTORS {
+ if valid & (1u32 << connector) != 0 {
+ self.close_blank_bracket(dev, connector as u8)?;
+ }
+ }
+
+ self.begin_cp_timeline();
+ let started = Instant::<Monotonic>::now();
+ let mut sent = 0u32;
+ let transaction = (|| -> Result<u32> {
+ for step in steps {
+ match *step {
+ DockWideStep::SetMode(slot) => {
+ let connector = connector_of(slot);
+ let timing = timings[connector].ok_or(EINVAL)?;
+ self.send_cp(dev, 0x48, 0, |ctr| {
+ crate::cp::set_mode(ctr, connector as u8, &timing)
+ })?;
+ self.programmed_timing.lock()[connector] = Some(timing);
+ if self.modeset_requested[connector].load(Ordering::Acquire)
+ != keys[connector]
+ {
+ return Ok(sent);
+ }
+ self.modeset_active[connector].store(keys[connector], Ordering::Release);
+ self.sustain_until.lock()[connector] = self.sustain_window(connector);
+ self.arm_stream_prologue(connector);
+ // A driven connector's stream opens with its pipe descriptor, not the idle
+ // open.
+ self.stream_open_pending
+ .fetch_and(!(1u32 << connector), Ordering::Release);
+ self.owe_keyframe(connector);
+ self.strip_hashes.lock()[connector] = None;
+ self.dirty_ttl.lock()[connector] = None;
+ sent |= 1u32 << connector;
+ }
+ DockWideStep::Marker(slot, sub, state) => {
+ self.stream_marker(dev, connector_of(slot) as u8, sub, state)?;
+ }
+ DockWideStep::Poll => self.poll_status(dev)?,
+ DockWideStep::Prologue(slot) => {
+ self.send_stream_prologue(dev, connector_of(slot) as u8)?;
+ }
+ DockWideStep::Ring(slot) => {
+ self.send_stream_ring(dev, connector_of(slot) as u8)?;
+ }
+ DockWideStep::Config(slot) => {
+ self.send_stream_config(dev, connector_of(slot) as u8)?;
+ }
+ DockWideStep::Carrier(slot) => {
+ let connector = connector_of(slot);
+ if sent & (1u32 << connector) == 0 {
+ continue;
+ }
+ self.submit_prompt_training(
+ dev,
+ connector as u8,
+ keys[connector],
+ prompts[connector].as_ref().ok_or(EINVAL)?,
+ ordinary_prompts[connector].as_ref().ok_or(EINVAL)?,
+ self.carrier_ms(PROMPT_TRAINING_OPEN_MS),
+ true,
+ )?;
+ }
+ DockWideStep::Stream(slot, frames) => {
+ // The vendor does not pause here, it keeps presenting. Its second connector
+ // is configured and its sink brought up while the first one's stream is
+ // running, so the records around this step reach a dock that is mid-frame
+ // -- which sleeping through reproduces on the wire and not at all in what
+ // the dock is doing. Present the connector's own flat surface: this
+ // transaction owns the endpoint exclusively and cannot take the
+ // compositor's live frame, but keeping the stream and its ring advancing is
+ // what the step is for.
+ let connector = connector_of(slot);
+ if sent & (1u32 << connector) == 0 {
+ continue;
+ }
+ self.submit_prompt_training(
+ dev,
+ connector as u8,
+ keys[connector],
+ prompts[connector].as_ref().ok_or(EINVAL)?,
+ ordinary_prompts[connector].as_ref().ok_or(EINVAL)?,
+ self.carrier_ms(
+ self.frame_period_ms().saturating_mul(i64::from(frames)),
+ ),
+ false,
+ )?;
+ }
+ }
+ }
+ Ok(sent)
+ })();
+ self.end_cp_timeline();
+
+ let sent = match transaction {
+ Ok(sent) => sent,
+ Err(e) => {
+ for connector in 0..MAX_CONNECTORS {
+ if valid & (1u32 << connector) != 0 {
+ self.modeset_active[connector].store(0, Ordering::Release);
+ self.programmed_timing.lock()[connector] = None;
+ }
+ }
+ // Unwind on `valid`: the transaction opens every activating connector's bracket
+ // before it reaches that connector's mode set, so a connector that failed early is
+ // still open on the dock.
+ for connector in 0..MAX_CONNECTORS {
+ if valid & (1u32 << connector) != 0 {
+ self.unwind_bracket(dev, connector as u8);
+ }
+ }
+ return Err(e);
+ }
+ };
+ if sent.count_ones() < wanted {
+ for connector in 0..MAX_CONNECTORS {
+ if valid & (1u32 << connector) != 0 {
+ self.modeset_active[connector].store(0, Ordering::Release);
+ self.programmed_timing.lock()[connector] = None;
+ self.unwind_bracket(dev, connector as u8);
+ }
+ }
+ return Ok(false);
+ }
+ vino_debug!(
+ "vino: dock-wide activation complete after {} ms (connectors 0x{:x})\n",
+ (Instant::<Monotonic>::now() - started).as_millis(),
+ sent
+ );
+ Ok(true)
+ }
+}
diff --git a/drivers/gpu/drm/vino/drm_sink/bracket.rs b/drivers/gpu/drm/vino/drm_sink/bracket.rs
new file mode 100644
index 000000000000..e18a698ca16b
--- /dev/null
+++ b/drivers/gpu/drm/vino/drm_sink/bracket.rs
@@ -0,0 +1,369 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Taking a connector's sink down and back up, and the brackets around a mode set.
+//!
+//! A dock does not simply accept a new timing. Its vendor drives the sink down, programs the
+//! timing and brings it back up, and the states and the gaps between them are what retrains the
+//! downstream link. Get the sequence wrong and the dock accepts every byte of every frame and
+//! lights nothing, with nothing on the wire to say so.
+
+use super::*;
+
+impl VinoDrmData {
+ /// Drive `connector` to black on the dock, then close its stream bracket.
+ ///
+ /// Runs on the command worker for [`KmsCmd::Blank`], i.e. after `atomic_disable` has already
+ /// zeroed this connector's mode generation. That zero is what makes the write legal: every
+ /// video path gates on `modeset_requested == modeset_active == want`, and passing `want = 0`
+ /// matches exactly the disabled state and nothing else -- so a re-enable racing this blank
+ /// flips both atomics to a real key and the submit loop drops out with `ENODEV` instead of
+ /// painting black over the freshly enabled mode.
+ ///
+ /// The dock's stream itself is still configured (vino never told it otherwise), so the black
+ /// frames are an ordinary accepted write, not a write onto a torn-down pipe.
+ pub(super) fn blank_connector(&self, dev: &BoundInterface<'_>, connector: u8) -> Result {
+ let socket = connector + 1;
+ let connector_index = connector as usize;
+ // A dock that wants its markers held takes two of them and then silence: no video, no mode
+ // set and no close bracket for as long as the output stays down. It is the same pair
+ // `modeset_bracket_pre` opens with, so the stream is held rather than torn down, and
+ // [`Self::close_blank_bracket`] owes the matching re-open before the connector is driven
+ // again. The scanout has already stopped, because `atomic_disable` zeroed this connector's
+ // mode generation before queueing this command.
+ //
+ // Sending such a dock the close bracket below instead re-enumerates it about two seconds
+ // later, which takes the whole desktop down with it.
+ if self.blank_bracket() == crate::profile::BlankBracket::MarkersHeld {
+ self.stream_marker(dev, connector, 0x2f, 1)?;
+ self.stream_marker(dev, connector, 0x2e, self.sink_down_state())?;
+ self.set_self_blanked(connector_index, true);
+ self.blank_bracket_open
+ .fetch_or(1u32 << connector_index, Ordering::Release);
+ vino_debug!("vino: socket {socket} blanked; bracket held open, stream idle\n");
+ return Ok(());
+ }
+ let Some(timing) = self.last_timing.lock()[connector_index] else {
+ // Never modeset, so there is nothing lit to blank.
+ return Ok(());
+ };
+ let geometry = self.geometry();
+ let padded_width =
+ (timing.hactive as usize + geometry.strip_w() - 1) & !(geometry.strip_w() - 1);
+ let padded_height =
+ (timing.vactive as usize + geometry.strip_h() - 1) & !(geometry.strip_h() - 1);
+ let frames =
+ crate::video::haar::black_frame_ep08(geometry, padded_width, padded_height, connector)?;
+ let ordinary_frames = crate::video::haar::black_frame_ep08_ordinary(
+ geometry,
+ padded_width,
+ padded_height,
+ connector,
+ )?;
+ // Present for long enough to reach every dock buffer. The dock is multi-buffered and a
+ // single presentation lands in one buffer only -- the same reason damage debt exists --
+ // so a one-shot blank leaves the other buffer holding the frozen desktop and the panel
+ // alternates between black and stale content.
+ let sent = self.submit_prompt_training(
+ dev,
+ connector,
+ 0,
+ &frames,
+ &ordinary_frames,
+ BLANK_PRESENT_MS,
+ false,
+ )?;
+ self.stream_marker(dev, connector, 0x2f, 0)?;
+ self.stream_marker(dev, connector, 0x2e, 0)?;
+ // Do not take the sink down for a connector whose monitor has already gone away.
+ //
+ // `atomic_disable` fires for both a DPMS-off and a monitor removal, and they need opposite
+ // treatment. Sending the power-down marker at a sink that is already gone is pointless, and
+ // setting `self_blanked` would make the presence watcher deliberately ignore that
+ // connector's silence, preventing a later replug from being detected.
+ let candidate = if self.connector_present(connector_index) {
+ self.sink_down_state()
+ } else {
+ vino_debug!(
+ "vino: socket {socket} blank skips the sink marker -- its monitor is already gone\n"
+ );
+ 0
+ };
+ if candidate != 0 {
+ // From here the dock will stop answering this connector's presence probe, exactly as it
+ // does for a real unplug. Claim the silence before causing it.
+ self.set_self_blanked(connector_index, true);
+ if let Err(e) = self.power_down_sink(dev, connector) {
+ self.set_self_blanked(connector_index, false);
+ return Err(e);
+ }
+ }
+ vino_debug!("vino: socket {socket} blanked on the dock ({sent} black presentation(s))\n");
+ Ok(())
+ }
+ /// Take one connector's downstream sink out of power, leaving the monitor with no signal.
+ ///
+ /// The dock goes on scanning out whatever it last decoded, so a driver that simply stops
+ /// sending pixels leaves the panel lit on a frozen image indefinitely. Only this sequence ends
+ /// that.
+ fn power_down_sink(&self, dev: &BoundInterface<'_>, connector: u8) -> Result {
+ let socket = connector + 1;
+ self.stream_marker(dev, connector, 0x2f, 1)?;
+ self.stream_marker(dev, connector, 0x2e, self.sink_down_state())?;
+ self.poll_status(dev)?;
+ self.stream_marker(dev, connector, 0x2f, 0)?;
+ vino_debug!("vino: socket {socket} downstream sink powered down\n");
+ Ok(())
+ }
+ /// Power down every sink this driver lit, before it stops being able to.
+ ///
+ /// Unbinding does not reach the dock: `atomic_disable` queues the blank on the command worker,
+ /// and teardown discards that queue and cancels the worker before it runs. The dock therefore
+ /// keeps scanning out the last frame it decoded and the monitors stay lit on a frozen desktop
+ /// until something else drives them.
+ ///
+ /// Best effort by construction. It runs on the disconnect path, where the device may already be
+ /// physically gone -- every transfer then fails immediately, which is the right outcome -- and
+ /// it must not be the reason unbinding blocks, so it does no work at all when no connector is
+ /// lit and its waits are the bounded ones the mode-set path already uses.
+ pub(crate) fn park_sinks(&self) {
+ let lit: u32 = self
+ .modeset_active
+ .iter()
+ .enumerate()
+ .fold(0, |mask, (h, active)| {
+ mask | (u32::from(active.load(Ordering::Acquire) != 0) << h)
+ });
+ // Nothing lit, or nothing to say it to: an unplug arrives here with the session already
+ // gone, and waiting for the video path to drain against a dead dock buys nothing.
+ if lit == 0 || self.check_cp_session().is_err() {
+ return;
+ }
+ // Stand the scanout workers down and let any frame already on the wire finish, so these
+ // markers do not land in the middle of one.
+ self.cmd_busy
+ .store(true, core::sync::atomic::Ordering::SeqCst);
+ self.wait_for_video_idle();
+ let Ok(link) = crate::usb_link::UsbLink::open(&self.io, self.endpoints) else {
+ return;
+ };
+ for connector in 0..MAX_CONNECTORS {
+ if lit & (1u32 << connector) == 0 {
+ continue;
+ }
+ if let Err(e) = self.power_down_sink(&link, connector as u8) {
+ vino_debug!(
+ "vino: socket {} sink power-down on unbind failed ({e:?})\n",
+ connector + 1
+ );
+ }
+ }
+ }
+ /// Drive one connector's stream bracket to the closed state.
+ ///
+ /// The dock holds a connector opened with `2e=3` until it is told otherwise: it stops driving
+ /// the sink and disengages that connector's EDID handler, so the connector then reads exactly
+ /// like an empty socket. Nothing but this sequence puts it back, and the dock keeps the state
+ /// across a USB re-enumeration, so it must be sent rather than inferred.
+ fn send_bracket_close(&self, dev: &BoundInterface<'_>, connector: u8) -> Result {
+ self.stream_marker(dev, connector, 0x2f, 1)?;
+ self.stream_marker(dev, connector, 0x2e, 0)?;
+ self.stream_marker(dev, connector, 0x2f, 0)?;
+ self.stream_marker(dev, connector, 0x2e, 0)
+ }
+ /// Assert the closed bracket state on a connector that is about to be probed.
+ ///
+ /// The dock's bracket state outlives this driver's record of it, so dedicated-pipe docks assert
+ /// it before probing. Ella is different: its vendor stream never performs a periodic reset, and
+ /// the four markers share EP02 with pixels. Resetting an idle socket while its sibling is live
+ /// drops the live sink, so its profile defers the close until no sibling is active.
+ ///
+ /// Best effort otherwise: a connector that is genuinely empty is not worth failing a probe
+ /// over.
+ pub(super) fn close_bracket_before_probe(&self, dev: &BoundInterface<'_>, connector: u8) {
+ let bit = 1u32 << connector;
+ let active_connectors = self
+ .modeset_active
+ .iter()
+ .enumerate()
+ .fold(0u32, |mask, (h, active)| {
+ mask | (u32::from(active.load(Ordering::Acquire) != 0) << h)
+ });
+ if !self
+ .probe_bracket()
+ .should_close(connector, active_connectors)
+ {
+ vino_debug!(
+ "vino: socket {} probe defers bracket reset beside active sibling(s) {:#x}\n",
+ connector + 1,
+ active_connectors & !bit
+ );
+ return;
+ }
+ if self.send_bracket_close(dev, connector).is_ok() {
+ self.blank_bracket_open.fetch_and(!bit, Ordering::AcqRel);
+ }
+ }
+ /// Put a connector back into the closed bracket state after a failed mode set.
+ ///
+ /// A mode set opens the bracket before it configures anything, so an error anywhere after that
+ /// point leaves the connector open on the dock with no record of it here. Best effort: this
+ /// runs on the error path, and the transport that just failed may fail again.
+ pub(super) fn unwind_bracket(&self, dev: &BoundInterface<'_>, connector: u8) {
+ // A dead session cannot carry the close, and every queued mode set fails against it, so
+ // attempting one per failure floods the log with a consequence of the disconnect rather
+ // than a cause. The dock is being re-established anyway; a fresh session closes the
+ // bracket in `reengage_connector` before it probes.
+ if !self.cp_link_alive() {
+ return;
+ }
+ if self.send_bracket_close(dev, connector).is_err() {
+ // The dock is still holding this connector. Say so once, rather than reporting only the
+ // error that got us here, because the connector now reads as an empty socket.
+ pr_warn!(
+ "vino: socket {socket} left open after a failed mode set; it will read as empty until a re-engage closes it\n",
+ socket = connector + 1
+ );
+ }
+ }
+ /// Close a blank bracket before this connector is driven again.
+ ///
+ /// Precedes the EDID probe and the mode set that follow. A connector that was never blanked
+ /// costs nothing here.
+ pub(super) fn close_blank_bracket(&self, dev: &BoundInterface<'_>, connector: u8) -> Result {
+ let socket = connector + 1;
+ let bit = 1u32 << connector;
+ if self.blank_bracket_open.load(Ordering::Acquire) & bit == 0 {
+ return Ok(());
+ }
+ // Clear the record only once the dock has been told, so a send that fails partway leaves
+ // the debt standing for the next attempt. Clearing first strands the connector open
+ // forever: the driver stops believing anything is owed while the dock goes on holding it.
+ self.send_bracket_close(dev, connector)?;
+ self.blank_bracket_open.fetch_and(!bit, Ordering::AcqRel);
+ // Closing the bracket restores the stream, not the sink. The blank powered the downstream
+ // sink down and nothing else turns it back on, so the vendor follows the closing markers
+ // with a full probe, EDID fetch and sink engage before it programs a timing. A dock woken
+ // without them comes back slowly, or not at all once the sink has been down long enough
+ // for the dock to have let go of it.
+ //
+ // Best effort: a connector whose monitor genuinely went away while it was blanked belongs
+ // to the presence watcher, and the mode set below is what reports a wake that failed.
+ //
+ // Not on a dock whose video shares the control pipe, for the same reason `sustain_window`
+ // is withheld there: the seven paced messages are taken directly from the pipe the wake is
+ // also driving pixels down, and this dock answers by going silent rather than by dropping
+ // a frame.
+ if !self.video_on_ctrl_pipe() && self.reengage_connector(dev, connector).is_err() {
+ vino_debug!("vino: socket {socket} wake re-engage failed; the mode set will retry\n");
+ }
+ // A wake is still not a cold plug: the re-engage above restores the sink, so the
+ // three-second keyframe window `sustain_window` grants a cold activation buys nothing here
+ // and costs about a gigabyte per connector. Sustained bandwidth is also what destabilises
+ // this dock, so it is never spent twice.
+ self.repair_connectors.fetch_or(bit, Ordering::Release);
+ vino_debug!("vino: socket {socket} blank bracket closed; wake runs as a repair\n");
+ Ok(())
+ }
+ /// Open the per-connector stream bracket before changing an active mode.
+ pub(super) fn modeset_bracket_pre(&self, dev: &BoundInterface<'_>, connector: u8) -> Result {
+ self.stream_marker(dev, connector, 0x2f, 1)?;
+ // Docks that want the sink torn down before it is configured say so in their profile.
+ // Where no state is carried the vendor sets the mode first and brackets behind it, and
+ // downing a sink that is about to be programmed would leave it down for the whole bracket.
+ if let Some(state) = self.pre_mode_sink_state() {
+ self.stream_marker(dev, connector, 0x2e, state)?;
+ }
+ self.poll_status(dev)
+ }
+ /// Sleep until an absolute millisecond offset from the mode-set anchor.
+ ///
+ /// Absolute deadlines keep scheduler delay from accumulating across the activation sequence.
+ pub(super) fn wait_mode_offset(anchor: Instant<Monotonic>, target_ms: i64) {
+ let elapsed_ms = (Instant::<Monotonic>::now() - anchor).as_millis();
+ if elapsed_ms < target_ms {
+ fsleep(Delta::from_millis(target_ms - elapsed_ms));
+ }
+ }
+ /// Sleep/spin until an exact microsecond offset in a short video-transport schedule.
+ ///
+ /// `fsleep` handles the bulk of the delay without burning a CPU; the small busy-wait tail
+ /// avoids scheduling a producer boundary hundreds of microseconds late. This is used only for
+ /// the four submissions of Navarro's one-shot prologue.
+ pub(super) fn wait_video_offset(anchor: Instant<Monotonic>, target_us: i64) {
+ const SPIN_MARGIN_US: i64 = 80;
+ let elapsed = anchor.elapsed().as_micros_ceil();
+ if elapsed >= target_us {
+ return;
+ }
+ if target_us - elapsed > SPIN_MARGIN_US {
+ fsleep(Delta::from_micros(target_us - elapsed - SPIN_MARGIN_US));
+ }
+ let elapsed = anchor.elapsed().as_micros_ceil();
+ if elapsed < target_us {
+ udelay(Delta::from_micros(target_us - elapsed));
+ }
+ }
+ /// Complete the stream-open markers and status polls up to the first video deadline.
+ pub(super) fn modeset_bracket_post_open(
+ &self,
+ dev: &BoundInterface<'_>,
+ connector: u8,
+ anchor: Instant<Monotonic>,
+ ) -> Result {
+ self.poll_status(dev)?;
+ Self::wait_mode_offset(anchor, 5);
+ self.stream_marker(dev, connector, 0x2f, 1)?;
+ Self::wait_mode_offset(anchor, 9);
+ self.stream_marker(dev, connector, 0x2e, self.post_mode_sink_state(0))?;
+ Self::wait_mode_offset(anchor, 12);
+ self.stream_marker(dev, connector, 0x2f, 1)?;
+ Self::wait_mode_offset(anchor, 14);
+ // `0x2e` state 3 takes the downstream sink down and 0 brings it back up. The vendor drives
+ // it down once here and then straight back up; repeating the 3 leaves the sink down for
+ // the rest of the bracket, which on DL-3x00 is a dock that accepts every byte of a frame
+ // and displays none of it.
+ self.stream_marker(dev, connector, 0x2e, self.post_mode_sink_state(1))?;
+ // The vendor's ring descriptor and decoder configuration land here, between the fourth
+ // marker and the fifth, so the closing `2e(connector, 0)` below is the last thing the dock
+ // sees before pixels. A dock told to bring its sink up after a frame has already gone out
+ // has been handed that frame with nothing scanning it out.
+ self.send_stream_prologue(dev, connector)?;
+ Self::wait_mode_offset(anchor, 20);
+ self.stream_marker(dev, connector, 0x2f, 1)?;
+ // The status poll shares the final `2f(1)` deadline.
+ self.poll_status(dev)?;
+ Self::wait_mode_offset(anchor, 26);
+ self.stream_marker(dev, connector, 0x2e, 0)?;
+ // There is a measured 63-ms quiet interval, then three polls at +89/+95/+110 ms. The last
+ // poll and first video bytes share one deadline.
+ Self::wait_mode_offset(anchor, 89);
+ self.poll_status(dev)?;
+ Self::wait_mode_offset(anchor, 95);
+ self.poll_status(dev)?;
+ Self::wait_mode_offset(anchor, PROMPT_VIDEO_MS);
+ self.poll_status(dev)
+ }
+ /// Close the post-mode-set bracket after prompt video has started.
+ ///
+ /// The first close marker is +13 ms from video and the second is +15 ms. Background keepalive
+ /// resumes immediately after this pair and supplies the continuing status dialogue.
+ ///
+ /// A dock without a video pipe has already closed its bracket: its last marker before the
+ /// strips is the sink-up, and the vendor sends nothing at all between a frame and the next
+ /// frame's opener. Closing again there puts two records into the one gap the vendor leaves
+ /// empty, one of them a marker state it never uses on this generation.
+ pub(super) fn modeset_bracket_post_close(
+ &self,
+ dev: &BoundInterface<'_>,
+ connector: u8,
+ anchor: Instant<Monotonic>,
+ ) -> Result {
+ if self.video_on_ctrl_pipe() {
+ return Ok(());
+ }
+ Self::wait_mode_offset(anchor, PROMPT_CLOSE_2F_MS);
+ self.stream_marker(dev, connector, 0x2f, 0)?;
+ Self::wait_mode_offset(anchor, PROMPT_CLOSE_2E_MS);
+ self.stream_marker(dev, connector, 0x2e, 0)
+ }
+}
diff --git a/drivers/gpu/drm/vino/drm_sink/cp_session.rs b/drivers/gpu/drm/vino/drm_sink/cp_session.rs
new file mode 100644
index 000000000000..36b3a46026a5
--- /dev/null
+++ b/drivers/gpu/drm/vino/drm_sink/cp_session.rs
@@ -0,0 +1,414 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! The control-plane session: sending sealed messages and keeping the link alive.
+//!
+//! The control plane is host-driven lockstep. Every message carries an authenticated counter that
+//! both ends step together, so a message sent out of turn, or a reply left undrained,
+//! desynchronises the session and every later message decrypts to nothing. That is why sending is
+//! serialised here rather than at the call sites.
+
+use super::*;
+
+impl VinoDrmData {
+ pub(crate) fn set_cp_engaged(&self, engaged: bool) {
+ self.cp_engaged
+ .store(engaged, core::sync::atomic::Ordering::SeqCst);
+ }
+ /// Whether the control session is still usable.
+ ///
+ /// Reads the flag, not the mutex: the caller is usually asking precisely because the link may
+ /// be stuck, and the stuck thread is holding that mutex.
+ pub(crate) fn cp_link_alive(&self) -> bool {
+ self.cp_session_live.load(Ordering::Acquire)
+ }
+ /// Record that the dock answered. Resets the silence deadline.
+ pub(super) fn note_cp_reply(&self) {
+ *self.cp_last_reply.lock() = Instant::<Monotonic>::now();
+ }
+ /// How long the dock has answered nothing, in milliseconds.
+ pub(super) fn cp_silent_for_ms(&self) -> i64 {
+ (Instant::<Monotonic>::now() - *self.cp_last_reply.lock()).as_millis()
+ }
+ /// Give up on the control session. Idempotent, and logs once.
+ ///
+ /// Only the flag is cleared here. Callers that hold `cp_link` drop its contents themselves;
+ /// the watchdog cannot, because the mutex is exactly what a wedged transfer is holding. Every
+ /// path consults the flag before the mutex, so an orphaned [`CpLink`] is unreachable, and
+ /// [`Self::shutdown`] frees it with the rest of the session state.
+ pub(super) fn abandon_cp_session(&self, silent_ms: i64) {
+ if self
+ .cp_session_live
+ .compare_exchange(true, false, Ordering::AcqRel, Ordering::Acquire)
+ .is_ok()
+ {
+ pr_warn!(
+ "vino: dock has answered nothing for {silent_ms} ms; abandoning the session\n"
+ );
+ }
+ }
+ /// Gate every control transaction, without taking `cp_link`.
+ ///
+ /// Checking the deadline here rather than under the mutex is the difference between noticing
+ /// the dock has gone and queueing another thread behind the transfer that proves it.
+ pub(super) fn check_cp_session(&self) -> Result {
+ // Teardown first. `disconnect()` waits for the workers, and a worker that starts a control
+ // transfer to a device already being disconnected waits for a completion that will not
+ // come; on a dock whose video shares this endpoint the scanout path issues those transfers
+ // too, so the window is wide. The pair deadlocks `usb_hub_wq` inside `usb_disconnect()`,
+ // which stops USB hotplug machine-wide and is only recoverable by rebooting.
+ if self.shutting_down.load(Ordering::Acquire) {
+ return Err(ENODEV);
+ }
+ if !self.cp_link_alive() {
+ return Err(ENODEV);
+ }
+ let silent_ms = self.cp_silent_for_ms();
+ if silent_ms >= self.cp_silence_limit_ms() {
+ self.abandon_cp_session(silent_ms);
+ return Err(ETIMEDOUT);
+ }
+ Ok(())
+ }
+ /// How long this dock may say nothing before its session is abandoned.
+ pub(super) fn cp_silence_limit_ms(&self) -> i64 {
+ if self.video_on_ctrl_pipe() {
+ CP_SILENCE_LIMIT_SHARED_MS
+ } else {
+ CP_SILENCE_LIMIT_MS
+ }
+ }
+ /// Ask the USB core to reset the dock after the control session has been abandoned.
+ ///
+ /// Without this the outputs stay down until the user unplugs the dock, because a session can
+ /// only be established through probe. It is the one recovery a stuck transfer cannot block:
+ /// the USB core runs it from its own work item. What turns it into a fresh session is
+ /// `post_reset` asking for the interface to be rebound -- a reset on its own leaves the
+ /// driver bound with nothing to drive.
+ pub(super) fn reset_after_wedge(&self) {
+ if self
+ .cp_reset_queued
+ .compare_exchange(false, true, Ordering::AcqRel, Ordering::Acquire)
+ .is_err()
+ {
+ return;
+ }
+ // A closed window means unplug is already under way, which needs no help from us.
+ let Ok(io) = self.io.enter() else {
+ return;
+ };
+ pr_warn!("vino: resetting the dock to recover the control session\n");
+ io.interface().queue_reset_device();
+ }
+ /// Whether the current device session has engaged content protection.
+ pub(crate) fn cp_engaged(&self) -> bool {
+ self.cp_engaged.load(core::sync::atomic::Ordering::Acquire)
+ }
+ /// Pause the background CP loop and let any iteration which passed its check finish. The mode
+ /// worker calls this before taking its timestamp anchor; the fixed delay therefore cannot move
+ /// any event relative to the mode-set itself.
+ pub(super) fn begin_cp_timeline(&self) {
+ self.cp_timeline_exclusive.store(true, Ordering::Release);
+ self.initial_modeset_quiet.store(false, Ordering::Release);
+ fsleep(Delta::from_millis(PROMPT_KEEPALIVE_QUIESCE_MS));
+ }
+ pub(super) fn end_cp_timeline(&self) {
+ self.cp_timeline_exclusive.store(false, Ordering::Release);
+ }
+ /// Used by `BringUp`'s long-lived keepalive worker. It deliberately remains cheap because that
+ /// worker checks it every millisecond while an activation sequence is in progress.
+ pub(crate) fn cp_timeline_exclusive(&self) -> bool {
+ self.cp_timeline_exclusive.load(Ordering::Acquire)
+ }
+ /// Publish the engaged CP session so the KMS callbacks can send runtime CP messages.
+ /// Called once by the bring-up work item after the dock acks (`acks > 0`). `wire_seq`/
+ /// `counter` are the next free values past the bring-up CP setup.
+ pub(crate) fn publish_session(
+ &self,
+ dev: &BoundInterface<'_>,
+ ks: &[u8; 16],
+ riv: &[u8; 8],
+ wire_seq: u32,
+ counter: u16,
+ ep84_depth: usize,
+ ) {
+ // EP84 must remain posted between runtime EP02 writes. A queue drained synchronously leaves
+ // the endpoint unposted between calls and can stall the control protocol.
+ // `ep84_depth` is the matched profile's `ep84_queue_depth`, so the runtime queue keeps the
+ // same number of reads posted that bring-up did.
+ let ep84_q = match dev.ctrl_in_queue(ep84_depth, 4096) {
+ Ok(q) => Some(q),
+ Err(e) => {
+ pr_warn!("vino: persistent EP84 queue open failed ({e:?}); using sync fallback\n");
+ None
+ }
+ };
+ *self.cp_link.lock() = Some(CpLink {
+ ks: kernel::crypto::Secret::new(*ks),
+ riv: *riv,
+ wire_seq,
+ counter,
+ ep84_q,
+ });
+ self.note_cp_reply();
+ self.cp_session_live.store(true, Ordering::Release);
+ }
+ /// Start the silence watchdog for the session just published.
+ ///
+ /// Separate from [`Self::publish_session`] because it needs an `ARef` to the device, and it
+ /// re-arms itself for the lifetime of the session.
+ pub(crate) fn start_cp_watchdog(&self, drm_dev: &VinoDrmDevice) {
+ if self.shutting_down.load(Ordering::Acquire) {
+ return;
+ }
+ let delay = kernel::time::msecs_to_jiffies(CP_WATCHDOG_PERIOD_MS);
+ let _ = workqueue::system().enqueue_delayed::<_, 5>(ARef::from(drm_dev), delay);
+ }
+ /// Seal and send one interactive CP message, advance the session, and pass its paired reply
+ /// to `consume`.
+ ///
+ /// `build(counter)` produces the inner message for the dock-echoed counter. The `cp_link`
+ /// mutex serialises the complete EP02/EP84 transaction with the KMS worker and keepalive.
+ /// Callers are sleepable; atomic callbacks queue commands instead of invoking this path.
+ fn send_cp_reply<T>(
+ &self,
+ dev: &BoundInterface<'_>,
+ id: u16,
+ tag_reserved: usize,
+ reserved_counter: Option<u16>,
+ build: impl FnOnce(u16) -> Result<KVec<u8>>,
+ consume: impl FnOnce(&[u8; 16], &[u8; 8], &[u8]) -> Result<T>,
+ ) -> Result<T> {
+ self.check_cp_session()?;
+ self.send_cp_locked(dev, id, tag_reserved, reserved_counter, build, consume)
+ }
+ #[allow(clippy::too_many_arguments)]
+ fn send_cp_locked<T>(
+ &self,
+ dev: &BoundInterface<'_>,
+ id: u16,
+ tag_reserved: usize,
+ reserved_counter: Option<u16>,
+ build: impl FnOnce(u16) -> Result<KVec<u8>>,
+ consume: impl FnOnce(&[u8; 16], &[u8; 8], &[u8]) -> Result<T>,
+ ) -> Result<T> {
+ let mut guard = self.cp_link.lock();
+ let Some(link) = (&mut *guard).as_mut() else {
+ return Err(ENODEV);
+ };
+ // DLM normally uses the next wire-order counter. Its cold Navarro mode transaction is the
+ // exception: per-connector workers reserve counters before their writes interleave, so the
+ // inner counter order differs from the monotonically advancing AES block sequence.
+ let request_counter = reserved_counter.unwrap_or(link.counter);
+ let msg = build(request_counter)?;
+ let inner_sub = if msg.len() >= 4 {
+ u16::from_le_bytes([msg[2], msg[3]])
+ } else {
+ 0
+ };
+ let content = &msg[..msg.len().saturating_sub(tag_reserved)];
+ let frame = crate::cp::seal_interactive(&link.ks, &link.riv, id, link.wire_seq, content)?;
+ let pipe = self.own_pipe();
+ // A shared video failure publishes the session dead while a CP writer may already be
+ // waiting for this pipe behind the failed frame. Recheck after acquiring it so that
+ // waiter cannot issue one last control record into the terminal stream before reset.
+ if !self.cp_link_alive() {
+ return Err(ENODEV);
+ }
+ if let Err(e) = dev.ctrl_send(&frame, crate::timeout(), GFP_KERNEL) {
+ let silent_ms = self.cp_silent_for_ms();
+ if silent_ms >= self.cp_silence_limit_ms() {
+ self.abandon_cp_session(silent_ms);
+ *guard = None;
+ }
+ return Err(e);
+ }
+ // The reply arrives on the other endpoint, so the pipe is free again the moment the
+ // request is out: holding it across the wait would stall a frame for the whole timeout.
+ drop(pipe);
+ link.wire_seq = link
+ .wire_seq
+ .wrapping_add(((content.len() + 15) / 16) as u32);
+ // A normal message consumes the next counter now. A reserved message consumed its counter
+ // when its logical worker queued the transaction, before independently queued EP02 writes
+ // interleaved; consuming it twice here would skip a value after the cold transaction.
+ if reserved_counter.is_none() {
+ link.counter = link.counter.wrapping_add(1);
+ }
+ // DLM keeps reading EP84 until it sees the reply whose inner counter echoes this request.
+ // Navarro also emits unprompted `id=2/sub=0x86` status pushes on the same endpoint;
+ // treating the first such push as the paired reply advances EP02 before the dock has
+ // completed the transaction. The dock then NAKs that write until the real reply is reaped,
+ // which is the exact 100-ms staircase visible in the failed captures. Consume pushes here
+ // and stop only at the echoed counter (or a bounded timeout for request classes which do
+ // not reply).
+ //
+ //
+ // Use the validated 4096-byte request size so larger logical replies arrive intact.
+ let mut reply = KVec::from_elem(0u8, 4096, GFP_KERNEL)?;
+ let deadline = Instant::<Monotonic>::now() + Delta::from_millis(64);
+ let mut matched = 0usize;
+ let (mut reaped, mut undecodable) = (0u32, 0u32);
+ let (mut seen_id, mut seen_sub, mut seen_counter) = (0u16, 0u16, 0u16);
+ loop {
+ let got = if let Some(q) = link.ep84_q.as_mut() {
+ match q.recv(dev.io(), &mut reply, crate::cp_reply_timeout()) {
+ Ok(Some(n)) => n,
+ Ok(None) => 0,
+ Err(_) => break,
+ }
+ } else {
+ dev.ctrl_recv(&mut reply, crate::cp_reply_timeout(), GFP_KERNEL)
+ .unwrap_or(0)
+ };
+ if got > 16 {
+ // During re-engagement an EDID push can precede the paired acknowledgment. Keep
+ // it for the waiting connector while continuing to wait for the echoed counter.
+ let target = self.edid_target.load(Ordering::Relaxed);
+ if target != NO_EDID_TARGET {
+ if let Ok(Some(blob)) =
+ crate::cp::parse_edid_from_reply(&link.ks, &link.riv, &reply[..got])
+ {
+ *self.edid_caught.lock() = Some(blob);
+ }
+ }
+ if let Some((reply_id, reply_sub, reply_counter)) =
+ crate::cp::decode_in_lenient(&link.ks, &link.riv, &reply[..got])
+ {
+ if reply_counter == request_counter {
+ matched = got;
+ break;
+ }
+ seen_id = reply_id;
+ seen_sub = reply_sub;
+ seen_counter = reply_counter;
+ if reply_id == 0x44 || crate::cp::edid_reply_len(reply_id).is_some() {
+ self.downstream_event.store(true, Ordering::Release);
+ }
+ } else {
+ undecodable += 1;
+ }
+ reaped += 1;
+ }
+ if (Instant::<Monotonic>::now() - deadline).as_millis() >= 0 {
+ break;
+ }
+ }
+ // Name the message that went unanswered, and say whether the dock was silent or merely
+ // unreadable. Without this a stalled control session can only be reported as ETIMEDOUT,
+ // which cannot distinguish "the dock sent nothing" from "the dock replied and vino could
+ // not decode it" -- and on the D6000 the wire shows 50 sealed replies arriving during an
+ // attempt that ends in ETIMEDOUT.
+ if matched == 0 {
+ // Name the *inner* sub as well as the wire id. "id=0x16 went unanswered" covers the
+ // EDID engage, the readiness kick and both stream/display markers, and which one the
+ // dock ignored is the whole diagnosis.
+ vino_debug!(
+ "vino: unanswered id={id:#06x} sub={inner_sub:#06x} ctr={request_counter}: reaped {reaped} reply/replies, {undecodable} undecodable, last decoded id={seen_id:#06x} sub={seen_sub:#06x} ctr={seen_counter}\n"
+ );
+ let silent_ms = self.cp_silent_for_ms();
+ if silent_ms >= self.cp_silence_limit_ms() {
+ self.abandon_cp_session(silent_ms);
+ *guard = None;
+ return Err(ETIMEDOUT);
+ }
+ } else {
+ self.note_cp_reply();
+ }
+ consume(&link.ks, &link.riv, &reply[..matched])
+ }
+ /// Seal and send one interactive CP message on EP02, advancing the session keystream.
+ pub(crate) fn send_cp(
+ &self,
+ dev: &BoundInterface<'_>,
+ id: u16,
+ tag_reserved: usize,
+ build: impl FnOnce(u16) -> Result<KVec<u8>>,
+ ) -> Result {
+ self.send_cp_reply(dev, id, tag_reserved, None, build, |_, _, _| Ok(()))
+ }
+ /// Send one CP message using a counter token previously consumed from the live allocator.
+ pub(super) fn send_cp_reserved(
+ &self,
+ dev: &BoundInterface<'_>,
+ id: u16,
+ inner_counter: u16,
+ build: impl FnOnce(u16) -> Result<KVec<u8>>,
+ ) -> Result {
+ self.send_cp_reply(dev, id, 0, Some(inner_counter), build, |_, _, _| Ok(()))
+ }
+ /// Consume `N` consecutive counters from the live session and return them as reservation
+ /// tokens. This models DLM's independently queued per-connector workers: allocation order
+ /// remains monotonic even when their actual EP02 writes interleave in a different order.
+ pub(super) fn reserve_cp_counters<const N: usize>(&self) -> Result<[u16; N]> {
+ let mut guard = self.cp_link.lock();
+ let Some(link) = (&mut *guard).as_mut() else {
+ return Err(ENODEV);
+ };
+ let mut counters = [0u16; N];
+ for counter in &mut counters {
+ *counter = link.counter;
+ link.counter = link.counter.wrapping_add(1);
+ }
+ Ok(counters)
+ }
+ /// Consume the dock's *unprompted* EP84 pushes, i.e. reads that are not the reply to any of our
+ /// writes. Returns how many frames were drained.
+ ///
+ /// The dock also emits capability and heartbeat frames without a paired request. The bounded
+ /// zero-timeout loop consumes those pushes without delaying keepalive or allowing a chatty dock
+ /// to monopolise the worker.
+ pub(crate) fn drain_cp_pushes(&self, dev: &BoundInterface<'_>, max: usize) -> usize {
+ if !self.cp_link_alive() {
+ return 0;
+ }
+ // Best-effort really does mean best-effort: if a transaction owns the link there is
+ // nothing to reap that it is not already reaping, and blocking here would put the
+ // keepalive behind a transfer that may never return.
+ let Some(mut guard) = self.cp_link.try_lock() else {
+ return 0;
+ };
+ let Some(link) = (&mut *guard).as_mut() else {
+ return 0;
+ };
+ let Ok(mut reply) = KVec::from_elem(0u8, 4096, GFP_KERNEL) else {
+ return 0;
+ };
+ let mut n = 0;
+ while n < max {
+ let got = match link.ep84_q.as_mut() {
+ // Every queue slot is already posted. One millisecond is enough to reap a
+ // completion without turning this best-effort reader into another control
+ // deadline. A zero-jiffy completion wait does not observe an already-signalled
+ // completion reliably, so it left pushes queued until the next EP02 write.
+ Some(q) => q.recv(dev.io(), &mut reply, Delta::from_millis(1)),
+ None => dev
+ .ctrl_recv(&mut reply, Delta::from_millis(1), GFP_KERNEL)
+ .map(Some),
+ };
+ // `Ok(None)` is the queue's timeout: nothing pending, so the dock has nothing more to
+ // say right now. Any error is treated the same -- this is best-effort drainage.
+ match got {
+ Ok(Some(len)) if len > 0 => {
+ n += 1;
+ // An unprompted push is the dock answering. The silence deadline asks whether
+ // it is talking at all, not whether it is answering us in particular, and on
+ // an idle lit link the heartbeats are most of what it says.
+ self.note_cp_reply();
+ if let Some((id, _, _)) =
+ crate::cp::decode_in_lenient(&link.ks, &link.riv, &reply[..len])
+ {
+ // An EDID-handler reply arriving with no probe outstanding is the dock
+ // reporting that a downstream sink changed -- it is the *only* thing it
+ // sent between a measured monitor replug and its own give-up reset. Treat
+ // it as "re-probe now" rather than waiting out the presence period.
+ if id == 0x44 || crate::cp::edid_reply_len(id).is_some() {
+ self.downstream_event.store(true, Ordering::Release);
+ }
+ }
+ }
+ _ => break,
+ }
+ }
+ n
+ }
+}
diff --git a/drivers/gpu/drm/vino/drm_sink/presence.rs b/drivers/gpu/drm/vino/drm_sink/presence.rs
new file mode 100644
index 000000000000..6e58f5729223
--- /dev/null
+++ b/drivers/gpu/drm/vino/drm_sink/presence.rs
@@ -0,0 +1,435 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! What is plugged into a connector, and keeping that answer honest.
+//!
+//! A dock reports presence through its EDID probe rather than through any hotplug interrupt, so
+//! everything here is polled and everything here can be wrong for a moment. A monitor still waking
+//! up, a connector the dock has stopped answering for, and a connector this driver blanked itself
+//! all look alike on the wire; telling them apart is what the debounce and repair paths do.
+
+use super::*;
+
+impl VinoDrmData {
+ /// Cache a connector's downstream EDID (read during probe). Bring-up publishes all connectors
+ /// with one hotplug only after both presence and EDID state are complete; firing here exposed
+ /// KWin to a transient no-EDID mode list (including synthetic 1920x1440) before the real EDID
+ /// arrived. Out-of-range connectors are ignored.
+ pub(crate) fn set_edid(&self, connector: usize, blob: KVec<u8>) {
+ let mut edids = self.cached_edids.lock();
+ let Some(slot) = edids.get_mut(connector) else {
+ return;
+ };
+ *slot = Some(blob);
+ }
+ /// Whether userspace has taken responsibility for describing `connector`'s sink, because the
+ /// dock cannot read it.
+ ///
+ /// A DP-to-HDMI converter that mangles or drops DDC leaves the dock unable to read the monitor
+ /// at all: the presence probe reports the socket occupied, but no `id=0x194` ever arrives, so
+ /// the connector stays disconnected and nothing is ever driven. The monitor is real, and its
+ /// EDID is readable from a working port on another machine.
+ ///
+ /// Rather than carry a second EDID source, this hands the connector to DRM's own override,
+ /// which already accepts a blob two ways -- `drm_kms_helper.edid_firmware=<connector>:<path>`
+ /// or a write to the connector's debugfs `edid_override`. The core applies it only to a
+ /// connector that is connected and whose `get_modes` returned none, so all this flag does is
+ /// make the connector report no modes of its own while no EDID has been read.
+ ///
+ /// It deliberately does not report the connector connected. Connecting a modeless connector
+ /// hands fbdev emulation a blank cheque, and its 640x480 default mode set puts this dock into a
+ /// 25 s re-enumeration loop. Userspace supplies the description and then forces the connector
+ /// on (`echo on > .../status`), in that order, which is also what makes the sequence race-free.
+ ///
+ /// An override describes the sink, not the link. A blob claiming a mode the converter cannot
+ /// carry gives a black screen just the same: this substitutes for a broken read, it does not
+ /// negotiate anything.
+ pub(crate) fn edid_from_userspace(&self, connector: usize) -> bool {
+ usize::from(*crate::module_parameters::edid_override.value()) & (1 << connector) != 0
+ }
+ /// Mark a connector connected from CP engagement alone (no raw EDID). Bring-up fires one
+ /// hotplug after every connector's EDID has also been cached, so the compositor never probes
+ /// partial state. Called once the connector's DISPLAY-CAP push confirms monitor presence.
+ pub(crate) fn set_connected(&self, connector: usize) {
+ if connector >= MAX_CONNECTORS {
+ return;
+ }
+ self.connectors_present
+ .fetch_or(1 << connector, core::sync::atomic::Ordering::Release);
+ }
+ /// Clear a connector's presence bit and cached EDID after monitor removal.
+ ///
+ /// `detect()` reports connected when either exists, so both must be cleared together.
+ pub(crate) fn set_disconnected(&self, connector: usize) {
+ let socket = connector + 1;
+ if connector >= MAX_CONNECTORS {
+ return;
+ }
+ // Tagged with the dock's connector count, which is what tells two bound docks apart in a
+ // single log.
+ if self.connectors_present.load(Ordering::Acquire) & (1u32 << connector) != 0 {
+ vino_debug!(
+ "vino: {}-connector dock: socket {socket} presence cleared\n",
+ self.connector_count()
+ );
+ }
+ self.connectors_present
+ .fetch_and(!(1u32 << connector), core::sync::atomic::Ordering::Release);
+ if let Some(slot) = self.cached_edids.lock().get_mut(connector) {
+ *slot = None;
+ }
+ }
+ fn send_reengage_step(
+ &self,
+ io: &BoundInterface<'_>,
+ id: u16,
+ gap_ms: i64,
+ build: impl FnOnce(u16) -> Result<KVec<u8>>,
+ ) -> Result {
+ self.send_cp(io, id, 0, build)?;
+ fsleep(Delta::from_millis(gap_ms));
+ Ok(())
+ }
+ /// Re-run one connector's EDID probe, fetch, engage, and capability query on the live CP link.
+ ///
+ /// Monitor removal tears down the dock's downstream sink, so a replug requires the engage
+ /// messages before a later mode set can start its pixel clock. The cached EDID is cleared on
+ /// removal and must be repopulated here before reporting the connector as present. Returns true
+ /// only when a valid EDID was received.
+ pub(crate) fn reengage_connector(
+ &self,
+ io: &BoundInterface<'_>,
+ connector: u8,
+ ) -> Result<bool> {
+ let socket = connector + 1;
+ self.set_self_blanked(connector as usize, false);
+ // A connector that is not answering is exactly a connector that may be sitting in an open
+ // bracket, where the dock has disengaged its EDID handler and every probe below would go
+ // unanswered. The state is the dock's and it survives a re-enumeration, so a fresh session
+ // cannot know it is owed: assert the closed state rather than infer it. A connector already
+ // closed ignores this.
+ self.close_bracket_before_probe(io, connector);
+ self.edid_target.store(connector as u32, Ordering::Release);
+ *self.edid_caught.lock() = None;
+ let result = (|| -> Result {
+ self.send_reengage_step(io, 0x15, 117, |c| {
+ crate::cp::get_edid_req_sub(c, 0x0020, connector)
+ })?;
+ self.send_reengage_step(io, 0x15, 115, |c| {
+ crate::cp::get_edid_req_sub(c, 0x0020, connector)
+ })?;
+ self.send_reengage_step(io, 0x16, 107, |c| {
+ crate::cp::edid_readiness_kick(c, connector)
+ })?;
+ self.send_reengage_step(io, 0x15, 11, |c| crate::cp::get_edid_req(c, connector))?;
+ self.send_reengage_step(io, 0x16, 118, |c| crate::cp::edid_engage_req(c, connector))?;
+ self.send_reengage_step(io, 0x16, 107, |c| crate::cp::edid_engage_req(c, connector))?;
+ self.send_reengage_step(io, 0x15, 11, |c| crate::cp::post_edid_query(c, connector))
+ })();
+ let caught = self.edid_caught.lock().take();
+ self.edid_target.store(NO_EDID_TARGET, Ordering::Release);
+ result?;
+ match caught.or_else(|| self.drain_for_edid(io)) {
+ Some(blob) => {
+ let n = blob.len();
+ // Say what the EDID claims to be, not just that one arrived. On unfamiliar
+ // hardware this is what distinguishes a real monitor from a block the dock
+ // synthesised for an empty port.
+ if blob.len() >= 12 {
+ let m = u16::from_be_bytes([blob[8], blob[9]]);
+ let vendor = [
+ b'@' + ((m >> 10) & 0x1f) as u8,
+ b'@' + ((m >> 5) & 0x1f) as u8,
+ b'@' + (m & 0x1f) as u8,
+ ];
+ vino_debug!(
+ "vino: socket {socket} EDID {n} B, vendor {}{}{} product {:#06x}\n",
+ vendor[0] as char,
+ vendor[1] as char,
+ vendor[2] as char,
+ u16::from_le_bytes([blob[10], blob[11]])
+ );
+ }
+ self.set_edid(connector as usize, blob);
+ Ok(true)
+ }
+ None => {
+ // Deliberately not published here, even under `edid_override`. A connector that
+ // reports connected with no modes is immediately mode-set by fbdev emulation at
+ // its own 640x480 default, and driving that at the dock resets it -- measured, in
+ // a 25 s re-enumeration loop. The connector stays disconnected until userspace has
+ // supplied the description AND forced the connector on; see `edid_from_userspace`.
+ if self.edid_from_userspace(connector as usize) {
+ pr_warn!(
+ "vino: socket {socket} has no EDID from the dock and is waiting for one from \
+ userspace (edid_override); it stays disconnected until then\n"
+ );
+ return Ok(false);
+ }
+ vino_debug!(
+ "vino: socket {socket} re-engaged but no EDID came back -- no monitor, or it is \
+ not ready yet\n"
+ );
+ Ok(false)
+ }
+ }
+ }
+ /// Drain EP84 looking for the `id=0x194` EDID the fetch above asks for, and return it.
+ ///
+ /// The real EDID only ever arrives as that push (never inside `id=0x4c`/`0x78`), and it can
+ /// land a few messages after the fetch, so this reads a bounded run of replies rather than just
+ /// the next one. Bounded twice over -- attempt count and per-read timeout -- because it runs on
+ /// the keepalive, which must not stall.
+ fn drain_for_edid(&self, dev: &BoundInterface<'_>) -> Option<KVec<u8>> {
+ self.check_cp_session().ok()?;
+ let mut guard = self.cp_link.lock();
+ let link = (&mut *guard).as_mut()?;
+ let mut reply = KVec::from_elem(0u8, 4096, GFP_KERNEL).ok()?;
+ for _ in 0..24 {
+ let got = match link.ep84_q.as_mut() {
+ Some(q) => match q.recv(dev.io(), &mut reply, Delta::from_millis(8)) {
+ Ok(Some(n)) if n > 16 => n,
+ _ => continue,
+ },
+ None => match dev.ctrl_recv(&mut reply, Delta::from_millis(8), GFP_KERNEL) {
+ Ok(n) if n > 16 => n,
+ _ => continue,
+ },
+ };
+ if let Ok(Some(blob)) =
+ crate::cp::parse_edid_from_reply(&link.ks, &link.riv, &reply[..got])
+ {
+ return Some(blob);
+ }
+ }
+ None
+ }
+ /// Stage 2 (runtime monitor hotplug): probe one physical connector.
+ ///
+ /// Navarro multiplexes two connectors per bulk endpoint, but its EDID selector and stream
+ /// record subfield are still per socket. Never collapse sockets 0/2 or 1/3 here: doing so
+ /// turns two independently connected monitors into one KMS connector.
+ pub(crate) fn probe_connector_present(
+ &self,
+ dev: &BoundInterface<'_>,
+ connector: u8,
+ ) -> Option<bool> {
+ if usize::from(connector) >= self.connector_count() {
+ return Some(false);
+ }
+ self.send_presence_probe(dev, connector, connector)
+ }
+ /// Probe one downstream connector. `connector` selects its own presence-change cell.
+ ///
+ /// Sends the EDID probe (`id=0x15 sub=0x20`, byte22 = connector selector -- the same selector
+ /// that unblocked the whole EDID path) and decodes the dock's sealed `0x45` reply. Returns
+ /// `Some(true/false)` on a decodable reply, `None` if CP is down or nothing decoded. Reuses the
+ /// live session `ks/riv/counter` exactly like `send_cp`, so it stays in CP lockstep.
+ fn send_presence_probe(
+ &self,
+ dev: &BoundInterface<'_>,
+ sel: u8,
+ connector: u8,
+ ) -> Option<bool> {
+ let socket = connector + 1;
+ self.check_cp_session().ok()?;
+ let mut guard = self.cp_link.lock();
+ let link = (&mut *guard).as_mut()?;
+ let request_counter = link.counter;
+ let msg = crate::cp::get_edid_req_sub(request_counter, 0x0020, sel).ok()?;
+ let frame =
+ crate::cp::seal_interactive(&link.ks, &link.riv, 0x15, link.wire_seq, &msg).ok()?;
+ if dev.ctrl_send(&frame, crate::timeout(), GFP_KERNEL).is_err() {
+ let silent_ms = self.cp_silent_for_ms();
+ if silent_ms >= self.cp_silence_limit_ms() {
+ self.abandon_cp_session(silent_ms);
+ *guard = None;
+ }
+ return None;
+ }
+ link.wire_seq = link.wire_seq.wrapping_add(((msg.len() + 15) / 16) as u32);
+ link.counter = link.counter.wrapping_add(1);
+ // Take the reply that answers this probe, not simply the next frame on EP84: the connectors
+ // are probed back to back, so a late reply or an unprompted push would otherwise be
+ // attributed to the wrong connector. The inner counter echoes the request. A round that
+ // never sees its own echo returns `None`, which the caller treats as "this poll learned
+ // nothing" rather than as an unplug.
+ let mut reply = KVec::from_elem(0u8, 4096, GFP_KERNEL).ok()?;
+ let deadline = Instant::<Monotonic>::now() + Delta::from_millis(64);
+ let got = loop {
+ let n = match link.ep84_q.as_mut() {
+ Some(q) => match q.recv(dev.io(), &mut reply, crate::cp_reply_timeout()) {
+ Ok(Some(n)) => n,
+ Ok(None) => 0,
+ Err(_) => return None,
+ },
+ None => dev
+ .ctrl_recv(&mut reply, crate::cp_reply_timeout(), GFP_KERNEL)
+ .unwrap_or(0),
+ };
+ if n > 16 {
+ match crate::cp::decode_in_lenient(&link.ks, &link.riv, &reply[..n]) {
+ Some((_, _, echoed)) if echoed == request_counter => break n,
+ // Undecodable frames are the dock's asynchronous pushes; keep draining.
+ _ => {}
+ }
+ }
+ if (Instant::<Monotonic>::now() - deadline).as_millis() >= 0 {
+ return None;
+ }
+ };
+ // Decode the downstream status at inner bytes 22..26 as well as the handler ID.
+ let (id, status, ready) =
+ crate::cp::probe_reply_status(&link.ks, &link.riv, &reply[..got])?;
+ self.note_cp_reply();
+ // Presence is bit 0x10 of inner byte 23, which lands in bits 8..15 of the status word:
+ // `05 11 27 00` for an occupied connector, `05 01 <20|21|60|61> 00` for an empty one.
+ // Which handler answered says nothing about it -- both docks reply `id=0x44` either way.
+ let present = status & 0x0000_1000 != 0;
+ // One line per *changed* answer per connector, so a steady link is silent and an unplug is
+ // unmissable. Both fields are packed into the same cell: the id alone cannot distinguish a
+ // dock that keeps saying `0x44` from one whose downstream state has moved underneath it.
+ let cell = ((id as u32) << 16) | (status & 0xffff);
+ let prev = self.presence_reply[connector as usize].swap(cell, Ordering::Relaxed);
+ if prev != cell {
+ // The dock moves the *other* connector's status word too when a sink appears or
+ // disappears, and it does so sooner than it pushes anything. `prev == 0` is this
+ // connector's first ever reply, which is bring-up, not an event.
+ if prev != 0 {
+ self.downstream_event.store(true, Ordering::Release);
+ }
+ // The decoded answer itself, not just the verdict derived from it. Without this a
+ // presence flap can only be read as "monitor disconnected", which says nothing about
+ // whether the dock changed its mind or vino changed the question. It is one line per
+ // *changed* reply per connector, so a steady link prints nothing at all.
+ // Tagged with this dock's video endpoints, which name the family uniquely: 08/0b is
+ // a DL-6xxx, 08/0a a DL-7400, 02/02 a DL-3x00. A connector count does not -- two of
+ // the three have two connectors, so once both are bound the tag distinguishes
+ // nothing and the same reading reads as either dock.
+ vino_debug!(
+ "vino: [video {:02x}/{:02x}] socket {socket} presence reply id={id:#06x} \
+ status={status:#010x} -> present={present} ready={ready} \
+ (was id={:#06x} status={:#06x})\n",
+ dev.endpoints.video[0].address(),
+ dev.endpoints.video[1].address(),
+ prev >> 16,
+ prev & 0xffff
+ );
+ }
+ Some(present)
+ }
+ /// Whether vino itself took `connector`'s sink down, so the presence watcher can tell its own
+ /// blank apart from a real unplug. See [`VinoDrmData::self_blanked`].
+ pub(crate) fn is_self_blanked(&self, connector: usize) -> bool {
+ self.self_blanked.load(Ordering::Acquire) & (1u32 << connector) != 0
+ }
+ pub(crate) fn set_self_blanked(&self, connector: usize, on: bool) {
+ if on {
+ self.self_blanked
+ .fetch_or(1u32 << connector, Ordering::Release);
+ } else {
+ self.self_blanked
+ .fetch_and(!(1u32 << connector), Ordering::Release);
+ }
+ }
+ /// How long this connector should hold the post-mode-set training cadence.
+ ///
+ /// A cold activation needs it: the dock will not program its downstream pixel clock without a
+ /// sustained stream. A repair does not -- the link is already trained and the sink was dropped
+ /// underneath us -- and running it there costs the dock three seconds of full keyframes at
+ /// `FRAME_PERIOD_MS`. Consumes the repair flag, so the window returns for the next real
+ /// bring-up.
+ pub(crate) fn sustain_window(&self, connector: usize) -> Option<Instant<Monotonic>> {
+ let bit = 1u32 << connector;
+ if self.repair_connectors.fetch_and(!bit, Ordering::AcqRel) & bit != 0 {
+ return None;
+ }
+ // A dock whose video shares the control pipe cannot be given this window. It exists to
+ // train a downstream link by presenting keyframes at frame cadence, which on a pipe of its
+ // own is bandwidth well spent; here it is bandwidth taken directly from the control plane,
+ // and the dock stops answering EP84 entirely rather than merely dropping frames.
+ if self.video_on_ctrl_pipe() {
+ return None;
+ }
+ Some(Instant::<Monotonic>::now() + Delta::from_millis(SUSTAIN_MS))
+ }
+ /// Re-drive every lit connector after the dock dropped a downstream sink underneath us.
+ ///
+ /// The presence flap is the dock really taking a sink down, and the only thing that used to
+ /// repair it was letting the DRM connector disappear so the compositor would re-enable the
+ /// output. That cure was worse than the disease: it re-lays-out userspace, and the mode set it
+ /// produces names one connector while its sibling is lit, which re-enumerates the dock. So vino
+ /// repairs the connector itself and leaves the connector alone.
+ ///
+ /// Every lit connector is re-queued, not just the one that flapped, and they go into one batch:
+ /// a mode set this dock accepts is one that names every connector at once
+ /// (`activate_dual_wake`). Zeroing the mode generation is what makes that path take them.
+ ///
+ /// Returns the number of connectors queued.
+ #[expect(
+ dead_code,
+ reason = "kept for the flap-repair experiment; see its doc comment"
+ )]
+ pub(super) fn repair_flapped_connector(&self, dev: &VinoDrmDevice, flapped: usize) -> u32 {
+ let mut queued = 0u32;
+ let mut cmds: [Option<crate::cp::Timing>; MAX_CONNECTORS] = [None; MAX_CONNECTORS];
+ for connector in 0..MAX_CONNECTORS {
+ let active = self.modeset_active[connector].load(Ordering::Acquire);
+ if active == 0 || self.modeset_requested[connector].load(Ordering::Acquire) != active {
+ continue;
+ }
+ let Some(timing) = self.last_timing.lock()[connector] else {
+ continue;
+ };
+ if timing_key(&timing) != active {
+ continue;
+ }
+ cmds[connector] = Some(timing);
+ }
+ if cmds.iter().flatten().count() == 0 {
+ return 0;
+ }
+ for connector in 0..MAX_CONNECTORS {
+ let Some(timing) = cmds[connector] else {
+ continue;
+ };
+ // The dock's copy of this connector is gone, so nothing it holds can be diffed against.
+ self.repair_connectors
+ .fetch_or(1u32 << connector, Ordering::Release);
+ self.modeset_active[connector].store(0, Ordering::Release);
+ self.owe_keyframe(connector);
+ self.strip_hashes.lock()[connector] = None;
+ self.dirty_ttl.lock()[connector] = None;
+ self.queue_cmd(
+ dev,
+ KmsCmd::ModeSet {
+ connector: connector as u8,
+ timing,
+ },
+ );
+ queued += 1;
+ }
+ pr_warn!("vino: connector {flapped} sink flap -- re-driving {queued} lit connector(s) together\n");
+ queued
+ }
+ /// Whether connector `connector`'s presence bit is currently set (for the keepalive to seed its
+ /// baseline before watching for runtime connect/remove transitions). Whether a monitor has
+ /// described itself on this socket.
+ ///
+ /// The dock recovers an EDID for a socket with something plugged into it and nothing at all for
+ /// an empty one, which on a family that cannot report downstream presence is the only presence
+ /// signal there is.
+ pub(crate) fn connector_has_edid(&self, connector: usize) -> bool {
+ self.cached_edids
+ .lock()
+ .get(connector)
+ .is_some_and(Option::is_some)
+ }
+ pub(crate) fn connector_present(&self, connector: usize) -> bool {
+ connector < MAX_CONNECTORS
+ && self
+ .connectors_present
+ .load(core::sync::atomic::Ordering::Acquire)
+ & (1u32 << connector)
+ != 0
+ }
+}
diff --git a/drivers/gpu/drm/vino/drm_sink/scanout.rs b/drivers/gpu/drm/vino/drm_sink/scanout.rs
new file mode 100644
index 000000000000..2edd29cffaf6
--- /dev/null
+++ b/drivers/gpu/drm/vino/drm_sink/scanout.rs
@@ -0,0 +1,1958 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Turning a committed framebuffer into bytes on a video endpoint.
+//!
+//! The compositor's atomic callback does no work beyond snapshotting: everything below runs on the
+//! deferred scanout worker. In order, a frame goes through damage selection against the previous
+//! frame's per-strip hashes, encoding (fanned across CPUs as [`EncodeChunk`]s), record framing, and
+//! submission through the connector's persistent URB queue.
+
+use super::mode_objects::rot_src;
+use super::*;
+
+/// Consecutive video stalls cleared before a connector is parked for a fresh mode-set.
+const VIDEO_STALL_LIMIT: u64 = 4;
+
+/// Compress and submit one coalesced primary-plane flip on the deferred worker. Keeping all slow
+/// work here makes the DRM atomic callback bounded to state inspection plus an `ARef` increment.
+pub(super) fn run_pending_scanout(
+ dev: &BoundInterface<'_>,
+ data: &VinoDrmData,
+ frame: PendingScanout,
+) {
+ use core::sync::atomic::Ordering::Relaxed;
+
+ let connector_index = frame.connector as usize;
+ if data.modeset_requested[connector_index].load(Ordering::Acquire) == 0 {
+ scanout_gate(
+ frame.connector,
+ "worker: connector has no mode-set requested",
+ );
+ return;
+ }
+ let requested_geometry_matches = data.last_timing.lock()[connector_index]
+ .is_some_and(|t| t.hactive as usize == frame.w && t.vactive as usize == frame.h);
+ if !requested_geometry_matches {
+ // stale framebuffer from a different-size mode generation
+ scanout_gate(
+ frame.connector,
+ "worker: framebuffer size differs from the cached mode",
+ );
+ return;
+ }
+ // Was this the mode-set's owed keyframe? Read before sending, since a successful send clears
+ // the bit.
+ let was_keyframe =
+ data.keyframe_pending.load(Ordering::Acquire) & (1u32 << frame.connector) != 0;
+ let settle_copy = was_keyframe.then(|| frame.clone());
+ let slot = frame.shadow_idx;
+ let generation = frame.shadow_generation;
+ let (source_w, source_h) = src_dims(frame.rotation, frame.w, frame.h);
+ let shadow = {
+ let mut pool = data.shadow[connector_index].lock();
+ // Split the validation so the counters say *which* invariant failed.
+ let in_range = slot < SHADOW_SLOTS;
+ let not_inflight = pool.inflight.is_none();
+ let gen_ok = in_range && pool.slots[slot].generation == generation;
+ let dims_ok = in_range
+ && pool.slots[slot]
+ .surface
+ .as_ref()
+ .is_some_and(|surface| surface.w == source_w && surface.h == source_h);
+ if !not_inflight {
+ scanout_gate(frame.connector, "slot busy: another encode inflight");
+ } else if !gen_ok {
+ scanout_gate(frame.connector, "slot generation moved under us");
+ } else if !dims_ok {
+ scanout_gate(frame.connector, "slot surface missing or wrong size");
+ }
+ let valid = in_range && not_inflight && gen_ok && dims_ok;
+ if !valid {
+ None
+ } else {
+ pool.inflight = Some(slot);
+ pool.slots[slot].surface.take()
+ }
+ };
+ let Some(shadow) = shadow else {
+ scanout_gate(
+ frame.connector,
+ "worker: committed surface is no longer available",
+ );
+ return;
+ };
+
+ // `pixels` and `hashes` are lent to the encoder and moved back below; the band is scratch that
+ // the encoder has no use for, so it just waits here to be reunited with them.
+ let ShadowSurface {
+ w: source_w,
+ h: source_h,
+ pixels,
+ hashes,
+ band,
+ } = shadow;
+ let color = data.color_snapshot(connector_index);
+ let direct = direct_pixel_map(frame.rotation, &color, source_w, source_h, frame.w, frame.h);
+ let src = match Arc::new(
+ PixelSource {
+ pixels,
+ pitch: source_w * 4,
+ w: source_w,
+ h: source_h,
+ output_w: frame.w,
+ output_h: frame.h,
+ rotation: frame.rotation,
+ color,
+ direct,
+ hashes,
+ depth: data.geometry_for_connector(frame.connector).depth(),
+ source_depth: data.connector_buffer_depth(frame.connector),
+ },
+ GFP_KERNEL,
+ ) {
+ Ok(src) => src,
+ Err(_) => {
+ let mut pool = data.shadow[connector_index].lock();
+ if pool.inflight == Some(slot) {
+ pool.inflight = None;
+ }
+ scanout_gate(frame.connector, "worker: pixel source allocation failed");
+ return;
+ }
+ };
+ let result = encode_and_send(
+ dev,
+ data,
+ frame.connector,
+ &src,
+ frame.rotation,
+ &frame.clips[..frame.nclips],
+ frame.w,
+ frame.h,
+ );
+ data.last_frame.lock()[connector_index] = Some(Instant::<Monotonic>::now());
+ let returned = Arc::into_unique_or_drop(src).map(|src| {
+ let mut src = core::pin::Pin::into_inner(src);
+ ShadowSurface {
+ w: source_w,
+ h: source_h,
+ pixels: core::mem::replace(&mut src.pixels, KVVec::new()),
+ hashes: core::mem::replace(&mut src.hashes, KVVec::new()),
+ band,
+ }
+ });
+ {
+ let mut pool = data.shadow[connector_index].lock();
+ if pool.inflight == Some(slot) {
+ if pool.slots[slot].generation == generation && pool.slots[slot].surface.is_none() {
+ pool.slots[slot].surface = returned;
+ }
+ pool.inflight = None;
+ }
+ }
+ match result {
+ Ok(()) => {
+ let n = data.scanout_fails[connector_index].swap(0, Relaxed);
+ data.scanout_skip[connector_index].store(0, Relaxed);
+ if n > 0 {
+ pr_info!("vino: connector {connector_index} scanout recovered after {n} failed frame(s)\n");
+ }
+ // Arm the one-shot settle repaint. A compositor that goes idle right after enabling an
+ // output can otherwise remain on the initial keyframe indefinitely.
+ if let Some(mut copy) = settle_copy {
+ copy.clips[0] = (0, 0, copy.w, copy.h);
+ copy.nclips = 1;
+ // During the post-mode-set training window, repaint at frame cadence so the dock
+ // receives the sustained stream needed to program the downstream pixel clock.
+ // Outside that window, use the bounded settle repaint.
+ let sustaining = data.sustain_until.lock()[connector_index]
+ .is_some_and(|until| (until - Instant::<Monotonic>::now()).as_millis() > 0);
+ // Training repaints at the fast cadence and is exempt from the budget; everything
+ // else charges one settle repaint against this connector's keyframe obligation and
+ // stops when it runs out. See `SETTLE_REPAINTS` for the unbounded keyframe loop
+ // that made a static desktop stream ~2.7 MB/s per connector. A dock that tears the
+ // link down over a silent video endpoint has to be re-fed whether or not anything
+ // changed, so its repaint is periodic and unbudgeted.
+ let keepalive = data.video_keepalive();
+ let unbudgeted = sustaining || keepalive;
+ let charged = unbudgeted
+ || data.settle_budget[connector_index]
+ .fetch_update(Relaxed, Relaxed, |b| b.checked_sub(1))
+ .is_ok();
+ if charged {
+ let delay = if sustaining {
+ data.frame_period_ms()
+ } else if keepalive {
+ NAVARRO_KEEPALIVE_MS
+ } else {
+ SETTLE_REPAINT_MS
+ };
+ // Only a repaint that exists to put bytes on the wire has to be a keyframe:
+ // training programs the downstream clock with them, and a keepalive dock tears
+ // its link down over a silent endpoint, so both need a frame whether or not
+ // anything changed. The ordinary settle repaint is there to replace whatever
+ // the compositor happened to have mapped when the mode set went out, and a
+ // keyframe has since reached every dock buffer -- so what it owes is the
+ // difference, which on a desktop that did not change is nothing at all.
+ // Sending a second full surface instead costs `dock_buffers` presentations of
+ // it, and on a dock that shares its control pipe that is the transfer the dock
+ // stops accepting.
+ let as_keyframe = sustaining || keepalive;
+ data.settle_repaint.lock()[connector_index] = Some((
+ Instant::<Monotonic>::now() + Delta::from_millis(delay),
+ copy,
+ as_keyframe,
+ ));
+ }
+ } else {
+ // Repaint the same framebuffer while strips still have retransmit debt. This is a
+ // delta, not a keyframe, and terminates after at most the profile's
+ // `damage_frames` accepted submissions because each pass decrements the ledger.
+ let owes = data.dirty_ttl.lock()[connector_index]
+ .as_ref()
+ .is_some_and(|debt| debt.iter().any(|&d| d > 0));
+ if owes {
+ data.settle_repaint.lock()[connector_index] = Some((
+ Instant::<Monotonic>::now() + Delta::from_millis(data.frame_period_ms()),
+ frame.clone(),
+ false,
+ ));
+ }
+ }
+ }
+ Err(e) => {
+ // Log at exponentially sparser points and back off future worker attempts. An error is
+ // transport state, not a reason to stall the compositor's pageflip path.
+ let n = data.scanout_fails[connector_index].fetch_add(1, Relaxed) + 1;
+ if n == 1 || n.is_power_of_two() {
+ pr_err!("vino: connector {connector_index} scanout frame failed ({e:?}) [x{n}] -- throttling\n");
+ }
+ data.scanout_skip[connector_index].store(core::cmp::min(n, 120), Relaxed);
+ // The failed queue was synchronously retired at the exact `q.send()` error site while
+ // that physical pipe was still owned. Give up after a few consecutive stalls and wait
+ // for the next mode set rather than repeatedly driving a pipe the dock is refusing.
+ if e == EPIPE || e == EPROTO {
+ if n == VIDEO_STALL_LIMIT + 1 {
+ pr_err!(
+ "vino: connector {connector_index} stalled {n} times; parking it until the next mode set\n"
+ );
+ data.modeset_active[connector_index].store(0, Ordering::Release);
+ data.programmed_timing.lock()[connector_index] = None;
+ }
+ }
+ }
+ }
+}
+
+/// Copy a whole cursor framebuffer for the dock.
+///
+/// The dock takes DRM `ARGB8888` unchanged; [`crate::cp::cursor_image`] owns the wire placement.
+/// The complete bitmap is sent every time rather than the helper's clipped rectangle: the dock is
+/// configured for a fixed cursor size (`mode_config.cursor_width/height`) and clips at the panel
+/// edge itself.
+pub(super) fn read_cursor_bgra(
+ fb: &kms::framebuffer::Framebuffer<VinoDrmDriver>,
+ w: usize,
+ h: usize,
+) -> Result<KVec<u8>> {
+ let vmap = fb.vmap::<VinoObject>()?;
+ let view = vmap.view();
+ let pitch = vmap.pitch();
+ let row = w.checked_mul(4).ok_or(EINVAL)?;
+ let len = row.checked_mul(h).ok_or(EINVAL)?;
+ let mut out = KVec::new();
+ out.resize(len, 0, GFP_KERNEL)?;
+ for dy in 0..h {
+ let src = dy.checked_mul(pitch).ok_or(EINVAL)?;
+ view.try_copy_to_slice(src, &mut out[dy * row..(dy + 1) * row])?;
+ }
+ Ok(out)
+}
+
+/// Source (framebuffer) dimensions for an output of `ow`x`oh` pixels under plane `rotation`.
+/// The 90/270 rotations swap width and height between the framebuffer and the displayed output;
+/// the others preserve them.
+pub(super) fn src_dims(rotation: plane::Rotation, ow: usize, oh: usize) -> (usize, usize) {
+ if matches!(
+ rotation.angle(),
+ plane::Rotation::ROTATE_90 | plane::Rotation::ROTATE_270
+ ) {
+ (oh, ow)
+ } else {
+ (ow, oh)
+ }
+}
+
+/// Copy a committed framebuffer into this connector's [`ShadowSurface`], reusing the existing
+/// allocation whenever the geometry is unchanged.
+///
+/// Runs in the atomic commit path, so everything else (damage selection, rotation, gamma, the
+/// codec) stays in the worker and reads this private surface instead of the compositor's live
+/// buffer.
+///
+/// The traversal is band-major: for each row of strips, the source's rows are pulled into
+/// [`ShadowSurface::band`] a full row per read, and only then are that band's strips hashed and --
+/// where the hash moved -- copied on into `pixels`. The obvious strip-major order costs far more
+/// for the same result, because a strip is 64 px wide but the source row is `pitch` bytes apart: it
+/// reads the source in `STRIP_W * 4`-byte fragments (57,600 of them per 1440p frame, versus 1,440
+/// full-row reads here), it walks those fragments against the row stride rather than sequentially,
+/// and it has to read a changed strip out of the source a second time to copy it, because the first
+/// read went to a fragment-sized scratch that could not be kept.
+///
+/// Strips whose hash is unchanged are still not written, so an idle desktop moves no more memory
+/// than the strip-major order would, and a busy one reads the source once.
+#[inline(never)]
+pub(super) fn snapshot_to_shadow(
+ geometry: crate::video::haar::Geometry,
+ slot: &mut Option<ShadowSurface>,
+ source: &kms::framebuffer::FramebufferVMapOwned<VinoObject>,
+ w: usize,
+ h: usize,
+) -> Result {
+ if w == 0 || h == 0 {
+ return Err(EINVAL);
+ }
+ let row = w.checked_mul(4).ok_or(EINVAL)?;
+ let need = row.checked_mul(h).ok_or(EINVAL)?;
+ // GEM dumb buffers pad the pitch, so the source stride is not necessarily `w * 4`.
+ let pitch = source.pitch();
+ let view = source.view();
+
+ let (sw, sh) = (geometry.strip_w(), geometry.strip_h());
+ let padded_width = (w + sw - 1) & !(sw - 1);
+ let padded_height = (h + sh - 1) & !(sh - 1);
+ let tiles_x = padded_width / sw;
+ let tiles_y = padded_height / sh;
+
+ // A freshly allocated surface holds zeros, not the previous frame, so nothing in it may be
+ // treated as already up to date however its stored hashes compare.
+ let band_len = sh.checked_mul(row).ok_or(EINVAL)?;
+ let mut fresh = false;
+ if !matches!(slot, Some(s) if s.w == w && s.h == h) {
+ let mut pixels: KVVec<u8> = KVVec::new();
+ pixels.resize(need, 0, GFP_KERNEL)?;
+ let mut hashes: KVVec<u64> = KVVec::new();
+ hashes.resize(tiles_x * tiles_y, 0, GFP_KERNEL)?;
+ let mut band: KVVec<u8> = KVVec::new();
+ band.resize(band_len, 0, GFP_KERNEL)?;
+ *slot = Some(ShadowSurface {
+ w,
+ h,
+ pixels,
+ hashes,
+ band,
+ });
+ fresh = true;
+ }
+ let shadow = slot.as_mut().ok_or(kernel::error::code::ENOMEM)?;
+ if shadow.hashes.len() != tiles_x * tiles_y {
+ shadow.hashes.resize(tiles_x * tiles_y, 0, GFP_KERNEL)?;
+ }
+ if shadow.band.len() != band_len {
+ shadow.band.resize(band_len, 0, GFP_KERNEL)?;
+ }
+
+ // Borrow the three buffers as disjoint fields: the band is read while `pixels` is written.
+ let ShadowSurface {
+ pixels,
+ hashes,
+ band,
+ ..
+ } = shadow;
+ for ty in 0..tiles_y {
+ let sy = ty * sh;
+ let y_end = (sy + sh).min(h);
+ // The final band is short whenever the height is not a whole number of strips.
+ let rows = y_end - sy;
+ // Pull the band out of the source through the checked I/O view, one full row per read.
+ for dy in 0..rows {
+ let dst = &mut band[dy * row..dy * row + row];
+ view.try_copy_to_slice((sy + dy) * pitch, dst)?;
+ }
+ for tx in 0..tiles_x {
+ let sx = tx * sw;
+ let x_end = (sx + sw).min(w);
+ let seed = 0x9e37_79b1_85eb_ca87u64
+ ^ (sx as u64).rotate_left(17)
+ ^ (sy as u64).rotate_left(43);
+ let mut hasher = xxhash::Xxh64::new(seed);
+ let bytes = (x_end - sx) * 4;
+ // Hash exactly the bytes, in the order, that the strip-major traversal did, so a
+ // surface's stored hashes stay comparable across this change.
+ if sx < x_end {
+ for dy in 0..rows {
+ let off = dy * row + sx * 4;
+ hasher.update(&band[off..off + bytes])?;
+ }
+ }
+ let hash = hasher.digest();
+ let idx = ty * tiles_x + tx;
+ if sx < x_end && (fresh || hashes[idx] != hash) {
+ for dy in 0..rows {
+ let src = dy * row + sx * 4;
+ let dst = (sy + dy) * row + sx * 4;
+ pixels[dst..dst + bytes].copy_from_slice(&band[src..src + bytes]);
+ }
+ }
+ hashes[idx] = hash;
+ }
+ }
+ Ok(())
+}
+
+/// Convert changed strip hashes into a compact set of damage rectangles.
+///
+/// The rectangles are built on the macro-tile grid, not the strip grid, because a touched
+/// macro-tile is resent whole either way -- see `Geometry::macro_w`. Describing damage at a
+/// granularity finer than it is transmitted at costs nothing and fragments the list by up to the
+/// sixteen strips a macro-tile holds, which is enough for a handful of scattered updates to
+/// exhaust the rectangle ceiling and take the whole surface with them.
+///
+/// Horizontal runs are joined, then equal runs on adjacent macro-rows are extended vertically. A
+/// frame still fragmented past the ceiling falls back to one full-output rectangle rather than
+/// growing an unbounded allocation or spending more time testing rectangles than encoding strips.
+#[inline(never)]
+pub(crate) fn changed_strip_rects(
+ geometry: crate::video::haar::Geometry,
+ old: &[u64],
+ new: &[u64],
+ padded_width: usize,
+ padded_height: usize,
+) -> Result<KVec<DamageRect>> {
+ const MAX_RECTS: usize = 128;
+ let tiles_x = padded_width >> geometry.strip_w_shift();
+ let tiles_y = padded_height >> geometry.strip_h_shift();
+ if old.len() != tiles_x * tiles_y || new.len() != old.len() {
+ return Err(EINVAL);
+ }
+ let (mw, mh) = (geometry.macro_w(), geometry.macro_h());
+ let per_x = mw / geometry.strip_w();
+ let per_y = mh / geometry.strip_h();
+ let macros_x = tiles_x.div_ceil(per_x);
+ let macros_y = tiles_y.div_ceil(per_y);
+ // The macro-tile grid as a changed/unchanged bitmap. One byte a tile: a 4K surface is 544 of
+ // them, so the map is cheaper than the rectangle list it replaces.
+ let mut touched: KVVec<u8> = KVVec::new();
+ touched.resize(macros_x * macros_y, 0, GFP_KERNEL)?;
+ for ty in 0..tiles_y {
+ for tx in 0..tiles_x {
+ if old[ty * tiles_x + tx] != new[ty * tiles_x + tx] {
+ touched[(ty / per_y) * macros_x + tx / per_x] = 1;
+ }
+ }
+ }
+ let mut rects: KVec<DamageRect> = KVec::new();
+ for my in 0..macros_y {
+ let mut mx = 0usize;
+ while mx < macros_x {
+ if touched[my * macros_x + mx] == 0 {
+ mx += 1;
+ continue;
+ }
+ let run_start = mx;
+ while mx < macros_x && touched[my * macros_x + mx] != 0 {
+ mx += 1;
+ }
+ let x0 = run_start * mw;
+ let x1 = (mx * mw).min(padded_width);
+ let y0 = my * mh;
+ let y1 = (y0 + mh).min(padded_height);
+ let mut merged = false;
+ for prior in rects.iter_mut().rev() {
+ if prior.0 == x0 && prior.2 == x1 && prior.3 == y0 {
+ prior.3 = y1;
+ merged = true;
+ break;
+ }
+ }
+ if !merged {
+ if rects.len() == MAX_RECTS {
+ let mut full: KVec<DamageRect> = KVec::new();
+ full.push((0, 0, padded_width, padded_height), GFP_KERNEL)?;
+ return Ok(full);
+ }
+ rects.push((x0, y0, x1, y1), GFP_KERNEL)?;
+ }
+ }
+ }
+ Ok(rects)
+}
+
+/// Maximum packet size of a SuperSpeed bulk endpoint.
+///
+/// Only its role as a divisor matters here: a transfer that is a whole number of these ends on a
+/// full packet and so terminates nothing. A device running below SuperSpeed uses a smaller value,
+/// which divides this one, so a transfer short by this measure is short by that one too.
+const BULK_MAX_PACKET: usize = 1024;
+
+/// How much of a frame's last transfer is split off behind it so the frame ends short.
+///
+/// Any value that is neither zero nor a multiple of [`BULK_MAX_PACKET`] works; a record stride is
+/// a multiple of sixteen, so this keeps the split on a stride boundary where a record allows one.
+const FRAME_TAIL_BYTES: usize = 16;
+
+/// Hard ceiling on work items per frame, purely to bound per-frame allocation.
+///
+/// Each chunk owns synchronization and a coordinate list, so the count must be
+/// bounded. At `ENCODE_MIN_STRIPS_PER_CHUNK`, a full 1440p frame needs about
+/// 112 chunks.
+const ENCODE_MAX_WORK_ITEMS: usize = 256;
+
+/// Fewest strips per chunk worth dispatching. Below this the allocation, enqueue and completion
+/// cost more than the strips themselves; a small delta stays on the serial path.
+const ENCODE_MIN_STRIPS_PER_CHUNK: usize = 32;
+
+/// Immutable driver-owned pixel source shared by parallel encode workers.
+pub(super) struct PixelSource {
+ pixels: KVVec<u8>,
+ pitch: usize,
+ /// Dimensions of the untransformed framebuffer snapshot.
+ w: usize,
+ h: usize,
+ /// Dimensions and transform of the image presented to the dock.
+ output_w: usize,
+ output_h: usize,
+ rotation: plane::Rotation,
+ color: Option<crate::color::ColorPipeline>,
+ /// True when an output pixel is the source pixel at the same coordinates: identity rotation, no
+ /// gamma table, and output dimensions equal to the snapshot's.
+ ///
+ /// Fullscreen video makes `px` the third-hottest symbol in the kernel (13.8% of the machine
+ /// on a 4K clip), because every one of the ~3.7 M pixels per frame pays a `rot_src` match and
+ /// a gamma branch that are constant for the whole frame. Deciding once per frame lets the
+ /// common case read straight out of the snapshot.
+ direct: bool,
+ /// Strip hashes computed during the snapshot -- see [`ShadowSurface::hashes`]. Carried through
+ /// so the encoder does not re-read the whole surface just to re-derive them.
+ hashes: KVVec<u64>,
+ /// Bits per channel of the snapshot's pixels, and therefore of the frame the codec produces.
+ ///
+ /// Both layouts this handles are four bytes per pixel, so the snapshot copy above is
+ /// depth-agnostic and only the unpack here differs: `XRGB8888` is three bytes in the low 24
+ /// bits, `XRGB2101010` three 10-bit fields in the low 30.
+ depth: crate::video::haar::Depth,
+ /// Bits per channel of the framebuffer itself, which decides how a pixel is decoded.
+ ///
+ /// Equal to `depth` except when userspace asked for a deeper link than the surface it handed
+ /// over, which is the ordinary way a compositor drives a ten-bit link from an eight-bit
+ /// desktop. Samples are widened after decoding so the codec always sees `depth`.
+ source_depth: crate::video::haar::Depth,
+}
+
+/// Whether the encoder can read output pixels straight out of the snapshot. See
+/// [`PixelSource::direct`].
+fn direct_pixel_map(
+ rotation: plane::Rotation,
+ color: &Option<crate::color::ColorPipeline>,
+ w: usize,
+ h: usize,
+ output_w: usize,
+ output_h: usize,
+) -> bool {
+ color.is_none() && rotation == plane::Rotation::ROTATE_0 && output_w == w && output_h == h
+}
+
+/// Widen an eight-bit sample to ten bits.
+///
+/// Replicating the top two bits into the low ones keeps both endpoints exact: black stays zero and
+/// full white stays full white, where a plain shift would land three codes short of it and tint
+/// every highlight.
+#[inline]
+fn widen_8_to_10(v: u16) -> u16 {
+ (v << 2) | (v >> 6)
+}
+
+impl PixelSource {
+ /// Split one packed 32-bit pixel into channels at this source's depth.
+ #[inline]
+ fn unpack(&self, p: u32) -> (u16, u16, u16) {
+ let (r, g, b) = self.unpack_source(p);
+ self.widen(r, g, b)
+ }
+
+ /// Widen a decoded sample from the framebuffer's depth to the link's.
+ #[inline]
+ fn widen(&self, r: u16, g: u16, b: u16) -> (u16, u16, u16) {
+ match (self.source_depth, self.depth) {
+ (crate::video::haar::Depth::Eight, crate::video::haar::Depth::Ten) => {
+ (widen_8_to_10(r), widen_8_to_10(g), widen_8_to_10(b))
+ }
+ _ => (r, g, b),
+ }
+ }
+
+ /// Split one packed 32-bit pixel into channels at the framebuffer's own depth.
+ #[inline]
+ fn unpack_source(&self, p: u32) -> (u16, u16, u16) {
+ match self.source_depth {
+ // Little-endian XRGB8888.
+ crate::video::haar::Depth::Eight => (
+ ((p >> 16) & 0xff) as u16,
+ ((p >> 8) & 0xff) as u16,
+ (p & 0xff) as u16,
+ ),
+ // XRGB2101010: two ignored bits, then R, G, B ten bits each.
+ crate::video::haar::Depth::Ten => (
+ ((p >> 20) & 0x3ff) as u16,
+ ((p >> 10) & 0x3ff) as u16,
+ (p & 0x3ff) as u16,
+ ),
+ }
+ }
+
+ /// Read one gamma-corrected pixel in untransformed framebuffer coordinates.
+ #[inline]
+ fn source_px(&self, sx: usize, sy: usize) -> (u16, u16, u16) {
+ if sx >= self.w || sy >= self.h {
+ return (0, 0, 0);
+ }
+ let off = sy * self.pitch + sx * 4;
+ // Bounds-checked once per pixel instead of the serial path's raw `read_unaligned`. The
+ // check is noise next to the 64-coefficient transform each pixel feeds.
+ let Some(chunk) = self.pixels.get(off..off + 4) else {
+ return (0, 0, 0);
+ };
+ let Ok(bytes) = <[u8; 4]>::try_from(chunk) else {
+ return (0, 0, 0);
+ };
+ let (r, g, b) = self.unpack(u32::from_le_bytes(bytes));
+ match &self.color {
+ // The colour pipeline's tables are 8-bit, so a 10-bit surface is corrected at 8-bit
+ // precision and scaled back. That loses up to two low bits of a corrected pixel -- but ignoring the
+ // correction instead would leave a compositor's night-colour shift silently unapplied
+ // on exactly the outputs most likely to be colour-managed. Revisit with a 10-bit LUT if
+ // banding is ever measured on a corrected HDR connector.
+ Some(pipeline) => match self.depth {
+ crate::video::haar::Depth::Eight => {
+ let (r, g, b) = pipeline.apply(r as u8, g as u8, b as u8);
+ (r as u16, g as u16, b as u16)
+ }
+ crate::video::haar::Depth::Ten => {
+ let (r, g, b) = pipeline.apply((r >> 2) as u8, (g >> 2) as u8, (b >> 2) as u8);
+ (
+ (r as u16) << 2 | (r as u16 >> 6),
+ (g as u16) << 2 | (g as u16 >> 6),
+ (b as u16) << 2 | (b as u16 >> 6),
+ )
+ }
+ },
+ None => (r, g, b),
+ }
+ }
+
+ /// Read one output pixel after applying the plane transform.
+ ///
+ /// Keeping the transform in the immutable shared source gives serial and parallel encoding
+ /// exactly the same sampler. Codec padding is black and never reads beyond the snapshot.
+ #[inline]
+ fn px(&self, dx: usize, dy: usize) -> (u16, u16, u16) {
+ if self.direct {
+ // The codec pads the surface up to whole strips and expects black outside the image, so
+ // the bounds check stays: without it a read past the row wraps into the next one.
+ if dx >= self.w || dy >= self.h {
+ return (0, 0, 0);
+ }
+ let off = dy * self.pitch + dx * 4;
+ let Some(chunk) = self.pixels.get(off..off + 4) else {
+ return (0, 0, 0);
+ };
+ if let crate::video::haar::Depth::Eight = self.source_depth {
+ // Little-endian XRGB8888: byte 0 is blue, 1 green, 2 red.
+ return self.widen(chunk[2] as u16, chunk[1] as u16, chunk[0] as u16);
+ }
+ let Ok(bytes) = <[u8; 4]>::try_from(chunk) else {
+ return (0, 0, 0);
+ };
+ return self.unpack(u32::from_le_bytes(bytes));
+ }
+ if dx >= self.output_w || dy >= self.output_h {
+ return (0, 0, 0);
+ }
+ let (sx, sy) = rot_src(self.rotation, dx, dy, self.w, self.h);
+ self.source_px(sx, sy)
+ }
+}
+
+/// vino's own workqueue for the parallel strip encode.
+///
+/// On the shared `system_unbound` pool the encode's CPU time is anonymous: the kernel
+/// composes worker thread names from the workqueue's, so shared-pool work appears only as
+/// `kworker/uN:M-events_unbound`, indistinguishable from every other user of that pool. On a queue
+/// of our own the same threads appear as `kworker/uN:M-vino_encode`, so `ps`/`top`/`perf`
+/// attribute the codec's cost to vino, and the fan-out does not compete with unrelated work for
+/// the shared pool's concurrency budget.
+///
+/// `WQ_UNBOUND` because strip encoding is pure compute with no CPU affinity worth preserving --
+/// that property is what let the fan-out reach ~7.4x. Allocated once on first use and never
+/// destroyed: it is driver-wide, costs one `workqueue_struct`, and outliving every `EncodeChunk` is
+/// exactly what makes the join safe.
+///
+/// `max_active` is the CPU count. An unbound queue built without it takes `WQ_DFL_ACTIVE`, which is
+/// half of `WQ_MAX_ACTIVE` and so far above any useful degree of parallelism that it does not bound
+/// the fan-out at all: a full frame is a few hundred chunks per connector, and every one of them
+/// becomes a runnable CPU-bound worker at once. On a machine that is already busy that is not
+/// parallelism, it is a thundering herd -- it evicts the caches the codec depends on and stalls
+/// unrelated work, including the compositor thread this driver is waiting on. Fine-grained chunks
+/// are still the right unit (see `encode_across_cpus`); how many run at once is a separate
+/// decision, and this is where it is made.
+///
+/// Falls back to `system_unbound` if the allocation ever fails, so a failure here costs the thread
+/// *name*, not the driver.
+fn encode_queue() -> Option<&'static workqueue::Queue> {
+ static ENCODE_WQ: kernel::sync::SetOnce<workqueue::OwnedQueue> = kernel::sync::SetOnce::new();
+ if let Some(q) = ENCODE_WQ.as_ref() {
+ return Some(q);
+ }
+ // A concurrent racer may win the `SetOnce`; its queue is dropped and the winner's is used.
+ if let Ok(q) = workqueue::Queue::new_unbound()
+ .max_active(kernel::cpu::nr_cpu_ids().max(1))
+ .build(kernel::c_str!("vino_encode"))
+ {
+ let _ = ENCODE_WQ.populate(q);
+ }
+ ENCODE_WQ.as_ref().map(|q| &**q)
+}
+
+/// Encode a batch of strips from `src`, in the order given.
+fn encode_coords(
+ geometry: crate::video::haar::Geometry,
+ src: &PixelSource,
+ coords: &[(usize, usize)],
+) -> Result<KVec<KVec<u8>>> {
+ let mut out = KVec::with_capacity(coords.len(), GFP_KERNEL)?;
+ for &(sx, sy) in coords.iter() {
+ let mut px = |dx, dy| src.px(dx, dy);
+ out.push(
+ crate::video::haar::colour_strip_at(geometry, sx, sy, &mut px)?,
+ GFP_KERNEL,
+ )?;
+ }
+ Ok(out)
+}
+
+/// One contiguous batch of strips, encoded on whichever CPU the unbound workqueue picks.
+///
+/// Chunks share nothing but the read-only [`PixelSource`]: each writes only its own `out`, so no
+/// locking is needed on the hot path and the results reassemble by chunk order.
+#[pin_data]
+struct EncodeChunk {
+ #[pin]
+ work: Work<EncodeChunk>,
+ #[pin]
+ done: Completion,
+ src: Arc<PixelSource>,
+ coords: KVec<(usize, usize)>,
+ /// The dock's strip layout, carried per chunk so two docks of different generations can
+ /// encode concurrently on the same workqueue.
+ geometry: crate::video::haar::Geometry,
+ /// Encoded strip bodies. Written once by the worker, read once by the joiner after `done`;
+ /// the lock is uncontended and taken twice per chunk per frame.
+ #[pin]
+ out: Mutex<KVec<KVec<u8>>>,
+}
+
+impl_has_work! {
+ impl HasWork<Self> for EncodeChunk { self.work }
+}
+
+impl EncodeChunk {
+ fn new(
+ geometry: crate::video::haar::Geometry,
+ src: Arc<PixelSource>,
+ coords: KVec<(usize, usize)>,
+ ) -> Result<Arc<Self>> {
+ Arc::pin_init(
+ pin_init!(EncodeChunk {
+ work <- new_work!("vino::EncodeChunk::work"),
+ done <- Completion::new(),
+ src,
+ coords,
+ geometry,
+ out <- new_mutex!(KVec::new(), "vino::EncodeChunk::out"),
+ }),
+ GFP_KERNEL,
+ )
+ }
+}
+
+impl WorkItem for EncodeChunk {
+ type Pointer = Arc<EncodeChunk>;
+
+ fn run(this: Arc<EncodeChunk>) {
+ if let Ok(strips) = encode_coords(this.geometry, &this.src, &this.coords) {
+ *this.out.lock() = strips;
+ }
+ // Complete unconditionally. On failure `out` stays short and the joiner detects that by
+ // length -- but it must never be left blocked on a completion that cannot fire.
+ this.done.complete_all();
+ }
+}
+
+/// Encode `coords` across CPUs and return the strip bodies in the same order.
+///
+/// Order is not a nicety: [`crate::video::haar::frame_records`] groups strips into one wire record
+/// per single-Y band and needs them x-ordered within a band, so the chunks are contiguous slices
+/// of the raster-ordered coordinate list and are reassembled strictly in chunk order.
+///
+/// Returns `Ok(None)` when the frame is too small to be worth splitting, so the caller falls
+/// through to the serial encoder rather than paying dispatch cost for a handful of strips.
+fn parallel_strip_encode(
+ geometry: crate::video::haar::Geometry,
+ src: &Arc<PixelSource>,
+ coords: &[(usize, usize)],
+) -> Result<Option<KVec<KVec<u8>>>> {
+ // Size chunks from the amount of work, not from the CPU count. Handing the queue more, smaller
+ // items than there are CPUs costs a little dispatch overhead but improves load balancing: with
+ // one chunk per CPU a single slow chunk holds up the whole join, whereas fine-grained items let
+ // idle workers pick up the remainder.
+ //
+ // This is only safe because `encode_queue` caps `max_active` at the CPU count. The chunk count
+ // is the unit of work; that cap is what keeps the count from also becoming the number of
+ // threads competing for the machine.
+ let nchunks = (coords.len() / ENCODE_MIN_STRIPS_PER_CHUNK).min(ENCODE_MAX_WORK_ITEMS);
+ if nchunks < 2 {
+ return Ok(None);
+ }
+ let per = coords.len().div_ceil(nchunks);
+
+ let mut chunks: KVec<Arc<EncodeChunk>> = KVec::with_capacity(nchunks, GFP_KERNEL)?;
+ let mut queued: KVec<bool> = KVec::with_capacity(nchunks, GFP_KERNEL)?;
+ let mut start = 0usize;
+ while start < coords.len() {
+ let end = (start + per).min(coords.len());
+ let mut mine: KVec<(usize, usize)> = KVec::with_capacity(end - start, GFP_KERNEL)?;
+ for &c in &coords[start..end] {
+ mine.push(c, GFP_KERNEL)?;
+ }
+ let chunk = EncodeChunk::new(geometry, src.clone(), mine)?;
+ // `enqueue` gives the item back if it is already pending -- impossible for one allocated
+ // a line ago, but if it ever happened, waiting on its completion would hang the scanout
+ // worker forever. Record it and encode that chunk inline instead.
+ let ok = encode_queue()
+ .map_or_else(
+ || workqueue::system_unbound().enqueue(chunk.clone()),
+ |q| q.enqueue(chunk.clone()),
+ )
+ .is_ok();
+ queued.push(ok, GFP_KERNEL)?;
+ chunks.push(chunk, GFP_KERNEL)?;
+ start = end;
+ }
+
+ // The scanout worker runs on the per-device scanout queue, so blocking here cannot deadlock
+ // against the separate unbound pool the chunks run on.
+ let mut strips: KVec<KVec<u8>> = KVec::with_capacity(coords.len(), GFP_KERNEL)?;
+ for (i, chunk) in chunks.iter().enumerate() {
+ let mine = if queued[i] {
+ chunk.done.wait_for_completion();
+ core::mem::take(&mut *chunk.out.lock())
+ } else {
+ encode_coords(chunk.geometry, &chunk.src, &chunk.coords)?
+ };
+ if mine.len() != chunk.coords.len() {
+ // A chunk failed to allocate. Sending a frame with strips missing would paint a
+ // partial image the dock would keep, so fail the whole encode and let the caller's
+ // retry/backoff handle it.
+ return Err(ENOMEM);
+ }
+ for s in mine {
+ strips.push(s, GFP_KERNEL)?;
+ }
+ }
+ Ok(Some(strips))
+}
+
+/// Verify that workqueue fan-out produces the same strip bytes as the serial transformed sampler.
+///
+/// This is kept behind the Vino KUnit option so the production driver carries no test allocation
+/// or dispatch path. The deliberately unaligned output also verifies that both paths produce
+/// identical black codec padding.
+#[cfg(CONFIG_DRM_VINO_KUNIT_TEST)]
+pub(crate) fn parallel_rotation_matches_serial(rotation: plane::Rotation) -> Result {
+ let (output_w, output_h) = (500usize, 123usize);
+ let (w, h) = src_dims(rotation, output_w, output_h);
+ let len = w
+ .checked_mul(h)
+ .and_then(|n| n.checked_mul(4))
+ .ok_or(EINVAL)?;
+ let mut pixels: KVVec<u8> = KVVec::new();
+ pixels.resize(len, 0, GFP_KERNEL)?;
+ for sy in 0..h {
+ for sx in 0..w {
+ let off = (sy * w + sx) * 4;
+ pixels[off] = ((sx * 3 + sy * 5) & 0xff) as u8;
+ pixels[off + 1] = ((sx * 7 + sy * 11) & 0xff) as u8;
+ pixels[off + 2] = ((sx * 13 + sy * 17) & 0xff) as u8;
+ pixels[off + 3] = 0xff;
+ }
+ }
+ let src = Arc::new(
+ PixelSource {
+ pixels,
+ pitch: w * 4,
+ w,
+ h,
+ output_w,
+ output_h,
+ rotation,
+ color: None,
+ direct: direct_pixel_map(rotation, &None, w, h, output_w, output_h),
+ hashes: KVVec::new(),
+ depth: crate::video::haar::Depth::Eight,
+ source_depth: crate::video::haar::Depth::Eight,
+ },
+ GFP_KERNEL,
+ )?;
+ let geometry = crate::video::haar::RIDGE_GEOMETRY;
+ let padded_width = output_w.next_multiple_of(geometry.strip_w());
+ let padded_height = output_h.next_multiple_of(geometry.strip_h());
+ let coords = crate::video::haar::all_strip_coords(geometry, padded_width, padded_height)?;
+
+ let mut serial: KVec<KVec<u8>> = KVec::with_capacity(coords.len(), GFP_KERNEL)?;
+ for &(strip_x, strip_y) in coords.iter() {
+ let mut px = |dx, dy| {
+ if dx >= output_w || dy >= output_h {
+ return (0, 0, 0);
+ }
+ let (sx, sy) = rot_src(rotation, dx, dy, w, h);
+ src.source_px(sx, sy)
+ };
+ serial.push(
+ crate::video::haar::colour_strip_at(geometry, strip_x, strip_y, &mut px)?,
+ GFP_KERNEL,
+ )?;
+ }
+
+ let parallel = parallel_strip_encode(geometry, &src, &coords)?.ok_or(EINVAL)?;
+ if serial.len() != parallel.len()
+ || serial
+ .iter()
+ .zip(parallel.iter())
+ .any(|(expected, actual)| expected[..] != actual[..])
+ {
+ return Err(EINVAL);
+ }
+ Ok(())
+}
+
+/// A frame that trips one of these is dropped between the compositor's commit and the wire.
+fn scanout_gate(connector: u8, reason: &str) {
+ vino_debug!("vino: scanout connector={connector} deferred: {reason}\n");
+}
+
+#[inline(never)]
+fn encode_and_send_haar(
+ dev: &BoundInterface<'_>,
+ data: &VinoDrmData,
+ connector: u8,
+ src: &Arc<PixelSource>,
+ rotation: plane::Rotation,
+ _clips: &[(usize, usize, usize, usize)],
+ w: usize,
+ h: usize,
+) -> Result {
+ let geometry = data.geometry_for_connector(connector);
+ let connector_index = connector as usize;
+ // The vendor sends the frames that open a stream without the steady-state record bit and
+ // every frame after them with it, whether they carry a whole surface or one damaged strip.
+ // The training window is that opening.
+ let opening = data.sustain_until.lock()[connector_index]
+ .is_some_and(|until| (until - Instant::<Monotonic>::now()).as_millis() > 0);
+ let geometry = if opening {
+ geometry.opening()
+ } else {
+ geometry
+ };
+ // Gate video on the matching mode-set reaching the dock. Plane updates run before the CRTC
+ // enable queues that mode-set, and the dock rejects video on an unconfigured stream. Deferring
+ // does not advance the codec sequence; the next scanout pass retries the frame.
+ let want = data.modeset_requested[connector_index].load(core::sync::atomic::Ordering::Acquire);
+ if want == 0 {
+ scanout_gate(connector, "no mode-set requested (modeset_requested == 0)");
+ return Ok(());
+ }
+ // A dock that cannot report downstream presence offers every connector it has, because its
+ // activation is one dock-wide transaction and a connector nobody described is not the same dock
+ // state as a connector that does not exist. The empty socket still recovers no EDID, and
+ // painting it spends a full surface per repaint on a sink that is not there -- on this family
+ // over the endpoint its control plane also needs. So configure the connector and leave it
+ // showing the black its carrier put there, rather than streaming to nothing.
+ if !data.reports_presence() && !data.connector_has_edid(connector_index) {
+ scanout_gate(connector, "no monitor has described this socket");
+ return Ok(());
+ }
+ let cached = data.last_timing.lock()[connector_index];
+ if !cached.is_some_and(|t| {
+ timing_key(&t) == want && t.hactive as usize == w && t.vactive as usize == h
+ }) {
+ scanout_gate(
+ connector,
+ "cached timing does not match the requested mode generation",
+ );
+ return Ok(());
+ }
+ if data.modeset_active[connector_index].load(core::sync::atomic::Ordering::Acquire) != want {
+ // A failed command-worker activation leaves the desired generation intact. This worker is
+ // sleepable, so retry the same transaction before submitting its pending framebuffer.
+ let timing = cached.ok_or(EINVAL)?;
+ data.activate_head(dev, connector, &timing, want)?;
+ // A successful inline retry has made this very commit safe to send: continue into the
+ // encoder instead of waiting for another page flip. A completely static connector may not
+ // receive another atomic update after its enabling commit.
+ if data.modeset_active[connector_index].load(core::sync::atomic::Ordering::Acquire) != want
+ {
+ scanout_gate(
+ connector,
+ "mode-set not active and the inline re-send did not land",
+ );
+ return Ok(());
+ }
+ }
+ let seq0 = data.scanout_seq.lock()[connector_index];
+ // Source dimensions (swapped from the output for 90/270 rotation).
+ let (sw, sh) = src_dims(rotation, w, h);
+ if src.w != sw
+ || src.h != sh
+ || src.output_w != w
+ || src.output_h != h
+ || src.rotation != rotation
+ {
+ return Err(EINVAL);
+ }
+ // Full keyframe vs damage delta. A mode-set requires a keyframe; rotation/reflection remains
+ // conservative because the content shadow is deliberately stored in unrotated framebuffer
+ // space. For identity rotation, compare the actual framebuffer instead of trusting optional
+ // FB_DAMAGE_CLIPS: KWin commonly changes framebuffer objects without publishing that blob.
+ let kf_bit = 1u32 << connector_index;
+ let identity = rotation.angle() == plane::Rotation::ROTATE_0
+ && !rotation.contains(plane::Rotation::REFLECT_X | plane::Rotation::REFLECT_Y);
+ let owes_keyframe = data
+ .keyframe_pending
+ .load(core::sync::atomic::Ordering::Acquire)
+ & kf_bit
+ != 0;
+ let mut full = owes_keyframe || !identity;
+ // The codec operates on complete 64x16 strips. Pad non-aligned modes to the next strip
+ // boundary; the mode-set retains the visible dimensions and the sampler supplies black for
+ // pixels outside them.
+ let padded_width = (w + geometry.strip_w() - 1) & !(geometry.strip_w() - 1);
+ let padded_height = (h + geometry.strip_h() - 1) & !(geometry.strip_h() - 1);
+ let mut content_hashes: Option<KVVec<u64>> = None;
+ let mut content_damage: KVec<DamageRect> = KVec::new();
+ // Strips whose pixels moved since the last accepted frame, before the retransmit debt and the
+ // macro-tile rounding widen that into what is actually sent. Reported next to the selected
+ // count so an oversized delta says which of the two widened it.
+ let mut moved_strips = 0usize;
+ if identity {
+ let expected = (padded_width >> geometry.strip_w_shift())
+ * (padded_height >> geometry.strip_h_shift());
+ if src.hashes.len() != expected {
+ return Err(EINVAL);
+ }
+ let mut hashes: KVVec<u64> = KVVec::new();
+ hashes.resize(expected, 0, GFP_KERNEL)?;
+ hashes.copy_from_slice(&src.hashes);
+ if !full {
+ let previous = data.strip_hashes.lock();
+ if let Some(state) = &previous[connector_index] {
+ if state.padded_width == padded_width && state.padded_height == padded_height {
+ // Charge every strip whose content moved with the profile's logical-frame
+ // debt, then select every strip that still owes a transmission -- including
+ // ones that changed on an earlier frame and have not reached every dock
+ // buffer. See `dirty_ttl` and `FrameDelivery`.
+ let mut ttl = data.dirty_ttl.lock();
+ if !ttl[connector_index]
+ .as_ref()
+ .is_some_and(|t| t.len() == hashes.len())
+ {
+ let mut fresh: KVVec<u8> = KVVec::new();
+ fresh.resize(hashes.len(), 0, GFP_KERNEL)?;
+ ttl[connector_index] = Some(fresh);
+ }
+ let debt = ttl[connector_index]
+ .as_mut()
+ .ok_or(kernel::error::code::ENOMEM)?;
+ let damage_frames = data.frame_delivery().damage_frames.max(1);
+ for i in 0..hashes.len() {
+ if state.hashes[i] != hashes[i] {
+ debt[i] = damage_frames;
+ moved_strips += 1;
+ }
+ }
+ // Reuse the hash differ: mark an owed strip by handing it a baseline value that
+ // cannot match, and an unowed one its own value.
+ let mut baseline: KVVec<u64> = KVVec::new();
+ baseline.resize(hashes.len(), 0, GFP_KERNEL)?;
+ for i in 0..hashes.len() {
+ baseline[i] = if debt[i] > 0 { !hashes[i] } else { hashes[i] };
+ }
+ content_damage = changed_strip_rects(
+ geometry,
+ &baseline,
+ &hashes,
+ padded_width,
+ padded_height,
+ )?;
+ } else {
+ full = true;
+ }
+ } else {
+ full = true;
+ }
+ }
+ content_hashes = Some(hashes);
+ }
+ if !full && content_damage.is_empty() {
+ scanout_gate(connector, "no keyframe owed and no strip content changed");
+ return Ok(());
+ }
+ // Serial fallback and parallel workers share the same transformed sampler.
+ let px = |dx: usize, dy: usize| src.px(dx, dy);
+ // Damage selection and encoded-strip reuse remain identity-only. Rotated and reflected frames
+ // are conservative full updates, but their independent strips can use the same workqueue
+ // fan-out as an identity keyframe.
+ // What the encoded bytes depend on besides the strip pixels themselves; see
+ // `StripHashState::tag`. Identity rotation is a precondition of caching at all, so it needs no
+ // representation here.
+ let encode_tag = {
+ let gamma = match &src.color {
+ Some(pipeline) => pipeline.tag(),
+ None => 0,
+ };
+ // The sample depth belongs here for the same reason the colour transform does, and is the
+ // more dangerous of the two: changing it re-maps every sample on the way into the codec
+ // and moves the escape ceiling, while leaving the framebuffer byte for byte identical. A
+ // cache keyed on the pixels alone therefore serves bodies encoded at the old depth inside
+ // a stream declared at the new one, and the dock decodes part of the frame as noise.
+ let deep = matches!(src.depth, crate::video::haar::Depth::Ten);
+ gamma ^ (u64::from(deep) * 0x9e37_79b9_7f4a_7c15)
+ };
+ // Strips carried over verbatim from the previous frame's encode, and the strips actually
+ // handed to the codec; kept for the post-send cache publish below.
+ let mut encoded: Option<(KVec<(usize, usize)>, KVec<KVec<u8>>)> = None;
+ let parallel = if !identity {
+ let coords = crate::video::haar::all_strip_coords(geometry, padded_width, padded_height)?;
+ match parallel_strip_encode(geometry, src, &coords)? {
+ Some(strips) => {
+ let records = if geometry.connector_selector_shift != 0 {
+ crate::video::haar::frame_records_navarro_ordinary(
+ geometry, &strips, connector,
+ )?
+ } else {
+ crate::video::haar::frame_records(geometry, &strips, connector)?
+ };
+ Some(records)
+ }
+ None => None,
+ }
+ } else {
+ let coords = if full {
+ crate::video::haar::all_strip_coords(geometry, padded_width, padded_height)?
+ } else {
+ crate::video::haar::damage_strip_coords(
+ geometry,
+ padded_width,
+ padded_height,
+ &content_damage,
+ )?
+ };
+ // What this frame costs the dock, and why. A delta that selects far more strips than moved
+ // was widened by the retransmit debt or by rounding out to whole macro-tiles; one that
+ // selects the surface from a handful of rectangles hit the rectangle ceiling instead.
+ vino_debug!(
+ "vino: scanout connector={} {} {}/{} strips from {} rect(s), {} moved\n",
+ connector,
+ if full { "keyframe" } else { "delta" },
+ coords.len(),
+ (padded_width >> geometry.strip_w_shift())
+ * (padded_height >> geometry.strip_h_shift()),
+ content_damage.len(),
+ moved_strips
+ );
+ // Reuse an encoded strip body when its pixels and gamma tag are unchanged. Encode only
+ // misses, then restore the required x-order within each Y band.
+ let tiles_x = padded_width >> geometry.strip_w_shift();
+ let mut reuse: KVec<Option<KVec<u8>>> = KVec::with_capacity(coords.len(), GFP_KERNEL)?;
+ let mut misses: KVec<(usize, usize)> = KVec::with_capacity(coords.len(), GFP_KERNEL)?;
+ {
+ let cache = data.strip_hashes.lock();
+ let usable = cache[connector_index].as_ref().filter(|c| {
+ c.padded_width == padded_width
+ && c.padded_height == padded_height
+ && c.tag == encode_tag
+ });
+ for &(sx, sy) in coords.iter() {
+ let idx =
+ (sy >> geometry.strip_h_shift()) * tiles_x + (sx >> geometry.strip_w_shift());
+ let hit = usable.and_then(|c| {
+ // Same pixels as when this body was produced, and a body was kept.
+ let same = c.hashes.get(idx).zip(content_hashes.as_ref()?.get(idx));
+ let body = c.bodies.get(idx)?;
+ (same.is_some_and(|(a, b)| a == b) && !body.is_empty()).then_some(body)
+ });
+ match hit {
+ Some(body) => {
+ let mut copy: KVec<u8> = KVec::with_capacity(body.len(), GFP_KERNEL)?;
+ copy.extend_from_slice(body, GFP_KERNEL)?;
+ reuse.push(Some(copy), GFP_KERNEL)?;
+ }
+ None => {
+ reuse.push(None, GFP_KERNEL)?;
+ misses.push((sx, sy), GFP_KERNEL)?;
+ }
+ }
+ }
+ }
+ let fresh = match parallel_strip_encode(geometry, src, &misses)? {
+ Some(s) => Some(s),
+ // Too few misses to be worth splitting: encode them here rather than dropping to
+ // the whole-frame serial path, which would re-encode the cache hits as well.
+ None if !misses.is_empty() => Some(encode_coords(geometry, src, &misses)?),
+ None => Some(KVec::new()),
+ };
+ match fresh {
+ Some(fresh) if fresh.len() == misses.len() => {
+ let mut strips: KVec<KVec<u8>> = KVec::with_capacity(coords.len(), GFP_KERNEL)?;
+ let mut next = fresh.into_iter();
+ for slot in reuse {
+ match slot {
+ Some(body) => strips.push(body, GFP_KERNEL)?,
+ None => strips.push(next.next().ok_or(EINVAL)?, GFP_KERNEL)?,
+ }
+ }
+ let records = if full && geometry.connector_selector_shift != 0 {
+ crate::video::haar::frame_records_navarro_ordinary(
+ geometry, &strips, connector,
+ )?
+ } else {
+ crate::video::haar::frame_records(geometry, &strips, connector)?
+ };
+ encoded = Some((coords, strips));
+ Some(records)
+ }
+ _ => None,
+ }
+ };
+ let frames = match parallel {
+ Some(r) => r,
+ None if full && geometry.connector_selector_shift != 0 => {
+ crate::video::haar::colour_frame_ep08_navarro_ordinary(
+ geometry,
+ padded_width,
+ padded_height,
+ connector,
+ px,
+ )?
+ }
+ None if full => crate::video::haar::colour_frame_ep08(
+ geometry,
+ padded_width,
+ padded_height,
+ connector,
+ px,
+ )?,
+ None => crate::video::haar::colour_frame_ep08_damage(
+ geometry,
+ padded_width,
+ padded_height,
+ connector,
+ &content_damage,
+ px,
+ )?,
+ };
+ // A damage delta that touched no aligned strip = nothing to send this flip: skip the write
+ // (no seq advance, no arm, keyframe obligation untouched). Full frames always have strips.
+ if frames.is_empty() {
+ scanout_gate(connector, "encoder produced zero records");
+ return Ok(());
+ }
+ if data.shutting_down.load(Ordering::Acquire)
+ || data.modeset_requested[connector_index].load(Ordering::Acquire) != want
+ || data.modeset_active[connector_index].load(Ordering::Acquire) != want
+ {
+ scanout_gate(
+ connector,
+ "mode generation changed between encode and submit",
+ );
+ return Ok(());
+ }
+ // A frame is one continuous bulk stream: intermediate transfers end on a full 1024-byte packet
+ // and only the final transfer is short. A short packet at a record boundary terminates the
+ // frame early and desynchronises the dock. The first frame after a mode set also prepends the
+ // connector's ten-record arm burst; clear that obligation only after a successful submission.
+ let connector_bit = 1u32 << connector;
+ // The 2560-byte arm burst appears only on frame zero after a mode set. Later frames begin
+ // directly with video records.
+ let arm = if data
+ .arm_prefix_pending
+ .load(core::sync::atomic::Ordering::Acquire)
+ & connector_bit
+ != 0
+ {
+ Some(data.build_stream_prefix_buf(connector_index)?)
+ } else {
+ None
+ };
+ let arm_len = arm.as_ref().map_or(0, |a| a.len());
+ // Revalidate at the actual wire boundary too. The encoded bytes and ARM prefix are specific to
+ // this mode generation; submitting them after a concurrent disable/re-enable poisons the next
+ // stream even though every USB URB can still complete successfully.
+ if data.shutting_down.load(Ordering::Acquire)
+ || data.modeset_requested[connector_index].load(Ordering::Acquire) != want
+ || data.modeset_active[connector_index].load(Ordering::Acquire) != want
+ {
+ vino_debug!(
+ "vino: scanout connector={} superseded before video submit; frame discarded\n",
+ connector
+ );
+ return Ok(());
+ }
+ // Preserve the last readiness-to-video adjacency from the VINO session that lit both panels.
+ // These are real CP status transactions (with EP84 replies drained by `send_cp`), paced at the
+ // required cadence, and only run for frame zero while the ARM prefix is present.
+ if arm.is_some() {
+ for _ in 0..VinoDrmData::PREWRITE_POLLS {
+ data.poll_status(dev)?;
+ fsleep(Delta::from_millis(VinoDrmData::PREWRITE_POLL_MS as i64));
+ }
+ vino_debug!(
+ "vino: inline pre-write paced poll ({}x @{}ms) before first video connector={}\n",
+ VinoDrmData::PREWRITE_POLLS,
+ VinoDrmData::PREWRITE_POLL_MS,
+ connector
+ );
+ }
+ // Frame zero starts with an arm record; later frames start with video records. Record fragments
+ // are allocation boundaries only and are joined into exact 64-KiB transfers below without a
+ // whole-frame coalescing allocation.
+ let frame_count = frames.len();
+ let image_len: usize = frames.iter().take(frame_count).map(|f| f.len()).sum();
+ // The DL7400's per-strip parameter map. DLM and Windows both include it in frame zero after
+ // at least some image records; the deterministic image-then-map ordering is used below. Ridge
+ // has no equivalent record and gets an empty slice.
+ let params: KVec<u8> = if geometry.connector_selector_shift == 0 {
+ KVec::new()
+ } else {
+ crate::video::haar::navarro_strip_params(
+ geometry,
+ connector,
+ padded_width,
+ padded_height,
+ &frames,
+ &mut data.strip_classes.lock()[connector_index],
+ )?
+ };
+ vino_debug!(
+ "vino: connector={} shift={} params={} B ({}x{} pad)\n",
+ connector,
+ geometry.connector_selector_shift,
+ params.len(),
+ padded_width,
+ padded_height
+ );
+ if arm.is_some() {
+ data.send_stream_open(dev, connector_index)?;
+ }
+ let startup = arm.is_some();
+ // A cold link requires a bounded back-to-back full-frame burst until the downstream clock is
+ // programmed. Reuse the encoded image and advance only its frame trailer and per-frame control
+ // sync. The arm prefix remains exclusive to presentation zero.
+ let training = full
+ && data.sustain_until.lock()[connector_index]
+ .is_some_and(|until| (until - Instant::<Monotonic>::now()).as_millis() > 0);
+ // A dock that shares its control pipe cannot be given the training window's presentation
+ // count: a whole-surface keyframe is 1.88 MB there, and eight back-to-back copies of it are
+ // 15 MB in one scanout against a dock DLM feeds at 0.86 MB/s. Measured, the dock accepts
+ // three such frames and then stops answering the control plane entirely. Every buffer still
+ // gets the keyframe; it just does not get it eight times.
+ // A keyframe has to initialise every buffer before its content shadow can become authoritative.
+ // Ordinary updates are different: DLM sends one Ella presentation per logical frame and lets
+ // successive frames walk the ring. The profile keeps those two requirements separate and the
+ // damage ledger schedules later logical frames if the compositor becomes idle.
+ let repeat_count = frame_presentation_count(
+ data.frame_delivery(),
+ full,
+ training,
+ data.video_on_ctrl_pipe(),
+ );
+ let first_opener_len = data
+ .build_frame_opener(connector, seq0, startup)
+ .as_ref()
+ .map_or(0, |o| o.len());
+ // Every ordinary Navarro presentation pairs its frame-sub records with one authenticated
+ // report on the stream sub. Build presentation zero once here so the diagnostic length and
+ // the bytes submitted below refer to the same live counter reservation. The prologue itself
+ // has no report; presentation one will allocate its own inside the loop.
+ let mut first_report = if startup {
+ None
+ } else {
+ data.build_stream_report_buf(connector_index, seq0)?
+ };
+ let first_report_len = first_report.as_ref().map_or(0, |r| r.len());
+ let first_wire_len = arm_len
+ + first_opener_len
+ + first_report_len
+ + params.len()
+ + image_len
+ + data.build_frame_trailer(connector, seq0).len();
+ let (rec_count, max_stride, max_strip) =
+ crate::video::haar::record_stats(&frames[..frame_count]);
+ vino_debug!(
+ "vino: connector={} chunks={} arm={} first={} presentations={} records={} max_stride={} max_strip={}\n",
+ connector,
+ frame_count,
+ arm_len,
+ first_wire_len,
+ repeat_count,
+ rec_count,
+ max_stride,
+ max_strip
+ );
+ // Split at 65536-byte boundaries, a multiple of the endpoint's 1024-byte maximum packet size,
+ // so only the final transfer terminates short. Submit through a persistent eight-deep queue to
+ // keep the frame continuous across transfer boundaries. Do not flush between frames: slot reuse
+ // reaps completions without introducing a pipeline gap.
+ const XFER: usize = VIDEO_XFER;
+ let pipe_i = dev.video_pipe_index(connector_index)?;
+ let mut last_wire_len = 0usize;
+ // Presentations that named a ring slot, which is what the frame counter counts. See
+ // `names_ring_slot`.
+ let mut named = 0u32;
+ for repeat in 0..repeat_count {
+ // Pace the copies apart on a dock whose control plane shares this endpoint. Back-to-back
+ // presentations hold it for as long as it takes to push several megabytes, which is
+ // exactly when the dock has to be able to answer, and it stops answering at all.
+ if repeat > 0 && data.video_on_ctrl_pipe() {
+ fsleep(Delta::from_millis(data.frame_period_ms()));
+ }
+ // A compositor mode change can arrive while the presentation is in flight. Never let the
+ // old frame cross the new mode generation.
+ if data.shutting_down.load(Ordering::Acquire)
+ || data.modeset_requested[connector_index].load(Ordering::Acquire) != want
+ || data.modeset_active[connector_index].load(Ordering::Acquire) != want
+ {
+ vino_debug!(
+ "vino: scanout connector={} superseded during presentation; stopped at {}/{}\n",
+ connector,
+ repeat,
+ repeat_count
+ );
+ return Ok(());
+ }
+
+ let repeat_seq = seq0.wrapping_add(named);
+ let frame_trailer = data.build_frame_trailer(connector, repeat_seq);
+ // Prefix ARM only to presentation zero. Every later presentation starts directly at the
+ // image records and carries a freshly advanced three-record frame trailer.
+ let arm_slice: &[u8] = if repeat == 0 {
+ arm.as_ref().map_or(&[], |a| &a[..])
+ } else {
+ &[]
+ };
+ let prologue_frame = startup && repeat == 0;
+ let frame_opener = data.build_frame_opener(connector, repeat_seq, prologue_frame);
+ let opener_slice: &[u8] = frame_opener.as_ref().map_or(&[], |o| &o[..]);
+ if super::names_ring_slot(opener_slice, &frame_trailer) {
+ named = named.wrapping_add(1);
+ }
+ let report = if prologue_frame {
+ None
+ } else if repeat == 0 {
+ first_report.take()
+ } else {
+ data.build_stream_report_buf(connector_index, repeat_seq)?
+ };
+ let report_slice: &[u8] = report.as_ref().map_or(&[], |r| &r[..]);
+ let wire_len = arm_slice.len()
+ + opener_slice.len()
+ + report_slice.len()
+ + params.len()
+ + image_len
+ + frame_trailer.len();
+ last_wire_len = wire_len;
+ {
+ // Take this connector's staging buffer while submitting, then restore it. The queue
+ // mutex stays locked for the complete frame: two connectors that address the same
+ // physical endpoint must never submit their record streams concurrently.
+ let mut staging = match data.video_staging.lock()[connector_index].take() {
+ Some(s) => s,
+ None => {
+ let mut s = KVec::new();
+ s.resize(XFER, 0, GFP_KERNEL)?;
+ s
+ }
+ };
+ let submitted = {
+ // One writer owns a shared pipe for the whole frame; see `own_pipe`.
+ let _pipe = data.own_pipe();
+ // A shared-pipe queue failure is terminal for its complete control/video session.
+ // It may have happened while this worker waited behind its sibling, so recheck
+ // under pipe ownership before recreating or touching the canonical queue.
+ if data.video_on_ctrl_pipe() && !data.cp_link_alive() {
+ data.video_staging.lock()[connector_index] = Some(staging);
+ return Err(ENODEV);
+ }
+ let mut queue_slot = data.video_q[pipe_i].lock();
+ if queue_slot.is_none() {
+ match dev.video_queue(connector_index, 8, XFER) {
+ Ok(q) => {
+ *queue_slot = Some(q);
+ vino_debug!(
+ "vino: connector={} endpoint={:#04x} persistent video queue opened (depth=8, {} B URBs)\n",
+ connector,
+ dev.endpoints.video[connector_index].address(),
+ XFER
+ );
+ }
+ // Nothing was taken from the shared pipe slot, but return this connector's
+ // staging allocation before propagating the open error.
+ Err(e) => {
+ data.video_staging.lock()[connector_index] = Some(staging);
+ return Err(e);
+ }
+ }
+ }
+ // Keep both borrows inside the mutex scope. Dropping the queue or unlocking it
+ // mid-frame would permit a second connector to interleave URBs on this endpoint.
+ let submit = |staging: &mut KVec<u8>, q: &mut crate::usb::BulkOutQueue| -> Result {
+ let staging = &mut staging[..];
+ let q = &mut *q;
+ // Scatter/gather cursor over
+ // [optional ARM][optional Navarro opener][authenticated stream report]
+ // [record chunks, with the parameter map among them][trailer]. Join only one
+ // transfer at a time in the reusable bounded staging allocation, avoiding a
+ // contiguous allocation spanning the complete frame.
+ let arm_parts = usize::from(!arm_slice.is_empty());
+ let opener_parts = usize::from(!opener_slice.is_empty());
+ let report_parts = usize::from(!report_slice.is_empty());
+ let param_parts = usize::from(!params.is_empty());
+ let trailer_parts = 1usize;
+ let opener_end = arm_parts + opener_parts;
+ let lead = opener_end + report_parts;
+ // The map describes the records around it and goes where the vendor puts it,
+ // part-way through them; see `param_map_chunk_split`. A dock with no map has
+ // no split to make.
+ let param_after = if param_parts == 0 {
+ frame_count
+ } else {
+ super::param_map_chunk_split(&frames[..frame_count])
+ };
+ let param_end = lead + param_after + param_parts;
+ let part_count = lead + frame_count + param_parts + trailer_parts;
+ let mut part_i = 0usize;
+ let split_tail = data.split_full_packet_frame();
+ let mut part_off = 0usize;
+ let mut wire_off = 0usize;
+ while wire_off < wire_len {
+ let mut data_len = (wire_len - wire_off).min(XFER);
+ // The dock delimits a frame by the short packet its last transfer ends on.
+ // A frame whose length is a whole number of maximum-size packets ends on a
+ // full one, delimits nothing, and is read as running into the next frame.
+ // Split one short tail off it so there is always a boundary; a record may
+ // span transfers, so where the split falls does not matter.
+ if split_tail
+ && data_len == wire_len - wire_off
+ && data_len % BULK_MAX_PACKET == 0
+ {
+ data_len -= FRAME_TAIL_BYTES;
+ }
+ let dst = &mut staging[..data_len];
+ let mut dst_off = 0usize;
+ while dst_off < dst.len() && part_i < part_count {
+ let part: &[u8] = if part_i < arm_parts {
+ arm_slice
+ } else if part_i < opener_end {
+ opener_slice
+ } else if part_i < lead {
+ report_slice
+ } else if part_i < lead + param_after {
+ &frames[part_i - lead][..]
+ } else if part_i < param_end {
+ ¶ms[..]
+ } else if part_i < part_count - trailer_parts {
+ &frames[part_i - lead - param_parts][..]
+ } else {
+ &frame_trailer[..]
+ };
+ let n = (part.len() - part_off).min(dst.len() - dst_off);
+ dst[dst_off..dst_off + n]
+ .copy_from_slice(&part[part_off..part_off + n]);
+ dst_off += n;
+ part_off += n;
+ if part_off == part.len() {
+ part_i += 1;
+ part_off = 0;
+ }
+ }
+ if let Err(e) = q.send(dev.io(), dst, crate::timeout()) {
+ // The queue is eight URBs deep and `send` reaps a completion when its
+ // slot is reused, so this offset is where the error surfaced, not
+ // where the dock objected -- that was about eight transfers earlier.
+ // Report the frame's shape instead: a dock refuses a malformed record
+ // by halting the endpoint, which arrives here as a transport error and
+ // gets blamed on the transport.
+ let (records, max_stride, max_strip) =
+ crate::video::haar::record_stats(&frames[..frame_count]);
+ pr_warn!(
+ "vino: scanout connector={} pipeline submit at off={}/{} failed; frame has {} records, largest stride {}, largest strip {}\n",
+ connector,
+ wire_off,
+ wire_len,
+ records,
+ max_stride,
+ max_strip
+ );
+ return Err(e);
+ }
+ wire_off += data_len;
+ }
+ Ok(())
+ };
+ let submitted = {
+ let mut queue = queue_slot.as_mut().get_mut().as_mut().ok_or(ENODEV)?;
+ submit(&mut staging, &mut queue)
+ };
+ match submitted {
+ Ok(()) => Ok(()),
+ Err(e) => {
+ // Stay inside the existing physical-pipe and canonical-queue guards. The
+ // queue drop kills and drains every later URB before halt-clear, so neither
+ // this connector's unframed tail nor a sibling's new frame can race
+ // recovery.
+ let clear_halt = (e == EPIPE || e == EPROTO)
+ && data.scanout_fails[connector_index].load(Ordering::Relaxed)
+ < VIDEO_STALL_LIMIT;
+ if let Err(recovery) = data.retire_failed_video_queue(
+ dev,
+ connector_index,
+ &mut *queue_slot,
+ e,
+ clear_halt,
+ ) {
+ pr_err!(
+ "vino: connector {connector_index} video queue recovery failed ({recovery:?})\n"
+ );
+ }
+ Err(e)
+ }
+ }
+ };
+ // Restore the per-connector staging allocation after the endpoint transaction.
+ data.video_staging.lock()[connector_index] = Some(staging);
+ submitted?;
+ }
+
+ // The ARM burst was delivered with presentation zero. Clear it immediately rather than
+ // after the whole replay: if a later copy fails, retrying ARM would corrupt a pipe that is
+ // already armed.
+ if repeat == 0 && startup {
+ data.arm_prefix_pending
+ .fetch_and(!connector_bit, core::sync::atomic::Ordering::Release);
+ // The cold-link requirement is measured from the start of continuous VIDEO, not from
+ // the earlier mode-set. Refresh the complete training window here so modeset bracket
+ // latency, cross-connector serialization, and encoder time cannot make it
+ // intermittently too short. Subsequent cadence-selected compositor flips and idle
+ // settle repaints are both promoted to full keyframes while this deadline is live.
+ // Re-armed from the first frame the dock actually received, but only where
+ // `sustain_window` granted one: a dock that shares its control pipe is granted none.
+ {
+ let mut sustain = data.sustain_until.lock();
+ if sustain[connector_index].is_some() {
+ sustain[connector_index] =
+ Some(Instant::<Monotonic>::now() + Delta::from_millis(3000));
+ }
+ }
+ vino_debug!(
+ "vino: scanout connector={} initial ARM+keyframe accepted ({} B on the wire)\n",
+ connector,
+ wire_len
+ );
+ // The dock expects two stream-commit messages on EP02 immediately after accepting the
+ // video arm burst.
+ for _ in 0..2 {
+ match data.send_cp(dev, 0x16, 0, |ctr| crate::cp::stream_commit(ctr, connector)) {
+ Ok(()) => vino_debug!("vino: stream-commit connector={} ok\n", connector),
+ Err(e) => pr_warn!(
+ "vino: stream-commit connector={} failed ({e:?})\n",
+ connector
+ ),
+ }
+ }
+ }
+
+ // A dedicated video endpoint can cheaply sample status after streaming. On a shared
+ // pipe, the long-lived keepalive already owns this same poll and DLM permits long runs of
+ // pixels without an inline CP transaction. Duplicating it here increases EP02 pressure
+ // and blocks the scanout worker waiting for a reply from the endpoint it just filled.
+ if !data.video_on_ctrl_pipe() {
+ let due = {
+ let mut last = data.last_status_poll.lock();
+ let due = last.is_none_or(|t| t.elapsed().as_millis() >= STATUS_POLL_MIN_MS);
+ if due {
+ *last = Some(Instant::<Monotonic>::now());
+ }
+ due
+ };
+ if due {
+ if let Err(e) =
+ data.send_cp(dev, 0x14, 0, |ctr| crate::cp::device_query_req(ctr, 0x000c))
+ {
+ vino_debug!(
+ "vino: scanout connector={} CP status poll failed ({e:?})\n",
+ connector
+ );
+ }
+ }
+ }
+ // Do not drain here. The eight-URB ring spans frame boundaries; `send()` reaps a
+ // completion when its slot is reused, so transport errors surface after the ring wraps
+ // without introducing a per-frame pipeline bubble.
+ }
+ // Publish the new codec sequence only after every URB for this frame was submitted. A stale
+ // generation or transport failure above leaves the old sequence intact for the next keyframe.
+ //
+ data.scanout_seq.lock()[connector_index] = seq0.wrapping_add(named);
+ // The USB path accepted the complete image. Publish its content shadow only now; every early
+ // return and transport error above deliberately leaves the previous dock-visible state intact.
+ // The frame reached the dock, so every strip it carried has paid one transmission. A full
+ // keyframe is presented twice and rewrites the whole surface, so it clears the ledger outright.
+ {
+ let mut ttl = data.dirty_ttl.lock();
+ if let Some(debt) = ttl[connector_index].as_mut() {
+ pay_damage_debt(debt, full);
+ }
+ }
+ // Publish the content shadow, and with it the encoded body of every strip this frame carried,
+ // so the retransmissions `damage_frames` owes can re-use the bytes instead of re-running the
+ // codec (see `StripHashState::bodies`). Bodies for strips this frame did not touch are carried
+ // forward from the previous state -- they are still what the dock holds, and a later debt pass
+ // may select them. Best-effort throughout: a failed allocation costs a cache miss, never a
+ // frame, so the hashes are published either way.
+ {
+ let mut state = data.strip_hashes.lock();
+ let carried = state[connector_index]
+ .take()
+ .filter(|c| {
+ c.padded_width == padded_width
+ && c.padded_height == padded_height
+ && c.tag == encode_tag
+ })
+ .map(|c| (c.bodies, c.hashes));
+ state[connector_index] = content_hashes.map(|hashes| {
+ let (mut bodies, old) = match carried {
+ Some((b, h)) => (b, Some(h)),
+ None => (KVec::new(), None),
+ };
+ // A carried body is only still valid if that strip's content has not moved since it
+ // was encoded. Every strip whose hash changes IS selected for this frame and so is
+ // overwritten below -- but do not rely on that invariant holding as the selection
+ // logic evolves: a body left paired with a newer hash would be served as a cache hit
+ // and paint stale pixels the dock would then keep, with nothing scheduled to repair
+ // it. Cheap to make airtight, and the failure it prevents is permanent corruption.
+ if let Some(old) = &old {
+ if old.len() == bodies.len() && old.len() == hashes.len() {
+ for i in 0..bodies.len() {
+ if old[i] != hashes[i] {
+ bodies[i] = KVec::new();
+ }
+ }
+ }
+ }
+ if bodies.len() != hashes.len() {
+ bodies = KVec::new();
+ let _ = bodies.reserve(hashes.len(), GFP_KERNEL);
+ while bodies.len() < hashes.len() && bodies.push(KVec::new(), GFP_KERNEL).is_ok() {}
+ }
+ if bodies.len() == hashes.len() {
+ if let Some((coords, strips)) = encoded {
+ let tiles_x = padded_width >> geometry.strip_w_shift();
+ for (&(sx, sy), body) in coords.iter().zip(strips) {
+ let idx = (sy >> geometry.strip_h_shift()) * tiles_x
+ + (sx >> geometry.strip_w_shift());
+ if let Some(slot) = bodies.get_mut(idx) {
+ *slot = body;
+ }
+ }
+ }
+ }
+ StripHashState {
+ padded_width,
+ padded_height,
+ hashes,
+ bodies,
+ tag: encode_tag,
+ }
+ });
+ }
+ // A full keyframe was accepted -- this connector may now send damage deltas until the next
+ // mode-set.
+ if full {
+ data.keyframe_pending
+ .fetch_and(!kf_bit, core::sync::atomic::Ordering::Release);
+ }
+
+ // Charge every presentation, not the frame once: each one is a separate copy on the wire and
+ // the dock decodes all of them.
+ data.charge_stream_budget(last_wire_len.saturating_mul(repeat_count as usize));
+
+ vino_debug!(
+ "vino: scanout connector={} frame ok ({} presentation(s), {} B final write)\n",
+ connector,
+ repeat_count,
+ last_wire_len
+ );
+ Ok(())
+}
+
+/// Convert the mapped XRGB8888 frame to RGB565, Vino-encode it against the previous frame,
+/// and bulk-write the resulting EP08 frame to the dock.
+pub(super) fn encode_and_send(
+ dev: &BoundInterface<'_>,
+ data: &VinoDrmData,
+ connector: u8,
+ src: &Arc<PixelSource>,
+ rotation: plane::Rotation,
+ // The client's changed rectangles (identity rotation only; empty means no pixel update).
+ // `encode_and_send_haar` uses these to send a damage delta (only changed strips) after the
+ // first full keyframe because the dock surface is undefined after a mode set.
+ clips: &[(usize, usize, usize, usize)],
+ w: usize,
+ h: usize,
+) -> Result {
+ // Non-64x16-aligned modes are padded to complete codec strips. The dock clips the padded image
+ // to the active timing, matching the validated 68-band wire layout for 1080-line modes.
+ encode_and_send_haar(dev, data, connector, src, rotation, clips, w, h)
+}
+
+// ---- Encoder ----------------------------------------------------------------
+
+#[cfg(CONFIG_DRM_VINO_KUNIT_TEST)]
+#[kunit_tests(vino_scanout)]
+mod tests {
+ use super::*;
+ use crate::*;
+
+ /// An eight-bit desktop driven over a ten-bit link must keep both endpoints exact.
+ ///
+ /// A plain shift leaves full white at 1020 of 1023, which tints every highlight and is the
+ /// kind of error that looks like a panel problem rather than an encoder one.
+ #[test]
+ fn widening_an_eight_bit_sample_keeps_the_endpoints() {
+ assert_eq!(widen_8_to_10(0), 0);
+ assert_eq!(widen_8_to_10(255), 1023);
+ // Mid grey stays mid grey rather than drifting low.
+ assert_eq!(widen_8_to_10(128), 514);
+ // Monotonic, and never outside the ten-bit range.
+ let mut previous = 0;
+ for v in 0..=255u16 {
+ let w = widen_8_to_10(v);
+ assert!(w <= 1023);
+ assert!(v == 0 || w > previous);
+ previous = w;
+ }
+ }
+
+ /// Scattered damage must cost the strips it touches, not the surface.
+ ///
+ /// The rectangle list is an intermediate representation between the per-strip hash comparison
+ /// and the strip coordinates the encoder is given, and it has a ceiling. Built on the strip
+ /// grid, an update spread across the screen fragments into more runs than that ceiling holds
+ /// and the whole surface is sent instead -- roughly thirty times the bytes, on a dock that
+ /// halts its endpoint when it is given too many. Built on the macro-tile grid, which is the
+ /// granularity a touched strip is resent at anyway, the same update stays inside it.
+ #[test]
+ fn scattered_damage_does_not_cost_the_whole_surface() -> Result {
+ let geometry = profile::PROFILE_ELLA.geometry();
+ // 1920x1088, this dock's whole top mode padded to strips: 30 x 68 = 2040 strips.
+ let (padded_width, padded_height) = (1920usize, 1088usize);
+ let tiles_x = padded_width >> geometry.strip_w_shift();
+ let tiles_y = padded_height >> geometry.strip_h_shift();
+ assert_eq!(tiles_x * tiles_y, 2040);
+
+ let mut old: KVVec<u64> = KVVec::new();
+ old.resize(tiles_x * tiles_y, 0, GFP_KERNEL)?;
+ let mut new: KVVec<u64> = KVVec::new();
+ new.resize(tiles_x * tiles_y, 0, GFP_KERNEL)?;
+
+ // Nothing moved: no rectangles, and the caller skips the write entirely.
+ assert!(changed_strip_rects(geometry, &old, &new, padded_width, padded_height)?.is_empty());
+
+ // Every other strip on every other band: 510 changed strips, and 510 separate runs on the
+ // strip grid -- four times the ceiling. On the macro-tile grid they collapse into whole
+ // rows of tiles.
+ let mut changed = 0usize;
+ for ty in (0..tiles_y).step_by(2) {
+ for tx in (0..tiles_x).step_by(2) {
+ new[ty * tiles_x + tx] = 1;
+ changed += 1;
+ }
+ }
+ assert_eq!(changed, 510);
+ let rects = changed_strip_rects(geometry, &old, &new, padded_width, padded_height)?;
+ assert!(!rects.is_empty());
+ let selected =
+ video::haar::damage_strip_coords(geometry, padded_width, padded_height, &rects)?.len();
+ // This pattern really does reach into every macro-tile, so the surface is the right
+ // answer; what matters is that it was reached by rounding and not by giving up.
+ assert_eq!(selected, 2040);
+
+ // A realistic update instead: a window redraw, a cursor and a clock. Three regions, and
+ // the cost is the macro-tiles they cover.
+ for h in new.iter_mut() {
+ *h = 0;
+ }
+ for (x0, y0, x1, y1) in [
+ (4usize, 8usize, 14usize, 30usize),
+ (20, 40, 22, 42),
+ (26, 2, 29, 4),
+ ] {
+ for ty in y0..y1 {
+ for tx in x0..x1 {
+ new[ty * tiles_x + tx] = 1;
+ }
+ }
+ }
+ let rects = changed_strip_rects(geometry, &old, &new, padded_width, padded_height)?;
+ // Every rectangle is macro-tile aligned, so the selector's own rounding adds nothing.
+ // Aligned on every edge the grid has one, and clamped to the surface where it does not:
+ // 1920 is seven whole macro-tiles and half of an eighth.
+ for &(x0, y0, x1, y1) in rects.iter() {
+ assert_eq!(x0 % geometry.macro_w(), 0);
+ assert_eq!(y0 % geometry.macro_h(), 0);
+ assert!(x1 == padded_width || x1 % geometry.macro_w() == 0);
+ assert!(y1 == padded_height || y1 % geometry.macro_h() == 0);
+ }
+ let selected =
+ video::haar::damage_strip_coords(geometry, padded_width, padded_height, &rects)?.len();
+ assert!(selected < 2040 / 2);
+ Ok(())
+ }
+
+ #[test]
+ fn parallel_encoder_matches_serial_for_every_plane_transform() -> Result {
+ use drm::kms::plane::Rotation;
+
+ let transforms = [
+ Rotation::ROTATE_0,
+ Rotation::ROTATE_90,
+ Rotation::ROTATE_180,
+ Rotation::ROTATE_270,
+ Rotation::ROTATE_0 | Rotation::REFLECT_X,
+ Rotation::ROTATE_90 | Rotation::REFLECT_X,
+ Rotation::ROTATE_180 | Rotation::REFLECT_X,
+ Rotation::ROTATE_270 | Rotation::REFLECT_X,
+ Rotation::ROTATE_0 | Rotation::REFLECT_Y,
+ Rotation::ROTATE_90 | Rotation::REFLECT_Y,
+ Rotation::ROTATE_180 | Rotation::REFLECT_Y,
+ Rotation::ROTATE_270 | Rotation::REFLECT_Y,
+ Rotation::ROTATE_0 | Rotation::REFLECT_X | Rotation::REFLECT_Y,
+ Rotation::ROTATE_90 | Rotation::REFLECT_X | Rotation::REFLECT_Y,
+ Rotation::ROTATE_180 | Rotation::REFLECT_X | Rotation::REFLECT_Y,
+ Rotation::ROTATE_270 | Rotation::REFLECT_X | Rotation::REFLECT_Y,
+ ];
+ for transform in transforms {
+ parallel_rotation_matches_serial(transform)?;
+ }
+ Ok(())
+ }
+}
diff --git a/drivers/gpu/drm/vino/drm_sink/stream.rs b/drivers/gpu/drm/vino/drm_sink/stream.rs
new file mode 100644
index 000000000000..1adb37de980e
--- /dev/null
+++ b/drivers/gpu/drm/vino/drm_sink/stream.rs
@@ -0,0 +1,470 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Opening a connector's video stream, and the records that frame it.
+//!
+//! Before any pixels reach a connector the dock has to be told what is coming: a ring of buffer
+//! addresses, a decoder configuration, and an opening that differs by family. Every record here is
+//! sealed with the connector's own video key, and the dock silently declines a stream whose
+//! opening it did not expect.
+
+use super::*;
+
+impl VinoDrmData {
+ /// Build one connector's cold video-arm burst, prepended to the first video frame after a mode
+ /// set. Records #0/#1/#4/#5 are plaintext and #6/#7 contain a fixed `type=4` body. Records
+ /// #2/#3/#8/#9 use this connector's video key and nonce, derived from the per-connector SKE
+ /// with `riv_h ^ (0x08 | connector)` in byte 7, and share one block counter. Records #8/#9
+ /// carry the decoder configuration and independent nonces. Build the per-connector video
+ /// stream-open this platform wants in place of the cold ARM burst.
+ ///
+ /// Sealed with the connector's video key like every other video-endpoint message, and prefixed
+ /// to the first frame after a mode set so it reaches the dock before any pixels. Build the
+ /// prefix that opens a connector's video stream after a mode set.
+ ///
+ /// Ridge prefixes a cold ARM burst to the first frame; Navarro prefixes a short sealed
+ /// stream-open. Both occupy the same slot ahead of any pixels, and every submission path must
+ /// pick between them the same way: sending one platform's opening to the other's dock leaves
+ /// the stream unopened, and the dock then watchdog-resets a few seconds later. Build the
+ /// records that close a frame, in this dock's format.
+ pub(super) fn build_frame_trailer(
+ &self,
+ connector: u8,
+ seq0: u32,
+ ) -> crate::video::haar::FrameTrailer {
+ let geometry = self.geometry();
+ if self.video_on_ctrl_pipe() {
+ crate::video::haar::FrameTrailer::one(&crate::video::haar::ella_frame_close(
+ geometry, connector, seq0,
+ ))
+ } else if self.uses_arm_burst() {
+ crate::video::haar::frame_trailer(geometry, connector, seq0)
+ } else {
+ crate::video::haar::navarro_frame_trailer(geometry, connector, seq0)
+ }
+ }
+ /// Build the record that starts a non-prologue DL7400 frame.
+ ///
+ /// Ridge carries its slot transition in the three-record trailer and the DL-3x00 in its single
+ /// closing record, so neither has an opener. Navarro terminates the old frame after its close
+ /// record and starts the next USB transfer with this opener instead.
+ pub(super) fn build_frame_opener(
+ &self,
+ connector: u8,
+ seq0: u32,
+ prologue: bool,
+ ) -> Option<KVec<u8>> {
+ if self.uses_arm_burst() || self.video_on_ctrl_pipe() || prologue {
+ return None;
+ }
+ let mut out = KVec::new();
+ out.extend_from_slice(
+ &crate::video::haar::navarro_frame_opener(self.geometry(), connector, seq0),
+ GFP_KERNEL,
+ )
+ .ok()
+ .map(|()| out)
+ }
+ /// Open a connector's video stream, once per mode generation, ahead of any pixels.
+ ///
+ /// Does nothing on a dock whose opening is the ARM burst carried with the first frame.
+ pub(super) fn send_stream_open(&self, dev: &BoundInterface<'_>, connector: usize) -> Result {
+ let bit = 1u32 << connector;
+ if self.stream_open_pending.load(Ordering::Acquire) & bit == 0 {
+ return Ok(());
+ }
+ let Some(open) = self.build_stream_open_buf(connector)? else {
+ self.stream_open_pending.fetch_and(!bit, Ordering::Release);
+ return Ok(());
+ };
+ let pipe_i = dev.video_pipe_index(connector)?;
+ let mut queue_slot = self.video_q[pipe_i].lock();
+ if queue_slot.is_none() {
+ *queue_slot = Some(dev.video_queue(connector, 8, VIDEO_XFER)?);
+ }
+ let queue = queue_slot
+ .as_mut()
+ .get_mut()
+ .as_mut()
+ .ok_or(kernel::error::code::ENODEV)?;
+ queue.send(dev.io(), &open, crate::timeout())?;
+ self.stream_open_pending.fetch_and(!bit, Ordering::Release);
+ vino_debug!("vino: connector {} video stream opened\n", connector);
+ Ok(())
+ }
+ /// Keep every engaged connector's video endpoint from going quiet long enough for the dock to
+ /// tear the link down.
+ ///
+ /// The DL7400 stops answering -- video *and* control -- about a second after the last byte on a
+ /// video endpoint, whatever it was doing before. A compositor with nothing to redraw leaves
+ /// vino silent well past that, so each connector that has not sent anything for
+ /// [`NAVARRO_KEEPALIVE_MS`] sends the same sealed report DLM pairs with every frame. Called
+ /// from the control keepalive, which already runs for the life of the session.
+ ///
+ /// Only connectors whose video queue is already open are fed: a connector that has never
+ /// streamed has nothing to keep alive, and opening a queue here would start a stream nothing
+ /// follows.
+ pub(crate) fn send_video_keepalive(&self, dev: &BoundInterface<'_>) {
+ if self.uses_arm_burst() || self.shutting_down.load(Ordering::Acquire) {
+ return;
+ }
+ let now = Instant::<Monotonic>::now();
+ for connector in 0..MAX_CONNECTORS {
+ if self.modeset_active[connector].load(Ordering::Acquire) == 0 {
+ continue;
+ }
+ let due = match self.last_video_at.lock()[connector] {
+ Some(last) => (now - last).as_millis() >= NAVARRO_KEEPALIVE_MS,
+ None => false,
+ };
+ if !due {
+ continue;
+ }
+ let frame = self.scanout_seq.lock()[connector];
+ let Ok(report) = self.build_stream_report_buf(connector, frame) else {
+ continue;
+ };
+ let Some(report) = report else { continue };
+ let Ok(pipe_i) = dev.video_pipe_index(connector) else {
+ continue;
+ };
+ let mut queue_slot = self.video_q[pipe_i].lock();
+ let Some(queue) = queue_slot.as_mut().get_mut().as_mut() else {
+ continue;
+ };
+ if queue.send(dev.io(), &report, crate::timeout()).is_ok() {
+ self.last_video_at.lock()[connector] = Some(Instant::<Monotonic>::now());
+ }
+ }
+ }
+ pub(super) fn build_stream_prefix_buf(&self, connector: usize) -> Result<KVec<u8>> {
+ if self.uses_arm_burst() {
+ return self.build_arm_burst_buf(connector);
+ }
+ if self.video_on_ctrl_pipe() {
+ // Sent inside the mode-set bracket instead, where the vendor puts it, so nothing is
+ // owed to the frame. The empty buffer still marks this as the frame that opens the
+ // generation. See `send_stream_prologue`.
+ return Ok(KVec::new());
+ }
+ self.build_navarro_prologue_buf(connector)
+ }
+ /// Write a connector's stream prologue on the control pipe, for a dock that has no video pipe.
+ ///
+ /// The ring descriptor and the decoder configuration are records like any other on such a
+ /// dock, so they can be ordered against the mode-set markers rather than glued to the front of
+ /// a frame. Docks with a pipe of their own carry theirs with the first frame, which is where
+ /// their own captures put it, and get nothing here.
+ pub(super) fn send_stream_prologue(&self, dev: &BoundInterface<'_>, connector: u8) -> Result {
+ self.send_stream_ring(dev, connector)?;
+ self.send_stream_config(dev, connector)?;
+ Ok(())
+ }
+ /// Whether `connector` still owes its shared-pipe stream prologue.
+ fn stream_prologue_pending(&self, connector: u8) -> bool {
+ self.video_on_ctrl_pipe()
+ && usize::from(connector) < MAX_CONNECTORS
+ && self.arm_prefix_pending.load(Ordering::Acquire) & (1u32 << connector) != 0
+ }
+ /// Send only the unsealed ring descriptor of an Ella stream prologue.
+ ///
+ /// DLM does not concatenate this record with the decoder configuration: status and marker
+ /// records sit between them during cold activation. Making the two records independent table
+ /// actions preserves that ordering while the conservative runtime path may still invoke both
+ /// through [`Self::send_stream_prologue`].
+ pub(super) fn send_stream_ring(&self, dev: &BoundInterface<'_>, connector: u8) -> Result {
+ if !self.stream_prologue_pending(connector) {
+ return Ok(());
+ }
+ let ring = crate::video::haar::ella_stream_open(self.geometry(), connector);
+ dev.ctrl_send(&ring, crate::timeout(), GFP_KERNEL)?;
+ vino_debug!(
+ "vino: connector={} stream ring descriptor sent inside the bracket ({} B)\n",
+ connector,
+ ring.len()
+ );
+ Ok(())
+ }
+ /// Send only the sealed decoder configuration of an Ella stream prologue.
+ pub(super) fn send_stream_config(&self, dev: &BoundInterface<'_>, connector: u8) -> Result {
+ if !self.stream_prologue_pending(connector) {
+ return Ok(());
+ }
+ let connector_index = usize::from(connector);
+ let config = self.build_ella_config_buf(connector_index)?;
+ dev.ctrl_send(&config, crate::timeout(), GFP_KERNEL)?;
+ vino_debug!(
+ "vino: connector={} decoder configuration sent inside the bracket ({} B)\n",
+ connector,
+ config.len()
+ );
+ Ok(())
+ }
+ /// The mode header a connector's stream states, built from the mode it was last given.
+ ///
+ /// The surface named here is the padded one the codec actually produces: a mode whose height
+ /// is not a whole number of strips is encoded as the next whole one, and the dock has to be
+ /// told the size it is going to be sent. Every captured mode on a dock with its own video pipe
+ /// is already a whole number of strips, so this only ever rounds on the shared-pipe dock,
+ /// whose 1080-line modes are stated as 1088.
+ /// Whether this connector is being driven at 30 bpp.
+ ///
+ /// The decoder configuration, the set-mode and the codec all state the depth, and all read it
+ /// from [`Self::connector_programmed_ten_bit`] so that they cannot disagree.
+ fn connector_ten_bit(&self, connector: usize) -> bool {
+ u8::try_from(connector).is_ok_and(|c| self.connector_programmed_ten_bit(c))
+ }
+
+ fn stream_mode_header(&self, connector: usize) -> Result<[u8; 26]> {
+ let timing = self
+ .last_timing
+ .lock()
+ .get(connector)
+ .copied()
+ .flatten()
+ .ok_or(ENODEV)?;
+ let geometry = self.geometry();
+ let pad = |value: u16, unit: usize| -> u16 {
+ let unit = unit.max(1) as u16;
+ value.div_ceil(unit).saturating_mul(unit)
+ };
+ Ok(crate::video_arm::mode_header(
+ pad(timing.hactive, geometry.strip_w()),
+ pad(timing.vactive, geometry.strip_h()),
+ self.layout_word(),
+ timing.ten_bit,
+ ))
+ }
+ /// This connector's video sealing key and nonce, as [`set_video_keys`](Self::set_video_keys)
+ /// stored them: the whitened SKE key at the front and the stream content nonce behind it.
+ fn video_seal_key(&self, connector: usize) -> Result<(kernel::crypto::Secret<16>, [u8; 8])> {
+ let keys = self.video_keys.lock();
+ let key = keys.get(connector).ok_or(EINVAL)?;
+ let mut vkey = kernel::crypto::Secret::zeroed();
+ vkey.copy_from_slice(&key[..16]);
+ let mut vnonce = [0u8; 8];
+ vnonce.copy_from_slice(&key[16..24]);
+ Ok((vkey, vnonce))
+ }
+ /// Build the sealed decoder configuration a connector owes ahead of its first frame on a dock
+ /// that shares its control pipe.
+ ///
+ /// The stream itself was announced during CP setup, where the plaintext markers and the sealed
+ /// open could be ordered against the rest of the sequence. What is left is what DLM sends after
+ /// the mode set. The configuration continues the block counter the setup open started, which is
+ /// why it must not be rebuilt from block zero.
+ fn build_ella_config_buf(&self, connector: usize) -> Result<KVec<u8>> {
+ let (vkey, vnonce) = self.video_seal_key(connector)?;
+ let header = self.stream_mode_header(connector)?;
+ let connector_selector = u8::try_from(connector).map_err(|_| EINVAL)?;
+ let geometry = self.geometry();
+ let stream = geometry.stream_id(connector_selector);
+ let config = crate::video_arm::build_config(
+ self.code_tables(),
+ &header,
+ &[],
+ self.connector_ten_bit(connector),
+ )?;
+ let seq = self.take_seal_seq(connector, config.len().div_ceil(16) as u32);
+ crate::cp::seal_video_arm(&vkey, &vnonce, stream, 0x0000, seq, &config)
+ }
+ /// Build the message a connector's video stream opens with, sent alone ahead of everything
+ /// else.
+ ///
+ /// Ridge has none: its ARM burst opens the stream from within the first frame's transfer.
+ ///
+ /// Navarro has none either, for a connector it is about to drive. The short sealed open does
+ /// exist on this dock, but both DLM captures send it only on the stream ids of the connectors
+ /// with no monitor -- `0x17` and `0x1f` while pixels went to connectors 0 and 1 -- each as the
+ /// first and only record on its own stream, sealed with that connector's own key at block 0. A
+ /// driven connector's sealed chain instead opens with the pipe descriptor at block 0.
+ ///
+ /// It must not go out on `stream_id | 0x10`, which for connector 0 is connector 2's stream id,
+ /// sealed with connector 0's key. That both signed another connector's stream with the wrong
+ /// key and, because the prologue then also started at block 0, used the connector's first
+ /// keystream block twice.
+ fn build_stream_open_buf(&self, connector: usize) -> Result<Option<KVec<u8>>> {
+ // A dock that carries video on the control pipe has already opened its streams: the same
+ // record went out inside the CP setup burst, where it can be ordered against the rest of
+ // setup. Sending a second one here would seal a block the dock has already accounted for.
+ if self.uses_arm_burst() || self.video_on_ctrl_pipe() {
+ return Ok(None);
+ }
+ let (vkey, vnonce) = self.video_seal_key(connector)?;
+ let content = crate::cp::navarro_stream_open();
+ let stream = self.geometry().stream_id(connector as u8);
+ let seq = self.take_seal_seq(connector, content.len().div_ceil(16) as u32);
+ Ok(Some(crate::cp::seal_video_arm(
+ &vkey, &vnonce, stream, 0x0002, seq, &content,
+ )?))
+ }
+ /// Build the sealed report a connector owes its stream for one frame.
+ ///
+ /// DLM pairs every frame on the frame sub with one of these on the stream sub, so a stream
+ /// that sends pixels and then falls silent on its stream sub is a stream the dock stops
+ /// believing in. Returns `None` on a dock whose frames carry no such record.
+ ///
+ /// The ordinary `aux=0x000c` form is what DLM sends for all but a handful of frames; the
+ /// `aux=0x0002` form restates the mode and goes out with the frame that carries the prologue,
+ /// which is the frame right after a mode set.
+ pub(super) fn build_stream_report_buf(
+ &self,
+ connector: usize,
+ frame: u32,
+ ) -> Result<Option<KVec<u8>>> {
+ if self.uses_arm_burst() {
+ return Ok(None);
+ }
+ let owed = self.stream_reports_owed.get(connector).ok_or(EINVAL)?;
+ if self.video_on_ctrl_pipe()
+ && (frame < STREAM_REPORT_FRAME
+ || owed
+ .fetch_update(Ordering::AcqRel, Ordering::Acquire, |n| n.checked_sub(1))
+ .is_err())
+ {
+ return Ok(None);
+ }
+ let (vkey, vnonce) = self.video_seal_key(connector)?;
+ let stream = self.geometry().stream_id(connector as u8);
+ let with_mode = self.video_on_ctrl_pipe()
+ || self.arm_prefix_pending.load(Ordering::Acquire) & (1u32 << connector) != 0;
+ let (aux, content): (u16, KVec<u8>) = if with_mode {
+ let header = self.stream_mode_header(connector)?;
+ let mut v = KVec::new();
+ if self.video_on_ctrl_pipe() {
+ v.extend_from_slice(&crate::cp::stream_report_mode_only(&header), GFP_KERNEL)?;
+ (0x0006, v)
+ } else {
+ v.extend_from_slice(&crate::cp::navarro_stream_report_mode(&header), GFP_KERNEL)?;
+ (0x0002, v)
+ }
+ } else {
+ let mut v = KVec::new();
+ v.extend_from_slice(&crate::cp::navarro_stream_report(), GFP_KERNEL)?;
+ (0x000c, v)
+ };
+ let seq = self.take_seal_seq(connector, content.len().div_ceil(16) as u32);
+ Ok(Some(crate::cp::seal_video_arm(
+ &vkey, &vnonce, stream, aux, seq, &content,
+ )?))
+ }
+ /// Build the DL7400 records that precede a connector's first frame.
+ ///
+ /// In wire order: two plaintext stream markers, the sealed pipe descriptor, a plaintext frame
+ /// marker, an unsealed record naming the connector's first and fifth ring addresses, and the
+ /// sealed decoder configuration. Both sealed records draw from the stream's running block
+ /// counter, so on a first arm the descriptor seals at block 0 and the configuration at block
+ /// 19 -- the descriptor's 304 bytes in blocks -- exactly as DLM's `0 -> 19 -> 88` chain does.
+ fn build_navarro_prologue_buf(&self, connector: usize) -> Result<KVec<u8>> {
+ let (vkey, vnonce) = self.video_seal_key(connector)?;
+ let connector_selector = connector as u8;
+ let geometry = self.geometry();
+ let stream = geometry.stream_id(connector_selector);
+ let frame_sub = u16::from(geometry.connector_selector(connector_selector));
+
+ let mut buf = KVec::with_capacity(1600, GFP_KERNEL)?;
+ for sub in [stream, stream | 0x0010] {
+ buf.extend_from_slice(
+ &crate::cp::stream_announce(sub, crate::cp::STREAM_ANNOUNCE_MARKER),
+ GFP_KERNEL,
+ )?;
+ }
+
+ let descriptor = crate::cp::navarro_pipe_descriptor(connector_selector)?;
+ let seal_seq = self.take_seal_seq(connector, descriptor.len().div_ceil(16) as u32);
+ let sealed =
+ crate::cp::seal_video_arm(&vkey, &vnonce, stream, 0x0000, seal_seq, &descriptor)?;
+ buf.extend_from_slice(&sealed, GFP_KERNEL)?;
+
+ buf.extend_from_slice(&crate::cp::stream_announce(frame_sub, 0), GFP_KERNEL)?;
+
+ // Unsealed type-4 record: the connector_selector's first and fifth ring addresses.
+ let mut ring = [0u8; 32];
+ ring[2..4].copy_from_slice(&0x001cu16.to_le_bytes());
+ ring[4..8].copy_from_slice(&4u32.to_le_bytes());
+ ring[8..10].copy_from_slice(&frame_sub.to_le_bytes());
+ ring[10..12].copy_from_slice(&0x0004u16.to_le_bytes());
+ ring[16..19].copy_from_slice(&[0x0a, 0x00, 0x04]);
+ ring[19] = frame_sub as u8;
+ ring[22..26]
+ .copy_from_slice(&crate::cp::navarro_pipe_ring(connector_selector, 0).to_le_bytes());
+ ring[26..30]
+ .copy_from_slice(&crate::cp::navarro_pipe_ring(connector_selector, 4).to_le_bytes());
+ buf.extend_from_slice(&ring, GFP_KERNEL)?;
+
+ let mut tail = [0u8; 14];
+ crate::rng::fill(&mut tail);
+ let config = crate::video_arm::build_config(
+ self.code_tables(),
+ &self.stream_mode_header(connector)?,
+ &tail,
+ self.connector_ten_bit(connector),
+ )?;
+ let seal_seq = self.take_seal_seq(connector, config.len().div_ceil(16) as u32);
+ buf.extend_from_slice(
+ &crate::cp::seal_video_arm(&vkey, &vnonce, stream, 0x000e, seal_seq, &config)?,
+ GFP_KERNEL,
+ )?;
+ Ok(buf)
+ }
+ fn build_arm_burst_buf(&self, connector: usize) -> Result<KVec<u8>> {
+ let (vkey, vnonce) = self.video_seal_key(connector)?;
+ let h = connector as u16;
+ // The sealed records share the video channel's running block counter:
+ // #2 seq0(+1), #3 seq1(+1), #8 seq2(+69), and #9 seq71.
+ let mut seal_seq: u32 = 0;
+ let mut buf = KVec::with_capacity(2560, GFP_KERNEL)?;
+ for (i, &(_wire_type, sub_base, aux, body_len)) in
+ crate::cp::VIDEO_ARM_BURST.iter().enumerate()
+ {
+ let sub = sub_base.wrapping_add(h);
+ match i {
+ 2 | 3 => {
+ // Sealed under the per-connector video key/nonce. Content = the six-byte stream
+ // marker + 10 host-random bytes; seq is the shared block counter (16 B = 1
+ // block each).
+ let content = crate::cp::stream_open(self.stream_marker_kind());
+ let frame =
+ crate::cp::seal_video_arm(&vkey, &vnonce, sub, aux, seal_seq, &content)?;
+ seal_seq += 1;
+ buf.extend_from_slice(&frame, GFP_KERNEL)?;
+ }
+ 6 | 7 => {
+ // type=4 but FIXED plaintext (not encrypted, no MAC): a 32-byte frame whose
+ // Its 16-byte body is fixed, with 0x10 at byte 11.
+ let mut f = [0u8; 32];
+ f[2..4].copy_from_slice(&0x001cu16.to_le_bytes());
+ f[4..8].copy_from_slice(&4u32.to_le_bytes());
+ f[8..10].copy_from_slice(&sub.to_le_bytes());
+ f[10..12].copy_from_slice(&aux.to_le_bytes());
+ f[16..32].copy_from_slice(&[
+ 0x0a, 0x00, 0x04, 0x00, 0, 0, 0, 0, 0, 0, 0, 0x10, 0, 0, 0, 0,
+ ]);
+ buf.extend_from_slice(&f, GFP_KERNEL)?;
+ }
+ 8 | 9 => {
+ debug_assert_eq!(body_len, 1104);
+ let mut nonce = [0u8; 14];
+ crate::rng::fill(&mut nonce);
+ let content = crate::video_arm::build_config(
+ self.code_tables(),
+ &self.stream_mode_header(connector)?,
+ &nonce,
+ self.connector_ten_bit(connector),
+ )?;
+ debug_assert_eq!(content.len(), body_len);
+ let frame =
+ crate::cp::seal_video_arm(&vkey, &vnonce, sub, aux, seal_seq, &content)?;
+ seal_seq += (body_len / 16) as u32;
+ buf.extend_from_slice(&frame, GFP_KERNEL)?;
+ }
+ _ => {
+ // wire_type==2 plaintext records (#0/#1/#4/#5).
+ let body = crate::cp::video_arm_plaintext_body(i, h);
+ let frame = crate::cp::video_arm_plain_frame(sub, &body);
+ buf.extend_from_slice(&frame, GFP_KERNEL)?;
+ }
+ }
+ }
+ Ok(buf)
+ }
+}
diff --git a/drivers/gpu/drm/vino/drm_sink/timeline.rs b/drivers/gpu/drm/vino/drm_sink/timeline.rs
new file mode 100644
index 000000000000..d6db1a2eb0e8
--- /dev/null
+++ b/drivers/gpu/drm/vino/drm_sink/timeline.rs
@@ -0,0 +1,571 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Measured bring-up and mode-set timelines.
+//!
+//! A cold dock will not light from a correct message sequence alone; the vendor's spacing is
+//! part of the contract, and a dock driven faster than this stays dark or resets. Every delay
+//! here was read off a capture of the vendor driver waking the same hardware.
+
+/// Dual-connector cold-wake timeline relative to the connector-0 mode set.
+///
+/// EP02 must remain quiet between `H1_MODE` and `QUIET_END`; the dock also requires a connector-1
+/// EDID probe and fetch before video starts.
+pub(crate) mod cold {
+ pub(crate) const H1_MODE: i64 = 29;
+ /// End of the silent window. Nothing may be sent on EP02 between `H1_MODE` and here.
+ pub(crate) const QUIET_END: i64 = 1016;
+ pub(crate) const H0_VIDEO: i64 = 1159;
+ pub(crate) const H1_VIDEO: i64 = 1233;
+ /// `(offset_ms, connector, sub, state)` stream markers. `sub` 0x2f/0x2e as on the wire.
+ pub(crate) const MARKERS: &[(i64, u8, u16, u8)] = &[
+ (17, 0, 0x2f, 1),
+ (21, 0, 0x2e, 3),
+ (1016, 0, 0x2f, 1),
+ (1021, 1, 0x2f, 1),
+ (1023, 0, 0x2e, 3),
+ (1029, 1, 0x2e, 3),
+ (1056, 0, 0x2f, 1),
+ (1057, 1, 0x2f, 1),
+ (1064, 0, 0x2e, 0),
+ (1124, 1, 0x2e, 3),
+ (1132, 1, 0x2f, 1),
+ (1135, 1, 0x2e, 0),
+ (1195, 0, 0x2f, 0),
+ (1208, 0, 0x2e, 0),
+ (1220, 1, 0x2f, 1),
+ (1225, 1, 0x2e, 0),
+ (1295, 1, 0x2f, 0),
+ (1298, 1, 0x2e, 0),
+ ];
+ /// `id=0x14 sub=0x0c` status polls.
+ pub(crate) const POLLS: &[i64] = &[
+ 5, 26, 1019, 1130, 1192, 1204, 1222, 1235, 1253, 1270, 1287, 1304,
+ ];
+ /// `(offset_ms, connector, is_fetch)` -- `false` is the `0x15/0x20` probe, `true` the
+ /// `0x15/0x21` fetch.
+ pub(crate) const EDID: &[(i64, u8, bool)] = &[(1033, 1, false), (1059, 1, true)];
+ /// Keep both carriers active until downstream clock programming completes.
+ pub(crate) const CARRIER_TAIL_MS: i64 = 800;
+}
+
+/// A dock's cold bring-up choreography, anchored on the first connector's mode set.
+///
+/// Ridge and Navarro differ in more than timing: Navarro opens each connector's bracket with a
+/// state-0 pair before the state-1/3 pair, spaces its two mode sets 757 ms apart rather than 29 ms,
+/// sets connector 0's mode a *second* time shortly before connector 0's video, and streams
+/// connector 1 first. Replaying one dock's timeline at the other leaves the endpoint unarmed. Every
+/// connector field below is a transcript slot, not a connector number: 0 is the first connector an
+/// activation brings up and 1 the second, whichever sockets they occupy. Both timelines were
+/// recorded with the panels in the first two sockets, where the two happen to coincide.
+/// [`VinoDrmData::activate_dual_wake`] resolves them.
+pub(super) struct ColdTimeline {
+ /// Offset of the second slot's mode set.
+ pub(crate) h1_mode: i64,
+ /// End of any silent window on EP02 after the second mode set.
+ pub(crate) quiet_end: i64,
+ /// Slots in the order they start streaming, with the offset each starts at.
+ pub(crate) video: &'static [(usize, i64)],
+ /// Mode sets repeated after the initial pair, as `(offset, slot)`.
+ pub(crate) remode: &'static [(i64, usize)],
+ /// `(offset_ms, slot, sub, state)` stream markers. `sub` 0x2f/0x2e as on the wire.
+ pub(crate) markers: &'static [(i64, u8, u16, u8)],
+ /// `id=0x14 sub=0x0c` status polls.
+ pub(crate) polls: &'static [i64],
+ /// `(offset_ms, slot, is_fetch)` EDID re-reads inside the bracket.
+ pub(crate) edid: &'static [(i64, u8, bool)],
+}
+
+/// Ridge's timeline, as replayed from a D6000 cold bring-up.
+pub(super) static COLD_RIDGE: ColdTimeline = ColdTimeline {
+ h1_mode: cold::H1_MODE,
+ quiet_end: cold::QUIET_END,
+ video: &[(0, cold::H0_VIDEO), (1, cold::H1_VIDEO)],
+ remode: &[],
+ markers: cold::MARKERS,
+ polls: cold::POLLS,
+ edid: cold::EDID,
+};
+
+/// Navarro's timeline, measured from a DLM cold bring-up and anchored on connector 0's real
+/// (`off23 = 2`) mode set, exactly as Ridge's is.
+pub(super) static COLD_NAVARRO: ColdTimeline = ColdTimeline {
+ h1_mode: 10,
+ // DLM polls continuously across this span; there is no silent window to preserve.
+ quiet_end: 11,
+ // Video is not a pair of one-shot events: DLM keeps connector 0's carrier alive throughout the
+ // still-open control bracket, starts connector 1, and continues both through the closing
+ // markers. A gap here makes the dock accept one frame and NAK the next forever. The activation
+ // path uses its pre-encoded carrier; normal scanout replaces it as soon as activation returns.
+ video: &[
+ (0, 122),
+ (0, 124),
+ (0, 134),
+ (0, 171),
+ (0, 192),
+ (0, 199),
+ (0, 235),
+ (0, 252),
+ (1, 272),
+ (0, 277),
+ (1, 293),
+ (1, 303),
+ ],
+ remode: &[],
+ markers: &[
+ (7, 0, 0x2f, 1),
+ (13, 0, 0x2e, 3),
+ (20, 1, 0x2f, 1),
+ (21, 0, 0x2f, 1),
+ (35, 0, 0x2e, 0),
+ (76, 1, 0x2e, 3),
+ (104, 1, 0x2f, 1),
+ (128, 1, 0x2e, 3),
+ (131, 0, 0x2f, 1),
+ (136, 0, 0x2e, 0),
+ (168, 1, 0x2f, 1),
+ (181, 1, 0x2e, 0),
+ (228, 0, 0x2f, 0),
+ (230, 0, 0x2e, 0),
+ (303, 1, 0x2f, 0),
+ (304, 1, 0x2e, 0),
+ ],
+ polls: &[17, 78, 120, 162, 179, 223, 267, 295, 297, 297],
+ // Navarro reads every connector's EDID before the anchor, not inside the bracket.
+ edid: &[],
+};
+
+/// One step of a DL-3x00 dock-wide activation.
+///
+/// Every connector field is an activation slot, not a connector number: slot 0 is the
+/// lowest-numbered activating connector and slot 1 the next, resolved to real connectors at the
+/// point of send, exactly as [`ColdTimeline`]'s are. Steps carry no offsets because the vendor's
+/// are separated by its own frames rather than by a clock; [`DockWideStep::Stream`] states the ones
+/// that matter.
+#[derive(Clone, Copy, PartialEq, Eq, Debug)]
+pub(crate) enum DockWideStep {
+ /// `0x48/0x22`, this slot's mode.
+ SetMode(u8),
+ /// `0x16/0x2e` or `0x16/0x2f`, with the state in byte 23.
+ Marker(u8, u16, u8),
+ /// `id=0x14 sub=0x0c` device status.
+ Poll,
+ /// The ring descriptor and decoder configuration that open this slot's stream as one unit.
+ ///
+ /// Kept for the conservative runtime re-arm. The cold Ella transcript separates these two
+ /// records with control traffic and uses [`DockWideStep::Ring`] / [`DockWideStep::Config`]
+ /// instead, so the table can state those producer boundaries exactly.
+ Prologue(u8),
+ /// The unsealed ring descriptor that starts this slot's stream generation.
+ Ring(u8),
+ /// The sealed decoder configuration that follows a slot's ring descriptor.
+ Config(u8),
+ /// This slot's activation carrier -- the flat surface its stream opens with.
+ Carrier(u8),
+ /// Frames the vendor presents on `slot` before its next control record.
+ ///
+ /// The second connector comes up behind a running stream, and this is that stream. A dock
+ /// whose activation is spaced by wall clock rather than by frames has nothing to put here.
+ Stream(u8, u32),
+}
+
+/// How the vendor brings both connectors of a DL-3x00 dock up, measured record for record.
+///
+/// Two rules distinguish it from a driver that simply repeats a per-connector bracket, and both are
+/// load-bearing. A single dock-wide transaction configures *both* connectors before any pixels:
+/// the mode sets are adjacent, and every marker between them addresses the first connector. And
+/// the second connector's sink is held down (`0x2e` state 3) across that transaction, coming up
+/// only once the first is streaming -- with no mode set of its own, because it already has one.
+///
+/// A second independent bracket instead is what the dock stops answering: it acknowledges
+/// everything up to the second bracket's first marker and nothing afterwards, which reads as a
+/// dead dock and ends with every scanout returning `ENODEV`.
+pub(crate) static ELLA_DOCK_WIDE: &[DockWideStep] = &[
+ DockWideStep::SetMode(0),
+ DockWideStep::Marker(0, 0x2f, 1),
+ DockWideStep::Marker(0, 0x2e, 3),
+ DockWideStep::SetMode(1),
+ DockWideStep::Marker(0, 0x2f, 1),
+ DockWideStep::Marker(0, 0x2e, 0),
+ DockWideStep::Marker(1, 0x2f, 1),
+ DockWideStep::Ring(0),
+ DockWideStep::Poll,
+ DockWideStep::Config(0),
+ DockWideStep::Marker(1, 0x2e, 3),
+ DockWideStep::Marker(0, 0x2f, 1),
+ DockWideStep::Marker(0, 0x2e, 0),
+ DockWideStep::Carrier(0),
+ DockWideStep::Stream(0, 2),
+ DockWideStep::Marker(1, 0x2f, 1),
+ DockWideStep::Marker(0, 0x2f, 0),
+ DockWideStep::Stream(0, 2),
+ DockWideStep::Marker(1, 0x2e, 0),
+ DockWideStep::Stream(0, 3),
+ DockWideStep::Poll,
+ DockWideStep::Ring(1),
+ DockWideStep::Marker(0, 0x2e, 0),
+ DockWideStep::Stream(0, 1),
+ DockWideStep::Config(1),
+ DockWideStep::Poll,
+ DockWideStep::Carrier(1),
+ // The second connector's sink is completed the same way the first one's was, and only once its
+ // own carrier is running: assert, bring up, release, bring up again. The first connector
+ // receives exactly this four-marker tail earlier in the transaction, and leaving it off the
+ // second one ends the transaction with that sink half raised.
+ DockWideStep::Marker(1, 0x2f, 1),
+ DockWideStep::Stream(1, 2),
+ DockWideStep::Poll,
+ DockWideStep::Marker(1, 0x2e, 0),
+ DockWideStep::Poll,
+ DockWideStep::Marker(1, 0x2f, 0),
+ DockWideStep::Poll,
+ DockWideStep::Marker(1, 0x2e, 0),
+];
+
+/// How the vendor reconfigures one connector of a DL-3x00 dock while the other one is lit.
+///
+/// This is not the dock-wide sequence with a connector left out. It is shorter, it takes the sink
+/// down *before* the mode set rather than after it, and it touches nothing belonging to the other
+/// connector -- which keeps streaming across the whole of it. Replaying the cold bracket here
+/// instead is what silences the dock: it stops answering at the first marker and does not answer
+/// again.
+///
+/// Measured twice, at two different resolutions, with identical shape both times.
+///
+/// The vendor also re-reads the sink's EDID between the sink-down and the mode set. That is left
+/// out: vino reads EDID on its own schedule, and a fetch issued inside a transaction has nowhere
+/// to deliver its reply.
+pub(crate) static ELLA_RUNTIME_MODE: &[DockWideStep] = &[
+ DockWideStep::Marker(0, 0x2f, 1),
+ DockWideStep::Marker(0, 0x2e, 3),
+ DockWideStep::SetMode(0),
+ DockWideStep::Marker(0, 0x2f, 1),
+ DockWideStep::Marker(0, 0x2e, 0),
+ DockWideStep::Poll,
+ DockWideStep::Prologue(0),
+ DockWideStep::Carrier(0),
+];
+
+/// Reservation-token slots for [`COLD_NAVARRO::markers`] and [`COLD_NAVARRO::polls`].
+///
+/// DLM assigns counters in its per-connector workers before their EP02 writes interleave. Wire AES
+/// sequence remains monotonic, but the echoed inner counters consequently do not: for example the
+/// wire order begins `n, n+1, n+3, n+2, n+5, n+4`. Navarro starts NAKing at the first flattened
+/// counter, so retain that allocation order. These numbers index live reservation tokens; they
+/// are not protocol counters and are never added to a captured/base counter.
+pub(super) static NAVARRO_MARKER_COUNTER_SLOTS: &[usize] =
+ &[1, 2, 4, 6, 8, 7, 10, 12, 11, 14, 16, 17, 20, 21, 26, 27];
+pub(super) static NAVARRO_POLL_COUNTER_SLOTS: &[usize] = &[5, 9, 13, 15, 18, 19, 22, 23, 24, 25];
+pub(super) const NAVARRO_COLD_COUNTERS: usize = 28;
+
+/// One operation in Navarro's cold sink-reset prelude. This is separate from [`ColdTimeline`]: it
+/// runs before the first real mode set and changes downstream EDID/sink state, whereas
+/// `ColdTimeline` brackets already-programmed streams.
+#[derive(Clone, Copy)]
+pub(super) enum NavarroColdOp {
+ Poll,
+ EdidState(u8, u8),
+ Probe(u8),
+ Fetch(u8),
+ /// Tear the downstream sink down. Offset 23 is the literal state `0xff`, not a connector.
+ SinkTeardown(u8),
+ /// Engage the downstream sink.
+ ///
+ /// `id=0x16 sub=0x23` names the connector twice, at offset 22 and at offset 23, which is why it
+ /// is [`crate::cp::edid_sink_state`]`(connector, connector)`. It is a distinct variant rather
+ /// than a state so that the remap below cannot mistake the second selector for a constant: the
+ /// dock acknowledges a mismatched pair and then never enables the sink.
+ Engage(u8),
+ PostEdid(u8),
+ Clear(u8),
+}
+
+impl NavarroColdOp {
+ /// Translate this op's transcript slot into the connector that slot stands for in this
+ /// activation.
+ ///
+ /// The captured sequence names connectors 0 and 1 because that is where DLM's panels were. It
+ /// describes the first and second connector being brought up, whichever sockets those are.
+ pub(crate) fn remap_head(self, remap: &impl Fn(u8) -> u8) -> Self {
+ match self {
+ Self::Poll => Self::Poll,
+ Self::EdidState(h, s) => Self::EdidState(remap(h), s),
+ Self::Probe(h) => Self::Probe(remap(h)),
+ Self::Fetch(h) => Self::Fetch(remap(h)),
+ Self::SinkTeardown(h) => Self::SinkTeardown(remap(h)),
+ Self::Engage(h) => Self::Engage(remap(h)),
+ Self::PostEdid(h) => Self::PostEdid(remap(h)),
+ Self::Clear(h) => Self::Clear(remap(h)),
+ }
+ }
+}
+
+/// Authenticated DLM transaction between Navarro's first clear pair and its first real mode.
+///
+/// Offsets are milliseconds from the first connector-0 clear in
+/// `navarro-dlm-today-124144/wire.pcapng`. Equal offsets deliberately retain wire order. Most
+/// importantly, DLM stops both EDID readers and sends sink state `0xff` immediately after the
+/// first clears, then re-reads and re-engages each sink before its second clear. Omitting this
+/// whole state transition left the video endpoints accepting one bulk transfer and NAKing every
+/// subsequent transfer.
+pub(super) static NAVARRO_COLD_PRELUDE: &[(i64, NavarroColdOp)] = &[
+ (2, NavarroColdOp::EdidState(0, 0)),
+ (3, NavarroColdOp::Probe(0)),
+ (5, NavarroColdOp::EdidState(1, 0)),
+ (5, NavarroColdOp::Probe(1)),
+ (7, NavarroColdOp::SinkTeardown(0)),
+ (7, NavarroColdOp::SinkTeardown(1)),
+ (8, NavarroColdOp::Poll),
+ (30, NavarroColdOp::Poll),
+ (50, NavarroColdOp::Poll),
+ (69, NavarroColdOp::Poll),
+ (87, NavarroColdOp::Poll),
+ (105, NavarroColdOp::Poll),
+ (123, NavarroColdOp::Poll),
+ (143, NavarroColdOp::Poll),
+ (162, NavarroColdOp::Poll),
+ (181, NavarroColdOp::Poll),
+ (201, NavarroColdOp::Poll),
+ (219, NavarroColdOp::Poll),
+ (237, NavarroColdOp::Poll),
+ (255, NavarroColdOp::Poll),
+ (273, NavarroColdOp::Poll),
+ (293, NavarroColdOp::Poll),
+ (312, NavarroColdOp::Poll),
+ (328, NavarroColdOp::Poll),
+ (329, NavarroColdOp::Poll),
+ (329, NavarroColdOp::Poll),
+ (329, NavarroColdOp::Poll),
+ (330, NavarroColdOp::Poll),
+ (330, NavarroColdOp::Poll),
+ (1216, NavarroColdOp::Poll),
+ (1233, NavarroColdOp::Poll),
+ (1315, NavarroColdOp::Poll),
+ (1650, NavarroColdOp::Poll),
+ (1667, NavarroColdOp::Poll),
+ (1750, NavarroColdOp::Poll),
+ (1755, NavarroColdOp::Probe(0)),
+ (1757, NavarroColdOp::EdidState(0, 1)),
+ (1758, NavarroColdOp::Probe(0)),
+ (1805, NavarroColdOp::Fetch(0)),
+ (1810, NavarroColdOp::Engage(0)),
+ (1822, NavarroColdOp::Clear(0)),
+ (1827, NavarroColdOp::Poll),
+ (1850, NavarroColdOp::Probe(1)),
+ (1853, NavarroColdOp::EdidState(1, 1)),
+ (1856, NavarroColdOp::Probe(1)),
+ (1902, NavarroColdOp::PostEdid(0)),
+ (1903, NavarroColdOp::Fetch(1)),
+ (1907, NavarroColdOp::Engage(1)),
+ (1930, NavarroColdOp::Clear(1)),
+ (1934, NavarroColdOp::Poll),
+ (1956, NavarroColdOp::Poll),
+ (1975, NavarroColdOp::Poll),
+ (1994, NavarroColdOp::Poll),
+ (2003, NavarroColdOp::Poll),
+ (2005, NavarroColdOp::Poll),
+ (2007, NavarroColdOp::PostEdid(1)),
+ (2016, NavarroColdOp::Poll),
+];
+
+/// Navarro tears both pipe descriptors down first, then executes
+/// [`NAVARRO_COLD_PRELUDE`] before programming the first real mode.
+pub(super) const NAVARRO_PRIME_CLEAR_H1_MS: i64 = 2;
+
+/// One status-poll interval, the gap the prelude's own trailing polls run at.
+pub(super) const NAVARRO_REAL_MODE_SETTLE_MS: i64 = 20;
+
+/// When the cold prelude's last operation is due.
+pub(crate) const NAVARRO_COLD_PRELUDE_END_MS: i64 =
+ NAVARRO_COLD_PRELUDE[NAVARRO_COLD_PRELUDE.len() - 1].0;
+
+/// Where the first real mode goes, measured from the start of activation.
+///
+/// DLM's capture puts it at 2978 ms, but DLM's own prelude finishes at 2016 ms; the ~960 ms
+/// between is dead air in that capture rather than something the dock asked for, and it is over
+/// half a 5.59 s enumerate-to-pixels bring-up. Follow the end of the prelude plus one poll
+/// interval, so the two cannot drift apart when the prelude changes.
+pub(crate) const NAVARRO_REAL_MODE_H0_MS: i64 =
+ NAVARRO_COLD_PRELUDE_END_MS + NAVARRO_REAL_MODE_SETTLE_MS;
+
+/// How long the KMS worker waits for the rest of a multi-connector atomic commit's mode sets.
+///
+/// Bounded so a genuine single-connector commit costs at most this before proceeding.
+pub(super) const MODESET_BATCH_SETTLE_MS: i64 = 20;
+
+/// Number of back-to-back presentations of one already-encoded full frame while a newly-mode-set
+/// downstream is training.
+pub(crate) const COLD_TRAINING_PRESENTATIONS: u32 = 8;
+
+#[cfg(CONFIG_DRM_VINO_KUNIT_TEST)]
+use kernel::prelude::kunit_tests;
+
+#[cfg(CONFIG_DRM_VINO_KUNIT_TEST)]
+#[kunit_tests(vino_timeline)]
+mod tests {
+ use super::*;
+ use crate::*;
+
+ /// A mode is bounded by link rate, not by refresh, except where the dock itself clamps.
+ ///
+ /// The first real mode follows the cold prelude; it does not sit at a copied capture offset.
+ ///
+ /// DLM's capture put its first real mode at 2978 ms while its own prelude finished at
+ /// 2016 ms. Deriving the offset keeps the two from drifting apart, and this pins the
+ /// direction: the mode must come *after* the prelude's last op, and must not reintroduce the
+ /// second of dead air that made it 53% of a bring-up.
+ #[test]
+ fn the_first_real_mode_follows_the_prelude_rather_than_the_capture() {
+ let last = NAVARRO_COLD_PRELUDE_END_MS;
+ assert_eq!(last, 2016);
+ assert!(NAVARRO_REAL_MODE_H0_MS > last);
+ assert!(NAVARRO_REAL_MODE_H0_MS - last <= 100);
+ }
+
+ /// The DL-3x00 dock-wide activation, step for step against the vendor's record stream.
+ ///
+ /// The order below is transcribed from a decrypted DLM bring-up; the vendor's frames between
+ /// its control records are what the settles stand in for. Two properties of it are what the
+ /// dock actually enforces, and both are checked separately afterwards, because a plausible
+ /// reordering breaks them without changing any single step.
+ #[test]
+ fn ella_dock_wide_matches_the_dlm_capture() -> Result {
+ use super::DockWideStep::*;
+ let want = [
+ SetMode(0), // vendor #86
+ Marker(0, 0x2f, 1), // #87
+ Marker(0, 0x2e, 3), // #88
+ SetMode(1), // #89, inside the first connector's bracket
+ Marker(0, 0x2f, 1), // #90
+ Marker(0, 0x2e, 0), // #91
+ Marker(1, 0x2f, 1), // #92
+ Ring(0), // #93 ring descriptor
+ Poll, // #94, between the two records of the prologue
+ Config(0), // #95 decoder configuration
+ Marker(1, 0x2e, 3), // #96, the second connector's sink held down
+ Marker(0, 0x2f, 1), // #97
+ Marker(0, 0x2e, 0), // #98, the last record before pixels
+ Carrier(0), // #99
+ Stream(0, 2), // #99..#227, the first connector streaming
+ Marker(1, 0x2f, 1), // #229
+ Marker(0, 0x2f, 0), // #247
+ Stream(0, 2), // #248..#429
+ Marker(1, 0x2e, 0), // #431, the second connector's sink up behind a running stream
+ Stream(0, 3), // #432..#629
+ Poll, // #632
+ Ring(1), // #633 ring descriptor, ahead of the marker
+ Marker(0, 0x2e, 0), // #634
+ Stream(0, 1), // one first-connector frame, #635..#732
+ Config(1), // #733 decoder configuration
+ Poll, // #734, the last record before the second connector's pixels
+ Carrier(1), // #735
+ // The second connector's sink is completed exactly as the first one's was, and only
+ // behind its own running carrier. Leaving this off ends the transaction with that
+ // sink half raised.
+ Marker(1, 0x2f, 1), // #737
+ Stream(1, 2), // #738..#742, two more carrier frames
+ Poll, // #743
+ Marker(1, 0x2e, 0), // #744
+ Poll, // #745
+ Marker(1, 0x2f, 0), // #746
+ Poll, // #747
+ Marker(1, 0x2e, 0), // #748
+ ];
+ assert_eq!(ELLA_DOCK_WIDE, &want[..]);
+ Ok(())
+ }
+
+ /// Both connectors are configured before either sends a pixel, and only one mode set each.
+ ///
+ /// A second bracket around a second mode set is what the dock stops answering: it acknowledges
+ /// every record up to that bracket's first marker and nothing afterwards.
+ #[test]
+ fn ella_dock_wide_sets_both_modes_before_any_pixels() -> Result {
+ use super::DockWideStep::*;
+ let mut modes = [0u32; 2];
+ for step in ELLA_DOCK_WIDE {
+ match *step {
+ SetMode(slot) => modes[usize::from(slot)] += 1,
+ // Every carrier finds both connectors already configured, and each exactly once.
+ Carrier(_) => assert_eq!(modes, [1, 1]),
+ _ => {}
+ }
+ }
+ assert_eq!(modes, [1, 1]);
+ Ok(())
+ }
+
+ /// The second connector's sink stays down across the first connector's carrier.
+ ///
+ /// The vendor takes it down inside the dock-wide bracket and brings it up only once the first
+ /// connector is streaming. A sink brought up early is a connector the dock is scanning out
+ /// while its stream has neither a ring descriptor nor a decoder configuration.
+ #[test]
+ fn ella_dock_wide_holds_the_second_sink_down_until_the_first_streams() -> Result {
+ use super::DockWideStep::*;
+ let mut down = false;
+ let mut streaming = false;
+ let mut up_after_streaming = false;
+ for step in ELLA_DOCK_WIDE {
+ match *step {
+ Marker(1, 0x2e, 3) => down = true,
+ Marker(1, 0x2e, 0) => {
+ // Up only after it was taken down, and only once the first is streaming.
+ assert!(down);
+ assert!(streaming);
+ up_after_streaming = true;
+ }
+ Carrier(0) => streaming = true,
+ // Neither the second stream's opening records nor its pixels reach a downed sink.
+ Carrier(1) => assert!(up_after_streaming),
+ Ring(1) | Config(1) | Prologue(1) => assert!(up_after_streaming),
+ _ => {}
+ }
+ }
+ assert!(up_after_streaming);
+ Ok(())
+ }
+
+ /// Reconfiguring one DL-3x00 connector while the other is lit, against the vendor's own.
+ ///
+ /// Transcribed from two vendor reconfigurations at different resolutions, whose records agree
+ /// step for step. The sink-down before the mode set is the part a reader is most likely to
+ /// think redundant with the cold table's, where it comes after.
+ #[test]
+ fn ella_runtime_mode_matches_the_dlm_capture() -> Result {
+ use super::DockWideStep::*;
+ let want = [
+ Marker(0, 0x2f, 1), // vendor #38348 / #44725
+ Marker(0, 0x2e, 3), // #38349 / #44726, sink down ahead of the mode set
+ SetMode(0), // #38353 / #44729
+ Marker(0, 0x2f, 1), // #38354 / #44730
+ Marker(0, 0x2e, 0), // #38355 / #44731, sink up
+ Poll, // #38356 / #44732
+ Prologue(0), // #38357 / #44733
+ Carrier(0), // #38358 / #44734 onward
+ ];
+ assert_eq!(ELLA_RUNTIME_MODE, &want[..]);
+ Ok(())
+ }
+
+ /// A runtime reconfiguration names one connector and never the other.
+ ///
+ /// The connector that is already lit keeps streaming across the whole sequence; a marker
+ /// addressed to it here would bracket a stream the dock is mid-frame on.
+ #[test]
+ fn ella_runtime_mode_touches_one_connector() -> Result {
+ use super::DockWideStep::*;
+ for step in ELLA_RUNTIME_MODE {
+ let slot = match *step {
+ SetMode(slot)
+ | Marker(slot, _, _)
+ | Prologue(slot)
+ | Ring(slot)
+ | Config(slot)
+ | Carrier(slot) => slot,
+ Poll | Stream(_, _) => continue,
+ };
+ assert_eq!(slot, 0);
+ }
+ Ok(())
+ }
+}
next prev parent reply other threads:[~2026-08-26 16:41 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-26 16:37 [PATCH v3 0/13] drm/vino: a Rust driver for DisplayLink DL3 docks Mike Lothian
2026-08-26 16:37 ` [PATCH v3 8/13] drm/vino: add the KMS device and the atomic path Mike Lothian
2026-08-26 16:37 ` Mike Lothian [this message]
2026-08-26 16:37 ` [PATCH v3 11/13] drm/vino: add the USB driver frontend Mike Lothian
2026-08-26 16:37 ` [PATCH v3 12/13] drm/vino: allow the driver to be built Mike Lothian
2026-08-26 16:37 ` [PATCH v3 13/13] Documentation/gpu: document the Vino driver Mike Lothian
2026-08-26 17:28 ` Randy Dunlap
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260826163913.7052-10-mike@fireburn.co.uk \
--to=mike@fireburn.co.uk \
--cc=airlied@gmail.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=gary@garyguo.net \
--cc=linux-kernel@vger.kernel.org \
--cc=lossin@kernel.org \
--cc=maarten.lankhorst@linux.intel.com \
--cc=mripard@kernel.org \
--cc=rust-for-linux@vger.kernel.org \
--cc=simona@ffwll.ch \
--cc=tzimmermann@suse.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox