Linux-ARM-Kernel Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v8 00/11] Add support for Wave6 video codec driver
@ 2026-11-19  8:59 Nas Chung
  2026-11-19  8:59 ` [PATCH v8 01/11] dt-bindings: media: nxp: Add Wave6 video codec device Nas Chung
                   ` (10 more replies)
  0 siblings, 11 replies; 18+ messages in thread
From: Nas Chung @ 2026-11-19  8:59 UTC (permalink / raw)
  To: mchehab, hverkuil, robh, krzk+dt, conor+dt, Frank.Li, s.hauer
  Cc: linux-media, devicetree, linux-kernel, imx, linux-arm-kernel,
	festevam, kernel, nicolas.dufresne, ming.qian, marek.vasut,
	jackson.lee, lafley.kim, Nas Chung

This patch series introduces support for the Chips&Media Wave6 video
codec IP, a completely different hardware architecture compared to Wave5.

The wave6 driver is a M2M stateful encoder/decoder driver.
It supports various video formats, including H.264 and H.265, for both
encoding and decoding.
While other versions of the Wave6 IP may support VP9 decoding and AV1
decoding and encoding those formats are not implemented or validated in
this driver at this time.

On NXP i.MX SoCs, the Wave6 hardware exposes one control register region
and four interface register regions for one shared video processing
engine.
VPU Control region, Manages shared resources such as firmware memory.
VPU Core region, Provides encoding and decoding capabilities.
The control and interface regions are distinct DMA requesters and can be
associated with separate IOMMU stream IDs, allowing DMA isolation between
them.

The interface regions are described as child nodes of the VPU node rather
than folded into the parent with reg-names/interrupt-names, because each
one is a separate DMA requester. This follows the same structure already
accepted for the Qualcomm video codec bindings, and was mentioned on the v6
thread.

The firmware tested by this driver has been upstreamed in linux-firmware:
- Path: cnm/wave633c_imx9_codec_fw.bin

This driver has been tested with GStreamer on:
- NXP i.MX95 board
- pre-silicon FPGA environment

Test results for decoder fluster with -j2 option:
- JVT-AVC_V1, Ran 77/135 tests successfully              in 24.180 secs
- JVT-FR-EXT, Ran 25/69 tests successfully               in 11.157 secs
- JCT-VC-HEVC_V1, Ran 132/147 tests successfully         in 45.534 secs
- All failures are due to unsupported hardware features:
-- 10bit, Resolutions higher than 4K, FMO, MBAFF
-- Extended profile, Field encoding and High422 sreams.

Test results for v4l2-compliance:
v4l2-compliance 1.31.0-5386, 64 bits, 64-bit time_t
v4l2-compliance SHA: 48316b8a20aa 2025-08-12 12:44:56

Compliance test for wave6-dec device /dev/video0:
                fail: v4l2-test-controls.cpp(1204): !have_source_change || !have_eos
        test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
Total for wave6-dec device /dev/video0: 48, Succeeded: 47, Failed: 1, Warnings: 0

Compliance test for wave6-enc device /dev/video1:
                fail: v4l2-test-controls.cpp(1193): node->codec_mask & STATEFUL_ENCODER
        test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
Total for wave6-enc device /dev/video1: 48, Succeeded: 47, Failed: 1, Warnings: 0

Note: the failures are all related with the eos event.

Changelog:

v8:
- Dropped "media: v4l2-common: Fix P010 format info", now applied upstream
- Split the core, decoder and encoder support into smaller patches so
  that every patch builds on its own
- Moved the performance profiling code into its own patch
- Built the control and core drivers into one wave6 module and replaced
  the function pointers between them with direct calls
- Reworked the decoder and encoder drain to follow the stateful codec
  interface, using the v4l2-mem2mem helpers in the encoder
- Replaced the instance STOP state with an ERROR state set on command
  failures
- Made the job timeout watchdog per instance
- Fixed runtime PM, probe, teardown and cooling device issues
- Range-checked the frame buffer registration counts
- Set DEFAULT_NUM_TICKS_POC_DIFF to 1 to align with the codec specification
- Added the imx list to MAINTAINERS and updated the copyright year
- Cleaned up the code: helper macros, relaxed register accessors, devm
  helpers, unused codes and includes, coding style

v7:
- Moved the sram property from imx95-19x19-evk.dts to imx95.dtsi
- Simplified the sram binding property to maxItems: 1
- Fixed the issues reported by the sashiko-bot review of v6:
  - Bounded frame buffer and aux buffer counts before indexing
  - Added vb2_is_busy() checks to the decoder S_FMT handlers
  - Fixed video_device lifetime and driver teardown ordering
  - Propagated hardware init, runtime PM and thermal failures upward
  - Fixed tracepoint field widths and debugfs handling
- Balanced the VPU reference on the core activation error path

v6:
- Fixed the existing P010 format_info entry
- Moved all encoder parameter validation from the HW layer into the V4L2
  layer
- Removed redundant HW-abstraction wrappers from the VPU API layer
- Computed frame stride and sizeimage in the V4L2 layer
- Added explicit instance lifecycle helpers to the core driver
- Enabled the VPU only on the tested imx95-19x19-evk board

RFC v5:
- Move all shared resources to the parent node
- Drop child compatible and use data-only interface child nodes
- Update the VPU driver to create child devices and load the core driver

v4:
- Fixed build issues reported by CI tools
- Updated commit messages to use imperative mood
- Avoided using the same name for both nodes and labels in devicetree
- Removed unused labels from YAML examples
- Added description for child(vpu-core) node
- Added iommus property to both parent(vpu) and child(vpu-core) nodes
- Updated probe() functions to use dev_err_probe() when returning
  -EPROBE_DEFER
- Added wave6_vpu prefix to trace functions
- Updated HEVC decoder profile control to report MAIN_STILL profile
- Fixed bug in multiple instance creation by pre-allocating work buffer
- Fixed interrupt handling by checking INSTANCE_INFO register and instance
  list

v3:
- Removed ambiguous SUPPORT_FOLLOWER feature
- Used WARN_ON() for unexpected programming errors
- Split thermal device code into wave6-vpu-thermal.c/h
- Dropped wave6_cooling_disable module parameter
- Replaced mutex_lock() with guard()
- Added lockdep_assert_held() to clarify locking regions
- Removed exported function due to dual-license and used function pointer
- Added documentation and validation for state transitions
- Added documentation for device structures
- Added patch to enable VPU device in imx95 DTS
- Updated DT bindings and driver to align with parent(vpu) and
  child(vpu-core)
- Replaced magic numbers with mask and offset macros when accessing
  registers
- Placed goto statements after an empty line
- Printed HW info (e.g. product_code) via dev_dbg() for debugging
- Replaced wave6_vpu_dec_give_command() with dedicated functions

v2:
- Refined DT bindings to better represent the hardware
- Reworked driver to align with the parent(VPU) and child(CTRL, CORE)
- Fixed build issues reported by CI tools (Smatch, Sparse, TRACE)
- Improved commit messages with clearer descriptions
- Added kernel-doc for exported functions
- Removed redundant print statements and unused code
- Reordered patches to prevent build failures

Nas Chung (11):
  dt-bindings: media: nxp: Add Wave6 video codec device
  media: chips-media: wave6: Add Wave6 control driver
  media: chips-media: wave6: Add Wave6 core driver
  media: chips-media: wave6: Add Wave6 decoder firmware interface
  media: chips-media: wave6: Add Wave6 m2m decoder
  media: chips-media: wave6: Add Wave6 encoder firmware interface
  media: chips-media: wave6: Add Wave6 m2m encoder
  media: chips-media: wave6: Add trace and debugfs
  media: chips-media: wave6: Add performance profiling
  media: chips-media: wave6: Add Wave6 thermal cooling device
  arm64: dts: freescale: imx95: Add video codec node

 .../bindings/media/nxp,imx95-vpu.yaml         |  162 +
 MAINTAINERS                                   |    9 +
 .../boot/dts/freescale/imx95-19x19-evk.dts    |   10 +
 arch/arm64/boot/dts/freescale/imx95.dtsi      |   37 +
 drivers/media/platform/chips-media/Kconfig    |    1 +
 drivers/media/platform/chips-media/Makefile   |    1 +
 .../media/platform/chips-media/wave6/Kconfig  |   16 +
 .../media/platform/chips-media/wave6/Makefile |   15 +
 .../platform/chips-media/wave6/wave6-hw.c     | 1934 ++++++++++++
 .../platform/chips-media/wave6/wave6-hw.h     |   58 +
 .../chips-media/wave6/wave6-regdefine.h       |  651 ++++
 .../platform/chips-media/wave6/wave6-trace.h  |  284 ++
 .../platform/chips-media/wave6/wave6-vdi.h    |  102 +
 .../chips-media/wave6/wave6-vpu-core.c        |  446 +++
 .../chips-media/wave6/wave6-vpu-core.h        |  130 +
 .../chips-media/wave6/wave6-vpu-dbg.c         |  184 ++
 .../chips-media/wave6/wave6-vpu-dbg.h         |   14 +
 .../chips-media/wave6/wave6-vpu-dec.c         | 1969 ++++++++++++
 .../chips-media/wave6/wave6-vpu-enc.c         | 2762 +++++++++++++++++
 .../chips-media/wave6/wave6-vpu-thermal.c     |  141 +
 .../chips-media/wave6/wave6-vpu-thermal.h     |   24 +
 .../chips-media/wave6/wave6-vpu-v4l2.c        |  500 +++
 .../platform/chips-media/wave6/wave6-vpu.c    |  824 +++++
 .../platform/chips-media/wave6/wave6-vpu.h    |  142 +
 .../platform/chips-media/wave6/wave6-vpuapi.c |  333 ++
 .../platform/chips-media/wave6/wave6-vpuapi.h | 1014 ++++++
 .../chips-media/wave6/wave6-vpuconfig.h       |   72 +
 .../chips-media/wave6/wave6-vpuerror.h        |  252 ++
 28 files changed, 12087 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/nxp,imx95-vpu.yaml
 create mode 100644 drivers/media/platform/chips-media/wave6/Kconfig
 create mode 100644 drivers/media/platform/chips-media/wave6/Makefile
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-hw.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-hw.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-regdefine.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-trace.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vdi.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-core.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuapi.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuerror.h


base-commit: aa0f4c41d4870a18e65772a1817af16427200371
-- 
2.41.0.windows.3



^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2026-11-19 21:05 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-19  8:59 [PATCH v8 00/11] Add support for Wave6 video codec driver Nas Chung
2026-11-19  8:59 ` [PATCH v8 01/11] dt-bindings: media: nxp: Add Wave6 video codec device Nas Chung
2026-11-19  8:59 ` [PATCH v8 02/11] media: chips-media: wave6: Add Wave6 control driver Nas Chung
2026-11-19 20:07   ` Frank Li
2026-11-19  8:59 ` [PATCH v8 03/11] media: chips-media: wave6: Add Wave6 core driver Nas Chung
2026-11-19 20:27   ` Frank Li
2026-11-19  8:59 ` [PATCH v8 04/11] media: chips-media: wave6: Add Wave6 decoder firmware interface Nas Chung
2026-11-19 20:37   ` Frank Li
2026-11-19  8:59 ` [PATCH v8 05/11] media: chips-media: wave6: Add Wave6 m2m decoder Nas Chung
2026-11-19  9:00 ` [PATCH v8 06/11] media: chips-media: wave6: Add Wave6 encoder firmware interface Nas Chung
2026-11-19  9:00 ` [PATCH v8 07/11] media: chips-media: wave6: Add Wave6 m2m encoder Nas Chung
2026-11-19  9:00 ` [PATCH v8 08/11] media: chips-media: wave6: Add trace and debugfs Nas Chung
2026-11-19  9:00 ` [PATCH v8 09/11] media: chips-media: wave6: Add performance profiling Nas Chung
2026-11-19 20:56   ` Frank Li
2026-11-19  9:00 ` [PATCH v8 10/11] media: chips-media: wave6: Add Wave6 thermal cooling device Nas Chung
2026-11-19 21:03   ` Frank Li
2026-11-19  9:00 ` [PATCH v8 11/11] arm64: dts: freescale: imx95: Add video codec node Nas Chung
2026-11-19 21:05   ` Frank Li

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox