* [PATCH v8 00/11] Add support for Wave6 video codec driver
@ 2026-11-19 8:59 Nas Chung
2026-11-19 8:59 ` [PATCH v8 01/11] dt-bindings: media: nxp: Add Wave6 video codec device Nas Chung
` (10 more replies)
0 siblings, 11 replies; 23+ messages in thread
From: Nas Chung @ 2026-11-19 8:59 UTC (permalink / raw)
To: mchehab, hverkuil, robh, krzk+dt, conor+dt, Frank.Li, s.hauer
Cc: linux-media, devicetree, linux-kernel, imx, linux-arm-kernel,
festevam, kernel, nicolas.dufresne, ming.qian, marek.vasut,
jackson.lee, lafley.kim, Nas Chung
This patch series introduces support for the Chips&Media Wave6 video
codec IP, a completely different hardware architecture compared to Wave5.
The wave6 driver is a M2M stateful encoder/decoder driver.
It supports various video formats, including H.264 and H.265, for both
encoding and decoding.
While other versions of the Wave6 IP may support VP9 decoding and AV1
decoding and encoding those formats are not implemented or validated in
this driver at this time.
On NXP i.MX SoCs, the Wave6 hardware exposes one control register region
and four interface register regions for one shared video processing
engine.
VPU Control region, Manages shared resources such as firmware memory.
VPU Core region, Provides encoding and decoding capabilities.
The control and interface regions are distinct DMA requesters and can be
associated with separate IOMMU stream IDs, allowing DMA isolation between
them.
The interface regions are described as child nodes of the VPU node rather
than folded into the parent with reg-names/interrupt-names, because each
one is a separate DMA requester. This follows the same structure already
accepted for the Qualcomm video codec bindings, and was mentioned on the v6
thread.
The firmware tested by this driver has been upstreamed in linux-firmware:
- Path: cnm/wave633c_imx9_codec_fw.bin
This driver has been tested with GStreamer on:
- NXP i.MX95 board
- pre-silicon FPGA environment
Test results for decoder fluster with -j2 option:
- JVT-AVC_V1, Ran 77/135 tests successfully in 24.180 secs
- JVT-FR-EXT, Ran 25/69 tests successfully in 11.157 secs
- JCT-VC-HEVC_V1, Ran 132/147 tests successfully in 45.534 secs
- All failures are due to unsupported hardware features:
-- 10bit, Resolutions higher than 4K, FMO, MBAFF
-- Extended profile, Field encoding and High422 sreams.
Test results for v4l2-compliance:
v4l2-compliance 1.31.0-5386, 64 bits, 64-bit time_t
v4l2-compliance SHA: 48316b8a20aa 2025-08-12 12:44:56
Compliance test for wave6-dec device /dev/video0:
fail: v4l2-test-controls.cpp(1204): !have_source_change || !have_eos
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
Total for wave6-dec device /dev/video0: 48, Succeeded: 47, Failed: 1, Warnings: 0
Compliance test for wave6-enc device /dev/video1:
fail: v4l2-test-controls.cpp(1193): node->codec_mask & STATEFUL_ENCODER
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
Total for wave6-enc device /dev/video1: 48, Succeeded: 47, Failed: 1, Warnings: 0
Note: the failures are all related with the eos event.
Changelog:
v8:
- Dropped "media: v4l2-common: Fix P010 format info", now applied upstream
- Split the core, decoder and encoder support into smaller patches so
that every patch builds on its own
- Moved the performance profiling code into its own patch
- Built the control and core drivers into one wave6 module and replaced
the function pointers between them with direct calls
- Reworked the decoder and encoder drain to follow the stateful codec
interface, using the v4l2-mem2mem helpers in the encoder
- Replaced the instance STOP state with an ERROR state set on command
failures
- Made the job timeout watchdog per instance
- Fixed runtime PM, probe, teardown and cooling device issues
- Range-checked the frame buffer registration counts
- Set DEFAULT_NUM_TICKS_POC_DIFF to 1 to align with the codec specification
- Added the imx list to MAINTAINERS and updated the copyright year
- Cleaned up the code: helper macros, relaxed register accessors, devm
helpers, unused codes and includes, coding style
v7:
- Moved the sram property from imx95-19x19-evk.dts to imx95.dtsi
- Simplified the sram binding property to maxItems: 1
- Fixed the issues reported by the sashiko-bot review of v6:
- Bounded frame buffer and aux buffer counts before indexing
- Added vb2_is_busy() checks to the decoder S_FMT handlers
- Fixed video_device lifetime and driver teardown ordering
- Propagated hardware init, runtime PM and thermal failures upward
- Fixed tracepoint field widths and debugfs handling
- Balanced the VPU reference on the core activation error path
v6:
- Fixed the existing P010 format_info entry
- Moved all encoder parameter validation from the HW layer into the V4L2
layer
- Removed redundant HW-abstraction wrappers from the VPU API layer
- Computed frame stride and sizeimage in the V4L2 layer
- Added explicit instance lifecycle helpers to the core driver
- Enabled the VPU only on the tested imx95-19x19-evk board
RFC v5:
- Move all shared resources to the parent node
- Drop child compatible and use data-only interface child nodes
- Update the VPU driver to create child devices and load the core driver
v4:
- Fixed build issues reported by CI tools
- Updated commit messages to use imperative mood
- Avoided using the same name for both nodes and labels in devicetree
- Removed unused labels from YAML examples
- Added description for child(vpu-core) node
- Added iommus property to both parent(vpu) and child(vpu-core) nodes
- Updated probe() functions to use dev_err_probe() when returning
-EPROBE_DEFER
- Added wave6_vpu prefix to trace functions
- Updated HEVC decoder profile control to report MAIN_STILL profile
- Fixed bug in multiple instance creation by pre-allocating work buffer
- Fixed interrupt handling by checking INSTANCE_INFO register and instance
list
v3:
- Removed ambiguous SUPPORT_FOLLOWER feature
- Used WARN_ON() for unexpected programming errors
- Split thermal device code into wave6-vpu-thermal.c/h
- Dropped wave6_cooling_disable module parameter
- Replaced mutex_lock() with guard()
- Added lockdep_assert_held() to clarify locking regions
- Removed exported function due to dual-license and used function pointer
- Added documentation and validation for state transitions
- Added documentation for device structures
- Added patch to enable VPU device in imx95 DTS
- Updated DT bindings and driver to align with parent(vpu) and
child(vpu-core)
- Replaced magic numbers with mask and offset macros when accessing
registers
- Placed goto statements after an empty line
- Printed HW info (e.g. product_code) via dev_dbg() for debugging
- Replaced wave6_vpu_dec_give_command() with dedicated functions
v2:
- Refined DT bindings to better represent the hardware
- Reworked driver to align with the parent(VPU) and child(CTRL, CORE)
- Fixed build issues reported by CI tools (Smatch, Sparse, TRACE)
- Improved commit messages with clearer descriptions
- Added kernel-doc for exported functions
- Removed redundant print statements and unused code
- Reordered patches to prevent build failures
Nas Chung (11):
dt-bindings: media: nxp: Add Wave6 video codec device
media: chips-media: wave6: Add Wave6 control driver
media: chips-media: wave6: Add Wave6 core driver
media: chips-media: wave6: Add Wave6 decoder firmware interface
media: chips-media: wave6: Add Wave6 m2m decoder
media: chips-media: wave6: Add Wave6 encoder firmware interface
media: chips-media: wave6: Add Wave6 m2m encoder
media: chips-media: wave6: Add trace and debugfs
media: chips-media: wave6: Add performance profiling
media: chips-media: wave6: Add Wave6 thermal cooling device
arm64: dts: freescale: imx95: Add video codec node
.../bindings/media/nxp,imx95-vpu.yaml | 162 +
MAINTAINERS | 9 +
.../boot/dts/freescale/imx95-19x19-evk.dts | 10 +
arch/arm64/boot/dts/freescale/imx95.dtsi | 37 +
drivers/media/platform/chips-media/Kconfig | 1 +
drivers/media/platform/chips-media/Makefile | 1 +
.../media/platform/chips-media/wave6/Kconfig | 16 +
.../media/platform/chips-media/wave6/Makefile | 15 +
.../platform/chips-media/wave6/wave6-hw.c | 1934 ++++++++++++
.../platform/chips-media/wave6/wave6-hw.h | 58 +
.../chips-media/wave6/wave6-regdefine.h | 651 ++++
.../platform/chips-media/wave6/wave6-trace.h | 284 ++
.../platform/chips-media/wave6/wave6-vdi.h | 102 +
.../chips-media/wave6/wave6-vpu-core.c | 446 +++
.../chips-media/wave6/wave6-vpu-core.h | 130 +
.../chips-media/wave6/wave6-vpu-dbg.c | 184 ++
.../chips-media/wave6/wave6-vpu-dbg.h | 14 +
.../chips-media/wave6/wave6-vpu-dec.c | 1969 ++++++++++++
.../chips-media/wave6/wave6-vpu-enc.c | 2762 +++++++++++++++++
.../chips-media/wave6/wave6-vpu-thermal.c | 141 +
.../chips-media/wave6/wave6-vpu-thermal.h | 24 +
.../chips-media/wave6/wave6-vpu-v4l2.c | 500 +++
.../platform/chips-media/wave6/wave6-vpu.c | 824 +++++
.../platform/chips-media/wave6/wave6-vpu.h | 142 +
.../platform/chips-media/wave6/wave6-vpuapi.c | 333 ++
.../platform/chips-media/wave6/wave6-vpuapi.h | 1014 ++++++
.../chips-media/wave6/wave6-vpuconfig.h | 72 +
.../chips-media/wave6/wave6-vpuerror.h | 252 ++
28 files changed, 12087 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/nxp,imx95-vpu.yaml
create mode 100644 drivers/media/platform/chips-media/wave6/Kconfig
create mode 100644 drivers/media/platform/chips-media/wave6/Makefile
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-hw.c
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-hw.h
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-regdefine.h
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-trace.h
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vdi.h
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-core.h
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.h
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.h
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu.c
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu.h
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuapi.c
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuerror.h
base-commit: aa0f4c41d4870a18e65772a1817af16427200371
--
2.41.0.windows.3
^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH v8 01/11] dt-bindings: media: nxp: Add Wave6 video codec device
2026-11-19 8:59 [PATCH v8 00/11] Add support for Wave6 video codec driver Nas Chung
@ 2026-11-19 8:59 ` Nas Chung
2026-11-19 8:59 ` [PATCH v8 02/11] media: chips-media: wave6: Add Wave6 control driver Nas Chung
` (9 subsequent siblings)
10 siblings, 0 replies; 23+ messages in thread
From: Nas Chung @ 2026-11-19 8:59 UTC (permalink / raw)
To: mchehab, hverkuil, robh, krzk+dt, conor+dt, Frank.Li, s.hauer
Cc: linux-media, devicetree, linux-kernel, imx, linux-arm-kernel,
festevam, kernel, nicolas.dufresne, ming.qian, marek.vasut,
jackson.lee, lafley.kim, Nas Chung, Krzysztof Kozlowski
Add documentation for the Chips&Media Wave6 video codec on NXP i.MX SoCs.
The hardware contains one control register region and four interface
register regions for a shared video processing engine. The control region
manages shared resources such as firmware memory, while each interface
region has its own MMIO range and interrupt.
The control region and each interface region are distinct DMA requesters
and can be associated with separate IOMMU stream IDs. Represent the
control region as the parent node and the interface register regions as
child nodes to describe these resources.
Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
---
.../bindings/media/nxp,imx95-vpu.yaml | 162 ++++++++++++++++++
MAINTAINERS | 8 +
2 files changed, 170 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/nxp,imx95-vpu.yaml
diff --git a/Documentation/devicetree/bindings/media/nxp,imx95-vpu.yaml b/Documentation/devicetree/bindings/media/nxp,imx95-vpu.yaml
new file mode 100644
index 000000000000..aa7dd8838af0
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/nxp,imx95-vpu.yaml
@@ -0,0 +1,162 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/nxp,imx95-vpu.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Chips&Media Wave6 Series multi-standard codec IP on NXP i.MX SoCs
+
+maintainers:
+ - Nas Chung <nas.chung@chipsnmedia.com>
+ - Jackson Lee <jackson.lee@chipsnmedia.com>
+
+description:
+ The Chips&Media Wave6 codec IP is a multi-standard video encoder/decoder.
+ On NXP i.MX SoCs, the Wave6 codec IP exposes one control register region and
+ four interface register regions for a shared video processing engine.
+ The parent node describes the control region, which has its own MMIO range and
+ manages shared resources such as firmware memory. The child nodes describe the
+ interface register regions. Each interface region has its own MMIO range and
+ interrupt.
+ The control region and the interface regions are distinct DMA requesters.
+ The control region and each interface region can be associated with separate
+ IOMMU stream IDs, allowing DMA isolation between them.
+
+properties:
+ compatible:
+ enum:
+ - nxp,imx95-vpu
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ items:
+ - description: VPU core clock
+ - description: VPU associated block clock
+
+ clock-names:
+ items:
+ - const: core
+ - const: vpublk
+
+ power-domains:
+ items:
+ - description: Main VPU power domain
+ - description: Performance power domain
+
+ power-domain-names:
+ items:
+ - const: vpu
+ - const: perf
+
+ memory-region:
+ maxItems: 1
+
+ sram:
+ maxItems: 1
+ description:
+ Used to store reference data, reducing DMA memory bandwidth.
+
+ iommus:
+ maxItems: 1
+
+ "#cooling-cells":
+ const: 2
+
+ "#address-cells":
+ const: 2
+
+ "#size-cells":
+ const: 2
+
+ ranges: true
+
+patternProperties:
+ "^interface@[0-9a-f]+$":
+ type: object
+ description:
+ An interface register region within the Chips&Media Wave6 codec IP.
+ Each region has its own MMIO range and interrupt and can be associated
+ with a separate IOMMU stream ID for DMA isolation.
+ additionalProperties: false
+
+ properties:
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ iommus:
+ maxItems: 1
+
+ required:
+ - reg
+ - interrupts
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - power-domains
+ - power-domain-names
+ - memory-region
+ - "#address-cells"
+ - "#size-cells"
+ - ranges
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/clock/nxp,imx95-clock.h>
+
+ soc {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ video-codec@4c4c0000 {
+ compatible = "nxp,imx95-vpu";
+ reg = <0x0 0x4c4c0000 0x0 0x10000>;
+ clocks = <&scmi_clk 115>,
+ <&vpu_blk_ctrl IMX95_CLK_VPUBLK_WAVE>;
+ clock-names = "core", "vpublk";
+ power-domains = <&scmi_devpd 21>,
+ <&scmi_perf 10>;
+ power-domain-names = "vpu", "perf";
+ memory-region = <&vpu_boot>;
+ sram = <&sram1>;
+ iommus = <&smmu 0x32>;
+ #cooling-cells = <2>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ interface@4c480000 {
+ reg = <0x0 0x4c480000 0x0 0x10000>;
+ interrupts = <GIC_SPI 299 IRQ_TYPE_LEVEL_HIGH>;
+ iommus = <&smmu 0x33>;
+ };
+
+ interface@4c490000 {
+ reg = <0x0 0x4c490000 0x0 0x10000>;
+ interrupts = <GIC_SPI 300 IRQ_TYPE_LEVEL_HIGH>;
+ iommus = <&smmu 0x34>;
+ };
+
+ interface@4c4a0000 {
+ reg = <0x0 0x4c4a0000 0x0 0x10000>;
+ interrupts = <GIC_SPI 301 IRQ_TYPE_LEVEL_HIGH>;
+ iommus = <&smmu 0x35>;
+ };
+
+ interface@4c4b0000 {
+ reg = <0x0 0x4c4b0000 0x0 0x10000>;
+ interrupts = <GIC_SPI 302 IRQ_TYPE_LEVEL_HIGH>;
+ iommus = <&smmu 0x36>;
+ };
+ };
+ };
diff --git a/MAINTAINERS b/MAINTAINERS
index c08aade46f61..e014d1b1c890 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -30125,6 +30125,14 @@ S: Maintained
F: Documentation/devicetree/bindings/media/cnm,wave521c.yaml
F: drivers/media/platform/chips-media/wave5/
+WAVE6 VPU CODEC DRIVER
+M: Nas Chung <nas.chung@chipsnmedia.com>
+M: Jackson Lee <jackson.lee@chipsnmedia.com>
+L: linux-media@vger.kernel.org
+L: imx@lists.linux.dev
+S: Maintained
+F: Documentation/devicetree/bindings/media/nxp,imx95-vpu.yaml
+
WHISKEYCOVE PMIC GPIO DRIVER
M: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
L: linux-gpio@vger.kernel.org
--
2.41.0.windows.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v8 02/11] media: chips-media: wave6: Add Wave6 control driver
2026-11-19 8:59 [PATCH v8 00/11] Add support for Wave6 video codec driver Nas Chung
2026-11-19 8:59 ` [PATCH v8 01/11] dt-bindings: media: nxp: Add Wave6 video codec device Nas Chung
@ 2026-11-19 8:59 ` Nas Chung
2026-11-19 9:26 ` sashiko-bot
2026-11-19 8:59 ` [PATCH v8 03/11] media: chips-media: wave6: Add Wave6 core driver Nas Chung
` (8 subsequent siblings)
10 siblings, 1 reply; 23+ messages in thread
From: Nas Chung @ 2026-11-19 8:59 UTC (permalink / raw)
To: mchehab, hverkuil, robh, krzk+dt, conor+dt, Frank.Li, s.hauer
Cc: linux-media, devicetree, linux-kernel, imx, linux-arm-kernel,
festevam, kernel, nicolas.dufresne, ming.qian, marek.vasut,
jackson.lee, lafley.kim, Nas Chung
Add the control driver for the Chips&Media Wave6 video codec IP.
The hardware contains one control register region and four interface
register regions for a shared video processing engine.
This driver handles the control region and manages shared resources such
as firmware loading, firmware memory allocation, and coordination required
by the interface register regions.
It also instantiates and coordinates with the `wave6-core` child devices
for firmware and power state management.
Tested-by: Ming Qian <ming.qian@oss.nxp.com>
Tested-by: Marek Vasut <marek.vasut@mailbox.org>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
---
MAINTAINERS | 1 +
drivers/media/platform/chips-media/Kconfig | 1 +
drivers/media/platform/chips-media/Makefile | 1 +
.../media/platform/chips-media/wave6/Kconfig | 13 +
.../media/platform/chips-media/wave6/Makefile | 4 +
.../chips-media/wave6/wave6-regdefine.h | 49 ++
.../platform/chips-media/wave6/wave6-vdi.h | 82 ++
.../platform/chips-media/wave6/wave6-vpu.c | 780 ++++++++++++++++++
.../platform/chips-media/wave6/wave6-vpu.h | 137 +++
.../chips-media/wave6/wave6-vpuconfig.h | 37 +
10 files changed, 1105 insertions(+)
create mode 100644 drivers/media/platform/chips-media/wave6/Kconfig
create mode 100644 drivers/media/platform/chips-media/wave6/Makefile
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-regdefine.h
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vdi.h
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu.c
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu.h
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
diff --git a/MAINTAINERS b/MAINTAINERS
index e014d1b1c890..4c472b30f78a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -30132,6 +30132,7 @@ L: linux-media@vger.kernel.org
L: imx@lists.linux.dev
S: Maintained
F: Documentation/devicetree/bindings/media/nxp,imx95-vpu.yaml
+F: drivers/media/platform/chips-media/wave6/
WHISKEYCOVE PMIC GPIO DRIVER
M: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
diff --git a/drivers/media/platform/chips-media/Kconfig b/drivers/media/platform/chips-media/Kconfig
index ad350eb6b1fc..8ef7fc8029a4 100644
--- a/drivers/media/platform/chips-media/Kconfig
+++ b/drivers/media/platform/chips-media/Kconfig
@@ -4,3 +4,4 @@ comment "Chips&Media media platform drivers"
source "drivers/media/platform/chips-media/coda/Kconfig"
source "drivers/media/platform/chips-media/wave5/Kconfig"
+source "drivers/media/platform/chips-media/wave6/Kconfig"
diff --git a/drivers/media/platform/chips-media/Makefile b/drivers/media/platform/chips-media/Makefile
index 6b5d99de8b54..b9a07a91c9d6 100644
--- a/drivers/media/platform/chips-media/Makefile
+++ b/drivers/media/platform/chips-media/Makefile
@@ -2,3 +2,4 @@
obj-y += coda/
obj-y += wave5/
+obj-y += wave6/
diff --git a/drivers/media/platform/chips-media/wave6/Kconfig b/drivers/media/platform/chips-media/wave6/Kconfig
new file mode 100644
index 000000000000..153a46ebdc6b
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/Kconfig
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config VIDEO_WAVE6_VPU
+ tristate "Chips&Media Wave6 Codec Driver"
+ depends on V4L_MEM2MEM_DRIVERS
+ depends on VIDEO_DEV && OF
+ depends on ARCH_MXC || COMPILE_TEST
+ select GENERIC_ALLOCATOR
+ help
+ Chips&Media Wave6 stateful codec driver.
+ The wave6 driver manages shared resources such as firmware memory.
+ To compile this driver as a module, choose M here: the
+ module will be called wave6.
diff --git a/drivers/media/platform/chips-media/wave6/Makefile b/drivers/media/platform/chips-media/wave6/Makefile
new file mode 100644
index 000000000000..255fc90bc642
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/Makefile
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0
+
+wave6-objs += wave6-vpu.o
+obj-$(CONFIG_VIDEO_WAVE6_VPU) += wave6.o
diff --git a/drivers/media/platform/chips-media/wave6/wave6-regdefine.h b/drivers/media/platform/chips-media/wave6/wave6-regdefine.h
new file mode 100644
index 000000000000..e879a9743cb3
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-regdefine.h
@@ -0,0 +1,49 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
+/*
+ * Wave6 series multi-standard codec IP - wave6 register definitions
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#ifndef __WAVE6_REGDEFINE_H__
+#define __WAVE6_REGDEFINE_H__
+
+#include <linux/bitfield.h>
+
+enum wave6_command {
+ W6_CMD_INIT_VPU = 0x0001,
+ W6_CMD_WAKEUP_VPU = 0x0002,
+ W6_CMD_SLEEP_VPU = 0x0004,
+ W6_CMD_INIT_WORK_BUF = 0x1000,
+};
+
+#define W6_REG_BASE 0x00000000
+
+#define W6_VPU_VCPU_CUR_PC (W6_REG_BASE + 0x0004)
+#define W6_VPU_HOST_INT_REQ (W6_REG_BASE + 0x0038)
+#define HOST_INT_REQ_ON BIT(0)
+#define W6_VPU_REMAP_CTRL_GB (W6_REG_BASE + 0x0060)
+#define REMAP_CTRL_ON BIT(31)
+#define REMAP_CTRL_INDEX(x) FIELD_PREP(GENMASK(15, 12), (x))
+#define REMAP_CTRL_PAGE_SIZE_ON BIT(11)
+#define REMAP_CTRL_PAGE_SIZE(x) FIELD_PREP(GENMASK(8, 0), ((x) >> 12))
+#define W6_VPU_REMAP_VADDR_GB (W6_REG_BASE + 0x0064)
+#define W6_VPU_REMAP_PADDR_GB (W6_REG_BASE + 0x0068)
+#define W6_VPU_REMAP_CORE_START_GB (W6_REG_BASE + 0x006C)
+#define REMAP_CORE_START_ON BIT(0)
+#define W6_VPU_BUSY_STATUS (W6_REG_BASE + 0x0070)
+#define BUSY_STATUS_SET BIT(0)
+#define W6_VPU_RET_PRODUCT_CODE (W6_REG_BASE + 0x0094)
+
+/* COMMON */
+#define W6_COMMAND_GB (W6_REG_BASE + 0x104)
+#define W6_COMMAND (W6_REG_BASE + 0x200)
+#define W6_CMD_INIT_VPU_SEC_AXI_BASE_CORE0 (W6_REG_BASE + 0x364)
+#define W6_CMD_INIT_VPU_SEC_AXI_SIZE_CORE0 (W6_REG_BASE + 0x368)
+#define W6_CMD_SET_WORK_BUF_ADDR (W6_REG_BASE + 0x5F0)
+#define W6_CMD_SET_WORK_BUF_SIZE (W6_REG_BASE + 0x5F4)
+#define SET_WORK_BUF_SIZE_ACK 0
+#define W6_RET_SUCCESS (W6_REG_BASE + 0x208)
+#define W6_RET_FAIL_REASON (W6_REG_BASE + 0x20C)
+
+#endif /* __WAVE6_REGDEFINE_H__ */
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vdi.h b/drivers/media/platform/chips-media/wave6/wave6-vdi.h
new file mode 100644
index 000000000000..69ebf0ebcb13
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-vdi.h
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
+/*
+ * Wave6 series multi-standard codec IP - low level access interface
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#ifndef __WAVE6_VDI_H__
+#define __WAVE6_VDI_H__
+
+#include <linux/string.h>
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/io.h>
+
+/**
+ * struct vpu_buf - VPU buffer for a coherent DMA buffer
+ * @size: Buffer size
+ * @daddr: Mapped address for device access
+ * @vaddr: Kernel virtual address
+ * @dev: Device pointer for DMA API
+ *
+ * Represents a buffer allocated via dma_alloc_coherent().
+ */
+struct vpu_buf {
+ size_t size;
+ dma_addr_t daddr;
+ void *vaddr;
+ struct device *dev;
+};
+
+static inline void wave6_vdi_writel(void __iomem *base, u32 addr, u32 data)
+{
+ writel(data, base + addr);
+}
+
+static inline void wave6_vdi_writel_relaxed(void __iomem *base, u32 addr,
+ u32 data)
+{
+ writel_relaxed(data, base + addr);
+}
+
+static inline unsigned int wave6_vdi_readl(void __iomem *base, u32 addr)
+{
+ return readl(base + addr);
+}
+
+static inline unsigned int wave6_vdi_readl_relaxed(void __iomem *base, u32 addr)
+{
+ return readl_relaxed(base + addr);
+}
+
+static inline int wave6_vdi_alloc_dma(struct device *dev, struct vpu_buf *vb)
+{
+ void *vaddr;
+ dma_addr_t daddr;
+
+ if (!vb || !vb->size)
+ return -EINVAL;
+
+ vaddr = dma_alloc_coherent(dev, vb->size, &daddr, GFP_KERNEL);
+ if (!vaddr)
+ return -ENOMEM;
+
+ vb->vaddr = vaddr;
+ vb->daddr = daddr;
+ vb->dev = dev;
+
+ return 0;
+}
+
+static inline void wave6_vdi_free_dma(struct vpu_buf *vb)
+{
+ if (!vb || !vb->size || !vb->vaddr)
+ return;
+
+ dma_free_coherent(vb->dev, vb->size, vb->vaddr, vb->daddr);
+
+ memset(vb, 0, sizeof(*vb));
+}
+
+#endif /* __WAVE6_VDI_H__ */
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu.c b/drivers/media/platform/chips-media/wave6/wave6-vpu.c
new file mode 100644
index 000000000000..9160ce53062c
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu.c
@@ -0,0 +1,780 @@
+// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause)
+/*
+ * Wave6 series multi-standard codec IP - wave6 driver
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/clk.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/property.h>
+#include <linux/of_irq.h>
+#include <linux/firmware.h>
+#include <linux/interrupt.h>
+#include <linux/pm_runtime.h>
+#include <linux/pm_domain.h>
+#include <linux/dma-mapping.h>
+#include <linux/iopoll.h>
+#include <linux/genalloc.h>
+
+#include "wave6-vpuconfig.h"
+#include "wave6-regdefine.h"
+#include "wave6-vpu.h"
+
+static const struct wave6_vpu_resource wave633c_data = {
+ .fw_name = "cnm/wave633c_imx9_codec_fw.bin",
+ /* For HEVC, AVC, 4096x4096, 8bit */
+ .sram_size = 0x14800,
+};
+
+static const struct wave6_vpu_core_resource wave633c_core_data = {
+ .codec_types = WAVE633_CODEC_TYPE,
+ .compatible_fw_version = WAVE633_COMPATIBLE_FW_VERSION,
+};
+
+static const char *wave6_vpu_state_name(enum wave6_vpu_state state)
+{
+ switch (state) {
+ case WAVE6_VPU_STATE_OFF:
+ return "off";
+ case WAVE6_VPU_STATE_PREPARE:
+ return "prepare";
+ case WAVE6_VPU_STATE_ON:
+ return "on";
+ case WAVE6_VPU_STATE_SLEEP:
+ return "sleep";
+ default:
+ return "unknown";
+ }
+}
+
+static bool wave6_vpu_valid_transition(struct wave6_vpu_device *vpu,
+ enum wave6_vpu_state next)
+{
+ switch (vpu->state) {
+ case WAVE6_VPU_STATE_OFF:
+ /* to PREPARE: first boot attempt */
+ /* to ON: already booted before, skipping boot */
+ if (next == WAVE6_VPU_STATE_PREPARE ||
+ next == WAVE6_VPU_STATE_ON)
+ return true;
+ break;
+ case WAVE6_VPU_STATE_PREPARE:
+ /* to OFF: boot failed */
+ /* to ON: boot successful */
+ if (next == WAVE6_VPU_STATE_OFF ||
+ next == WAVE6_VPU_STATE_ON)
+ return true;
+ break;
+ case WAVE6_VPU_STATE_ON:
+ /* to OFF: sleep failed */
+ /* to SLEEP: sleep successful */
+ if (next == WAVE6_VPU_STATE_OFF ||
+ next == WAVE6_VPU_STATE_SLEEP)
+ return true;
+ break;
+ case WAVE6_VPU_STATE_SLEEP:
+ /* to OFF: resume failed */
+ /* to ON: resume successful */
+ if (next == WAVE6_VPU_STATE_OFF ||
+ next == WAVE6_VPU_STATE_ON)
+ return true;
+ break;
+ }
+
+ dev_err(vpu->dev, "invalid transition: %s -> %s\n",
+ wave6_vpu_state_name(vpu->state), wave6_vpu_state_name(next));
+
+ return false;
+}
+
+static void wave6_vpu_set_state(struct wave6_vpu_device *vpu,
+ enum wave6_vpu_state state)
+{
+ if (!wave6_vpu_valid_transition(vpu, state))
+ return;
+
+ dev_dbg(vpu->dev, "set state: %s -> %s\n",
+ wave6_vpu_state_name(vpu->state), wave6_vpu_state_name(state));
+
+ vpu->state = state;
+}
+
+static int wave6_vpu_wait_busy(void __iomem *core_base)
+{
+ u32 val;
+
+ return read_poll_timeout(wave6_vdi_readl, val, !val,
+ W6_VPU_POLL_DELAY_US, W6_VPU_POLL_TIMEOUT,
+ false, core_base, W6_VPU_BUSY_STATUS);
+}
+
+static int wave6_vpu_check_result(void __iomem *core_base)
+{
+ if (wave6_vdi_readl(core_base, W6_RET_SUCCESS))
+ return 0;
+
+ return wave6_vdi_readl(core_base, W6_RET_FAIL_REASON);
+}
+
+static u32 wave6_vpu_get_code_buf_size(struct wave6_vpu_device *vpu)
+{
+ return min_t(u32, vpu->code_buf.size, W6_MAX_CODE_BUF_SIZE);
+}
+
+static void wave6_vpu_remap_code_buf(struct wave6_vpu_device *vpu)
+{
+ dma_addr_t code_base = vpu->code_buf.dma_addr;
+ u32 reg_val;
+
+ for (u32 i = 0; i < wave6_vpu_get_code_buf_size(vpu) / W6_MAX_REMAP_PAGE_SIZE; i++) {
+ reg_val = REMAP_CTRL_ON |
+ REMAP_CTRL_INDEX(i) |
+ REMAP_CTRL_PAGE_SIZE_ON |
+ REMAP_CTRL_PAGE_SIZE(W6_MAX_REMAP_PAGE_SIZE);
+ wave6_vdi_writel(vpu->reg_base, W6_VPU_REMAP_CTRL_GB, reg_val);
+ wave6_vdi_writel(vpu->reg_base, W6_VPU_REMAP_VADDR_GB,
+ i * W6_MAX_REMAP_PAGE_SIZE);
+ wave6_vdi_writel(vpu->reg_base, W6_VPU_REMAP_PADDR_GB,
+ code_base + i * W6_MAX_REMAP_PAGE_SIZE);
+ }
+}
+
+static int wave6_vpu_init_code_buf(struct wave6_vpu_device *vpu)
+{
+ void *vaddr;
+
+ if (vpu->code_buf.size < W6_CODE_BUF_SIZE)
+ return dev_err_probe(vpu->dev, -EINVAL,
+ "code buf size (%zu) is too small\n",
+ vpu->code_buf.size);
+
+ vaddr = devm_memremap(vpu->dev,
+ vpu->code_buf.phys_addr,
+ vpu->code_buf.size,
+ MEMREMAP_WC);
+ if (IS_ERR(vaddr))
+ return dev_err_probe(vpu->dev, PTR_ERR(vaddr),
+ "failed to map code buf\n");
+
+ vpu->code_buf.dma_addr = dma_map_phys(vpu->dev,
+ vpu->code_buf.phys_addr,
+ vpu->code_buf.size,
+ DMA_BIDIRECTIONAL,
+ DMA_ATTR_SKIP_CPU_SYNC);
+ if (dma_mapping_error(vpu->dev, vpu->code_buf.dma_addr))
+ return dev_err_probe(vpu->dev, -ENOMEM,
+ "failed to DMA map code buf\n");
+
+ vpu->code_buf.vaddr = vaddr;
+
+ return 0;
+}
+
+static void wave6_vpu_allocate_work_buffers(struct wave6_vpu_device *vpu)
+{
+ struct vpu_buf *buf;
+
+ for (int i = 0; i < MAX_NUM_INSTANCE; i++) {
+ buf = &vpu->work_buffers[i];
+ buf->size = W637DEC_WORKBUF_SIZE_FOR_CQ;
+
+ if (wave6_vdi_alloc_dma(vpu->dev, buf)) {
+ dev_warn(vpu->dev, "Failed to allocate work_buffers\n");
+ return;
+ }
+
+ vpu->work_buffers_alloc++;
+ }
+}
+
+static void wave6_vpu_free_work_buffers(struct wave6_vpu_device *vpu)
+{
+ for (int i = 0; i < vpu->work_buffers_alloc; i++)
+ wave6_vdi_free_dma(&vpu->work_buffers[i]);
+
+ vpu->work_buffers_alloc = 0;
+ vpu->work_buffers_avail = 0;
+}
+
+static int wave6_vpu_init_work_buf(struct wave6_vpu_device *vpu,
+ void __iomem *core_base)
+{
+ int ret;
+
+ lockdep_assert_held(&vpu->lock);
+
+ wave6_vdi_writel(core_base, W6_VPU_BUSY_STATUS, BUSY_STATUS_SET);
+ wave6_vdi_writel(core_base, W6_COMMAND, W6_CMD_INIT_WORK_BUF);
+ wave6_vdi_writel(core_base, W6_VPU_HOST_INT_REQ, HOST_INT_REQ_ON);
+
+ ret = wave6_vpu_wait_busy(core_base);
+ if (ret) {
+ dev_err(vpu->dev, "init work buf failed\n");
+ return -EINVAL;
+ }
+
+ ret = wave6_vpu_check_result(core_base);
+ if (ret) {
+ dev_err(vpu->dev, "init work buf failed, reason 0x%x\n", ret);
+ return -EIO;
+ }
+
+ vpu->work_buffers_avail = vpu->work_buffers_alloc;
+
+ return 0;
+}
+
+static int wave6_vpu_init_vpu(struct wave6_vpu_device *vpu,
+ void __iomem *core_base)
+{
+ int ret;
+
+ lockdep_assert_held(&vpu->lock);
+
+ /* try init directly as firmware is running */
+ if (wave6_vdi_readl(core_base, W6_VPU_VCPU_CUR_PC))
+ goto init_done;
+
+ wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_PREPARE);
+
+ wave6_vpu_remap_code_buf(vpu);
+
+ wave6_vdi_writel(core_base, W6_VPU_BUSY_STATUS, BUSY_STATUS_SET);
+ wave6_vdi_writel(core_base, W6_CMD_INIT_VPU_SEC_AXI_BASE_CORE0,
+ vpu->sram_buf.dma_addr);
+ wave6_vdi_writel(core_base, W6_CMD_INIT_VPU_SEC_AXI_SIZE_CORE0,
+ vpu->sram_buf.size);
+ wave6_vdi_writel(vpu->reg_base, W6_COMMAND_GB, W6_CMD_INIT_VPU);
+ wave6_vdi_writel(vpu->reg_base, W6_VPU_REMAP_CORE_START_GB,
+ REMAP_CORE_START_ON);
+
+ ret = wave6_vpu_wait_busy(core_base);
+ if (ret) {
+ dev_err(vpu->dev, "init vpu timeout\n");
+ wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_OFF);
+ return -EINVAL;
+ }
+
+ ret = wave6_vpu_check_result(core_base);
+ if (ret) {
+ dev_err(vpu->dev, "init vpu fail, reason 0x%x\n", ret);
+ wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_OFF);
+ return -EIO;
+ }
+
+init_done:
+ ret = wave6_vpu_init_work_buf(vpu, core_base);
+ if (ret)
+ return ret;
+
+ wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_ON);
+
+ return 0;
+}
+
+static int wave6_vpu_sleep(struct wave6_vpu_device *vpu,
+ void __iomem *core_base)
+{
+ int ret;
+
+ lockdep_assert_held(&vpu->lock);
+
+ if (!wave6_vdi_readl(core_base, W6_VPU_VCPU_CUR_PC)) {
+ wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_OFF);
+ return 0;
+ }
+
+ wave6_vdi_writel(core_base, W6_VPU_BUSY_STATUS, BUSY_STATUS_SET);
+ wave6_vdi_writel(core_base, W6_COMMAND, W6_CMD_SLEEP_VPU);
+ wave6_vdi_writel(core_base, W6_VPU_HOST_INT_REQ, HOST_INT_REQ_ON);
+
+ ret = wave6_vpu_wait_busy(core_base);
+ if (ret) {
+ dev_err(vpu->dev, "sleep vpu timeout\n");
+ wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_OFF);
+ return -EINVAL;
+ }
+
+ ret = wave6_vpu_check_result(core_base);
+ if (ret) {
+ dev_err(vpu->dev, "sleep vpu fail, reason 0x%x\n", ret);
+ wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_OFF);
+ return -EIO;
+ }
+
+ wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_SLEEP);
+
+ return 0;
+}
+
+static int wave6_vpu_wakeup(struct wave6_vpu_device *vpu,
+ void __iomem *core_base)
+{
+ int ret;
+
+ lockdep_assert_held(&vpu->lock);
+
+ /* try wakeup directly as firmware is running */
+ if (wave6_vdi_readl(core_base, W6_VPU_VCPU_CUR_PC))
+ goto wakeup_done;
+
+ wave6_vpu_remap_code_buf(vpu);
+
+ wave6_vdi_writel(core_base, W6_VPU_BUSY_STATUS, BUSY_STATUS_SET);
+ wave6_vdi_writel(core_base, W6_CMD_INIT_VPU_SEC_AXI_BASE_CORE0,
+ vpu->sram_buf.dma_addr);
+ wave6_vdi_writel(core_base, W6_CMD_INIT_VPU_SEC_AXI_SIZE_CORE0,
+ vpu->sram_buf.size);
+ wave6_vdi_writel(vpu->reg_base, W6_COMMAND_GB, W6_CMD_WAKEUP_VPU);
+ wave6_vdi_writel(vpu->reg_base, W6_VPU_REMAP_CORE_START_GB,
+ REMAP_CORE_START_ON);
+
+ ret = wave6_vpu_wait_busy(core_base);
+ if (ret) {
+ dev_err(vpu->dev, "wakeup vpu timeout\n");
+ wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_OFF);
+ return -EINVAL;
+ }
+
+ ret = wave6_vpu_check_result(core_base);
+ if (ret) {
+ dev_err(vpu->dev, "wakeup vpu fail, reason 0x%x\n", ret);
+ wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_OFF);
+ return -EIO;
+ }
+
+wakeup_done:
+ wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_ON);
+
+ return 0;
+}
+
+static int wave6_vpu_try_boot(struct wave6_vpu_device *vpu,
+ void __iomem *core_base)
+{
+ u32 product_code;
+ int ret;
+
+ lockdep_assert_held(&vpu->lock);
+
+ if (vpu->state != WAVE6_VPU_STATE_OFF && vpu->state != WAVE6_VPU_STATE_SLEEP)
+ return 0;
+
+ product_code = wave6_vdi_readl(core_base, W6_VPU_RET_PRODUCT_CODE);
+ if (!wave6_is_product_w_series(product_code)) {
+ dev_err(vpu->dev, "unknown product : %08x\n", product_code);
+ return -EINVAL;
+ }
+
+ if (vpu->state == WAVE6_VPU_STATE_SLEEP) {
+ ret = wave6_vpu_wakeup(vpu, core_base);
+ return ret;
+ }
+
+ ret = wave6_vpu_init_vpu(vpu, core_base);
+
+ return ret;
+}
+
+int wave6_vpu_get(struct wave6_vpu_device *vpu, void __iomem *core_base)
+{
+ int ret;
+
+ guard(mutex)(&vpu->lock);
+
+ /* Only the first core executes boot; others return */
+ if (atomic_inc_return(&vpu->core_count) > 1)
+ return 0;
+
+ ret = pm_runtime_resume_and_get(vpu->dev);
+ if (ret)
+ goto error_pm;
+
+ ret = wave6_vpu_try_boot(vpu, core_base);
+ if (ret)
+ goto error_boot;
+
+ return 0;
+
+error_boot:
+ pm_runtime_put_sync(vpu->dev);
+error_pm:
+ atomic_dec(&vpu->core_count);
+
+ return ret;
+}
+
+void wave6_vpu_put(struct wave6_vpu_device *vpu, void __iomem *core_base)
+{
+ guard(mutex)(&vpu->lock);
+
+ /* Only the last core executes sleep; others return */
+ if (atomic_dec_return(&vpu->core_count) > 0)
+ return;
+
+ wave6_vpu_sleep(vpu, core_base);
+
+ pm_runtime_put_sync(vpu->dev);
+}
+
+void wave6_vpu_require_work_buffer(struct wave6_vpu_device *vpu,
+ void __iomem *core_base)
+{
+ struct vpu_buf *vb;
+ u32 size;
+
+ size = wave6_vdi_readl(core_base, W6_CMD_SET_WORK_BUF_SIZE);
+ if (!size)
+ return;
+
+ if (WARN_ON(size > W637DEC_WORKBUF_SIZE_FOR_CQ))
+ goto exit;
+
+ if (WARN_ON(vpu->work_buffers_avail == 0))
+ goto exit;
+
+ vpu->work_buffers_avail--;
+ vb = &vpu->work_buffers[vpu->work_buffers_avail];
+
+ wave6_vdi_writel(core_base, W6_CMD_SET_WORK_BUF_ADDR, vb->daddr);
+
+exit:
+ wave6_vdi_writel(core_base, W6_CMD_SET_WORK_BUF_SIZE, SET_WORK_BUF_SIZE_ACK);
+}
+
+static int wave6_vpu_create_cores(struct wave6_vpu_device *vpu)
+{
+ struct device_node *child;
+ int num_cores = 0;
+
+ for_each_available_child_of_node(vpu->dev->of_node, child) {
+ struct platform_device_info info = {};
+ struct platform_device *pdev;
+ struct resource res[2];
+ int irq;
+
+ if (num_cores >= W6_VPU_MAX_NUM_CORE) {
+ of_node_put(child);
+ break;
+ }
+
+ if (of_address_to_resource(child, 0, &res[0])) {
+ dev_warn(vpu->dev, "%pOF: missing reg property\n", child);
+ continue;
+ }
+
+ irq = of_irq_get(child, 0);
+ if (irq < 0) {
+ dev_warn(vpu->dev, "%pOF: missing interrupts property\n", child);
+ continue;
+ }
+ res[1] = DEFINE_RES_IRQ(irq);
+
+ info.fwnode = of_fwnode_handle(child);
+ info.parent = vpu->dev;
+ info.name = WAVE6_VPU_CORE_PLATFORM_DRIVER_NAME;
+ info.id = num_cores;
+ info.dma_mask = DMA_BIT_MASK(32);
+ info.res = res;
+ info.num_res = ARRAY_SIZE(res);
+ info.data = &wave633c_core_data;
+ info.size_data = sizeof(wave633c_core_data);
+
+ pdev = platform_device_register_full(&info);
+ if (IS_ERR(pdev)) {
+ dev_err(vpu->dev, "Failed to register core %d\n",
+ num_cores);
+ continue;
+ }
+
+ vpu->core_pdevs[num_cores] = pdev;
+ num_cores++;
+ }
+
+ return num_cores;
+}
+
+static void wave6_vpu_destroy_cores(struct wave6_vpu_device *vpu)
+{
+ for (int i = 0; i < W6_VPU_MAX_NUM_CORE; i++) {
+ struct platform_device *pdev = vpu->core_pdevs[i];
+
+ if (!pdev)
+ continue;
+
+ platform_device_unregister(pdev);
+ vpu->core_pdevs[i] = NULL;
+ }
+}
+
+static void wave6_vpu_release(struct wave6_vpu_device *vpu)
+{
+ guard(mutex)(&vpu->lock);
+
+ wave6_vpu_destroy_cores(vpu);
+ wave6_vpu_free_work_buffers(vpu);
+ if (vpu->sram_pool && vpu->sram_buf.vaddr) {
+ dma_unmap_resource(vpu->dev,
+ vpu->sram_buf.dma_addr,
+ vpu->sram_buf.size,
+ DMA_BIDIRECTIONAL,
+ 0);
+ gen_pool_free(vpu->sram_pool,
+ (unsigned long)vpu->sram_buf.vaddr,
+ vpu->sram_buf.size);
+ }
+ if (vpu->code_buf.dma_addr)
+ dma_unmap_phys(vpu->dev,
+ vpu->code_buf.dma_addr,
+ vpu->code_buf.size,
+ DMA_BIDIRECTIONAL,
+ DMA_ATTR_SKIP_CPU_SYNC);
+}
+
+static void wave6_vpu_init_firmware(struct wave6_vpu_device *vpu,
+ const struct firmware *fw)
+{
+ guard(mutex)(&vpu->lock);
+
+ if (!fw || !fw->data) {
+ dev_err(vpu->dev, "No firmware.\n");
+ return;
+ }
+
+ if (fw->size + W6_EXTRA_CODE_BUF_SIZE > wave6_vpu_get_code_buf_size(vpu)) {
+ dev_err(vpu->dev, "firmware size (%zd > %zd) is too big\n",
+ fw->size, vpu->code_buf.size);
+ return;
+ }
+
+ memcpy(vpu->code_buf.vaddr, fw->data, fw->size);
+
+ if (!wave6_vpu_create_cores(vpu))
+ dev_err(vpu->dev, "Failed to create VPU cores\n");
+}
+
+static void wave6_vpu_load_firmware(const struct firmware *fw, void *context)
+{
+ struct wave6_vpu_device *vpu = context;
+
+ wave6_vpu_init_firmware(vpu, fw);
+ release_firmware(fw);
+ complete_all(&vpu->fw_done);
+}
+
+static void wave6_vpu_detach_pm_domains(struct wave6_vpu_device *vpu)
+{
+ if (!vpu->num_pm_domains)
+ return;
+
+ for (int i = 0; i < vpu->num_pm_domains; i++) {
+ struct device *pd_dev = vpu->pd_list->pd_devs[i];
+
+ if (!IS_ERR_OR_NULL(pd_dev) && !pm_runtime_suspended(pd_dev))
+ pm_runtime_force_suspend(pd_dev);
+ }
+
+ dev_pm_domain_detach_list(vpu->pd_list);
+ vpu->pd_list = NULL;
+ vpu->num_pm_domains = 0;
+}
+
+static int wave6_vpu_attach_pm_domains(struct wave6_vpu_device *vpu)
+{
+ int ret;
+
+ vpu->num_pm_domains = of_count_phandle_with_args(vpu->dev->of_node,
+ "power-domains",
+ "#power-domain-cells");
+ if (vpu->num_pm_domains < 0) {
+ dev_err(vpu->dev, "No power domains defined for vpu node\n");
+ return vpu->num_pm_domains;
+ }
+
+ if (vpu->num_pm_domains == 1) {
+ /* genpd_dev_pm_attach() attach automatically if count is 1 */
+ vpu->num_pm_domains = 0;
+ return 0;
+ }
+
+ ret = dev_pm_domain_attach_list(vpu->dev, NULL, &vpu->pd_list);
+ if (ret < 0) {
+ dev_err(vpu->dev, "Can't attach pm domains, ret = %d\n", ret);
+ return ret;
+ }
+
+ vpu->num_pm_domains = ret;
+
+ return 0;
+}
+
+static int wave6_vpu_probe(struct platform_device *pdev)
+{
+ struct device_node *np;
+ struct wave6_vpu_device *vpu;
+ const struct wave6_vpu_resource *res;
+ int ret;
+
+ ret = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
+ if (ret < 0) {
+ dev_err(&pdev->dev, "failed to set DMA mask: %d\n", ret);
+ return ret;
+ }
+
+ res = device_get_match_data(&pdev->dev);
+ if (!res)
+ return -ENODEV;
+
+ vpu = devm_kzalloc(&pdev->dev, sizeof(*vpu), GFP_KERNEL);
+ if (!vpu)
+ return -ENOMEM;
+
+ ret = devm_mutex_init(&pdev->dev, &vpu->lock);
+ if (ret)
+ return ret;
+
+ atomic_set(&vpu->core_count, 0);
+ dev_set_drvdata(&pdev->dev, vpu);
+ vpu->dev = &pdev->dev;
+ vpu->res = res;
+ vpu->reg_base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(vpu->reg_base))
+ return PTR_ERR(vpu->reg_base);
+
+ ret = devm_clk_bulk_get_all(&pdev->dev, &vpu->clks);
+ if (ret < 0)
+ return dev_err_probe(&pdev->dev, ret, "failed to get clocks\n");
+
+ vpu->num_clks = ret;
+
+ ret = wave6_vpu_attach_pm_domains(vpu);
+ if (ret)
+ return dev_err_probe(&pdev->dev, ret, "failed to attach pm domains\n");
+
+ np = of_parse_phandle(pdev->dev.of_node, "memory-region", 0);
+ if (np) {
+ struct resource mem;
+
+ ret = of_address_to_resource(np, 0, &mem);
+ of_node_put(np);
+ if (!ret) {
+ vpu->code_buf.phys_addr = mem.start;
+ vpu->code_buf.size = resource_size(&mem);
+ ret = wave6_vpu_init_code_buf(vpu);
+ if (ret)
+ goto error_detach_pm_domains;
+ } else {
+ dev_warn(&pdev->dev, "memory-region is not available.\n");
+ }
+ }
+
+ vpu->sram_pool = of_gen_pool_get(pdev->dev.of_node, "sram", 0);
+ if (vpu->sram_pool) {
+ vpu->sram_buf.size = vpu->res->sram_size;
+ vpu->sram_buf.vaddr = gen_pool_dma_alloc(vpu->sram_pool,
+ vpu->sram_buf.size,
+ &vpu->sram_buf.phys_addr);
+ if (!vpu->sram_buf.vaddr) {
+ vpu->sram_buf.size = 0;
+ } else {
+ vpu->sram_buf.dma_addr = dma_map_resource(&pdev->dev,
+ vpu->sram_buf.phys_addr,
+ vpu->sram_buf.size,
+ DMA_BIDIRECTIONAL,
+ 0);
+ if (dma_mapping_error(&pdev->dev, vpu->sram_buf.dma_addr)) {
+ gen_pool_free(vpu->sram_pool,
+ (unsigned long)vpu->sram_buf.vaddr,
+ vpu->sram_buf.size);
+ memset(&vpu->sram_buf, 0, sizeof(vpu->sram_buf));
+ }
+ }
+ }
+
+ wave6_vpu_allocate_work_buffers(vpu);
+
+ pm_runtime_enable(&pdev->dev);
+
+ init_completion(&vpu->fw_done);
+ ret = firmware_request_nowait_nowarn(THIS_MODULE,
+ vpu->res->fw_name,
+ &pdev->dev,
+ GFP_KERNEL,
+ vpu,
+ wave6_vpu_load_firmware);
+ if (ret) {
+ dev_err(&pdev->dev, "request firmware fail, ret = %d\n", ret);
+ goto error;
+ }
+
+ return 0;
+
+error:
+ wave6_vpu_release(vpu);
+ pm_runtime_disable(&pdev->dev);
+error_detach_pm_domains:
+ wave6_vpu_detach_pm_domains(vpu);
+
+ return ret;
+}
+
+static void wave6_vpu_remove(struct platform_device *pdev)
+{
+ struct wave6_vpu_device *vpu = dev_get_drvdata(&pdev->dev);
+
+ wait_for_completion(&vpu->fw_done);
+ wave6_vpu_release(vpu);
+ pm_runtime_disable(vpu->dev);
+ wave6_vpu_detach_pm_domains(vpu);
+}
+
+static int wave6_vpu_runtime_suspend(struct device *dev)
+{
+ struct wave6_vpu_device *vpu = dev_get_drvdata(dev);
+
+ clk_bulk_disable_unprepare(vpu->num_clks, vpu->clks);
+
+ return 0;
+}
+
+static int wave6_vpu_runtime_resume(struct device *dev)
+{
+ struct wave6_vpu_device *vpu = dev_get_drvdata(dev);
+
+ return clk_bulk_prepare_enable(vpu->num_clks, vpu->clks);
+}
+
+static const struct dev_pm_ops wave6_vpu_pm_ops = {
+ RUNTIME_PM_OPS(wave6_vpu_runtime_suspend,
+ wave6_vpu_runtime_resume, NULL)
+ SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+ pm_runtime_force_resume)
+};
+
+static const struct of_device_id wave6_vpu_ids[] = {
+ { .compatible = "nxp,imx95-vpu", .data = &wave633c_data },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, wave6_vpu_ids);
+
+static struct platform_driver wave6_vpu_driver = {
+ .driver = {
+ .name = WAVE6_VPU_PLATFORM_DRIVER_NAME,
+ .of_match_table = wave6_vpu_ids,
+ .pm = pm_ptr(&wave6_vpu_pm_ops),
+ .suppress_bind_attrs = true,
+ },
+ .probe = wave6_vpu_probe,
+ .remove = wave6_vpu_remove,
+};
+
+module_platform_driver(wave6_vpu_driver);
+MODULE_DESCRIPTION("chips&media Wave6 VPU driver");
+MODULE_AUTHOR("CHIPS&MEDIA INC");
+MODULE_FIRMWARE("cnm/wave633c_imx9_codec_fw.bin");
+MODULE_LICENSE("Dual BSD/GPL");
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu.h b/drivers/media/platform/chips-media/wave6/wave6-vpu.h
new file mode 100644
index 000000000000..96d67e931e71
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu.h
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
+/*
+ * Wave6 series multi-standard codec IP - wave6 driver
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#ifndef __WAVE6_VPU_H__
+#define __WAVE6_VPU_H__
+
+#include <linux/completion.h>
+#include <linux/device.h>
+
+#include "wave6-vpuconfig.h"
+#include "wave6-vdi.h"
+
+#define WAVE6_VPU_PLATFORM_DRIVER_NAME "wave6-vpu"
+#define WAVE6_VPU_CORE_PLATFORM_DRIVER_NAME "wave6-vpu-core"
+
+struct wave6_vpu_device;
+
+/**
+ * enum wave6_vpu_state - VPU states
+ * @WAVE6_VPU_STATE_OFF: VPU is powered off
+ * @WAVE6_VPU_STATE_PREPARE: VPU is booting
+ * @WAVE6_VPU_STATE_ON: VPU is running
+ * @WAVE6_VPU_STATE_SLEEP: VPU is in a sleep mode
+ */
+enum wave6_vpu_state {
+ WAVE6_VPU_STATE_OFF,
+ WAVE6_VPU_STATE_PREPARE,
+ WAVE6_VPU_STATE_ON,
+ WAVE6_VPU_STATE_SLEEP
+};
+
+/**
+ * struct wave6_vpu_dma_buf - VPU buffer from reserved memory or gen_pool
+ * @size: Buffer size
+ * @dma_addr: Mapped address for device access
+ * @vaddr: Kernel virtual address
+ * @phys_addr: Physical address of the reserved memory region or gen_pool
+ *
+ * Represents a buffer allocated from pre-reserved device memory regions or
+ * SRAM via gen_pool_dma_alloc(). Used for code and SRAM buffers only.
+ * Managed by the VPU device.
+ */
+struct wave6_vpu_dma_buf {
+ size_t size;
+ dma_addr_t dma_addr;
+ void *vaddr;
+ phys_addr_t phys_addr;
+};
+
+/**
+ * struct wave6_vpu_resource - VPU device compatible data
+ * @fw_name: Firmware name for the device
+ * @sram_size: Required SRAM size
+ */
+struct wave6_vpu_resource {
+ const char *fw_name;
+ u32 sram_size;
+};
+
+#define WAVE6_IS_ENC BIT(0)
+#define WAVE6_IS_DEC BIT(1)
+
+#define WAVE633_CODEC_TYPE (WAVE6_IS_ENC | WAVE6_IS_DEC)
+#define WAVE633_COMPATIBLE_FW_VERSION 0x2010000
+
+/**
+ * struct wave6_vpu_core_resource - VPU CORE device compatible data
+ * @codec_types: Bitmask of supported codec types
+ * @compatible_fw_version: Firmware version compatible with driver
+ */
+struct wave6_vpu_core_resource {
+ int codec_types;
+ u32 compatible_fw_version;
+};
+
+/**
+ * struct wave6_vpu_device - VPU driver structure
+ * @dev: Platform device pointer
+ * @reg_base: Base address of MMIO registers
+ * @clks: Array of clock handles
+ * @num_clks: Number of entries in @clks
+ * @state: Device state
+ * @lock: Mutex protecting device data, register access
+ * @fw_done: Completed when the firmware request callback has returned
+ * @res: Device compatible data
+ * @sram_pool: Genalloc pool for SRAM allocations
+ * @sram_buf: Optional SRAM buffer
+ * @code_buf: Firmware code buffer
+ * @work_buffers: Array of work buffers
+ * @work_buffers_alloc: Number of allocated work buffers
+ * @work_buffers_avail: Number of available work buffers
+ * @core_count: Number of VPU cores holding a wave6_vpu_get() reference
+ * @num_pm_domains: Number of attached power domains
+ * @pd_list: List of attached power domains
+ * @core_pdevs: VPU core platform devices
+ *
+ * Buffers such as @sram_buf, @code_buf, and @work_buffers are managed
+ * by the VPU device and accessed exclusively by the firmware.
+ */
+struct wave6_vpu_device {
+ struct device *dev;
+ void __iomem *reg_base;
+ struct clk_bulk_data *clks;
+ int num_clks;
+ enum wave6_vpu_state state;
+ struct mutex lock; /* Protects device data, register access */
+
+ struct completion fw_done;
+
+ const struct wave6_vpu_resource *res;
+ struct gen_pool *sram_pool;
+ struct wave6_vpu_dma_buf sram_buf;
+ struct wave6_vpu_dma_buf code_buf;
+
+ /* Allocates per-instance, used for storing instance-specific data. */
+ struct vpu_buf work_buffers[MAX_NUM_INSTANCE];
+ u32 work_buffers_alloc;
+ u32 work_buffers_avail;
+
+ atomic_t core_count;
+
+ int num_pm_domains;
+ struct dev_pm_domain_list *pd_list;
+
+ struct platform_device *core_pdevs[W6_VPU_MAX_NUM_CORE];
+};
+
+int wave6_vpu_get(struct wave6_vpu_device *vpu, void __iomem *core_base);
+void wave6_vpu_put(struct wave6_vpu_device *vpu, void __iomem *core_base);
+void wave6_vpu_require_work_buffer(struct wave6_vpu_device *vpu,
+ void __iomem *core_base);
+
+#endif /* __WAVE6_VPU_H__ */
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h b/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
new file mode 100644
index 000000000000..1d4e016c2a58
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
+/*
+ * Wave6 series multi-standard codec IP - product config definitions
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#ifndef __WAVE6_VPUCONFIG_H__
+#define __WAVE6_VPUCONFIG_H__
+
+#define WAVE617_CODE 0x6170
+#define WAVE627_CODE 0x6270
+#define WAVE633_CODE 0x6330
+#define WAVE637_CODE 0x6370
+#define WAVE663_CODE 0x6630
+#define WAVE677_CODE 0x6670
+
+#define W6_VPU_MAX_NUM_CORE 4
+#define MAX_NUM_INSTANCE 32
+
+#define W6_VPU_POLL_DELAY_US 10
+#define W6_VPU_POLL_TIMEOUT 300000
+
+#define W6_MAX_REMAP_PAGE_SIZE (1024 * 1024)
+#define W6_MAX_CODE_BUF_SIZE (4 * 1024 * 1024)
+#define W6_CODE_BUF_SIZE (1 * 1024 * 1024)
+#define W6_EXTRA_CODE_BUF_SIZE (256 * 1024)
+#define W637DEC_WORKBUF_SIZE_FOR_CQ (3 * 512 * 1024)
+
+static inline bool wave6_is_product_w_series(u32 code)
+{
+ return code == WAVE617_CODE || code == WAVE627_CODE ||
+ code == WAVE633_CODE || code == WAVE637_CODE ||
+ code == WAVE663_CODE || code == WAVE677_CODE;
+}
+
+#endif /* __WAVE6_VPUCONFIG_H__ */
--
2.41.0.windows.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v8 03/11] media: chips-media: wave6: Add Wave6 core driver
2026-11-19 8:59 [PATCH v8 00/11] Add support for Wave6 video codec driver Nas Chung
2026-11-19 8:59 ` [PATCH v8 01/11] dt-bindings: media: nxp: Add Wave6 video codec device Nas Chung
2026-11-19 8:59 ` [PATCH v8 02/11] media: chips-media: wave6: Add Wave6 control driver Nas Chung
@ 2026-11-19 8:59 ` Nas Chung
2026-11-19 9:32 ` sashiko-bot
2026-11-19 8:59 ` [PATCH v8 04/11] media: chips-media: wave6: Add Wave6 decoder firmware interface Nas Chung
` (7 subsequent siblings)
10 siblings, 1 reply; 23+ messages in thread
From: Nas Chung @ 2026-11-19 8:59 UTC (permalink / raw)
To: mchehab, hverkuil, robh, krzk+dt, conor+dt, Frank.Li, s.hauer
Cc: linux-media, devicetree, linux-kernel, imx, linux-arm-kernel,
festevam, kernel, nicolas.dufresne, ming.qian, marek.vasut,
jackson.lee, lafley.kim, Nas Chung
Add the core driver for the Chips&Media Wave6 video codec IP.
The hardware contains one control register region and four interface
register regions for a shared video processing engine. The core driver
handles the interface register regions, each with its own MMIO range
and interrupt, while relying on the control driver for firmware loading
and shared resource management.
It configures the V4L2 mem2mem devices and communicates with the Wave6
hardware to perform video processing tasks.
Tested-by: Ming Qian <ming.qian@oss.nxp.com>
Tested-by: Marek Vasut <marek.vasut@mailbox.org>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
---
.../media/platform/chips-media/wave6/Kconfig | 2 +
.../media/platform/chips-media/wave6/Makefile | 5 +-
.../platform/chips-media/wave6/wave6-hw.c | 181 ++++++++
.../platform/chips-media/wave6/wave6-hw.h | 22 +
.../chips-media/wave6/wave6-regdefine.h | 65 +++
.../platform/chips-media/wave6/wave6-vdi.h | 20 +
.../chips-media/wave6/wave6-vpu-core.c | 392 ++++++++++++++++++
.../chips-media/wave6/wave6-vpu-core.h | 106 +++++
.../chips-media/wave6/wave6-vpu-v4l2.c | 384 +++++++++++++++++
.../platform/chips-media/wave6/wave6-vpu.c | 24 +-
.../platform/chips-media/wave6/wave6-vpu.h | 2 +
.../platform/chips-media/wave6/wave6-vpuapi.h | 381 +++++++++++++++++
.../chips-media/wave6/wave6-vpuconfig.h | 6 +
13 files changed, 1588 insertions(+), 2 deletions(-)
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-hw.c
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-hw.h
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-core.h
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
diff --git a/drivers/media/platform/chips-media/wave6/Kconfig b/drivers/media/platform/chips-media/wave6/Kconfig
index 153a46ebdc6b..6d87d26fa9fc 100644
--- a/drivers/media/platform/chips-media/wave6/Kconfig
+++ b/drivers/media/platform/chips-media/wave6/Kconfig
@@ -5,6 +5,8 @@ config VIDEO_WAVE6_VPU
depends on V4L_MEM2MEM_DRIVERS
depends on VIDEO_DEV && OF
depends on ARCH_MXC || COMPILE_TEST
+ select VIDEOBUF2_DMA_CONTIG
+ select V4L2_MEM2MEM_DEV
select GENERIC_ALLOCATOR
help
Chips&Media Wave6 stateful codec driver.
diff --git a/drivers/media/platform/chips-media/wave6/Makefile b/drivers/media/platform/chips-media/wave6/Makefile
index 255fc90bc642..7bfc28ba5523 100644
--- a/drivers/media/platform/chips-media/wave6/Makefile
+++ b/drivers/media/platform/chips-media/wave6/Makefile
@@ -1,4 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
-wave6-objs += wave6-vpu.o
+wave6-objs += wave6-vpu.o \
+ wave6-vpu-core.o \
+ wave6-vpu-v4l2.o \
+ wave6-hw.o
obj-$(CONFIG_VIDEO_WAVE6_VPU) += wave6.o
diff --git a/drivers/media/platform/chips-media/wave6/wave6-hw.c b/drivers/media/platform/chips-media/wave6/wave6-hw.c
new file mode 100644
index 000000000000..fcf5be294ef5
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-hw.c
@@ -0,0 +1,181 @@
+// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause)
+/*
+ * Wave6 series multi-standard codec IP - wave6 backend interface
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#include <linux/iopoll.h>
+
+#include "wave6-vpu-core.h"
+#include "wave6-hw.h"
+#include "wave6-regdefine.h"
+
+#define wave6_print_reg_err(core, reason) \
+ _wave6_print_reg_err((core), (reason), __func__)
+
+void wave6_vpu_writel(struct vpu_core_device *core, u32 addr, u32 data)
+{
+ wave6_vdi_writel(core->reg_base, addr, data);
+}
+
+u32 wave6_vpu_readl(struct vpu_core_device *core, u32 addr)
+{
+ u32 data;
+
+ data = wave6_vdi_readl(core->reg_base, addr);
+
+ return data;
+}
+
+void wave6_vpu_writel_relaxed(struct vpu_core_device *core, u32 addr, u32 data)
+{
+ wave6_vdi_writel_relaxed(core->reg_base, addr, data);
+}
+
+u32 wave6_vpu_readl_relaxed(struct vpu_core_device *core, u32 addr)
+{
+ u32 data;
+
+ data = wave6_vdi_readl_relaxed(core->reg_base, addr);
+
+ return data;
+}
+
+static void _wave6_print_reg_err(struct vpu_core_device *core, u32 fail_reason,
+ const char *func)
+{
+ dev_err(core->dev, "%s: failure: 0x%x\n", func, fail_reason);
+}
+
+static int wave6_wait_vpu_busy(struct vpu_core_device *core, unsigned int addr)
+{
+ u32 data;
+
+ lockdep_assert_held(&core->hw_lock);
+
+ return read_poll_timeout(vpu_read_reg, data, !data,
+ W6_VPU_POLL_DELAY_US, W6_VPU_POLL_TIMEOUT,
+ false, core, addr);
+}
+
+void wave6_vpu_enable_interrupt(struct vpu_core_device *core)
+{
+ u32 data;
+
+ lockdep_assert_held(&core->hw_lock);
+
+ data = BIT(W6_INT_BIT_INIT_SEQ);
+ data |= BIT(W6_INT_BIT_DEC_PIC);
+ data |= BIT(W6_INT_BIT_BSBUF_ERROR);
+ data |= BIT(W6_INT_BIT_REQ_WORK_BUF);
+ vpu_write_reg(core, W6_VPU_VINT_ENABLE, data);
+}
+
+bool wave6_vpu_is_init(struct vpu_core_device *core)
+{
+ lockdep_assert_held(&core->hw_lock);
+
+ return vpu_read_reg(core, W6_VPU_VCPU_CUR_PC) != 0;
+}
+
+static u32 wave6_vpu_get_product_id(struct vpu_core_device *core)
+{
+ u32 val;
+
+ lockdep_assert_held(&core->hw_lock);
+
+ val = vpu_read_reg_relaxed(core, W6_VPU_RET_PRODUCT_CODE);
+
+ switch (val) {
+ case WAVE617_CODE:
+ return PRODUCT_ID_617;
+ case WAVE627_CODE:
+ return PRODUCT_ID_627;
+ case WAVE633_CODE:
+ case WAVE637_CODE:
+ case WAVE663_CODE:
+ case WAVE677_CODE:
+ return PRODUCT_ID_637;
+ default:
+ dev_err(core->dev, "Invalid product (%x)\n", val);
+ break;
+ }
+
+ return PRODUCT_ID_NONE;
+}
+
+static void wave6_send_command(struct vpu_core_device *core, u32 id, u32 std, u32 cmd)
+{
+ u32 reg_val;
+
+ lockdep_assert_held(&core->hw_lock);
+
+ reg_val = INSTANCE_INFO_CODEC_STD(std);
+ if (cmd != W6_CMD_CREATE_INSTANCE)
+ reg_val |= INSTANCE_INFO_ID(id);
+ vpu_write_reg_relaxed(core, W6_CMD_INSTANCE_INFO, reg_val);
+
+ vpu_write_reg_relaxed(core, W6_VPU_BUSY_STATUS, BUSY_STATUS_SET);
+ vpu_write_reg_relaxed(core, W6_COMMAND, cmd);
+ vpu_write_reg(core, W6_VPU_HOST_INT_REQ, HOST_INT_REQ_ON);
+}
+
+static int wave6_send_query(struct vpu_core_device *core, u32 id, u32 std,
+ enum wave6_query_option query_opt)
+{
+ u32 reg_val;
+ int ret;
+
+ lockdep_assert_held(&core->hw_lock);
+
+ vpu_write_reg_relaxed(core, W6_QUERY_OPTION, query_opt);
+ wave6_send_command(core, id, std, W6_CMD_QUERY);
+
+ ret = wave6_wait_vpu_busy(core, W6_VPU_BUSY_STATUS);
+ if (ret) {
+ dev_err(core->dev, "query timed out opt=0x%x\n", query_opt);
+ return ret;
+ }
+
+ if (!vpu_read_reg_relaxed(core, W6_RET_SUCCESS)) {
+ reg_val = vpu_read_reg_relaxed(core, W6_RET_FAIL_REASON);
+ wave6_print_reg_err(core, reg_val);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+int wave6_vpu_get_version(struct vpu_core_device *core)
+{
+ struct vpu_attr *attr = &core->attr;
+ u32 std_def1, conf_feature;
+ int ret;
+
+ lockdep_assert_held(&core->hw_lock);
+
+ ret = wave6_send_query(core, 0, 0, W6_QUERY_OPT_GET_VPU_INFO);
+ if (ret)
+ return ret;
+
+ attr->product_id = wave6_vpu_get_product_id(core);
+ attr->product_code = vpu_read_reg_relaxed(core, W6_VPU_RET_PRODUCT_CODE);
+ attr->product_version = vpu_read_reg_relaxed(core, W6_RET_PRODUCT_VERSION);
+ attr->fw_version = vpu_read_reg_relaxed(core, W6_RET_FW_API_VERSION);
+ attr->fw_revision = vpu_read_reg_relaxed(core, W6_RET_FW_VERSION);
+ attr->hw_version = vpu_read_reg_relaxed(core, W6_RET_CONF_HW_VERSION);
+ std_def1 = vpu_read_reg_relaxed(core, W6_RET_STD_DEF1);
+ conf_feature = vpu_read_reg_relaxed(core, W6_RET_CONF_FEATURE);
+
+ attr->support_decoders = STD_DEF1_HEVC_DEC(std_def1) ? BIT(W_HEVC_DEC) : 0;
+ attr->support_decoders |= STD_DEF1_AVC_DEC(std_def1) ? BIT(W_AVC_DEC) : 0;
+ attr->support_encoders = STD_DEF1_HEVC_ENC(std_def1) ? BIT(W_HEVC_ENC) : 0;
+ attr->support_encoders |= STD_DEF1_AVC_ENC(std_def1) ? BIT(W_AVC_ENC) : 0;
+ attr->support_hevc10bit_dec = CONF_FEATURE_HEVC10BIT_DEC(conf_feature);
+ attr->support_avc10bit_dec = CONF_FEATURE_AVC10BIT_DEC(conf_feature);
+ attr->support_hevc10bit_enc = CONF_FEATURE_HEVC10BIT_ENC(conf_feature);
+ attr->support_avc10bit_enc = CONF_FEATURE_AVC10BIT_ENC(conf_feature);
+
+ return 0;
+}
diff --git a/drivers/media/platform/chips-media/wave6/wave6-hw.h b/drivers/media/platform/chips-media/wave6/wave6-hw.h
new file mode 100644
index 000000000000..bf8b6dcb54c2
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-hw.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
+/*
+ * Wave6 series multi-standard codec IP - wave6 backend interface
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#ifndef __WAVE6_HW_H__
+#define __WAVE6_HW_H__
+
+enum product_id {
+ PRODUCT_ID_617,
+ PRODUCT_ID_627,
+ PRODUCT_ID_637,
+ PRODUCT_ID_NONE,
+};
+
+bool wave6_vpu_is_init(struct vpu_core_device *core);
+int wave6_vpu_get_version(struct vpu_core_device *core);
+void wave6_vpu_enable_interrupt(struct vpu_core_device *core);
+
+#endif /* __WAVE6_HW_H__ */
diff --git a/drivers/media/platform/chips-media/wave6/wave6-regdefine.h b/drivers/media/platform/chips-media/wave6/wave6-regdefine.h
index e879a9743cb3..aa9b972c82d1 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-regdefine.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-regdefine.h
@@ -14,14 +14,51 @@ enum wave6_command {
W6_CMD_INIT_VPU = 0x0001,
W6_CMD_WAKEUP_VPU = 0x0002,
W6_CMD_SLEEP_VPU = 0x0004,
+ W6_CMD_CREATE_INSTANCE = 0x0008,
+ W6_CMD_FLUSH_INSTANCE = 0x0010,
+ W6_CMD_DESTROY_INSTANCE = 0x0020,
+ W6_CMD_INIT_SEQ = 0x0040,
+ W6_CMD_SET_FB = 0x0080,
+ W6_CMD_DEC_PIC = 0x0100,
+ W6_CMD_DEC_SET_DISP = 0x0400,
W6_CMD_INIT_WORK_BUF = 0x1000,
+ W6_CMD_QUERY = 0x4000,
+};
+
+enum wave6_query_option {
+ W6_QUERY_OPT_GET_VPU_INFO = 0,
+ W6_QUERY_OPT_GET_RESULT = 2,
+ W6_QUERY_OPT_GET_FLUSH_CMD_INFO = 10,
+};
+
+enum wave6_interrupt_bit {
+ W6_INT_BIT_INIT_VPU = 0,
+ W6_INT_BIT_WAKEUP_VPU = 1,
+ W6_INT_BIT_SLEEP_VPU = 2,
+ W6_INT_BIT_CREATE_INSTANCE = 3,
+ W6_INT_BIT_FLUSH_INSTANCE = 4,
+ W6_INT_BIT_DESTROY_INSTANCE = 5,
+ W6_INT_BIT_INIT_SEQ = 6,
+ W6_INT_BIT_SET_FB = 7,
+ W6_INT_BIT_DEC_PIC = 8,
+ W6_INT_BIT_SET_DISP = 10,
+ W6_INT_BIT_REQ_WORK_BUF = 12,
+ W6_INT_BIT_BSBUF_ERROR = 15,
};
#define W6_REG_BASE 0x00000000
+#define W6_CMD_REG_BASE 0x00000200
+#define W6_CMD_REG_END 0x00000600
#define W6_VPU_VCPU_CUR_PC (W6_REG_BASE + 0x0004)
+#define W6_VPU_VINT_REASON_CLEAR (W6_REG_BASE + 0x0034)
#define W6_VPU_HOST_INT_REQ (W6_REG_BASE + 0x0038)
#define HOST_INT_REQ_ON BIT(0)
+#define W6_VPU_VINT_CLEAR (W6_REG_BASE + 0x003C)
+#define VINT_CLEAR BIT(0)
+#define W6_VPU_VPU_INT_STS (W6_REG_BASE + 0x0044)
+#define W6_VPU_VINT_ENABLE (W6_REG_BASE + 0x0048)
+#define W6_VPU_VINT_REASON (W6_REG_BASE + 0x004C)
#define W6_VPU_REMAP_CTRL_GB (W6_REG_BASE + 0x0060)
#define REMAP_CTRL_ON BIT(31)
#define REMAP_CTRL_INDEX(x) FIELD_PREP(GENMASK(15, 12), (x))
@@ -38,6 +75,10 @@ enum wave6_command {
/* COMMON */
#define W6_COMMAND_GB (W6_REG_BASE + 0x104)
#define W6_COMMAND (W6_REG_BASE + 0x200)
+#define W6_QUERY_OPTION (W6_REG_BASE + 0x204)
+#define W6_CMD_INSTANCE_INFO (W6_REG_BASE + 0x210)
+#define INSTANCE_INFO_CODEC_STD(x) FIELD_PREP(GENMASK(31, 16), (x))
+#define INSTANCE_INFO_ID(x) FIELD_PREP(GENMASK(15, 0), (x))
#define W6_CMD_INIT_VPU_SEC_AXI_BASE_CORE0 (W6_REG_BASE + 0x364)
#define W6_CMD_INIT_VPU_SEC_AXI_SIZE_CORE0 (W6_REG_BASE + 0x368)
#define W6_CMD_SET_WORK_BUF_ADDR (W6_REG_BASE + 0x5F0)
@@ -45,5 +86,29 @@ enum wave6_command {
#define SET_WORK_BUF_SIZE_ACK 0
#define W6_RET_SUCCESS (W6_REG_BASE + 0x208)
#define W6_RET_FAIL_REASON (W6_REG_BASE + 0x20C)
+#define W6_RET_INT_INSTANCE_INFO (W6_REG_BASE + 0x21C)
+#define INT_INSTANCE_INFO_CLEAR 0
+#define W6_RET_INSTANCE_ID (W6_REG_BASE + 0x220)
+
+/* COMMON - QUERY : GET_VPU_INFO */
+#define W6_RET_FW_VERSION (W6_REG_BASE + 0x300)
+#define W6_RET_PRODUCT_NAME (W6_REG_BASE + 0x304)
+#define W6_RET_PRODUCT_VERSION (W6_REG_BASE + 0x308)
+#define W6_RET_STD_DEF0 (W6_REG_BASE + 0x30C)
+#define W6_RET_STD_DEF1 (W6_REG_BASE + 0x310)
+#define STD_DEF1_AVC_DEC(x) FIELD_GET(GENMASK(3, 3), (x))
+#define STD_DEF1_HEVC_DEC(x) FIELD_GET(GENMASK(2, 2), (x))
+#define STD_DEF1_AVC_ENC(x) FIELD_GET(GENMASK(1, 1), (x))
+#define STD_DEF1_HEVC_ENC(x) FIELD_GET(GENMASK(0, 0), (x))
+#define W6_RET_CONF_FEATURE (W6_REG_BASE + 0x314)
+#define CONF_FEATURE_AVC10BIT_ENC(x) FIELD_GET(GENMASK(11, 11), (x))
+#define CONF_FEATURE_AVC10BIT_DEC(x) FIELD_GET(GENMASK(9, 9), (x))
+#define CONF_FEATURE_HEVC10BIT_ENC(x) FIELD_GET(GENMASK(3, 3), (x))
+#define CONF_FEATURE_HEVC10BIT_DEC(x) FIELD_GET(GENMASK(1, 1), (x))
+#define W6_RET_CONF_DATE (W6_REG_BASE + 0x318)
+#define W6_RET_CONF_HW_VERSION (W6_REG_BASE + 0x31C)
+#define W6_RET_CONF_TYPE (W6_REG_BASE + 0x320)
+#define W6_RET_FW_API_VERSION (W6_REG_BASE + 0x32C)
+#define W6_RET_SHA_ID (W6_REG_BASE + 0x330)
#endif /* __WAVE6_REGDEFINE_H__ */
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vdi.h b/drivers/media/platform/chips-media/wave6/wave6-vdi.h
index 69ebf0ebcb13..67704fdbe27d 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vdi.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-vdi.h
@@ -13,6 +13,26 @@
#include <linux/dma-mapping.h>
#include <linux/io.h>
+enum endian_mode {
+ VDI_128BIT_BIG_ENDIAN = 0,
+ VDI_128BIT_BE_BYTE_SWAP,
+ VDI_128BIT_BE_WORD_SWAP,
+ VDI_128BIT_BE_WORD_BYTE_SWAP,
+ VDI_128BIT_BE_DWORD_SWAP,
+ VDI_128BIT_BE_DWORD_BYTE_SWAP,
+ VDI_128BIT_BE_DWORD_WORD_SWAP,
+ VDI_128BIT_BE_DWORD_WORD_BYTE_SWAP,
+ VDI_128BIT_LE_DWORD_WORD_BYTE_SWAP,
+ VDI_128BIT_LE_DWORD_WORD_SWAP,
+ VDI_128BIT_LE_DWORD_BYTE_SWAP,
+ VDI_128BIT_LE_DWORD_SWAP,
+ VDI_128BIT_LE_WORD_BYTE_SWAP,
+ VDI_128BIT_LE_WORD_SWAP,
+ VDI_128BIT_LE_BYTE_SWAP,
+ VDI_128BIT_LITTLE_ENDIAN = 15,
+ VDI_ENDIAN_MAX
+};
+
/**
* struct vpu_buf - VPU buffer for a coherent DMA buffer
* @size: Buffer size
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
new file mode 100644
index 000000000000..67c6666579d9
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
@@ -0,0 +1,392 @@
+// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause)
+/*
+ * Wave6 series multi-standard codec IP - wave6 core driver
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#include <linux/kernel.h>
+#include <linux/platform_device.h>
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/pm_runtime.h>
+#include <linux/iopoll.h>
+
+#include "wave6-vpu-core.h"
+#include "wave6-regdefine.h"
+#include "wave6-vpuconfig.h"
+#include "wave6-hw.h"
+
+static irqreturn_t wave6_vpu_core_irq(int irq, void *dev_id)
+{
+ struct vpu_core_device *core = dev_id;
+ struct vpu_irq irq_info;
+
+ if (!vpu_read_reg(core, W6_VPU_VPU_INT_STS))
+ return IRQ_NONE;
+
+ irq_info.status = vpu_read_reg(core, W6_VPU_VINT_REASON);
+ irq_info.inst_idc = vpu_read_reg(core, W6_RET_INT_INSTANCE_INFO);
+
+ vpu_write_reg(core, W6_RET_INT_INSTANCE_INFO, INT_INSTANCE_INFO_CLEAR);
+ vpu_write_reg(core, W6_VPU_VINT_REASON_CLEAR, irq_info.status);
+ vpu_write_reg(core, W6_VPU_VINT_CLEAR, VINT_CLEAR);
+
+ if (irq_info.status & BIT(W6_INT_BIT_REQ_WORK_BUF)) {
+ wave6_vpu_require_work_buffer(core->vpu, core->reg_base);
+
+ return IRQ_HANDLED;
+ }
+
+ if (kfifo_in(&core->irq_fifo, &irq_info, sizeof(struct vpu_irq)) !=
+ sizeof(struct vpu_irq))
+ dev_err(core->dev, "IRQ overflow status 0x%x idc 0x%x\n",
+ irq_info.status, irq_info.inst_idc);
+
+ return IRQ_WAKE_THREAD;
+}
+
+static struct vpu_instance *wave6_vpu_core_get_instance(struct vpu_core_device *core,
+ u32 inst_idc)
+{
+ struct vpu_instance *inst;
+
+ guard(spinlock)(&core->inst_lock);
+
+ list_for_each_entry(inst, &core->instances, list) {
+ if ((BIT(inst->id) & inst_idc) && inst->enable)
+ return inst;
+ }
+
+ return NULL;
+}
+
+void wave6_vpu_enable_instance(struct vpu_instance *inst)
+{
+ scoped_guard(spinlock, &inst->dev->inst_lock)
+ inst->enable = true;
+}
+
+void wave6_vpu_disable_instance(struct vpu_instance *inst)
+{
+ scoped_guard(spinlock, &inst->dev->inst_lock)
+ inst->enable = false;
+
+ synchronize_irq(inst->dev->irq);
+ cancel_delayed_work_sync(&inst->task_timer);
+}
+
+static irqreturn_t wave6_vpu_core_irq_thread(int irq, void *dev_id)
+{
+ struct vpu_core_device *core = dev_id;
+ struct vpu_instance *inst;
+ struct vpu_irq irq_info;
+
+ while (kfifo_len(&core->irq_fifo)) {
+ bool error = false;
+
+ if (!kfifo_out(&core->irq_fifo, &irq_info, sizeof(struct vpu_irq)))
+ break;
+
+ inst = wave6_vpu_core_get_instance(core, irq_info.inst_idc);
+ if (!inst)
+ continue;
+
+ if (irq_info.status & BIT(W6_INT_BIT_INIT_SEQ)) {
+ complete(&inst->irq_done);
+ continue;
+ }
+
+ if (irq_info.status & BIT(W6_INT_BIT_BSBUF_ERROR))
+ error = true;
+
+ if (inst->ops && inst->ops->finish_process)
+ inst->ops->finish_process(inst, error);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static int wave6_vpu_core_check_state(struct vpu_core_device *core)
+{
+ u32 val;
+ int ret;
+
+ guard(mutex)(&core->hw_lock);
+
+ ret = read_poll_timeout(vpu_read_reg, val, val != 0,
+ W6_VPU_POLL_DELAY_US, W6_VPU_POLL_TIMEOUT,
+ false, core, W6_VPU_VCPU_CUR_PC);
+ if (ret)
+ return ret;
+
+ wave6_vpu_enable_interrupt(core);
+ ret = wave6_vpu_get_version(core);
+ if (ret) {
+ dev_err(core->dev, "wave6_vpu_get_version fail\n");
+ return ret;
+ }
+
+ dev_dbg(core->dev, "product 0x%x, fw_ver %d.%d.%d(r%d), hw_ver 0x%x\n",
+ core->attr.product_code,
+ FW_VERSION_MAJOR(core->attr.fw_version),
+ FW_VERSION_MINOR(core->attr.fw_version),
+ FW_VERSION_REL(core->attr.fw_version),
+ core->attr.fw_revision,
+ core->attr.hw_version);
+
+ if (core->attr.fw_version < core->res->compatible_fw_version)
+ dev_err(core->dev, "fw version is too low (< v%d.%d.%d)\n",
+ FW_VERSION_MAJOR(core->res->compatible_fw_version),
+ FW_VERSION_MINOR(core->res->compatible_fw_version),
+ FW_VERSION_REL(core->res->compatible_fw_version));
+
+ return 0;
+}
+
+void wave6_vpu_core_activate(struct vpu_core_device *core)
+{
+ atomic_inc(&core->active_count);
+}
+
+void wave6_vpu_core_deactivate(struct vpu_core_device *core)
+{
+ atomic_dec(&core->active_count);
+}
+
+static int wave6_vpu_core_wait_activated(struct vpu_core_device *core)
+{
+ if (!atomic_read(&core->active_count))
+ return 0;
+
+ return wave6_vpu_core_check_state(core);
+}
+
+static void wave6_vpu_core_unregister_v4l2_dev(void *data)
+{
+ struct vpu_core_device *core = data;
+
+ v4l2_device_unregister(&core->v4l2_dev);
+}
+
+static void wave6_vpu_core_release_m2m_dev(void *data)
+{
+ wave6_vpu_release_m2m_dev(data);
+}
+
+static void wave6_vpu_core_free_temp_vbuf(void *data)
+{
+ struct vpu_core_device *core = data;
+
+ wave6_vdi_free_dma(&core->temp_vbuf);
+}
+
+static int wave6_vpu_core_probe(struct platform_device *pdev)
+{
+ struct vpu_core_device *core;
+ const struct wave6_vpu_core_resource *res;
+ int ret;
+
+ res = dev_get_platdata(&pdev->dev);
+ if (!res)
+ return -ENODEV;
+
+ ret = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
+ if (ret < 0) {
+ dev_err(&pdev->dev, "failed to set DMA mask: %d\n", ret);
+ return ret;
+ }
+
+ core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL);
+ if (!core)
+ return -ENOMEM;
+
+ ret = devm_mutex_init(&pdev->dev, &core->dev_lock);
+ if (ret)
+ return ret;
+
+ ret = devm_mutex_init(&pdev->dev, &core->hw_lock);
+ if (ret)
+ return ret;
+
+ spin_lock_init(&core->inst_lock);
+ INIT_LIST_HEAD(&core->instances);
+ dev_set_drvdata(&pdev->dev, core);
+ core->dev = &pdev->dev;
+ core->res = res;
+
+ core->vpu = dev_get_drvdata(pdev->dev.parent);
+
+ core->reg_base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(core->reg_base))
+ return PTR_ERR(core->reg_base);
+
+ ret = devm_clk_bulk_get_all(&pdev->dev, &core->clks);
+ if (ret < 0)
+ return dev_err_probe(&pdev->dev, ret, "failed to get clocks\n");
+
+ core->num_clks = ret;
+
+ ret = kfifo_init(&core->irq_fifo, core->irq_buffer,
+ sizeof(core->irq_buffer));
+ if (ret)
+ return ret;
+
+ core->irq = platform_get_irq(pdev, 0);
+ if (core->irq < 0)
+ return core->irq;
+
+ ret = devm_request_threaded_irq(&pdev->dev, core->irq,
+ wave6_vpu_core_irq,
+ wave6_vpu_core_irq_thread,
+ 0, "vpu_irq", core);
+ if (ret) {
+ dev_err(&pdev->dev, "failed to request IRQ: %d\n", ret);
+ return ret;
+ }
+
+ ret = v4l2_device_register(&pdev->dev, &core->v4l2_dev);
+ if (ret) {
+ dev_err(&pdev->dev, "failed to register v4l2_dev: %d\n", ret);
+ return ret;
+ }
+
+ ret = devm_add_action_or_reset(&pdev->dev, wave6_vpu_core_unregister_v4l2_dev, core);
+ if (ret)
+ return ret;
+
+ ret = wave6_vpu_init_m2m_dev(core);
+ if (ret)
+ return ret;
+
+ ret = devm_add_action_or_reset(&pdev->dev, wave6_vpu_core_release_m2m_dev, core);
+ if (ret)
+ return ret;
+
+ core->temp_vbuf.size = ALIGN(W6_TEMPBUF_SIZE, 4096);
+ ret = wave6_vdi_alloc_dma(core->dev, &core->temp_vbuf);
+ if (ret) {
+ dev_err(&pdev->dev, "failed to allocate temp_vbuf: %d\n", ret);
+ return ret;
+ }
+
+ ret = devm_add_action_or_reset(&pdev->dev, wave6_vpu_core_free_temp_vbuf, core);
+ if (ret)
+ return ret;
+
+ ret = devm_pm_runtime_enable(&pdev->dev);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int wave6_vpu_core_runtime_suspend(struct device *dev)
+{
+ struct vpu_core_device *core = dev_get_drvdata(dev);
+
+ if (WARN_ON(!core))
+ return -ENODEV;
+
+ if (core->vpu_held) {
+ wave6_vpu_put(core->vpu, core->reg_base);
+ core->vpu_held = false;
+ }
+
+ clk_bulk_disable_unprepare(core->num_clks, core->clks);
+
+ return 0;
+}
+
+static int wave6_vpu_core_runtime_resume(struct device *dev)
+{
+ struct vpu_core_device *core = dev_get_drvdata(dev);
+ int ret;
+
+ if (WARN_ON(!core))
+ return -ENODEV;
+
+ ret = clk_bulk_prepare_enable(core->num_clks, core->clks);
+ if (ret) {
+ dev_err(dev, "failed to enable clocks: %d\n", ret);
+ return ret;
+ }
+
+ if (atomic_read(&core->active_count)) {
+ ret = wave6_vpu_get(core->vpu, core->reg_base);
+ if (ret)
+ goto error_get_vpu;
+
+ core->vpu_held = true;
+ }
+
+ ret = wave6_vpu_core_wait_activated(core);
+ if (ret)
+ goto error_activate;
+
+ return 0;
+
+error_activate:
+ if (core->vpu_held) {
+ wave6_vpu_put(core->vpu, core->reg_base);
+ core->vpu_held = false;
+ }
+error_get_vpu:
+ clk_bulk_disable_unprepare(core->num_clks, core->clks);
+
+ return ret;
+}
+
+static int wave6_vpu_core_suspend(struct device *dev)
+{
+ struct vpu_core_device *core = dev_get_drvdata(dev);
+ int ret;
+
+ v4l2_m2m_suspend(core->m2m_dev);
+
+ ret = pm_runtime_force_suspend(dev);
+ if (ret)
+ v4l2_m2m_resume(core->m2m_dev);
+
+ return ret;
+}
+
+static int wave6_vpu_core_resume(struct device *dev)
+{
+ struct vpu_core_device *core = dev_get_drvdata(dev);
+ int ret;
+
+ ret = pm_runtime_force_resume(dev);
+ if (ret)
+ return ret;
+
+ v4l2_m2m_resume(core->m2m_dev);
+
+ return 0;
+}
+
+static const struct dev_pm_ops wave6_vpu_core_pm_ops = {
+ RUNTIME_PM_OPS(wave6_vpu_core_runtime_suspend,
+ wave6_vpu_core_runtime_resume, NULL)
+ SYSTEM_SLEEP_PM_OPS(wave6_vpu_core_suspend,
+ wave6_vpu_core_resume)
+};
+
+static struct platform_driver wave6_vpu_core_driver = {
+ .driver = {
+ .name = WAVE6_VPU_CORE_PLATFORM_DRIVER_NAME,
+ .pm = pm_ptr(&wave6_vpu_core_pm_ops),
+ .suppress_bind_attrs = true,
+ },
+ .probe = wave6_vpu_core_probe,
+};
+
+int __init wave6_vpu_core_register_driver(void)
+{
+ return platform_driver_register(&wave6_vpu_core_driver);
+}
+
+void wave6_vpu_core_unregister_driver(void)
+{
+ platform_driver_unregister(&wave6_vpu_core_driver);
+}
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.h b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.h
new file mode 100644
index 000000000000..788b1403c1d3
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
+/*
+ * Wave6 series multi-standard codec IP - wave6 core driver
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#ifndef __WAVE6_VPU_CORE_H__
+#define __WAVE6_VPU_CORE_H__
+
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fh.h>
+#include <media/videobuf2-v4l2.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "wave6-vpuapi.h"
+
+#define vpu_write_reg(CORE, ADDR, DATA) wave6_vpu_writel(CORE, ADDR, DATA)
+#define vpu_write_reg_relaxed(CORE, ADDR, DATA) wave6_vpu_writel_relaxed(CORE, ADDR, DATA)
+#define vpu_read_reg(CORE, ADDR) wave6_vpu_readl(CORE, ADDR)
+#define vpu_read_reg_relaxed(CORE, ADDR) wave6_vpu_readl_relaxed(CORE, ADDR)
+
+struct vpu_buffer {
+ struct v4l2_m2m_buffer v4l2_m2m_buf;
+ bool consumed;
+ bool used;
+ bool error;
+};
+
+enum vpu_fmt_type {
+ VPU_FMT_TYPE_CODEC = 0,
+ VPU_FMT_TYPE_RAW = 1
+};
+
+#define VPU_FMT_FLAG_CBCR_INTERLEAVED BIT(0)
+#define VPU_FMT_FLAG_CRCB_ORDER BIT(1)
+
+struct vpu_format {
+ unsigned int v4l2_pix_fmt;
+ unsigned int max_width;
+ unsigned int min_width;
+ unsigned int max_height;
+ unsigned int min_height;
+ unsigned int num_planes;
+ enum frame_buffer_format fb_fmt;
+ enum endian_mode endian;
+ unsigned int flags;
+};
+
+static inline struct vpu_instance *wave6_fh_to_vpu_inst(struct v4l2_fh *vfh)
+{
+ return container_of(vfh, struct vpu_instance, v4l2_fh);
+}
+
+static inline struct vpu_instance *wave6_file_to_vpu_inst(struct file *filp)
+{
+ return wave6_fh_to_vpu_inst(file_to_v4l2_fh(filp));
+}
+
+static inline struct vpu_instance *wave6_ctrl_to_vpu_inst(struct v4l2_ctrl *vctrl)
+{
+ return container_of(vctrl->handler, struct vpu_instance, v4l2_ctrl_hdl);
+}
+
+static inline struct vpu_buffer *wave6_to_vpu_buf(struct vb2_v4l2_buffer *vbuf)
+{
+ return container_of(vbuf, struct vpu_buffer, v4l2_m2m_buf.vb);
+}
+
+static inline bool wave6_vpu_both_queues_are_streaming(struct vpu_instance *inst)
+{
+ struct vb2_queue *vq_cap = v4l2_m2m_get_dst_vq(inst->v4l2_fh.m2m_ctx);
+ struct vb2_queue *vq_out = v4l2_m2m_get_src_vq(inst->v4l2_fh.m2m_ctx);
+
+ return vb2_is_streaming(vq_cap) && vb2_is_streaming(vq_out);
+}
+
+u32 wave6_vpu_get_consumed_fb_num(struct vpu_instance *inst);
+void wave6_vpu_core_activate(struct vpu_core_device *core);
+void wave6_vpu_core_deactivate(struct vpu_core_device *core);
+void wave6_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,
+ unsigned int width,
+ unsigned int height);
+struct vb2_v4l2_buffer *wave6_get_dst_buf_by_addr(struct vpu_instance *inst,
+ dma_addr_t addr);
+dma_addr_t wave6_get_dma_addr(struct vb2_v4l2_buffer *buf,
+ unsigned int plane_no);
+enum codec_std wave6_to_codec_std(enum vpu_instance_type type, unsigned int v4l2_pix_fmt);
+const char *wave6_vpu_instance_state_name(enum vpu_instance_state state);
+void wave6_vpu_set_instance_state(struct vpu_instance *inst,
+ enum vpu_instance_state state);
+int wave6_vpu_wait_interrupt(struct vpu_instance *inst, unsigned int timeout);
+void wave6_vpu_device_run_timeout(struct work_struct *work);
+void wave6_vpu_finish_job(struct vpu_instance *inst);
+int wave6_vpu_init_m2m_dev(struct vpu_core_device *core);
+void wave6_vpu_release_m2m_dev(struct vpu_core_device *core);
+int wave6_vpu_subscribe_event(struct v4l2_fh *fh,
+ const struct v4l2_event_subscription *sub);
+void wave6_vpu_return_buffers(struct vpu_instance *inst,
+ unsigned int type, enum vb2_buffer_state state);
+void wave6_vpu_enable_instance(struct vpu_instance *inst);
+void wave6_vpu_disable_instance(struct vpu_instance *inst);
+
+#endif /* __WAVE6_VPU_CORE_H__ */
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
new file mode 100644
index 000000000000..75371566de38
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
@@ -0,0 +1,384 @@
+// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause)
+/*
+ * Wave6 series multi-standard codec IP - v4l2 driver helper interface
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#include "wave6-vpu.h"
+#include "wave6-vpu-core.h"
+
+void wave6_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,
+ unsigned int width,
+ unsigned int height)
+{
+ const struct v4l2_format_info *fmt_info;
+ unsigned int stride_y;
+
+ pix_mp->width = width;
+ pix_mp->height = height;
+ pix_mp->flags = 0;
+ pix_mp->field = V4L2_FIELD_NONE;
+ memset(pix_mp->reserved, 0, sizeof(pix_mp->reserved));
+
+ fmt_info = v4l2_format_info(pix_mp->pixelformat);
+ if (!fmt_info) {
+ pix_mp->plane_fmt[0].bytesperline = 0;
+ if (!pix_mp->plane_fmt[0].sizeimage)
+ pix_mp->plane_fmt[0].sizeimage = width * height;
+
+ return;
+ }
+
+ stride_y = width * fmt_info->bpp[0];
+ if (pix_mp->plane_fmt[0].bytesperline <= W6_MAX_PIC_STRIDE)
+ stride_y = max(stride_y, pix_mp->plane_fmt[0].bytesperline);
+ stride_y = round_up(stride_y, W6_PIC_STRIDE_ALIGNMENT);
+ pix_mp->plane_fmt[0].bytesperline = stride_y;
+ pix_mp->plane_fmt[0].sizeimage = stride_y * height;
+
+ stride_y = DIV_ROUND_UP(stride_y, fmt_info->bpp[0]);
+
+ for (int i = 1; i < fmt_info->comp_planes; i++) {
+ unsigned int stride_c, sizeimage_c;
+
+ stride_c = DIV_ROUND_UP(stride_y, fmt_info->hdiv) *
+ fmt_info->bpp[i];
+ sizeimage_c = stride_c * DIV_ROUND_UP(height, fmt_info->vdiv);
+
+ if (fmt_info->mem_planes == 1) {
+ pix_mp->plane_fmt[0].sizeimage += sizeimage_c;
+ } else {
+ pix_mp->plane_fmt[i].bytesperline = stride_c;
+ pix_mp->plane_fmt[i].sizeimage = sizeimage_c;
+ }
+ }
+}
+
+dma_addr_t wave6_get_dma_addr(struct vb2_v4l2_buffer *buf, unsigned int plane_no)
+{
+ return vb2_dma_contig_plane_dma_addr(&buf->vb2_buf, plane_no) +
+ buf->planes[plane_no].data_offset;
+}
+
+struct vb2_v4l2_buffer *wave6_get_dst_buf_by_addr(struct vpu_instance *inst,
+ dma_addr_t addr)
+{
+ struct vb2_v4l2_buffer *vb2_v4l2_buf;
+ struct v4l2_m2m_buffer *v4l2_m2m_buf;
+ struct vb2_v4l2_buffer *dst_buf = NULL;
+
+ v4l2_m2m_for_each_dst_buf(inst->v4l2_fh.m2m_ctx, v4l2_m2m_buf) {
+ vb2_v4l2_buf = &v4l2_m2m_buf->vb;
+ if (addr == wave6_get_dma_addr(vb2_v4l2_buf, 0)) {
+ dst_buf = vb2_v4l2_buf;
+ break;
+ }
+ }
+
+ return dst_buf;
+}
+
+enum codec_std wave6_to_codec_std(enum vpu_instance_type type, unsigned int v4l2_pix_fmt)
+{
+ enum codec_std std = STD_UNKNOWN;
+
+ if (v4l2_pix_fmt == V4L2_PIX_FMT_H264)
+ std = (type == VPU_INST_TYPE_DEC) ? W_AVC_DEC : W_AVC_ENC;
+ else if (v4l2_pix_fmt == V4L2_PIX_FMT_HEVC)
+ std = (type == VPU_INST_TYPE_DEC) ? W_HEVC_DEC : W_HEVC_ENC;
+
+ return std;
+}
+
+const char *wave6_vpu_instance_state_name(enum vpu_instance_state state)
+{
+ switch (state) {
+ case VPU_INST_STATE_NONE:
+ return "none";
+ case VPU_INST_STATE_OPEN:
+ return "open";
+ case VPU_INST_STATE_INIT_SEQ:
+ return "init_seq";
+ case VPU_INST_STATE_PIC_RUN:
+ return "pic_run";
+ case VPU_INST_STATE_SEEK:
+ return "seek";
+ case VPU_INST_STATE_ERROR:
+ return "error";
+ default:
+ return "unknown";
+ }
+}
+
+static bool wave6_vpu_valid_instance_transition(struct vpu_instance *inst,
+ enum vpu_instance_state next)
+{
+ bool is_decoder = (inst->type == VPU_INST_TYPE_DEC);
+
+ switch (inst->state) {
+ case VPU_INST_STATE_NONE:
+ /* to OPEN: instance created */
+ if (next == VPU_INST_STATE_OPEN)
+ return true;
+ break;
+ case VPU_INST_STATE_OPEN:
+ if (is_decoder) {
+ /* to NONE: instance destroyed */
+ /* to INIT_SEQ: sequence header parsed */
+ /* to SEEK: output stop_streaming called */
+ if (next == VPU_INST_STATE_NONE ||
+ next == VPU_INST_STATE_INIT_SEQ ||
+ next == VPU_INST_STATE_SEEK)
+ return true;
+ }
+ break;
+ case VPU_INST_STATE_INIT_SEQ:
+ if (is_decoder) {
+ /* to NONE: instance destroyed */
+ /* to PIC_RUN: frame buffers registered */
+ /* to SEEK: output stop_streaming called */
+ if (next == VPU_INST_STATE_NONE ||
+ next == VPU_INST_STATE_PIC_RUN ||
+ next == VPU_INST_STATE_SEEK)
+ return true;
+ }
+ break;
+ case VPU_INST_STATE_PIC_RUN:
+ if (is_decoder) {
+ /* to NONE: instance destroyed */
+ /* to INIT_SEQ: new sequence header parsed */
+ /* to SEEK: output stop_streaming called */
+ /* to ERROR: decode command failed */
+ if (next == VPU_INST_STATE_NONE ||
+ next == VPU_INST_STATE_INIT_SEQ ||
+ next == VPU_INST_STATE_SEEK ||
+ next == VPU_INST_STATE_ERROR)
+ return true;
+ }
+ break;
+ case VPU_INST_STATE_SEEK:
+ if (is_decoder) {
+ /* to NONE: instance destroyed */
+ /* to OPEN/INIT_SEQ/PIC_RUN/ERROR: */
+ /* previous state restored on output start_streaming */
+ if (next == VPU_INST_STATE_NONE ||
+ next == VPU_INST_STATE_OPEN ||
+ next == VPU_INST_STATE_INIT_SEQ ||
+ next == VPU_INST_STATE_PIC_RUN ||
+ next == VPU_INST_STATE_ERROR)
+ return true;
+ }
+ break;
+ case VPU_INST_STATE_ERROR:
+ if (is_decoder) {
+ /* to NONE: instance destroyed */
+ /* to SEEK: output stop_streaming called */
+ if (next == VPU_INST_STATE_NONE ||
+ next == VPU_INST_STATE_SEEK)
+ return true;
+ }
+ break;
+ }
+
+ dev_err(inst->dev->dev, "[%d] invalid transition: %s -> %s\n",
+ inst->id,
+ wave6_vpu_instance_state_name(inst->state),
+ wave6_vpu_instance_state_name(next));
+
+ return false;
+}
+
+void wave6_vpu_set_instance_state(struct vpu_instance *inst,
+ enum vpu_instance_state state)
+{
+ if (!wave6_vpu_valid_instance_transition(inst, state))
+ return;
+
+ dev_dbg(inst->dev->dev, "[%d] %s -> %s\n",
+ inst->id,
+ wave6_vpu_instance_state_name(inst->state),
+ wave6_vpu_instance_state_name(state));
+
+ inst->state = state;
+}
+
+int wave6_vpu_wait_interrupt(struct vpu_instance *inst, unsigned int timeout)
+{
+ int ret;
+
+ ret = wait_for_completion_timeout(&inst->irq_done,
+ msecs_to_jiffies(timeout));
+ if (!ret)
+ return -ETIMEDOUT;
+
+ reinit_completion(&inst->irq_done);
+
+ return 0;
+}
+
+int wave6_vpu_subscribe_event(struct v4l2_fh *fh,
+ const struct v4l2_event_subscription *sub)
+{
+ struct vpu_instance *inst = wave6_fh_to_vpu_inst(fh);
+ bool is_decoder = inst->type == VPU_INST_TYPE_DEC;
+
+ dev_dbg(inst->dev->dev, "%s: [%s] type: %d id: %d | flags: %d\n",
+ __func__, is_decoder ? "decoder" : "encoder", sub->type,
+ sub->id, sub->flags);
+
+ switch (sub->type) {
+ case V4L2_EVENT_SOURCE_CHANGE:
+ if (is_decoder)
+ return v4l2_src_change_event_subscribe(fh, sub);
+ return -EINVAL;
+ case V4L2_EVENT_CTRL:
+ return v4l2_ctrl_subscribe_event(fh, sub);
+ default:
+ return -EINVAL;
+ }
+}
+
+void wave6_vpu_return_buffers(struct vpu_instance *inst,
+ unsigned int type, enum vb2_buffer_state state)
+{
+ struct vb2_v4l2_buffer *buf;
+
+ if (V4L2_TYPE_IS_OUTPUT(type)) {
+ while ((buf = v4l2_m2m_src_buf_remove(inst->v4l2_fh.m2m_ctx)))
+ v4l2_m2m_buf_done(buf, state);
+ } else {
+ while ((buf = v4l2_m2m_dst_buf_remove(inst->v4l2_fh.m2m_ctx))) {
+ for (int i = 0; i < inst->dst_fmt.num_planes; i++)
+ vb2_set_plane_payload(&buf->vb2_buf, i, 0);
+ v4l2_m2m_buf_done(buf, state);
+ }
+ }
+}
+
+u32 wave6_vpu_get_consumed_fb_num(struct vpu_instance *inst)
+{
+ struct vb2_v4l2_buffer *vb2_v4l2_buf;
+ struct v4l2_m2m_buffer *v4l2_m2m_buf;
+ struct vpu_buffer *vpu_buf;
+ u32 num = 0;
+
+ v4l2_m2m_for_each_dst_buf(inst->v4l2_fh.m2m_ctx, v4l2_m2m_buf) {
+ vb2_v4l2_buf = &v4l2_m2m_buf->vb;
+ vpu_buf = wave6_to_vpu_buf(vb2_v4l2_buf);
+ if (vpu_buf->consumed)
+ num++;
+ }
+
+ return num;
+}
+
+static bool wave6_vpu_check_fb_available(struct vpu_instance *inst)
+{
+ struct vb2_v4l2_buffer *vb2_v4l2_buf;
+ struct v4l2_m2m_buffer *v4l2_m2m_buf;
+ struct vpu_buffer *vpu_buf;
+
+ v4l2_m2m_for_each_dst_buf(inst->v4l2_fh.m2m_ctx, v4l2_m2m_buf) {
+ vb2_v4l2_buf = &v4l2_m2m_buf->vb;
+ vpu_buf = wave6_to_vpu_buf(vb2_v4l2_buf);
+
+ if (!vpu_buf->used)
+ return true;
+ }
+
+ return false;
+}
+
+static int wave6_vpu_job_ready(void *priv)
+{
+ struct vpu_instance *inst = priv;
+
+ dev_dbg(inst->dev->dev, "[%d]%s: state %d\n",
+ inst->id, __func__, inst->state);
+
+ if (inst->state == VPU_INST_STATE_OPEN)
+ return 1;
+ if (inst->state < VPU_INST_STATE_PIC_RUN)
+ return 0;
+ if (inst->state == VPU_INST_STATE_ERROR)
+ return 0;
+ if (!wave6_vpu_check_fb_available(inst))
+ return 0;
+
+ return 1;
+}
+
+void wave6_vpu_device_run_timeout(struct work_struct *work)
+{
+ struct vpu_instance *inst = container_of(to_delayed_work(work),
+ struct vpu_instance, task_timer);
+ struct vb2_v4l2_buffer *src_buf = NULL;
+ struct vb2_v4l2_buffer *dst_buf = NULL;
+
+ dev_err(inst->dev->dev, "[%d] sequence %d timeout\n", inst->id, inst->sequence);
+ src_buf = v4l2_m2m_src_buf_remove(inst->v4l2_fh.m2m_ctx);
+ if (src_buf) {
+ v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
+ if (inst->type == VPU_INST_TYPE_DEC)
+ inst->processed_buf_num++;
+ inst->error_buf_num++;
+ }
+
+ dst_buf = v4l2_m2m_dst_buf_remove(inst->v4l2_fh.m2m_ctx);
+ if (dst_buf)
+ v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
+
+ vb2_queue_error(v4l2_m2m_get_src_vq(inst->v4l2_fh.m2m_ctx));
+ vb2_queue_error(v4l2_m2m_get_dst_vq(inst->v4l2_fh.m2m_ctx));
+
+ v4l2_m2m_job_finish(inst->dev->m2m_dev, inst->v4l2_fh.m2m_ctx);
+}
+
+static void wave6_vpu_device_run(void *priv)
+{
+ struct vpu_instance *inst = priv;
+ int ret;
+
+ dev_dbg(inst->dev->dev, "[%d]%s: state %d\n",
+ inst->id, __func__, inst->state);
+
+ ret = inst->ops->prepare_process(inst);
+ if (ret) {
+ v4l2_m2m_job_finish(inst->dev->m2m_dev, inst->v4l2_fh.m2m_ctx);
+ return;
+ }
+
+ schedule_delayed_work(&inst->task_timer, msecs_to_jiffies(W6_VPU_TIMEOUT));
+
+ ret = inst->ops->start_process(inst);
+ if (ret)
+ wave6_vpu_finish_job(inst);
+}
+
+void wave6_vpu_finish_job(struct vpu_instance *inst)
+{
+ cancel_delayed_work(&inst->task_timer);
+ v4l2_m2m_job_finish(inst->dev->m2m_dev, inst->v4l2_fh.m2m_ctx);
+}
+
+static const struct v4l2_m2m_ops wave6_vpu_m2m_ops = {
+ .device_run = wave6_vpu_device_run,
+ .job_ready = wave6_vpu_job_ready,
+};
+
+int wave6_vpu_init_m2m_dev(struct vpu_core_device *core)
+{
+ core->m2m_dev = v4l2_m2m_init(&wave6_vpu_m2m_ops);
+ if (IS_ERR(core->m2m_dev)) {
+ dev_err(core->dev, "v4l2_m2m_init fail\n");
+ return PTR_ERR(core->m2m_dev);
+ }
+
+ return 0;
+}
+
+void wave6_vpu_release_m2m_dev(struct vpu_core_device *core)
+{
+ v4l2_m2m_release(core->m2m_dev);
+}
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu.c b/drivers/media/platform/chips-media/wave6/wave6-vpu.c
index 9160ce53062c..54a6567bc64a 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu.c
@@ -773,7 +773,29 @@ static struct platform_driver wave6_vpu_driver = {
.remove = wave6_vpu_remove,
};
-module_platform_driver(wave6_vpu_driver);
+static int __init wave6_vpu_init(void)
+{
+ int ret;
+
+ ret = wave6_vpu_core_register_driver();
+ if (ret)
+ return ret;
+
+ ret = platform_driver_register(&wave6_vpu_driver);
+ if (ret)
+ wave6_vpu_core_unregister_driver();
+
+ return ret;
+}
+module_init(wave6_vpu_init);
+
+static void __exit wave6_vpu_exit(void)
+{
+ platform_driver_unregister(&wave6_vpu_driver);
+ wave6_vpu_core_unregister_driver();
+}
+module_exit(wave6_vpu_exit);
+
MODULE_DESCRIPTION("chips&media Wave6 VPU driver");
MODULE_AUTHOR("CHIPS&MEDIA INC");
MODULE_FIRMWARE("cnm/wave633c_imx9_codec_fw.bin");
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu.h b/drivers/media/platform/chips-media/wave6/wave6-vpu.h
index 96d67e931e71..35a1d9f8a003 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu.h
@@ -133,5 +133,7 @@ int wave6_vpu_get(struct wave6_vpu_device *vpu, void __iomem *core_base);
void wave6_vpu_put(struct wave6_vpu_device *vpu, void __iomem *core_base);
void wave6_vpu_require_work_buffer(struct wave6_vpu_device *vpu,
void __iomem *core_base);
+int wave6_vpu_core_register_driver(void);
+void wave6_vpu_core_unregister_driver(void);
#endif /* __WAVE6_VPU_H__ */
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
new file mode 100644
index 000000000000..62dd4969afaf
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
@@ -0,0 +1,381 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
+/*
+ * Wave6 series multi-standard codec IP - wave6 helper interface
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#ifndef __WAVE6_VPUAPI_H__
+#define __WAVE6_VPUAPI_H__
+
+#include <linux/bitfield.h>
+#include <linux/kfifo.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/v4l2-ctrls.h>
+
+#include "wave6-vpuconfig.h"
+#include "wave6-vdi.h"
+#include "wave6-vpu.h"
+
+struct vpu_attr;
+
+enum vpu_instance_type {
+ VPU_INST_TYPE_DEC,
+ VPU_INST_TYPE_ENC
+};
+
+/**
+ * enum vpu_instance_state - VPU instance states
+ * @VPU_INST_STATE_NONE: Instance is not created or has been destroyed
+ * @VPU_INST_STATE_OPEN: Instance is created
+ * @VPU_INST_STATE_INIT_SEQ: For decoder, sequence header is parsed
+ * For encoder, parameter configuration is complete
+ * @VPU_INST_STATE_PIC_RUN: Instance is decoding or encoding
+ * @VPU_INST_STATE_SEEK: Ready to seek a decoding resume point
+ * Valid only for decoder
+ * @VPU_INST_STATE_ERROR: A command failed, no more jobs are run
+ */
+enum vpu_instance_state {
+ VPU_INST_STATE_NONE,
+ VPU_INST_STATE_OPEN,
+ VPU_INST_STATE_INIT_SEQ,
+ VPU_INST_STATE_PIC_RUN,
+ VPU_INST_STATE_SEEK,
+ VPU_INST_STATE_ERROR
+};
+
+#define WAVE6_MAX_FBS 31
+
+enum codec_std {
+ W_HEVC_DEC = 0x00,
+ W_HEVC_ENC = 0x01,
+ W_AVC_DEC = 0x02,
+ W_AVC_ENC = 0x03,
+ STD_UNKNOWN = 0xFF
+};
+
+#define FW_VERSION_MAJOR_MASK 0xFF000000
+#define FW_VERSION_MINOR_MASK 0x00FF0000
+#define FW_VERSION_REL_MASK 0x0000FFFF
+#define FW_VERSION_MAJOR(x) FIELD_GET(FW_VERSION_MAJOR_MASK, (x))
+#define FW_VERSION_MINOR(x) FIELD_GET(FW_VERSION_MINOR_MASK, (x))
+#define FW_VERSION_REL(x) FIELD_GET(FW_VERSION_REL_MASK, (x))
+
+enum chroma_format_idc {
+ C_FMT_IDC_YUV400,
+ C_FMT_IDC_YUV420,
+ C_FMT_IDC_YUV422,
+ C_FMT_IDC_YUV444,
+ C_FMT_IDC_RGB
+};
+
+enum frame_buffer_format {
+ FORMAT_ERR = -1,
+
+ FORMAT_420 = 0,
+ FORMAT_422,
+ FORMAT_224,
+ FORMAT_444,
+ FORMAT_400,
+
+ FORMAT_420_P10_16BIT_MSB = 5,
+ FORMAT_420_P10_16BIT_LSB,
+ FORMAT_420_P10_32BIT_MSB,
+ FORMAT_420_P10_32BIT_LSB,
+
+ FORMAT_422_P10_16BIT_MSB,
+ FORMAT_422_P10_16BIT_LSB,
+ FORMAT_422_P10_32BIT_MSB,
+ FORMAT_422_P10_32BIT_LSB,
+
+ FORMAT_444_P10_16BIT_MSB,
+ FORMAT_444_P10_16BIT_LSB,
+ FORMAT_444_P10_32BIT_MSB,
+ FORMAT_444_P10_32BIT_LSB,
+
+ FORMAT_400_P10_16BIT_MSB,
+ FORMAT_400_P10_16BIT_LSB,
+ FORMAT_400_P10_32BIT_MSB,
+ FORMAT_400_P10_32BIT_LSB,
+
+ FORMAT_YUYV,
+ FORMAT_YUYV_P10_16BIT_MSB,
+ FORMAT_YUYV_P10_16BIT_LSB,
+ FORMAT_YUYV_P10_32BIT_MSB,
+ FORMAT_YUYV_P10_32BIT_LSB,
+
+ FORMAT_YVYU,
+ FORMAT_YVYU_P10_16BIT_MSB,
+ FORMAT_YVYU_P10_16BIT_LSB,
+ FORMAT_YVYU_P10_32BIT_MSB,
+ FORMAT_YVYU_P10_32BIT_LSB,
+
+ FORMAT_UYVY,
+ FORMAT_UYVY_P10_16BIT_MSB,
+ FORMAT_UYVY_P10_16BIT_LSB,
+ FORMAT_UYVY_P10_32BIT_MSB,
+ FORMAT_UYVY_P10_32BIT_LSB,
+
+ FORMAT_VYUY,
+ FORMAT_VYUY_P10_16BIT_MSB,
+ FORMAT_VYUY_P10_16BIT_LSB,
+ FORMAT_VYUY_P10_32BIT_MSB,
+ FORMAT_VYUY_P10_32BIT_LSB,
+
+ FORMAT_RGB_32BIT_PACKED = 90,
+ FORMAT_YUV444_32BIT_PACKED,
+ FORMAT_RGB_P10_32BIT_PACKED,
+ FORMAT_YUV444_P10_32BIT_PACKED,
+
+ FORMAT_RGB_24BIT_PACKED = 95,
+ FORMAT_YUV444_24BIT_PACKED,
+ FORMAT_YUV444_24BIT,
+
+ FORMAT_MAX,
+};
+
+enum pic_type {
+ PIC_TYPE_I = 0,
+ PIC_TYPE_P = 1,
+ PIC_TYPE_B = 2,
+ PIC_TYPE_IDR = 5,
+ PIC_TYPE_MAX
+};
+
+enum display_mode {
+ DISP_MODE_DISP_ORDER,
+ DISP_MODE_DEC_ORDER,
+};
+
+enum sw_reset_mode {
+ SW_RESET_SAFETY,
+ SW_RESET_FORCE,
+ SW_RESET_ON_BOOT
+};
+
+enum tiled_map_type {
+ LINEAR_FRAME_MAP = 0,
+ COMPRESSED_FRAME_MAP = 17,
+};
+
+enum aux_buffer_type {
+ AUX_BUF_FBC_Y_TBL,
+ AUX_BUF_FBC_C_TBL,
+ AUX_BUF_MV_COL,
+ AUX_BUF_SUB_SAMPLE,
+ AUX_BUF_TYPE_MAX,
+};
+
+struct vpu_attr {
+ u32 product_id;
+ u32 product_code;
+ u32 product_version;
+ u32 fw_version;
+ u32 fw_revision;
+ u32 hw_version;
+ u32 support_decoders;
+ u32 support_encoders;
+ bool support_avc10bit_dec;
+ bool support_hevc10bit_dec;
+ bool support_avc10bit_enc;
+ bool support_hevc10bit_enc;
+ bool support_dual_core;
+};
+
+struct frame_buffer {
+ dma_addr_t buf_y;
+ dma_addr_t buf_cb;
+ dma_addr_t buf_cr;
+ enum tiled_map_type map_type;
+ unsigned int stride;
+ unsigned int stride_c;
+ unsigned int height;
+ u32 luma_bit_depth: 4;
+ u32 chroma_bit_depth: 4;
+ enum chroma_format_idc c_fmt_idc;
+};
+
+struct vpu_rect {
+ u32 left;
+ u32 top;
+ u32 right;
+ u32 bottom;
+};
+
+struct instance_buffer {
+ dma_addr_t temp_base;
+ u32 temp_size;
+};
+
+struct dec_scaler_info {
+ bool enable;
+ int width;
+ int height;
+};
+
+/**
+ * struct vpu_irq - VPU interrupt information
+ * @status: Value of the VPU interrupt status
+ * @inst_idc: Bitmask of instance IDs for this interrupt
+ */
+struct vpu_irq {
+ u32 status;
+ u32 inst_idc;
+};
+
+/**
+ * struct vpu_core_device - VPU core driver structure
+ * @dev: Platform device pointer
+ * @vpu: Parent VPU driver structure
+ * @v4l2_dev: V4L2 device
+ * @m2m_dev: V4L2 mem2mem device
+ * @instances: List of VPU instances
+ * @inst_lock: Spinlock protecting instance list
+ * @dev_lock: Mutex protecting video_device, src, dst vb2_queue
+ * @hw_lock: Mutex protecting register access
+ * @attr: Hardware attributes retrieved after boot
+ * @reg_base: Base address of MMIO registers
+ * @temp_vbuf: TEMP buffer
+ * @clks: Array of clock handles
+ * @num_clks: Number of entries in @clks
+ * @irq_fifo: kfifo storing interrupt information
+ * @irq_buffer: Buffer used by @irq_fifo
+ * @irq: Interrupt number
+ * @res: Device compatible data
+ * @active_count: Number of decoder/encoder instances using the core
+ * @vpu_held: Core holds a reference taken by wave6_vpu_get()
+ */
+struct vpu_core_device {
+ struct device *dev;
+ struct wave6_vpu_device *vpu;
+ struct v4l2_device v4l2_dev;
+ struct v4l2_m2m_dev *m2m_dev;
+ struct list_head instances;
+ spinlock_t inst_lock; /* Protects instance list */
+ struct mutex dev_lock; /* Protects video_device, src, dst vb2_queue */
+ struct mutex hw_lock; /* Protects register access */
+ struct vpu_attr attr;
+ void __iomem *reg_base;
+
+ /* Allocates per-core, used during decode/encode process. */
+ struct vpu_buf temp_vbuf;
+
+ struct clk_bulk_data *clks;
+ int num_clks;
+
+ /* Stores interrupt information quickly, consumed in irq_thread. */
+ struct kfifo irq_fifo;
+ struct vpu_irq irq_buffer[MAX_NUM_INSTANCE];
+ int irq;
+
+ const struct wave6_vpu_core_resource *res;
+ atomic_t active_count;
+ bool vpu_held;
+};
+
+struct vpu_instance;
+
+struct vpu_instance_ops {
+ int (*prepare_process)(struct vpu_instance *inst);
+ int (*start_process)(struct vpu_instance *inst);
+ void (*finish_process)(struct vpu_instance *inst, bool error);
+};
+
+/**
+ * struct vpu_instance - VPU instance structure
+ * @list: List head for VPU core device instance list
+ * @v4l2_fh: V4L2 file handler
+ * @v4l2_ctrl_hdl: V4L2 control handler
+ * @dev: VPU core driver structure
+ * @irq_done: Completion for command finish
+ * @task_timer: Watchdog of the running decode/encode job
+ * @lock: Protects the drain state
+ * @enable: Instance enable flag
+ * @src_fmt: V4L2 pixel format for source
+ * @dst_fmt: V4L2 pixel format for destination
+ * @crop: Crop rectangle
+ * @scaler_info: Decoder scaler information
+ * @colorspace: V4L2 colorspace
+ * @xfer_func: V4L2 transfer function
+ * @ycbcr_enc: V4L2 YCbCr encoding
+ * @quantization: V4L2 quantization
+ * @state: Instance state
+ * @state_in_seek: Previous state before seek
+ * @type: Instance type (decoder or encoder)
+ * @ops: Instance operations
+ * @std: Codec standard
+ * @id: Instance id
+ * @frame_buf: Metadata for reference frame buffers
+ * @frame_vbuf: Reference frame buffers
+ * @aux_vbuf: Auxiliary buffers
+ * @fbc_buf_registered: Reference frame buffers registration flag
+ * @queued_src_buf_num: number of queued source buffers
+ * @queued_dst_buf_num: number of queued destination buffers
+ * @processed_buf_num: number of processed buffers
+ * @error_buf_num: number of error buffers
+ * @sequence: V4L2 buffers sequence number
+ * @drain_src_buf_num: Number of OUTPUT buffers left to decode in the drain
+ * @drain: Decoder drain is in progress
+ * @next_dst_buf_last: Next queued CAPTURE buffer is returned as the last one
+ * @cbcr_interleave: CbCr interleave format flag
+ * @nv21: NV21 format flag
+ * @disp_mode: Display mode for decoder V4L2 control
+ */
+struct vpu_instance {
+ struct list_head list;
+ struct v4l2_fh v4l2_fh;
+ struct v4l2_ctrl_handler v4l2_ctrl_hdl;
+ struct vpu_core_device *dev;
+
+ /* Signals when INIT_SEQ, SET_PARAM command is done. */
+ struct completion irq_done;
+
+ /* Detects hangs during decode/encode and aborts if needed. */
+ struct delayed_work task_timer;
+
+ /* Protects the drain state. */
+ struct mutex lock;
+
+ bool enable;
+
+ struct v4l2_pix_format_mplane src_fmt;
+ struct v4l2_pix_format_mplane dst_fmt;
+ struct v4l2_rect crop;
+ struct dec_scaler_info scaler_info;
+ enum v4l2_colorspace colorspace;
+ enum v4l2_xfer_func xfer_func;
+ enum v4l2_ycbcr_encoding ycbcr_enc;
+ enum v4l2_quantization quantization;
+
+ enum vpu_instance_state state;
+ enum vpu_instance_state state_in_seek;
+ enum vpu_instance_type type;
+ const struct vpu_instance_ops *ops;
+
+ enum codec_std std;
+ u32 id;
+ struct frame_buffer frame_buf[WAVE6_MAX_FBS];
+ struct vpu_buf frame_vbuf[WAVE6_MAX_FBS];
+ struct vpu_buf aux_vbuf[AUX_BUF_TYPE_MAX][WAVE6_MAX_FBS];
+ bool fbc_buf_registered;
+ u32 queued_src_buf_num;
+ u32 queued_dst_buf_num;
+ u32 processed_buf_num;
+ u32 error_buf_num;
+ u32 sequence;
+ u32 drain_src_buf_num;
+ bool drain;
+ bool next_dst_buf_last;
+ bool cbcr_interleave;
+ bool nv21;
+
+ enum display_mode disp_mode;
+};
+
+void wave6_vpu_writel(struct vpu_core_device *core, u32 addr, u32 data);
+void wave6_vpu_writel_relaxed(struct vpu_core_device *core, u32 addr, u32 data);
+u32 wave6_vpu_readl(struct vpu_core_device *core, u32 addr);
+u32 wave6_vpu_readl_relaxed(struct vpu_core_device *core, u32 addr);
+
+#endif /* __WAVE6_VPUAPI_H__ */
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h b/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
index 1d4e016c2a58..fccb854a8a8c 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
@@ -18,13 +18,19 @@
#define W6_VPU_MAX_NUM_CORE 4
#define MAX_NUM_INSTANCE 32
+#define W6_MAX_PIC_STRIDE (4096 * 4)
+#define W6_PIC_STRIDE_ALIGNMENT 32
+
#define W6_VPU_POLL_DELAY_US 10
#define W6_VPU_POLL_TIMEOUT 300000
+#define W6_BOOT_WAIT_TIMEOUT 10000
+#define W6_VPU_TIMEOUT 6000
#define W6_MAX_REMAP_PAGE_SIZE (1024 * 1024)
#define W6_MAX_CODE_BUF_SIZE (4 * 1024 * 1024)
#define W6_CODE_BUF_SIZE (1 * 1024 * 1024)
#define W6_EXTRA_CODE_BUF_SIZE (256 * 1024)
+#define W6_TEMPBUF_SIZE (3 * 1024 * 1024)
#define W637DEC_WORKBUF_SIZE_FOR_CQ (3 * 512 * 1024)
static inline bool wave6_is_product_w_series(u32 code)
--
2.41.0.windows.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v8 04/11] media: chips-media: wave6: Add Wave6 decoder firmware interface
2026-11-19 8:59 [PATCH v8 00/11] Add support for Wave6 video codec driver Nas Chung
` (2 preceding siblings ...)
2026-11-19 8:59 ` [PATCH v8 03/11] media: chips-media: wave6: Add Wave6 core driver Nas Chung
@ 2026-11-19 8:59 ` Nas Chung
2026-11-19 9:22 ` sashiko-bot
2026-11-19 8:59 ` [PATCH v8 05/11] media: chips-media: wave6: Add Wave6 m2m decoder Nas Chung
` (6 subsequent siblings)
10 siblings, 1 reply; 23+ messages in thread
From: Nas Chung @ 2026-11-19 8:59 UTC (permalink / raw)
To: mchehab, hverkuil, robh, krzk+dt, conor+dt, Frank.Li, s.hauer
Cc: linux-media, devicetree, linux-kernel, imx, linux-arm-kernel,
festevam, kernel, nicolas.dufresne, ming.qian, marek.vasut,
jackson.lee, lafley.kim, Nas Chung
Add an interface layer to manage hardware register configuration
and communication with the Chips&Media Wave6 video codec IP for
video decoding.
The interface provides low-level helper functions used by the
Wave6 core driver to implement video decoding operations.
It handles command submission to the firmware via MMIO registers,
and waits for a response by polling the firmware busy flag.
The decoder commands cover instance creation, flush and destruction,
sequence initialization, frame buffer and display buffer registration,
and picture decoding.
Sequence initialization and picture decoding complete asynchronously:
the interface interrupt reports completion and a query command reads
back the result.
Tested-by: Ming Qian <ming.qian@oss.nxp.com>
Tested-by: Marek Vasut <marek.vasut@mailbox.org>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
---
.../media/platform/chips-media/wave6/Makefile | 1 +
.../platform/chips-media/wave6/wave6-hw.c | 753 ++++++++++++++++++
.../platform/chips-media/wave6/wave6-hw.h | 27 +
.../chips-media/wave6/wave6-regdefine.h | 219 +++++
.../platform/chips-media/wave6/wave6-vpuapi.c | 202 +++++
.../platform/chips-media/wave6/wave6-vpuapi.h | 168 ++++
.../chips-media/wave6/wave6-vpuconfig.h | 5 +
.../chips-media/wave6/wave6-vpuerror.h | 252 ++++++
8 files changed, 1627 insertions(+)
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuapi.c
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuerror.h
diff --git a/drivers/media/platform/chips-media/wave6/Makefile b/drivers/media/platform/chips-media/wave6/Makefile
index 7bfc28ba5523..08f11724b81c 100644
--- a/drivers/media/platform/chips-media/wave6/Makefile
+++ b/drivers/media/platform/chips-media/wave6/Makefile
@@ -3,5 +3,6 @@
wave6-objs += wave6-vpu.o \
wave6-vpu-core.o \
wave6-vpu-v4l2.o \
+ wave6-vpuapi.o \
wave6-hw.o
obj-$(CONFIG_VIDEO_WAVE6_VPU) += wave6.o
diff --git a/drivers/media/platform/chips-media/wave6/wave6-hw.c b/drivers/media/platform/chips-media/wave6/wave6-hw.c
index fcf5be294ef5..872a540df021 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-hw.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-hw.c
@@ -48,6 +48,103 @@ static void _wave6_print_reg_err(struct vpu_core_device *core, u32 fail_reason,
dev_err(core->dev, "%s: failure: 0x%x\n", func, fail_reason);
}
+static void wave6_dec_set_display_buffer(struct vpu_instance *inst, struct frame_buffer fb)
+{
+ struct dec_info *p_dec_info = &inst->codec_info->dec_info;
+
+ for (int i = 0; i < WAVE6_MAX_FBS; i++) {
+ if (!p_dec_info->disp_buf[i].buf_y) {
+ p_dec_info->disp_buf[i] = fb;
+ break;
+ }
+ }
+}
+
+static int wave6_dec_get_display_buffer(struct vpu_instance *inst, dma_addr_t addr)
+{
+ struct dec_info *p_dec_info = &inst->codec_info->dec_info;
+
+ for (int i = 0; i < WAVE6_MAX_FBS; i++) {
+ if (p_dec_info->disp_buf[i].buf_y == addr)
+ return i;
+ }
+
+ return -1;
+}
+
+static void wave6_dec_remove_display_buffer(struct vpu_instance *inst,
+ dma_addr_t addr)
+{
+ struct dec_info *p_dec_info = &inst->codec_info->dec_info;
+
+ for (int i = 0; i < WAVE6_MAX_FBS; i++) {
+ if (p_dec_info->disp_buf[i].buf_y == addr) {
+ p_dec_info->disp_buf[i].buf_y = 0;
+ break;
+ }
+ }
+}
+
+static enum chroma_format_idc get_chroma_format_idc(enum frame_buffer_format fmt)
+{
+ switch (fmt) {
+ case FORMAT_420:
+ case FORMAT_420_P10_16BIT_MSB:
+ case FORMAT_420_P10_16BIT_LSB:
+ case FORMAT_420_P10_32BIT_MSB:
+ case FORMAT_420_P10_32BIT_LSB:
+ return C_FMT_IDC_YUV420;
+ case FORMAT_422:
+ case FORMAT_422_P10_16BIT_MSB:
+ case FORMAT_422_P10_16BIT_LSB:
+ case FORMAT_422_P10_32BIT_MSB:
+ case FORMAT_422_P10_32BIT_LSB:
+ case FORMAT_YUYV:
+ case FORMAT_YVYU:
+ case FORMAT_UYVY:
+ case FORMAT_VYUY:
+ case FORMAT_YUYV_P10_16BIT_MSB:
+ case FORMAT_YVYU_P10_16BIT_MSB:
+ case FORMAT_UYVY_P10_16BIT_MSB:
+ case FORMAT_VYUY_P10_16BIT_MSB:
+ case FORMAT_YUYV_P10_16BIT_LSB:
+ case FORMAT_YVYU_P10_16BIT_LSB:
+ case FORMAT_UYVY_P10_16BIT_LSB:
+ case FORMAT_VYUY_P10_16BIT_LSB:
+ case FORMAT_YUYV_P10_32BIT_MSB:
+ case FORMAT_YVYU_P10_32BIT_MSB:
+ case FORMAT_UYVY_P10_32BIT_MSB:
+ case FORMAT_VYUY_P10_32BIT_MSB:
+ case FORMAT_YUYV_P10_32BIT_LSB:
+ case FORMAT_YVYU_P10_32BIT_LSB:
+ case FORMAT_UYVY_P10_32BIT_LSB:
+ case FORMAT_VYUY_P10_32BIT_LSB:
+ return C_FMT_IDC_YUV422;
+ case FORMAT_444:
+ case FORMAT_444_P10_16BIT_MSB:
+ case FORMAT_444_P10_16BIT_LSB:
+ case FORMAT_444_P10_32BIT_MSB:
+ case FORMAT_444_P10_32BIT_LSB:
+ return C_FMT_IDC_YUV444;
+ case FORMAT_400:
+ case FORMAT_400_P10_16BIT_MSB:
+ case FORMAT_400_P10_16BIT_LSB:
+ case FORMAT_400_P10_32BIT_MSB:
+ case FORMAT_400_P10_32BIT_LSB:
+ case FORMAT_YUV444_24BIT:
+ return C_FMT_IDC_YUV400;
+ case FORMAT_RGB_24BIT_PACKED:
+ case FORMAT_YUV444_24BIT_PACKED:
+ case FORMAT_RGB_32BIT_PACKED:
+ case FORMAT_RGB_P10_32BIT_PACKED:
+ case FORMAT_YUV444_32BIT_PACKED:
+ case FORMAT_YUV444_P10_32BIT_PACKED:
+ return C_FMT_IDC_RGB;
+ default:
+ return C_FMT_IDC_YUV400;
+ }
+}
+
static int wave6_wait_vpu_busy(struct vpu_core_device *core, unsigned int addr)
{
u32 data;
@@ -179,3 +276,659 @@ int wave6_vpu_get_version(struct vpu_core_device *core)
return 0;
}
+
+int wave6_vpu_build_up_dec_param(struct vpu_instance *inst,
+ struct dec_open_param *param)
+{
+ struct dec_info *p_dec_info = &inst->codec_info->dec_info;
+ u32 reg_val;
+ int ret;
+
+ lockdep_assert_held(&inst->dev->hw_lock);
+
+ p_dec_info->sec_axi.use_dec_ip = true;
+ p_dec_info->sec_axi.use_dec_lf_row = true;
+ switch (inst->std) {
+ case W_HEVC_DEC:
+ p_dec_info->seq_change_mask = SEQ_CHANGE_ENABLE_ALL_HEVC;
+ break;
+ case W_AVC_DEC:
+ p_dec_info->seq_change_mask = SEQ_CHANGE_ENABLE_ALL_AVC;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_TEMP_BASE,
+ param->inst_buffer.temp_base);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_TEMP_SIZE,
+ param->inst_buffer.temp_size);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_BS_PARAM, VPU_STREAM_ENDIAN);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_ADDR_EXT, param->ext_addr_vcpu);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_DISP_MODE, param->disp_mode);
+
+ reg_val = CREATE_INST_CORE_INFO_CQ_DEPTH(COMMAND_QUEUE_DEPTH) |
+ CREATE_INST_CORE_INFO_CORE_IDC(SINGLE_CORE_IDC) |
+ CREATE_INST_CORE_INFO_CORE_NUM(SINGLE_CORE);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_CORE_INFO, reg_val);
+
+ reg_val = CREATE_INST_PRIORITY_SECURITY_FLAG(param->is_secure_inst) |
+ CREATE_INST_PRIORITY_VALUE(param->inst_priority);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_PRIORITY, reg_val);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_TIMEOUT_CYCLE_COUNT,
+ W6_VPU_TIMEOUT_CYCLE_COUNT);
+
+ wave6_send_command(inst->dev, 0, inst->std, W6_CMD_CREATE_INSTANCE);
+ ret = wave6_wait_vpu_busy(inst->dev, W6_VPU_BUSY_STATUS);
+ if (ret) {
+ dev_err(inst->dev->dev, "%s: timeout\n", __func__);
+ return ret;
+ }
+
+ if (!vpu_read_reg_relaxed(inst->dev, W6_RET_SUCCESS)) {
+ u32 reason_code = vpu_read_reg_relaxed(inst->dev, W6_RET_FAIL_REASON);
+
+ wave6_print_reg_err(inst->dev, reason_code);
+ return -EIO;
+ }
+
+ inst->id = vpu_read_reg_relaxed(inst->dev, W6_RET_INSTANCE_ID);
+
+ return 0;
+}
+
+int wave6_vpu_dec_issue_seq_init(struct vpu_instance *inst)
+{
+ struct dec_info *p_dec_info;
+ u32 reg_val;
+ int ret;
+
+ guard(mutex)(&inst->dev->hw_lock);
+
+ p_dec_info = &inst->codec_info->dec_info;
+
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_INIT_SEQ_BS_RD_PTR, p_dec_info->stream_rd_ptr);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_INIT_SEQ_BS_WR_PTR, p_dec_info->stream_wr_ptr);
+
+ reg_val = DEC_PIC_BS_OPTION_STREAM_END(p_dec_info->stream_end) |
+ DEC_PIC_BS_OPTION_EXPLICIT_END_ON;
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_INIT_SEQ_BS_OPTION, reg_val);
+ reg_val = INIT_SEQ_OPTION_MODE(W6_INIT_SEQ_OPT_NORMAL);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_INIT_SEQ_OPTION, reg_val);
+
+ wave6_send_command(inst->dev, inst->id, inst->std, W6_CMD_INIT_SEQ);
+ ret = wave6_wait_vpu_busy(inst->dev, W6_VPU_BUSY_STATUS);
+ if (ret) {
+ dev_err(inst->dev->dev, "%s: timeout\n", __func__);
+ return ret;
+ }
+
+ if (!vpu_read_reg_relaxed(inst->dev, W6_RET_SUCCESS)) {
+ u32 reason_code = vpu_read_reg_relaxed(inst->dev, W6_RET_FAIL_REASON);
+
+ wave6_print_reg_err(inst->dev, reason_code);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static u32 wave6_dec_hevc_profile(u32 profile, u32 profile_comp)
+{
+ if (profile)
+ return profile;
+
+ if ((profile_comp & PROFILE_COMPATIBILITY_MAIN) &&
+ (profile_comp & PROFILE_COMPATIBILITY_MAIN10))
+ return HEVC_PROFILE_MAIN;
+ if (profile_comp & PROFILE_COMPATIBILITY_MAIN10)
+ return HEVC_PROFILE_MAIN10;
+ if (profile_comp & PROFILE_COMPATIBILITY_STILL_PICTURE)
+ return HEVC_PROFILE_STILLPICTURE;
+
+ return HEVC_PROFILE_MAIN;
+}
+
+static u32 wave6_dec_avc_profile(u32 profile)
+{
+ switch (profile) {
+ case PROFILE_H264_BP:
+ return H264_PROFILE_BP;
+ case PROFILE_H264_HP:
+ return H264_PROFILE_HP;
+ case PROFILE_H264_MP:
+ return H264_PROFILE_MP;
+ case PROFILE_H264_HIGH10:
+ return H264_PROFILE_HIGH10;
+ case PROFILE_H264_EXTENDED:
+ return H264_PROFILE_EXTENDED;
+ default:
+ return H264_PROFILE_BP;
+ }
+}
+
+static void wave6_get_dec_seq_result(struct vpu_instance *inst, struct dec_seq_info *info)
+{
+ u32 profile, profile_comp;
+ u32 reg_val;
+
+ lockdep_assert_held(&inst->dev->hw_lock);
+
+ info->rd_ptr = wave6_vpu_dec_get_rd_ptr(inst);
+
+ reg_val = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_PIC_SIZE);
+ info->pic_width = DEC_PIC_SIZE_WIDTH(reg_val);
+ info->pic_height = DEC_PIC_SIZE_HEIGHT(reg_val);
+
+ info->min_frame_buffer_count = vpu_read_reg_relaxed(inst->dev,
+ W6_RET_DEC_NUM_REQUIRED_FBC_FB);
+ info->frame_buf_delay = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_NUM_REORDER_DELAY);
+ info->req_mv_buffer_count = vpu_read_reg_relaxed(inst->dev,
+ W6_RET_DEC_NUM_REQUIRED_COL_BUF);
+
+ reg_val = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_CROP_TOP_BOTTOM);
+ info->pic_crop_rect.top = DEC_CROP_TOP(reg_val);
+ info->pic_crop_rect.bottom = info->pic_height - DEC_CROP_BOTTOM(reg_val);
+
+ reg_val = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_CROP_LEFT_RIGHT);
+ info->pic_crop_rect.left = DEC_CROP_LEFT(reg_val);
+ info->pic_crop_rect.right = info->pic_width - DEC_CROP_RIGHT(reg_val);
+
+ info->f_rate_numerator = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_FRAME_RATE_NR);
+ info->f_rate_denominator = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_FRAME_RATE_DR);
+
+ reg_val = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_COLOR_SAMPLE);
+ info->luma_bit_depth = DEC_COLOR_SAMPLE_L_BIT_DEPTH(reg_val);
+ info->chroma_bit_depth = DEC_COLOR_SAMPLE_C_BIT_DEPTH(reg_val);
+ info->c_fmt_idc = DEC_COLOR_SAMPLE_C_FMT_IDC(reg_val);
+ info->aspect_rate_info = DEC_COLOR_SAMPLE_ASPECT_RATIO_IDC(reg_val);
+ info->is_ext_sar = info->aspect_rate_info == H264_VUI_SAR_IDC_EXTENDED;
+ if (info->is_ext_sar)
+ info->aspect_rate_info = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_ASPECT_RATIO);
+ info->bitrate = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_BIT_RATE);
+
+ reg_val = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_SEQ_PARAM);
+ info->level = DEC_SEQ_PARAM_LEVEL(reg_val);
+ info->tier = DEC_SEQ_PARAM_TIER(reg_val);
+
+ profile = DEC_SEQ_PARAM_PROFILE(reg_val);
+ profile_comp = DEC_SEQ_PARAM_PROFILE_COMPATIBILITY(reg_val);
+ if (inst->std == W_HEVC_DEC)
+ info->profile = wave6_dec_hevc_profile(profile, profile_comp);
+ else if (inst->std == W_AVC_DEC)
+ info->profile = wave6_dec_avc_profile(profile);
+
+ reg_val = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_COLOR_CONFIG);
+ if (reg_val) {
+ info->color.video_signal_type_present = true;
+ info->color.color_description_present = DEC_COLOR_CONFIG_COLOR_PRESENT(reg_val);
+ info->color.color_primaries = DEC_COLOR_CONFIG_COLOR_PRIMARIES(reg_val);
+ info->color.transfer_characteristics = DEC_COLOR_CONFIG_TRANS_CHAR(reg_val);
+ info->color.matrix_coefficients = DEC_COLOR_CONFIG_MATRIX_COEFF(reg_val);
+ info->color.color_range = DEC_COLOR_CONFIG_COLOR_RANGE(reg_val);
+ } else {
+ info->color.video_signal_type_present = false;
+ }
+}
+
+int wave6_vpu_dec_get_seq_info(struct vpu_instance *inst, struct dec_seq_info *info)
+{
+ int ret;
+
+ lockdep_assert_held(&inst->dev->hw_lock);
+
+ ret = wave6_send_query(inst->dev, inst->id, inst->std, W6_QUERY_OPT_GET_RESULT);
+ if (ret)
+ return ret;
+
+ if (vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_DECODING_SUCCESS) != 1) {
+ info->err_reason = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_ERR_INFO);
+ ret = -EIO;
+ } else {
+ info->warn_info = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_WARN_INFO);
+ }
+
+ wave6_get_dec_seq_result(inst, info);
+
+ return ret;
+}
+
+static bool wave6_check_aux_vbuf(struct vpu_instance *inst,
+ enum aux_buffer_type type, size_t count)
+{
+ for (size_t i = 0; i < count; i++) {
+ if (!inst->aux_vbuf[type][i].daddr)
+ return false;
+ }
+
+ return true;
+}
+
+int wave6_vpu_dec_register_frame_buffer(struct vpu_instance *inst)
+{
+ size_t fbc_remain, mv_remain, fbc_idx = 0, mv_idx = 0;
+ size_t k, group_num, fbc_count, mv_count;
+ struct dec_info *p_dec_info;
+ struct frame_buffer *fb_arr;
+ dma_addr_t fbc_cr_tbl_addr;
+ u32 reg_val;
+ int ret;
+
+ guard(mutex)(&inst->dev->hw_lock);
+
+ p_dec_info = &inst->codec_info->dec_info;
+
+ fb_arr = &inst->frame_buf[0];
+ fbc_count = p_dec_info->seq_info.min_frame_buffer_count;
+ mv_count = p_dec_info->seq_info.req_mv_buffer_count;
+ if (!fbc_count || !mv_count)
+ return -EINVAL;
+
+ if (!wave6_check_aux_vbuf(inst, AUX_BUF_FBC_Y_TBL, fbc_count) ||
+ !wave6_check_aux_vbuf(inst, AUX_BUF_FBC_C_TBL, fbc_count) ||
+ !wave6_check_aux_vbuf(inst, AUX_BUF_MV_COL, mv_count))
+ return -EINVAL;
+
+ reg_val = SET_FB_PIC_SIZE_WIDTH(p_dec_info->seq_info.pic_width) |
+ SET_FB_PIC_SIZE_HEIGHT(p_dec_info->seq_info.pic_height);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_PIC_SIZE, reg_val);
+ reg_val = SET_FB_PIC_INFO_C_FMT_IDC(p_dec_info->seq_info.c_fmt_idc) |
+ SET_FB_PIC_INFO_L_BIT_DEPTH(p_dec_info->seq_info.luma_bit_depth) |
+ SET_FB_PIC_INFO_C_BIT_DEPTH(p_dec_info->seq_info.chroma_bit_depth);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_PIC_INFO, reg_val);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_DEFAULT_CDF, 0);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_SEGMAP, 0);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_MV_COL_PRE_ENT, 0);
+
+ fbc_remain = fbc_count;
+ mv_remain = mv_count;
+ group_num = DIV_ROUND_UP(max(fbc_count, mv_count), MAX_SET_FB_GROUP_SIZE);
+
+ for (size_t i = 0; i < group_num; i++) {
+ u32 set_fbc_num = min(fbc_remain, MAX_SET_FB_GROUP_SIZE);
+ u32 set_mv_num = min(mv_remain, MAX_SET_FB_GROUP_SIZE);
+ u32 fbc_start_no = i * MAX_SET_FB_GROUP_SIZE;
+ u32 mv_start_no = i * MAX_SET_FB_GROUP_SIZE;
+ bool last_group = i == group_num - 1;
+ bool first_group = i == 0;
+ u32 fbc_end_no, mv_end_no;
+
+ fbc_end_no = fbc_start_no + set_fbc_num - 1;
+ mv_end_no = mv_start_no + set_mv_num - 1;
+
+ reg_val = SET_FB_OPTION_ENDIAN(VDI_128BIT_BIG_ENDIAN);
+ if (first_group)
+ reg_val |= SET_FB_OPTION_START;
+ if (last_group)
+ reg_val |= SET_FB_OPTION_END;
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_OPTION, reg_val);
+
+ reg_val = SET_FB_NUM_FBC_START_IDX(fbc_start_no) |
+ SET_FB_NUM_FBC_END_IDX(fbc_end_no) |
+ SET_FB_NUM_MV_COL_START_IDX(mv_start_no) |
+ SET_FB_NUM_MV_COL_END_IDX(mv_end_no);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_NUM, reg_val);
+
+ for (k = 0; k < set_fbc_num; k++) {
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_FBC_Y(k),
+ fb_arr[fbc_idx].buf_y);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_FBC_C(k),
+ fb_arr[fbc_idx].buf_cb);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_FBC_CR(k),
+ fb_arr[fbc_idx].buf_cr);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_FBC_Y_OFFSET(k),
+ inst->aux_vbuf[AUX_BUF_FBC_Y_TBL][fbc_idx].daddr);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_FBC_C_OFFSET(k),
+ inst->aux_vbuf[AUX_BUF_FBC_C_TBL][fbc_idx].daddr);
+ fbc_cr_tbl_addr = inst->aux_vbuf[AUX_BUF_FBC_C_TBL][fbc_idx].daddr +
+ (inst->aux_vbuf[AUX_BUF_FBC_C_TBL][fbc_idx].size >> 1);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_FBC_CR_OFFSET(k),
+ fbc_cr_tbl_addr);
+ fbc_idx++;
+ }
+ fbc_remain -= k;
+
+ for (k = 0; k < set_mv_num; k++) {
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_MV_COL(k),
+ inst->aux_vbuf[AUX_BUF_MV_COL][mv_idx].daddr);
+ mv_idx++;
+ }
+ mv_remain -= k;
+
+ wave6_send_command(inst->dev, inst->id, inst->std, W6_CMD_SET_FB);
+ ret = wave6_wait_vpu_busy(inst->dev, W6_VPU_BUSY_STATUS);
+ if (ret) {
+ dev_err(inst->dev->dev, "%s: timeout\n", __func__);
+ return ret;
+ }
+
+ if (!vpu_read_reg_relaxed(inst->dev, W6_RET_SUCCESS))
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static u32 wave6_dec_out_fmt(enum frame_buffer_format fmt)
+{
+ return SET_DISP_COMMON_PIC_INFO_C_FMT_IDC(get_chroma_format_idc(fmt)) |
+ SET_DISP_COMMON_PIC_INFO_OUT_JUSTIFY(WTL_RIGHT_JUSTIFIED) |
+ SET_DISP_COMMON_PIC_INFO_OUT_PIXEL(WTL_PIXEL_8BIT);
+}
+
+int wave6_vpu_dec_register_display_buffer(struct vpu_instance *inst, struct frame_buffer fb)
+{
+ struct dec_info *p_dec_info;
+ u32 reg_val;
+ int ret;
+
+ guard(mutex)(&inst->dev->hw_lock);
+
+ p_dec_info = &inst->codec_info->dec_info;
+
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_SET_DISP_SCL_PARAM,
+ inst->scaler_info.enable);
+ reg_val = SET_DISP_SCL_PIC_SIZE_WIDTH(inst->scaler_info.width) |
+ SET_DISP_SCL_PIC_SIZE_HEIGHT(inst->scaler_info.height);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_SET_DISP_SCL_PIC_SIZE, reg_val);
+ reg_val = SET_DISP_PIC_SIZE_WIDTH(p_dec_info->seq_info.pic_width) |
+ SET_DISP_PIC_SIZE_HEIGHT(p_dec_info->seq_info.pic_height);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_SET_DISP_PIC_SIZE, reg_val);
+
+ reg_val = SET_DISP_COMMON_PIC_INFO_BWB_ON |
+ wave6_dec_out_fmt(p_dec_info->out_fmt) |
+ SET_DISP_COMMON_PIC_INFO_PIXEL_ORDER(PIXEL_ORDER_INCREASING) |
+ SET_DISP_COMMON_PIC_INFO_CBCR_ORDER(inst->nv21) |
+ SET_DISP_COMMON_PIC_INFO_2PLANE(inst->cbcr_interleave) |
+ SET_DISP_COMMON_PIC_INFO_STRIDE(fb.stride);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_SET_DISP_COMMON_PIC_INFO, reg_val);
+ reg_val = SET_DISP_OPTION_ENDIAN(VDI_128BIT_BIG_ENDIAN);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_SET_DISP_OPTION, reg_val);
+ reg_val = SET_DISP_PIC_INFO_L_BIT_DEPTH(fb.luma_bit_depth) |
+ SET_DISP_PIC_INFO_C_BIT_DEPTH(fb.chroma_bit_depth) |
+ SET_DISP_PIC_INFO_C_FMT_IDC(fb.c_fmt_idc);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_SET_DISP_PIC_INFO, reg_val);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_SET_DISP_Y_BASE, fb.buf_y);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_SET_DISP_CB_BASE, fb.buf_cb);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_SET_DISP_CR_BASE, fb.buf_cr);
+
+ wave6_send_command(inst->dev, inst->id, inst->std, W6_CMD_DEC_SET_DISP);
+ ret = wave6_wait_vpu_busy(inst->dev, W6_VPU_BUSY_STATUS);
+ if (ret) {
+ dev_err(inst->dev->dev, "%s: timeout\n", __func__);
+ return ret;
+ }
+
+ if (!vpu_read_reg_relaxed(inst->dev, W6_RET_SUCCESS))
+ return -EIO;
+
+ wave6_dec_set_display_buffer(inst, fb);
+
+ return 0;
+}
+
+int wave6_vpu_dec_start_one_frame(struct vpu_instance *inst, struct dec_param *param,
+ u32 *fail_res)
+{
+ struct dec_info *p_dec_info;
+ u32 reg_val;
+ int ret;
+
+ guard(mutex)(&inst->dev->hw_lock);
+
+ p_dec_info = &inst->codec_info->dec_info;
+
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_PIC_BS_RD_PTR, p_dec_info->stream_rd_ptr);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_PIC_BS_WR_PTR, p_dec_info->stream_wr_ptr);
+
+ reg_val = DEC_PIC_BS_OPTION_STREAM_END(p_dec_info->stream_end) |
+ DEC_PIC_BS_OPTION_EXPLICIT_END_ON;
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_PIC_BS_OPTION, reg_val);
+
+ reg_val = DEC_PIC_SEC_AXI_IP_ENABLE(p_dec_info->sec_axi.use_dec_ip) |
+ DEC_PIC_SEC_AXI_LF_ENABLE(p_dec_info->sec_axi.use_dec_lf_row);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_PIC_SEC_AXI, reg_val);
+
+ reg_val = DEC_PIC_OPTION_MODE(W6_DEC_PIC_OPT_NORMAL);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_PIC_OPTION, reg_val);
+
+ reg_val = DEC_PIC_TEMPORAL_ID_PLUS1_SPATIAL(DECODE_ALL_SPATIAL_LAYERS) |
+ DEC_PIC_TEMPORAL_ID_PLUS1(DECODE_ALL_TEMPORAL_LAYERS);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_PIC_TEMPORAL_ID_PLUS1, reg_val);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_PIC_SEQ_CHANGE_ENABLE_FLAG,
+ p_dec_info->seq_change_mask);
+ reg_val = lower_32_bits(param->timestamp);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_PIC_TIMESTAMP_LOW, reg_val);
+ reg_val = upper_32_bits(param->timestamp);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_DEC_PIC_TIMESTAMP_HIGH, reg_val);
+
+ wave6_send_command(inst->dev, inst->id, inst->std, W6_CMD_DEC_PIC);
+ ret = wave6_wait_vpu_busy(inst->dev, W6_VPU_BUSY_STATUS);
+ if (ret) {
+ dev_err(inst->dev->dev, "%s: timeout\n", __func__);
+ return ret;
+ }
+
+ if (!vpu_read_reg_relaxed(inst->dev, W6_RET_SUCCESS)) {
+ *fail_res = vpu_read_reg_relaxed(inst->dev, W6_RET_FAIL_REASON);
+ wave6_print_reg_err(inst->dev, *fail_res);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+int wave6_vpu_dec_get_output_info(struct vpu_instance *inst, struct dec_output_info *info)
+{
+ int decoded_idx = -1, disp_idx = -1;
+ struct dec_info *p_dec_info;
+ unsigned long idc;
+ u32 reg_val, i;
+ int ret;
+
+ if (WARN_ON(!info))
+ return -EINVAL;
+
+ guard(mutex)(&inst->dev->hw_lock);
+
+ p_dec_info = &inst->codec_info->dec_info;
+
+ ret = wave6_send_query(inst->dev, inst->id, inst->std, W6_QUERY_OPT_GET_RESULT);
+ if (ret) {
+ info->rd_ptr = p_dec_info->stream_rd_ptr;
+ info->wr_ptr = p_dec_info->stream_wr_ptr;
+ return ret;
+ }
+
+ info->decoding_success = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_DECODING_SUCCESS);
+ if (!info->decoding_success)
+ info->error_reason = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_ERR_INFO);
+ else
+ info->warn_info = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_WARN_INFO);
+
+ reg_val = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_PIC_TYPE);
+ info->ctu_size = DEC_PIC_TYPE_CTU_SIZE(reg_val);
+ info->nal_type = DEC_PIC_TYPE_NAL_UNIT_TYPE(reg_val);
+
+ if (reg_val & DEC_PIC_TYPE_B)
+ info->pic_type = PIC_TYPE_B;
+ else if (reg_val & DEC_PIC_TYPE_P)
+ info->pic_type = PIC_TYPE_P;
+ else if (reg_val & DEC_PIC_TYPE_I)
+ info->pic_type = PIC_TYPE_I;
+ else
+ info->pic_type = PIC_TYPE_MAX;
+ if (inst->std == W_HEVC_DEC) {
+ if (info->pic_type == PIC_TYPE_I &&
+ (info->nal_type == H265_NAL_UNIT_TYPE_IDR_W_RADL ||
+ info->nal_type == H265_NAL_UNIT_TYPE_IDR_N_LP))
+ info->pic_type = PIC_TYPE_IDR;
+ } else if (inst->std == W_AVC_DEC) {
+ if (info->pic_type == PIC_TYPE_I &&
+ info->nal_type == H264_NAL_UNIT_TYPE_IDR_PICTURE)
+ info->pic_type = PIC_TYPE_IDR;
+ }
+
+ reg_val = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_DECODED_FLAG);
+ if (reg_val) {
+ dma_addr_t addr = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_DECODED_ADDR);
+
+ decoded_idx = wave6_dec_get_display_buffer(inst, addr);
+ info->frame_decoded_addr = addr;
+ info->frame_decoded = true;
+ }
+
+ reg_val = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_DISPLAY_FLAG);
+ if (reg_val) {
+ dma_addr_t addr = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_DISPLAY_ADDR);
+
+ disp_idx = wave6_dec_get_display_buffer(inst, addr);
+ info->frame_display_addr = addr;
+ info->frame_display = true;
+ }
+
+ idc = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_DISP_IDC);
+ for_each_set_bit(i, &idc, WAVE6_MAX_FBS) {
+ dma_addr_t addr;
+
+ addr = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_DISP_LINEAR_ADDR(i));
+
+ info->disp_frame_addr[info->disp_frame_num] = addr;
+ info->disp_frame_num++;
+ }
+
+ idc = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_RELEASE_IDC);
+ for_each_set_bit(i, &idc, WAVE6_MAX_FBS) {
+ dma_addr_t addr;
+
+ addr = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_DISP_LINEAR_ADDR(i));
+
+ wave6_dec_remove_display_buffer(inst, addr);
+ info->release_disp_frame_addr[info->release_disp_frame_num] = addr;
+ info->release_disp_frame_num++;
+ }
+
+ if (inst->std == W_HEVC_DEC) {
+ info->decoded_poc = -1;
+ info->display_poc = -1;
+ if (decoded_idx >= 0)
+ info->decoded_poc = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_PIC_POC);
+ } else if (inst->std == W_AVC_DEC) {
+ info->decoded_poc = -1;
+ info->display_poc = -1;
+ if (decoded_idx >= 0)
+ info->decoded_poc = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_PIC_POC);
+ }
+
+ reg_val = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_PIC_SIZE);
+ info->dec_pic_width = DEC_PIC_SIZE_WIDTH(reg_val);
+ info->dec_pic_height = DEC_PIC_SIZE_HEIGHT(reg_val);
+
+ info->rd_ptr = wave6_vpu_dec_get_rd_ptr(inst);
+ info->wr_ptr = p_dec_info->stream_wr_ptr;
+ info->byte_pos_frame_start = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_AU_START_POS);
+ info->byte_pos_frame_end = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_AU_END_POS);
+ info->stream_end = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_STREAM_END);
+ info->notification_flags = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_NOTIFICATION);
+ info->last_frame_in_au = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_LAST_FRAME_FLAG);
+
+ reg_val = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_TIMESTAMP_LOW);
+ info->timestamp = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_TIMESTAMP_HIGH);
+ info->timestamp = (info->timestamp << 32) | reg_val;
+
+ if (decoded_idx >= 0 && decoded_idx < WAVE6_MAX_FBS)
+ p_dec_info->dec_out_info[decoded_idx].decoded_poc = info->decoded_poc;
+
+ if (disp_idx >= 0 && disp_idx < WAVE6_MAX_FBS) {
+ info->display_poc = p_dec_info->dec_out_info[disp_idx].decoded_poc;
+ info->disp_pic_width = p_dec_info->dec_out_info[disp_idx].dec_pic_width;
+ info->disp_pic_height = p_dec_info->dec_out_info[disp_idx].dec_pic_height;
+ }
+
+ info->sequence_no = p_dec_info->seq_info.sequence_no;
+ if (decoded_idx >= 0 && decoded_idx < WAVE6_MAX_FBS)
+ p_dec_info->dec_out_info[decoded_idx] = *info;
+
+ if (info->notification_flags & DEC_NOTI_FLAG_SEQ_CHANGE) {
+ wave6_get_dec_seq_result(inst, &p_dec_info->seq_info);
+ p_dec_info->seq_info.sequence_no++;
+ }
+
+ return 0;
+}
+
+int wave6_vpu_dec_fini_seq(struct vpu_instance *inst, u32 *fail_res)
+{
+ int ret;
+
+ lockdep_assert_held(&inst->dev->hw_lock);
+
+ wave6_send_command(inst->dev, inst->id, inst->std, W6_CMD_DESTROY_INSTANCE);
+ ret = wave6_wait_vpu_busy(inst->dev, W6_VPU_BUSY_STATUS);
+ if (ret)
+ return -ETIMEDOUT;
+
+ if (!vpu_read_reg_relaxed(inst->dev, W6_RET_SUCCESS)) {
+ *fail_res = vpu_read_reg_relaxed(inst->dev, W6_RET_FAIL_REASON);
+ wave6_print_reg_err(inst->dev, *fail_res);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+dma_addr_t wave6_vpu_dec_get_rd_ptr(struct vpu_instance *inst)
+{
+ lockdep_assert_held(&inst->dev->hw_lock);
+
+ return vpu_read_reg(inst->dev, W6_RET_DEC_BS_RD_PTR);
+}
+
+int wave6_vpu_dec_flush_instance(struct vpu_instance *inst)
+{
+ struct dec_info *p_dec_info;
+ unsigned long remove_idc;
+ int ret, index;
+ u32 using_idc;
+
+ guard(mutex)(&inst->dev->hw_lock);
+
+ p_dec_info = &inst->codec_info->dec_info;
+ if (!p_dec_info->seq_info_obtained)
+ return -EINVAL;
+
+ wave6_send_command(inst->dev, inst->id, inst->std, W6_CMD_FLUSH_INSTANCE);
+ ret = wave6_wait_vpu_busy(inst->dev, W6_VPU_BUSY_STATUS);
+ if (ret)
+ return -ETIMEDOUT;
+
+ if (!vpu_read_reg_relaxed(inst->dev, W6_RET_SUCCESS)) {
+ u32 reg_val;
+
+ reg_val = vpu_read_reg_relaxed(inst->dev, W6_RET_FAIL_REASON);
+ wave6_print_reg_err(inst->dev, reg_val);
+ return -EIO;
+ }
+
+ ret = wave6_send_query(inst->dev, inst->id, inst->std, W6_QUERY_OPT_GET_FLUSH_CMD_INFO);
+ if (ret)
+ return ret;
+
+ remove_idc = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_FLUSH_CMD_BUF_STATE_UNUSED_IDC) |
+ vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_FLUSH_CMD_BUF_STATE_USED_IDC);
+ if (remove_idc)
+ dev_dbg(inst->dev->dev, "%s: remove_idc 0x%lx\n", __func__, remove_idc);
+
+ using_idc = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_FLUSH_CMD_BUF_STATE_USING_IDC);
+ if (using_idc)
+ dev_dbg(inst->dev->dev, "%s: using_idc 0x%x\n", __func__, using_idc);
+
+ for_each_set_bit(index, &remove_idc, WAVE6_MAX_FBS) {
+ dma_addr_t addr;
+
+ addr = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_FLUSH_CMD_DISP_ADDR(index));
+ wave6_dec_remove_display_buffer(inst, addr);
+ }
+
+ return 0;
+}
diff --git a/drivers/media/platform/chips-media/wave6/wave6-hw.h b/drivers/media/platform/chips-media/wave6/wave6-hw.h
index bf8b6dcb54c2..b163c6bb6724 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-hw.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-hw.h
@@ -15,8 +15,35 @@ enum product_id {
PRODUCT_ID_NONE,
};
+#define COMMAND_QUEUE_DEPTH 1
+#define SINGLE_CORE_IDC 1
+#define SINGLE_CORE 1
+
+#define MAX_SET_FB_GROUP_SIZE 16
+
+#define DECODE_ALL_TEMPORAL_LAYERS 0
+#define DECODE_ALL_SPATIAL_LAYERS 0
+
+#define PIXEL_ORDER_INCREASING 1
+
+#define WTL_RIGHT_JUSTIFIED 0
+#define WTL_LEFT_JUSTIFIED 1
+#define WTL_PIXEL_8BIT 0
+#define WTL_PIXEL_16BIT 1
+#define WTL_PIXEL_32BIT 2
+
+#define H265_NAL_UNIT_TYPE_IDR_W_RADL 19
+#define H265_NAL_UNIT_TYPE_IDR_N_LP 20
+
+#define H264_NAL_UNIT_TYPE_IDR_PICTURE 5
+
bool wave6_vpu_is_init(struct vpu_core_device *core);
int wave6_vpu_get_version(struct vpu_core_device *core);
void wave6_vpu_enable_interrupt(struct vpu_core_device *core);
+int wave6_vpu_build_up_dec_param(struct vpu_instance *inst, struct dec_open_param *param);
+int wave6_vpu_dec_get_seq_info(struct vpu_instance *inst, struct dec_seq_info *info);
+int wave6_vpu_dec_fini_seq(struct vpu_instance *inst, u32 *fail_res);
+dma_addr_t wave6_vpu_dec_get_rd_ptr(struct vpu_instance *inst);
+
#endif /* __WAVE6_HW_H__ */
diff --git a/drivers/media/platform/chips-media/wave6/wave6-regdefine.h b/drivers/media/platform/chips-media/wave6/wave6-regdefine.h
index aa9b972c82d1..0651f5c1e652 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-regdefine.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-regdefine.h
@@ -25,6 +25,18 @@ enum wave6_command {
W6_CMD_QUERY = 0x4000,
};
+enum wave6_init_seq_option {
+ W6_INIT_SEQ_OPT_NORMAL = 1,
+ W6_INIT_SEQ_OPT_W_THUMBNAIL = 17,
+};
+
+enum wave6_dec_pic_option {
+ W6_DEC_PIC_OPT_NORMAL = 0,
+ W6_DEC_PIC_OPT_W_THUMBNAIL = 16,
+ W6_DEC_PIC_OPT_SKIP_NON_IRAP = 17,
+ W6_DEC_PIC_OPT_SKIP_NON_REF_PIC = 19,
+};
+
enum wave6_query_option {
W6_QUERY_OPT_GET_VPU_INFO = 0,
W6_QUERY_OPT_GET_RESULT = 2,
@@ -111,4 +123,211 @@ enum wave6_interrupt_bit {
#define W6_RET_FW_API_VERSION (W6_REG_BASE + 0x32C)
#define W6_RET_SHA_ID (W6_REG_BASE + 0x330)
+/* COMMON - CREATE_INSTANCE */
+#define W6_CMD_CREATE_INST_BS_PARAM (W6_REG_BASE + 0x310)
+#define W6_CMD_CREATE_INST_ADDR_EXT (W6_REG_BASE + 0x318)
+#define W6_CMD_CREATE_INST_DISP_MODE (W6_REG_BASE + 0x31C)
+#define W6_CMD_CREATE_INST_CORE_INFO (W6_REG_BASE + 0x330)
+#define CREATE_INST_CORE_INFO_CQ_DEPTH(x) FIELD_PREP(GENMASK(10, 8), (x))
+#define CREATE_INST_CORE_INFO_CORE_IDC(x) FIELD_PREP(GENMASK(7, 4), (x))
+#define CREATE_INST_CORE_INFO_CORE_NUM(x) FIELD_PREP(GENMASK(3, 0), (x))
+#define W6_CMD_CREATE_INST_PRIORITY (W6_REG_BASE + 0x334)
+#define CREATE_INST_PRIORITY_SECURITY_FLAG(x) FIELD_PREP(GENMASK(8, 8), (x))
+#define CREATE_INST_PRIORITY_VALUE(x) FIELD_PREP(GENMASK(4, 0), (x))
+#define W6_CMD_CREATE_INST_TEMP_BASE (W6_REG_BASE + 0x348)
+#define W6_CMD_CREATE_INST_TEMP_SIZE (W6_REG_BASE + 0x34C)
+#define W6_CMD_CREATE_INST_TIMEOUT_CYCLE_COUNT (W6_REG_BASE + 0x3A8)
+
+/* COMMON - SET_FB */
+#define W6_CMD_SET_FB_OPTION (W6_REG_BASE + 0x204)
+#define SET_FB_OPTION_ENDIAN(x) FIELD_PREP(GENMASK(19, 16), (x))
+#define SET_FB_OPTION_END BIT(4)
+#define SET_FB_OPTION_START BIT(3)
+#define W6_CMD_SET_FB_PIC_INFO (W6_REG_BASE + 0x300)
+#define SET_FB_PIC_INFO_C_FMT_IDC(x) FIELD_PREP(GENMASK(26, 25), (x))
+#define SET_FB_PIC_INFO_L_BIT_DEPTH(x) FIELD_PREP(GENMASK(24, 21), (x))
+#define SET_FB_PIC_INFO_C_BIT_DEPTH(x) FIELD_PREP(GENMASK(20, 17), (x))
+#define W6_CMD_SET_FB_PIC_SIZE (W6_REG_BASE + 0x304)
+#define SET_FB_PIC_SIZE_WIDTH(x) FIELD_PREP(GENMASK(31, 16), (x))
+#define SET_FB_PIC_SIZE_HEIGHT(x) FIELD_PREP(GENMASK(15, 0), (x))
+#define W6_CMD_SET_FB_NUM (W6_REG_BASE + 0x308)
+#define SET_FB_NUM_FBC_START_IDX(x) FIELD_PREP(GENMASK(28, 24), (x))
+#define SET_FB_NUM_FBC_END_IDX(x) FIELD_PREP(GENMASK(20, 16), (x))
+#define SET_FB_NUM_MV_COL_START_IDX(x) FIELD_PREP(GENMASK(9, 5), (x))
+#define SET_FB_NUM_MV_COL_END_IDX(x) FIELD_PREP(GENMASK(4, 0), (x))
+
+/* W6_CMD_SET_FB_FBC_Y(x)
+ * Address range: 0x310, 0x328, ..., 0x478 (step = 0x18)
+ */
+#define W6_CMD_SET_FB_FBC_Y(x) (W6_REG_BASE + 0x310 + ((x) * 0x18))
+
+/* W6_CMD_SET_FB_FBC_C(x)
+ * Address range: 0x314, 0x32C, ..., 0x47C (step = 0x18)
+ */
+#define W6_CMD_SET_FB_FBC_C(x) (W6_REG_BASE + 0x314 + ((x) * 0x18))
+
+/* W6_CMD_SET_FB_FBC_Y_OFFSET(x)
+ * Address range: 0x318, 0x330, ..., 0x480 (step = 0x18)
+ */
+#define W6_CMD_SET_FB_FBC_Y_OFFSET(x) (W6_REG_BASE + 0x318 + ((x) * 0x18))
+
+/* W6_CMD_SET_FB_FBC_C_OFFSET(x)
+ * Address range: 0x31C, 0x334, ..., 0x484 (step = 0x18)
+ */
+#define W6_CMD_SET_FB_FBC_C_OFFSET(x) (W6_REG_BASE + 0x31C + ((x) * 0x18))
+
+/* W6_CMD_SET_FB_MV_COL(x)
+ * Address range: 0x320, 0x338, ..., 0x488 (step = 0x18)
+ */
+#define W6_CMD_SET_FB_MV_COL(x) (W6_REG_BASE + 0x320 + ((x) * 0x18))
+
+#define W6_CMD_SET_FB_DEFAULT_CDF (W6_REG_BASE + 0x494)
+#define W6_CMD_SET_FB_SEGMAP (W6_REG_BASE + 0x498)
+#define W6_CMD_SET_FB_MV_COL_PRE_ENT (W6_REG_BASE + 0x4DC)
+
+/* W6_CMD_SET_FB_FBC_CR(x)
+ * Address range: 0x4F0, 0x4F8, ..., 0x568 (step = 0x8)
+ */
+#define W6_CMD_SET_FB_FBC_CR(x) (W6_REG_BASE + 0x4F0 + ((x) * 0x8))
+
+/* W6_CMD_SET_FB_FBC_CR_OFFSET(x)
+ * Address range: 0x4F4, 0x4FC, ..., 0x56C (step = 0x8)
+ */
+#define W6_CMD_SET_FB_FBC_CR_OFFSET(x) (W6_REG_BASE + 0x4F4 + ((x) * 0x8))
+
+/* DECODER - INIT_SEQ */
+#define W6_CMD_DEC_INIT_SEQ_OPTION (W6_REG_BASE + 0x204)
+#define INIT_SEQ_OPTION_MODE(x) FIELD_PREP(GENMASK(5, 0), (x))
+#define W6_CMD_DEC_INIT_SEQ_BS_RD_PTR (W6_REG_BASE + 0x300)
+#define W6_CMD_DEC_INIT_SEQ_BS_WR_PTR (W6_REG_BASE + 0x304)
+#define W6_CMD_DEC_INIT_SEQ_BS_OPTION (W6_REG_BASE + 0x308)
+
+/* DECODER - SET_DISP */
+#define W6_CMD_DEC_SET_DISP_OPTION (W6_REG_BASE + 0x204)
+#define SET_DISP_OPTION_ENDIAN(x) FIELD_PREP(GENMASK(19, 16), (x))
+#define W6_CMD_DEC_SET_DISP_COMMON_PIC_INFO (W6_REG_BASE + 0x300)
+#define SET_DISP_COMMON_PIC_INFO_BWB_ON BIT(28)
+#define SET_DISP_COMMON_PIC_INFO_C_FMT_IDC(x) FIELD_PREP(GENMASK(25, 24), (x))
+#define SET_DISP_COMMON_PIC_INFO_PIXEL_ORDER(x) FIELD_PREP(GENMASK(23, 23), (x))
+#define SET_DISP_COMMON_PIC_INFO_OUT_JUSTIFY(x) FIELD_PREP(GENMASK(22, 22), (x))
+#define SET_DISP_COMMON_PIC_INFO_OUT_PIXEL(x) FIELD_PREP(GENMASK(21, 20), (x))
+#define SET_DISP_COMMON_PIC_INFO_CBCR_ORDER(x) FIELD_PREP(GENMASK(17, 17), (x))
+#define SET_DISP_COMMON_PIC_INFO_2PLANE(x) FIELD_PREP(GENMASK(16, 16), (x))
+#define SET_DISP_COMMON_PIC_INFO_STRIDE(x) FIELD_PREP(GENMASK(15, 0), (x))
+#define W6_CMD_DEC_SET_DISP_PIC_SIZE (W6_REG_BASE + 0x304)
+#define SET_DISP_PIC_SIZE_WIDTH(x) FIELD_PREP(GENMASK(31, 16), (x))
+#define SET_DISP_PIC_SIZE_HEIGHT(x) FIELD_PREP(GENMASK(15, 0), (x))
+#define W6_CMD_DEC_SET_DISP_PIC_INFO (W6_REG_BASE + 0x308)
+#define SET_DISP_PIC_INFO_L_BIT_DEPTH(x) FIELD_PREP(GENMASK(25, 22), (x))
+#define SET_DISP_PIC_INFO_C_BIT_DEPTH(x) FIELD_PREP(GENMASK(21, 18), (x))
+#define SET_DISP_PIC_INFO_C_FMT_IDC(x) FIELD_PREP(GENMASK(17, 16), (x))
+#define W6_CMD_DEC_SET_DISP_Y_BASE (W6_REG_BASE + 0x30C)
+#define W6_CMD_DEC_SET_DISP_CB_BASE (W6_REG_BASE + 0x310)
+#define W6_CMD_DEC_SET_DISP_CR_BASE (W6_REG_BASE + 0x314)
+#define W6_CMD_DEC_SET_DISP_SCL_PARAM (W6_REG_BASE + 0x318)
+#define W6_CMD_DEC_SET_DISP_SCL_PIC_SIZE (W6_REG_BASE + 0x31C)
+#define SET_DISP_SCL_PIC_SIZE_WIDTH(x) FIELD_PREP(GENMASK(31, 16), (x))
+#define SET_DISP_SCL_PIC_SIZE_HEIGHT(x) FIELD_PREP(GENMASK(15, 0), (x))
+
+/* DECODER - DEC_PIC */
+#define W6_CMD_DEC_PIC_OPTION (W6_REG_BASE + 0x204)
+#define DEC_PIC_OPTION_MODE(x) FIELD_PREP(GENMASK(5, 0), (x))
+#define W6_CMD_DEC_PIC_BS_RD_PTR (W6_REG_BASE + 0x300)
+#define W6_CMD_DEC_PIC_BS_WR_PTR (W6_REG_BASE + 0x304)
+#define W6_CMD_DEC_PIC_BS_OPTION (W6_REG_BASE + 0x308)
+#define DEC_PIC_BS_OPTION_STREAM_END(x) FIELD_PREP(GENMASK(1, 1), (x))
+#define DEC_PIC_BS_OPTION_EXPLICIT_END_ON BIT(0)
+#define W6_CMD_DEC_PIC_SEC_AXI (W6_REG_BASE + 0x30C)
+#define DEC_PIC_SEC_AXI_IP_ENABLE(x) FIELD_PREP(GENMASK(1, 1), (x))
+#define DEC_PIC_SEC_AXI_LF_ENABLE(x) FIELD_PREP(GENMASK(0, 0), (x))
+#define W6_CMD_DEC_PIC_SEQ_CHANGE_ENABLE_FLAG (W6_REG_BASE + 0x310)
+#define W6_CMD_DEC_PIC_TEMPORAL_ID_PLUS1 (W6_REG_BASE + 0x318)
+#define DEC_PIC_TEMPORAL_ID_PLUS1_SPATIAL(x) FIELD_PREP(GENMASK(11, 9), (x))
+#define DEC_PIC_TEMPORAL_ID_PLUS1(x) FIELD_PREP(GENMASK(7, 0), (x))
+#define W6_CMD_DEC_PIC_TIMESTAMP_LOW (W6_REG_BASE + 0x354)
+#define W6_CMD_DEC_PIC_TIMESTAMP_HIGH (W6_REG_BASE + 0x358)
+
+/* DECODER - QUERY : GET_RESULT */
+#define W6_RET_DEC_BS_RD_PTR (W6_REG_BASE + 0x30C)
+#define W6_RET_DEC_SEQ_PARAM (W6_REG_BASE + 0x310)
+#define DEC_SEQ_PARAM_TIER(x) FIELD_GET(GENMASK(29, 29), (x))
+#define DEC_SEQ_PARAM_PROFILE(x) FIELD_GET(GENMASK(28, 24), (x))
+#define PROFILE_H264_BP 2
+#define PROFILE_H264_HP 4
+#define PROFILE_H264_MP 13
+#define PROFILE_H264_HIGH10 14
+#define PROFILE_H264_EXTENDED 24
+#define DEC_SEQ_PARAM_PROFILE_COMPATIBILITY(x) FIELD_GET(GENMASK(19, 12), (x))
+#define PROFILE_COMPATIBILITY_MAIN BIT(1)
+#define PROFILE_COMPATIBILITY_MAIN10 BIT(2)
+#define PROFILE_COMPATIBILITY_STILL_PICTURE BIT(3)
+#define DEC_SEQ_PARAM_LEVEL(x) FIELD_GET(GENMASK(7, 0), (x))
+#define W6_RET_DEC_COLOR_SAMPLE (W6_REG_BASE + 0x314)
+#define DEC_COLOR_SAMPLE_ASPECT_RATIO_IDC(x) FIELD_GET(GENMASK(23, 16), (x))
+#define DEC_COLOR_SAMPLE_C_FMT_IDC(x) FIELD_GET(GENMASK(11, 8), (x))
+#define DEC_COLOR_SAMPLE_C_BIT_DEPTH(x) FIELD_GET(GENMASK(7, 4), (x))
+#define DEC_COLOR_SAMPLE_L_BIT_DEPTH(x) FIELD_GET(GENMASK(3, 0), (x))
+#define W6_RET_DEC_ASPECT_RATIO (W6_REG_BASE + 0x318)
+#define W6_RET_DEC_BIT_RATE (W6_REG_BASE + 0x31C)
+#define W6_RET_DEC_FRAME_RATE_NR (W6_REG_BASE + 0x320)
+#define W6_RET_DEC_FRAME_RATE_DR (W6_REG_BASE + 0x324)
+#define W6_RET_DEC_NUM_REQUIRED_FBC_FB (W6_REG_BASE + 0x328)
+#define W6_RET_DEC_NUM_REORDER_DELAY (W6_REG_BASE + 0x32C)
+#define W6_RET_DEC_NOTIFICATION (W6_REG_BASE + 0x334)
+#define W6_RET_DEC_PIC_SIZE (W6_REG_BASE + 0x33C)
+#define DEC_PIC_SIZE_WIDTH(x) FIELD_GET(GENMASK(31, 16), (x))
+#define DEC_PIC_SIZE_HEIGHT(x) FIELD_GET(GENMASK(15, 0), (x))
+#define W6_RET_DEC_CROP_TOP_BOTTOM (W6_REG_BASE + 0x340)
+#define DEC_CROP_TOP(x) FIELD_GET(GENMASK(31, 16), (x))
+#define DEC_CROP_BOTTOM(x) FIELD_GET(GENMASK(15, 0), (x))
+#define W6_RET_DEC_CROP_LEFT_RIGHT (W6_REG_BASE + 0x344)
+#define DEC_CROP_LEFT(x) FIELD_GET(GENMASK(31, 16), (x))
+#define DEC_CROP_RIGHT(x) FIELD_GET(GENMASK(15, 0), (x))
+#define W6_RET_DEC_AU_START_POS (W6_REG_BASE + 0x348)
+#define W6_RET_DEC_AU_END_POS (W6_REG_BASE + 0x34C)
+#define W6_RET_DEC_PIC_TYPE (W6_REG_BASE + 0x350)
+#define DEC_PIC_TYPE_CTU_SIZE(x) (16 << FIELD_GET(GENMASK(11, 10), (x)))
+#define DEC_PIC_TYPE_NAL_UNIT_TYPE(x) FIELD_GET(GENMASK(9, 4), (x))
+#define DEC_PIC_TYPE_B BIT(2)
+#define DEC_PIC_TYPE_P BIT(1)
+#define DEC_PIC_TYPE_I BIT(0)
+#define W6_RET_DEC_PIC_POC (W6_REG_BASE + 0x354)
+#define W6_RET_DEC_RECOVERY_POINT (W6_REG_BASE + 0x358)
+#define W6_RET_DEC_DECODED_ADDR (W6_REG_BASE + 0x360)
+#define W6_RET_DEC_DISPLAY_ADDR (W6_REG_BASE + 0x364)
+#define W6_RET_DEC_ERR_CTB_NUM (W6_REG_BASE + 0x370)
+#define W6_RET_DEC_DISPLAY_FLAG (W6_REG_BASE + 0x3A8)
+#define W6_RET_DEC_RELEASE_IDC (W6_REG_BASE + 0x3AC)
+#define W6_RET_DEC_DISP_IDC (W6_REG_BASE + 0x3B0)
+#define W6_RET_DEC_STREAM_END (W6_REG_BASE + 0x3C0)
+#define W6_RET_DEC_DECODED_FLAG (W6_REG_BASE + 0x3C4)
+#define W6_RET_DEC_WARN_INFO (W6_REG_BASE + 0x3CC)
+#define W6_RET_DEC_ERR_INFO (W6_REG_BASE + 0x3D0)
+#define W6_RET_DEC_DECODING_SUCCESS (W6_REG_BASE + 0x3D4)
+#define W6_RET_DEC_TIMESTAMP_LOW (W6_REG_BASE + 0x3D8)
+#define W6_RET_DEC_TIMESTAMP_HIGH (W6_REG_BASE + 0x3DC)
+#define W6_RET_DEC_LAST_FRAME_FLAG (W6_REG_BASE + 0x3E0)
+#define W6_RET_DEC_NUM_REQUIRED_COL_BUF (W6_REG_BASE + 0x3E4)
+
+/* W6_RET_DEC_DISP_LINEAR_ADDR(x)
+ * Address range: 0x3E8, 0x3EC, ..., 0x460 (step = 0x4)
+ */
+#define W6_RET_DEC_DISP_LINEAR_ADDR(x) (W6_REG_BASE + 0x3E8 + ((x) * 0x4))
+#define W6_RET_DEC_COLOR_CONFIG (W6_REG_BASE + 0x57C)
+#define DEC_COLOR_CONFIG_COLOR_RANGE(x) FIELD_GET(GENMASK(25, 25), (x))
+#define DEC_COLOR_CONFIG_MATRIX_COEFF(x) FIELD_GET(GENMASK(24, 17), (x))
+#define DEC_COLOR_CONFIG_TRANS_CHAR(x) FIELD_GET(GENMASK(16, 9), (x))
+#define DEC_COLOR_CONFIG_COLOR_PRIMARIES(x) FIELD_GET(GENMASK(8, 1), (x))
+#define DEC_COLOR_CONFIG_COLOR_PRESENT(x) FIELD_GET(GENMASK(0, 0), (x))
+
+/* DECODER - QUERY : GET_FLUSH_CMD_INFO */
+
+/* W6_RET_DEC_FLUSH_CMD_DISP_ADDR(x)
+ * Address range: 0x300, 0x304, ..., 0x378 (step = 0x4)
+ */
+#define W6_RET_DEC_FLUSH_CMD_DISP_ADDR(x) (W6_REG_BASE + 0x300 + ((x) * 0x4))
+#define W6_RET_DEC_FLUSH_CMD_BUF_STATE_UNUSED_IDC (W6_REG_BASE + 0x57C)
+#define W6_RET_DEC_FLUSH_CMD_BUF_STATE_USED_IDC (W6_REG_BASE + 0x580)
+#define W6_RET_DEC_FLUSH_CMD_BUF_STATE_USING_IDC (W6_REG_BASE + 0x584)
+
#endif /* __WAVE6_REGDEFINE_H__ */
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.c b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.c
new file mode 100644
index 000000000000..c8d1138334b3
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.c
@@ -0,0 +1,202 @@
+// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause)
+/*
+ * Wave6 series multi-standard codec IP - wave6 helper interface
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#include <linux/bug.h>
+
+#include "wave6-vpuapi.h"
+#include "wave6-hw.h"
+
+int wave6_vpu_dec_open(struct vpu_instance *inst, struct dec_open_param *pop)
+{
+ struct dec_info *p_dec_info;
+ int ret;
+ struct vpu_core_device *core = inst->dev;
+
+ guard(mutex)(&core->hw_lock);
+
+ if (!wave6_vpu_is_init(core))
+ return -ENODEV;
+
+ inst->codec_info = kzalloc_obj(*inst->codec_info);
+ if (!inst->codec_info)
+ return -ENOMEM;
+
+ p_dec_info = &inst->codec_info->dec_info;
+ memcpy(&p_dec_info->open_param, pop, sizeof(struct dec_open_param));
+
+ ret = wave6_vpu_build_up_dec_param(inst, pop);
+ if (ret) {
+ kfree(inst->codec_info);
+ inst->codec_info = NULL;
+ }
+
+ return ret;
+}
+
+int wave6_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res)
+{
+ int ret;
+
+ if (WARN_ON(!inst->codec_info))
+ return -EINVAL;
+
+ guard(mutex)(&inst->dev->hw_lock);
+
+ ret = wave6_vpu_dec_fini_seq(inst, fail_res);
+ if (ret)
+ dev_warn(inst->dev->dev, "dec seq end timed out\n");
+
+ kfree(inst->codec_info);
+ inst->codec_info = NULL;
+
+ return ret;
+}
+
+int wave6_vpu_dec_complete_seq_init(struct vpu_instance *inst,
+ struct dec_seq_info *info)
+{
+ struct dec_info *p_dec_info = &inst->codec_info->dec_info;
+ int ret;
+
+ guard(mutex)(&inst->dev->hw_lock);
+
+ ret = wave6_vpu_dec_get_seq_info(inst, info);
+ if (!ret)
+ p_dec_info->seq_info_obtained = true;
+
+ info->rd_ptr = wave6_vpu_dec_get_rd_ptr(inst);
+ info->wr_ptr = p_dec_info->stream_wr_ptr;
+
+ p_dec_info->seq_info = *info;
+
+ return ret;
+}
+
+int wave6_vpu_dec_get_aux_buffer_size(struct vpu_instance *inst,
+ struct aux_buffer_size_info info, u32 *size)
+{
+ struct dec_info *p_dec_info = &inst->codec_info->dec_info;
+ int width = info.width;
+ int height = info.height;
+ int buf_size, twice;
+
+ if (info.type == AUX_BUF_FBC_Y_TBL) {
+ switch (inst->std) {
+ case W_HEVC_DEC:
+ case W_AVC_DEC:
+ buf_size = WAVE6_FBC_LUMA_TABLE_SIZE(width, height);
+ break;
+ default:
+ return -EINVAL;
+ }
+ buf_size = ALIGN(buf_size, 16);
+ } else if (info.type == AUX_BUF_FBC_C_TBL) {
+ if (p_dec_info->seq_info.c_fmt_idc == C_FMT_IDC_YUV422)
+ twice = 2;
+ else if (p_dec_info->seq_info.c_fmt_idc == C_FMT_IDC_YUV444)
+ twice = 4;
+ else
+ twice = 1;
+
+ switch (inst->std) {
+ case W_HEVC_DEC:
+ case W_AVC_DEC:
+ buf_size = WAVE6_FBC_CHROMA_TABLE_SIZE(width, height);
+ break;
+ default:
+ return -EINVAL;
+ }
+ buf_size = buf_size * twice;
+ buf_size = ALIGN(buf_size, 16);
+ } else if (info.type == AUX_BUF_MV_COL) {
+ switch (inst->std) {
+ case W_HEVC_DEC:
+ buf_size = WAVE6_DEC_HEVC_MVCOL_BUF_SIZE(width, height);
+ break;
+ case W_AVC_DEC:
+ buf_size = WAVE6_DEC_AVC_MVCOL_BUF_SIZE(width, height);
+ break;
+ default:
+ return -EINVAL;
+ }
+ buf_size = ALIGN(buf_size, 16);
+ } else {
+ return -EINVAL;
+ }
+
+ *size = buf_size;
+
+ return 0;
+}
+
+void wave6_vpu_dec_get_bitstream_buffer(struct vpu_instance *inst,
+ dma_addr_t *p_rd_ptr,
+ dma_addr_t *p_wr_ptr)
+{
+ struct dec_info *p_dec_info = &inst->codec_info->dec_info;
+ dma_addr_t rd_ptr;
+ dma_addr_t wr_ptr;
+
+ guard(mutex)(&inst->dev->hw_lock);
+
+ rd_ptr = wave6_vpu_dec_get_rd_ptr(inst);
+ wr_ptr = p_dec_info->stream_wr_ptr;
+
+ if (p_rd_ptr)
+ *p_rd_ptr = rd_ptr;
+ if (p_wr_ptr)
+ *p_wr_ptr = wr_ptr;
+}
+
+int wave6_vpu_dec_update_bitstream_buffer(struct vpu_instance *inst, int size)
+{
+ struct dec_info *p_dec_info = &inst->codec_info->dec_info;
+ dma_addr_t wr_ptr;
+ dma_addr_t rd_ptr;
+
+ wr_ptr = p_dec_info->stream_wr_ptr;
+ rd_ptr = p_dec_info->stream_rd_ptr;
+
+ if (size > 0) {
+ if (wr_ptr < rd_ptr && rd_ptr <= wr_ptr + size)
+ return -EINVAL;
+
+ wr_ptr += size;
+
+ p_dec_info->stream_wr_ptr = wr_ptr;
+ p_dec_info->stream_rd_ptr = rd_ptr;
+ }
+
+ p_dec_info->stream_end = size == 0;
+
+ return 0;
+}
+
+void wave6_vpu_dec_set_rd_ptr(struct vpu_instance *inst, dma_addr_t addr,
+ bool update_wr_ptr)
+{
+ struct dec_info *p_dec_info = &inst->codec_info->dec_info;
+
+ p_dec_info->stream_rd_ptr = addr;
+ if (update_wr_ptr)
+ p_dec_info->stream_wr_ptr = addr;
+}
+
+void wave6_vpu_dec_reset_frame_buffer_info(struct vpu_instance *inst)
+{
+ struct dec_info *p_dec_info = &inst->codec_info->dec_info;
+
+ for (int i = 0; i < WAVE6_MAX_FBS; i++) {
+ wave6_vdi_free_dma(&inst->frame_vbuf[i]);
+ memset(&inst->frame_buf[i], 0, sizeof(struct frame_buffer));
+ memset(&p_dec_info->disp_buf[i], 0, sizeof(struct frame_buffer));
+
+ wave6_vdi_free_dma(&inst->aux_vbuf[AUX_BUF_MV_COL][i]);
+ wave6_vdi_free_dma(&inst->aux_vbuf[AUX_BUF_FBC_Y_TBL][i]);
+ wave6_vdi_free_dma(&inst->aux_vbuf[AUX_BUF_FBC_C_TBL][i]);
+ }
+}
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
index 62dd4969afaf..7d6a734f64b4 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
@@ -14,6 +14,7 @@
#include <media/v4l2-mem2mem.h>
#include <media/v4l2-ctrls.h>
+#include "wave6-vpuerror.h"
#include "wave6-vpuconfig.h"
#include "wave6-vdi.h"
#include "wave6-vpu.h"
@@ -47,6 +48,15 @@ enum vpu_instance_state {
#define WAVE6_MAX_FBS 31
+#define WAVE6_DEC_HEVC_MVCOL_BUF_SIZE(_w, _h) \
+ ((ALIGN((_w), 256) / 16) * (ALIGN((_h), 64) / 16) * 1 * 16)
+#define WAVE6_DEC_AVC_MVCOL_BUF_SIZE(_w, _h) \
+ ((ALIGN((_w), 64) / 16) * (ALIGN((_h), 16) / 16) * 5 * 16)
+#define WAVE6_FBC_LUMA_TABLE_SIZE(_w, _h) \
+ (ALIGN((_w), 256) * ALIGN((_h), 64) / 32)
+#define WAVE6_FBC_CHROMA_TABLE_SIZE(_w, _h) \
+ (ALIGN(((_w) / 2), 256) * ALIGN((_h), 64) / 32)
+
enum codec_std {
W_HEVC_DEC = 0x00,
W_HEVC_ENC = 0x01,
@@ -55,6 +65,36 @@ enum codec_std {
STD_UNKNOWN = 0xFF
};
+#define HEVC_PROFILE_MAIN 1
+#define HEVC_PROFILE_MAIN10 2
+#define HEVC_PROFILE_STILLPICTURE 3
+#define HEVC_PROFILE_MAIN10_STILLPICTURE 2
+
+#define H264_PROFILE_BP 1
+#define H264_PROFILE_MP 2
+#define H264_PROFILE_EXTENDED 3
+#define H264_PROFILE_HP 4
+#define H264_PROFILE_HIGH10 5
+
+#define H264_VUI_SAR_IDC_EXTENDED 255
+
+#define SEQ_CHANGE_ENABLE_PROFILE BIT(5)
+#define SEQ_CHANGE_ENABLE_SIZE BIT(16)
+#define SEQ_CHANGE_ENABLE_BIT_DEPTH BIT(18)
+#define SEQ_CHANGE_ENABLE_DPB_COUNT BIT(19)
+
+#define SEQ_CHANGE_ENABLE_ALL_HEVC (SEQ_CHANGE_ENABLE_PROFILE | \
+ SEQ_CHANGE_ENABLE_SIZE | \
+ SEQ_CHANGE_ENABLE_BIT_DEPTH | \
+ SEQ_CHANGE_ENABLE_DPB_COUNT)
+
+#define SEQ_CHANGE_ENABLE_ALL_AVC (SEQ_CHANGE_ENABLE_SIZE | \
+ SEQ_CHANGE_ENABLE_BIT_DEPTH | \
+ SEQ_CHANGE_ENABLE_DPB_COUNT)
+
+#define DEC_NOTI_FLAG_NO_FB BIT(1)
+#define DEC_NOTI_FLAG_SEQ_CHANGE BIT(0)
+
#define FW_VERSION_MAJOR_MASK 0xFF000000
#define FW_VERSION_MINOR_MASK 0x00FF0000
#define FW_VERSION_REL_MASK 0x0000FFFF
@@ -208,12 +248,116 @@ struct instance_buffer {
u32 temp_size;
};
+struct color_param {
+ u32 color_range;
+ u32 matrix_coefficients;
+ u32 transfer_characteristics;
+ u32 color_primaries;
+ bool color_description_present;
+ bool video_signal_type_present;
+};
+
+struct sec_axi_info {
+ bool use_dec_ip;
+ bool use_dec_lf_row;
+};
+
+struct aux_buffer_size_info {
+ int width;
+ int height;
+ enum aux_buffer_type type;
+};
+
struct dec_scaler_info {
bool enable;
int width;
int height;
};
+struct dec_open_param {
+ enum display_mode disp_mode;
+ u32 ext_addr_vcpu: 8;
+ u32 is_secure_inst: 1;
+ u32 inst_priority: 5;
+ struct instance_buffer inst_buffer;
+};
+
+struct dec_seq_info {
+ u32 pic_width;
+ u32 pic_height;
+ u32 f_rate_numerator;
+ u32 f_rate_denominator;
+ struct vpu_rect pic_crop_rect;
+ u32 min_frame_buffer_count: 5;
+ u32 req_mv_buffer_count: 5;
+ u32 frame_buf_delay;
+ u32 profile;
+ u32 level;
+ u32 tier;
+ bool is_ext_sar;
+ u32 aspect_rate_info;
+ u32 bitrate;
+ enum chroma_format_idc c_fmt_idc;
+ u32 luma_bit_depth;
+ u32 chroma_bit_depth;
+ u32 err_reason;
+ int warn_info;
+ dma_addr_t rd_ptr;
+ dma_addr_t wr_ptr;
+ unsigned int sequence_no;
+ struct color_param color;
+};
+
+struct dec_param {
+ u64 timestamp;
+};
+
+struct dec_output_info {
+ int nal_type;
+ int pic_type;
+ int disp_pic_width;
+ int disp_pic_height;
+ int dec_pic_width;
+ int dec_pic_height;
+ int decoded_poc;
+ int display_poc;
+ dma_addr_t rd_ptr;
+ dma_addr_t wr_ptr;
+ dma_addr_t byte_pos_frame_start;
+ dma_addr_t byte_pos_frame_end;
+ dma_addr_t frame_decoded_addr;
+ dma_addr_t frame_display_addr;
+ int error_reason;
+ int warn_info;
+ unsigned int sequence_no;
+ dma_addr_t release_disp_frame_addr[WAVE6_MAX_FBS];
+ dma_addr_t disp_frame_addr[WAVE6_MAX_FBS];
+ u64 timestamp;
+ u32 notification_flags;
+ u32 release_disp_frame_num: 5;
+ u32 disp_frame_num: 5;
+ u32 ctu_size: 2;
+ bool frame_display;
+ bool frame_decoded;
+ bool stream_end;
+ bool last_frame_in_au;
+ bool decoding_success;
+};
+
+struct dec_info {
+ struct dec_open_param open_param;
+ struct dec_seq_info seq_info;
+ dma_addr_t stream_wr_ptr;
+ dma_addr_t stream_rd_ptr;
+ bool stream_end;
+ struct frame_buffer disp_buf[WAVE6_MAX_FBS];
+ bool seq_info_obtained;
+ struct sec_axi_info sec_axi;
+ struct dec_output_info dec_out_info[WAVE6_MAX_FBS];
+ int seq_change_mask;
+ enum frame_buffer_format out_fmt;
+};
+
/**
* struct vpu_irq - VPU interrupt information
* @status: Value of the VPU interrupt status
@@ -306,6 +450,7 @@ struct vpu_instance_ops {
* @ops: Instance operations
* @std: Codec standard
* @id: Instance id
+ * @codec_info: Codec-specific information
* @frame_buf: Metadata for reference frame buffers
* @frame_vbuf: Reference frame buffers
* @aux_vbuf: Auxiliary buffers
@@ -355,6 +500,9 @@ struct vpu_instance {
enum codec_std std;
u32 id;
+ union {
+ struct dec_info dec_info;
+ } *codec_info;
struct frame_buffer frame_buf[WAVE6_MAX_FBS];
struct vpu_buf frame_vbuf[WAVE6_MAX_FBS];
struct vpu_buf aux_vbuf[AUX_BUF_TYPE_MAX][WAVE6_MAX_FBS];
@@ -378,4 +526,24 @@ void wave6_vpu_writel_relaxed(struct vpu_core_device *core, u32 addr, u32 data);
u32 wave6_vpu_readl(struct vpu_core_device *core, u32 addr);
u32 wave6_vpu_readl_relaxed(struct vpu_core_device *core, u32 addr);
+int wave6_vpu_dec_open(struct vpu_instance *inst, struct dec_open_param *pop);
+int wave6_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res);
+int wave6_vpu_dec_issue_seq_init(struct vpu_instance *inst);
+int wave6_vpu_dec_complete_seq_init(struct vpu_instance *inst, struct dec_seq_info *info);
+int wave6_vpu_dec_get_aux_buffer_size(struct vpu_instance *inst,
+ struct aux_buffer_size_info info, u32 *size);
+int wave6_vpu_dec_register_frame_buffer(struct vpu_instance *inst);
+int wave6_vpu_dec_register_display_buffer(struct vpu_instance *inst, struct frame_buffer fb);
+int wave6_vpu_dec_start_one_frame(struct vpu_instance *inst, struct dec_param *param,
+ u32 *fail_res);
+int wave6_vpu_dec_get_output_info(struct vpu_instance *inst, struct dec_output_info *info);
+void wave6_vpu_dec_set_rd_ptr(struct vpu_instance *inst, dma_addr_t addr,
+ bool update_wr_ptr);
+void wave6_vpu_dec_reset_frame_buffer_info(struct vpu_instance *inst);
+void wave6_vpu_dec_get_bitstream_buffer(struct vpu_instance *inst,
+ dma_addr_t *p_rd_ptr,
+ dma_addr_t *p_wr_ptr);
+int wave6_vpu_dec_update_bitstream_buffer(struct vpu_instance *inst, int size);
+int wave6_vpu_dec_flush_instance(struct vpu_instance *inst);
+
#endif /* __WAVE6_VPUAPI_H__ */
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h b/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
index fccb854a8a8c..d1b53f509dfa 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
@@ -25,12 +25,17 @@
#define W6_VPU_POLL_TIMEOUT 300000
#define W6_BOOT_WAIT_TIMEOUT 10000
#define W6_VPU_TIMEOUT 6000
+#define W6_VPU_TIMEOUT_CYCLE_COUNT (8000000 * 4 * 4)
+
+#define HOST_ENDIAN VDI_128BIT_LITTLE_ENDIAN
+#define VPU_STREAM_ENDIAN HOST_ENDIAN
#define W6_MAX_REMAP_PAGE_SIZE (1024 * 1024)
#define W6_MAX_CODE_BUF_SIZE (4 * 1024 * 1024)
#define W6_CODE_BUF_SIZE (1 * 1024 * 1024)
#define W6_EXTRA_CODE_BUF_SIZE (256 * 1024)
#define W6_TEMPBUF_SIZE (3 * 1024 * 1024)
+#define W637DEC_WORKBUF_SIZE (2 * 512 * 1024)
#define W637DEC_WORKBUF_SIZE_FOR_CQ (3 * 512 * 1024)
static inline bool wave6_is_product_w_series(u32 code)
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuerror.h b/drivers/media/platform/chips-media/wave6/wave6-vpuerror.h
new file mode 100644
index 000000000000..152a2617182c
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpuerror.h
@@ -0,0 +1,252 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
+/*
+ * Wave6 series multi-standard codec IP - wave6 vpu error values
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#ifndef __WAVE6_VPUERROR_H__
+#define __WAVE6_VPUERROR_H__
+
+#include <linux/bits.h>
+
+/* WAVE6 COMMON SYSTEM ERROR (FAIL_REASON) */
+#define WAVE6_SYSERR_QUEUEING_FAIL BIT(0)
+#define WAVE6_SYSERR_DECODER_FUSE BIT(1)
+#define WAVE6_SYSERR_INSTRUCTION_ACCESS_VIOLATION BIT(2)
+#define WAVE6_SYSERR_PRIVILEGE_VIOLATION BIT(3)
+#define WAVE6_SYSERR_DATA_ADDR_ALIGNMENT BIT(4)
+#define WAVE6_SYSERR_DATA_ACCESS_VIOLATION BIT(5)
+#define WAVE6_SYSERR_ACCESS_VIOLATION_HW BIT(6)
+#define WAVE6_SYSERR_INSTRUCTION_ADDR_ALIGNMENT BIT(7)
+#define WAVE6_SYSERR_UNKNOWN BIT(8)
+#define WAVE6_SYSERR_BUS_ERROR BIT(9)
+#define WAVE6_SYSERR_DOUBLE_FAULT BIT(10)
+#define WAVE6_SYSERR_RESULT_NOT_READY BIT(11)
+#define WAVE6_SYSERR_VPU_STILL_RUNNING BIT(12)
+#define WAVE6_SYSERR_UNKNOWN_CMD BIT(13)
+#define WAVE6_SYSERR_UNKNOWN_CODEC_STD BIT(14)
+#define WAVE6_SYSERR_UNKNOWN_QUERY_OPTION BIT(15)
+#define WAVE6_SYSERR_WATCHDOG_TIMEOUT BIT(17)
+#define WAVE6_SYSERR_NOT_SUPPORT BIT(20)
+#define WAVE6_SYSERR_TEMP_SEC_BUF_OVERFLOW BIT(21)
+#define WAVE6_SYSERR_NOT_SUPPORT_PROFILE BIT(22)
+#define WAVE6_SYSERR_TIMEOUT_CODEC_FW BIT(30)
+
+/* WAVE6 ERROR ON DECODER (ERR_INFO) */
+#define HEVC_SPSERR_SEQ_PARAMETER_SET_ID 0x00001000
+#define HEVC_SPSERR_CHROMA_FORMAT_IDC 0x00001001
+#define HEVC_SPSERR_PIC_WIDTH_IN_LUMA_SAMPLES 0x00001002
+#define HEVC_SPSERR_PIC_HEIGHT_IN_LUMA_SAMPLES 0x00001003
+#define HEVC_SPSERR_CONF_WIN_LEFT_OFFSET 0x00001004
+#define HEVC_SPSERR_CONF_WIN_RIGHT_OFFSET 0x00001005
+#define HEVC_SPSERR_CONF_WIN_TOP_OFFSET 0x00001006
+#define HEVC_SPSERR_CONF_WIN_BOTTOM_OFFSET 0x00001007
+#define HEVC_SPSERR_BIT_DEPTH_LUMA_MINUS8 0x00001008
+#define HEVC_SPSERR_BIT_DEPTH_CHROMA_MINUS8 0x00001009
+#define HEVC_SPSERR_LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4 0x0000100A
+#define HEVC_SPSERR_SPS_MAX_DEC_PIC_BUFFERING 0x0000100B
+#define HEVC_SPSERR_SPS_MAX_NUM_REORDER_PICS 0x0000100C
+#define HEVC_SPSERR_SPS_MAX_LATENCY_INCREASE 0x0000100D
+#define HEVC_SPSERR_LOG2_MIN_LUMA_CODING_BLOCK_SIZE_MINUS3 0x0000100E
+#define HEVC_SPSERR_LOG2_DIFF_MAX_MIN_LUMA_CODING_BLOCK_SIZE 0x0000100F
+#define HEVC_SPSERR_LOG2_MIN_TRANSFORM_BLOCK_SIZE_MINUS2 0x00001010
+#define HEVC_SPSERR_LOG2_DIFF_MAX_MIN_TRANSFORM_BLOCK_SIZE 0x00001011
+#define HEVC_SPSERR_MAX_TRANSFORM_HIERARCHY_DEPTH_INTER 0x00001012
+#define HEVC_SPSERR_MAX_TRANSFORM_HIERARCHY_DEPTH_INTRA 0x00001013
+#define HEVC_SPSERR_SCALING_LIST 0x00001014
+#define HEVC_SPSERR_LOG2_DIFF_MIN_PCM_LUMA_CODING_BLOCK_SIZE_MINUS3 0x00001015
+#define HEVC_SPSERR_LOG2_DIFF_MAX_MIN_PCM_LUMA_CODING_BLOCK_SIZE 0x00001016
+#define HEVC_SPSERR_NUM_SHORTTERM_REF_PIC_SETS 0x00001017
+#define HEVC_SPSERR_NUM_LONGTERM_REF_PICS_SPS 0x00001018
+#define HEVC_SPSERR_GBU_PARSING_ERROR 0x00001019
+#define HEVC_SPSERR_EXTENSION_FLAG 0x0000101A
+#define HEVC_SPSERR_VUI_ERROR 0x0000101B
+#define HEVC_SPSERR_ACTIVATE_SPS 0x0000101C
+#define HEVC_SPSERR_PROFILE_SPACE 0x0000101D
+#define HEVC_PPSERR_PPS_PIC_PARAMETER_SET_ID 0x00002000
+#define HEVC_PPSERR_PPS_SEQ_PARAMETER_SET_ID 0x00002001
+#define HEVC_PPSERR_NUM_REF_IDX_L0_DEFAULT_ACTIVE_MINUS1 0x00002002
+#define HEVC_PPSERR_NUM_REF_IDX_L1_DEFAULT_ACTIVE_MINUS1 0x00002003
+#define HEVC_PPSERR_INIT_QP_MINUS26 0x00002004
+#define HEVC_PPSERR_DIFF_CU_QP_DELTA_DEPTH 0x00002005
+#define HEVC_PPSERR_PPS_CB_QP_OFFSET 0x00002006
+#define HEVC_PPSERR_PPS_CR_QP_OFFSET 0x00002007
+#define HEVC_PPSERR_NUM_TILE_COLUMNS_MINUS1 0x00002008
+#define HEVC_PPSERR_NUM_TILE_ROWS_MINUS1 0x00002009
+#define HEVC_PPSERR_COLUMN_WIDTH_MINUS1 0x0000200A
+#define HEVC_PPSERR_ROW_HEIGHT_MINUS1 0x0000200B
+#define HEVC_PPSERR_PPS_BETA_OFFSET_DIV2 0x0000200C
+#define HEVC_PPSERR_PPS_TC_OFFSET_DIV2 0x0000200D
+#define HEVC_PPSERR_SCALING_LIST 0x0000200E
+#define HEVC_PPSERR_LOG2_PARALLEL_MERGE_LEVEL_MINUS2 0x0000200F
+#define HEVC_PPSERR_NUM_TILE_COLUMNS_RANGE_OUT 0x00002010
+#define HEVC_PPSERR_NUM_TILE_ROWS_RANGE_OUT 0x00002011
+#define HEVC_PPSERR_MORE_RBSP_DATA_ERROR 0x00002012
+#define HEVC_PPSERR_PPS_PIC_PARAMETER_SET_ID_RANGE_OUT 0x00002013
+#define HEVC_PPSERR_PPS_SEQ_PARAMETER_SET_ID_RANGE_OUT 0x00002014
+#define HEVC_PPSERR_NUM_REF_IDX_L0_DEFAULT_ACTIVE_MINUS1_RANGE_OUT 0x00002015
+#define HEVC_PPSERR_NUM_REF_IDX_L1_DEFAULT_ACTIVE_MINUS1_RANGE_OUT 0x00002016
+#define HEVC_PPSERR_PPS_CB_QP_OFFSET_RANGE_OUT 0x00002017
+#define HEVC_PPSERR_PPS_CR_QP_OFFSET_RANGE_OUT 0x00002018
+#define HEVC_PPSERR_COLUMN_WIDTH_MINUS1_RANGE_OUT 0x00002019
+#define HEVC_PPSERR_ROW_HEIGHT_MINUS1_RANGE_OUT 0x00002020
+#define HEVC_PPSERR_PPS_BETA_OFFSET_DIV2_RANGE_OUT 0x00002021
+#define HEVC_PPSERR_PPS_TC_OFFSET_DIV2_RANGE_OUT 0x00002022
+#define HEVC_SHERR_SLICE_PIC_PARAMETER_SET_ID 0x00003000
+#define HEVC_SHERR_ACTIVATE_PPS 0x00003001
+#define HEVC_SHERR_ACTIVATE_SPS 0x00003002
+#define HEVC_SHERR_SLICE_TYPE 0x00003003
+#define HEVC_SHERR_FIRST_SLICE_IS_DEPENDENT_SLICE 0x00003004
+#define HEVC_SHERR_SHORTTERM_REF_PIC_SET_SPS_FLAG 0x00003005
+#define HEVC_SHERR_SHORTTERM_REF_PIC_SET 0x00003006
+#define HEVC_SHERR_SHORTTERM_REF_PIC_SET_IDX 0x00003007
+#define HEVC_SHERR_NUM_LONGTERM_SPS 0x00003008
+#define HEVC_SHERR_NUM_LONGTERM_PICS 0x00003009
+#define HEVC_SHERR_LT_IDX_SPS_IS_OUT_OF_RANGE 0x0000300A
+#define HEVC_SHERR_DELTA_POC_MSB_CYCLE_LT 0x0000300B
+#define HEVC_SHERR_NUM_REF_IDX_L0_ACTIVE_MINUS1 0x0000300C
+#define HEVC_SHERR_NUM_REF_IDX_L1_ACTIVE_MINUS1 0x0000300D
+#define HEVC_SHERR_COLLOCATED_REF_IDX 0x0000300E
+#define HEVC_SHERR_PRED_WEIGHT_TABLE 0x0000300F
+#define HEVC_SHERR_FIVE_MINUS_MAX_NUM_MERGE_CAND 0x00003010
+#define HEVC_SHERR_SLICE_QP_DELTA 0x00003011
+#define HEVC_SHERR_SLICE_QP_DELTA_IS_OUT_OF_RANGE 0x00003012
+#define HEVC_SHERR_SLICE_CB_QP_OFFSET 0x00003013
+#define HEVC_SHERR_SLICE_CR_QP_OFFSET 0x00003014
+#define HEVC_SHERR_SLICE_BETA_OFFSET_DIV2 0x00003015
+#define HEVC_SHERR_SLICE_TC_OFFSET_DIV2 0x00003016
+#define HEVC_SHERR_NUM_ENTRY_POINT_OFFSETS 0x00003017
+#define HEVC_SHERR_OFFSET_LEN_MINUS1 0x00003018
+#define HEVC_SHERR_SLICE_SEGMENT_HEADER_EXTENSION_LENGTH 0x00003019
+#define HEVC_SHERR_WRONG_POC_IN_STILL_PICTURE_PROFILE 0x0000301A
+#define HEVC_SHERR_SLICE_TYPE_ERROR_IN_STILL_PICTURE_PROFILE 0x0000301B
+#define HEVC_SHERR_PPS_ID_NOT_EQUAL_PREV_VALUE 0x0000301C
+#define HEVC_SPECERR_OVER_PICTURE_WIDTH_SIZE 0x00004000
+#define HEVC_SPECERR_OVER_PICTURE_HEIGHT_SIZE 0x00004001
+#define HEVC_SPECERR_OVER_CHROMA_FORMAT 0x00004002
+#define HEVC_SPECERR_OVER_BIT_DEPTH 0x00004003
+#define HEVC_SPECERR_OVER_BUFFER_OVER_FLOW 0x00004004
+#define HEVC_SPECERR_OVER_WRONG_BUFFER_ACCESS 0x00004005
+#define HEVC_ETCERR_INIT_SEQ_SPS_NOT_FOUND 0x00005000
+#define HEVC_ETCERR_DEC_PIC_VCL_NOT_FOUND 0x00005001
+#define HEVC_ETCERR_NO_VALID_SLICE_IN_AU 0x00005002
+#define HEVC_ETCERR_INPLACE_V 0x0000500F
+
+#define AVC_SPSERR_SEQ_PARAMETER_SET_ID 0x00001000
+#define AVC_SPSERR_CHROMA_FORMAT_IDC 0x00001001
+#define AVC_SPSERR_PIC_WIDTH_IN_LUMA_SAMPLES 0x00001002
+#define AVC_SPSERR_PIC_HEIGHT_IN_LUMA_SAMPLES 0x00001003
+#define AVC_SPSERR_CONF_WIN_LEFT_OFFSET 0x00001004
+#define AVC_SPSERR_CONF_WIN_RIGHT_OFFSET 0x00001005
+#define AVC_SPSERR_CONF_WIN_TOP_OFFSET 0x00001006
+#define AVC_SPSERR_CONF_WIN_BOTTOM_OFFSET 0x00001007
+#define AVC_SPSERR_BIT_DEPTH_LUMA_MINUS8 0x00001008
+#define AVC_SPSERR_BIT_DEPTH_CHROMA_MINUS8 0x00001009
+#define AVC_SPSERR_SPS_MAX_DEC_PIC_BUFFERING 0x0000100B
+#define AVC_SPSERR_SPS_MAX_NUM_REORDER_PICS 0x0000100C
+#define AVC_SPSERR_SCALING_LIST 0x00001014
+#define AVC_SPSERR_GBU_PARSING_ERROR 0x00001019
+#define AVC_SPSERR_VUI_ERROR 0x0000101B
+#define AVC_SPSERR_ACTIVATE_SPS 0x0000101C
+#define AVC_PPSERR_PPS_PIC_PARAMETER_SET_ID 0x00002000
+#define AVC_PPSERR_PPS_SEQ_PARAMETER_SET_ID 0x00002001
+#define AVC_PPSERR_NUM_REF_IDX_L0_DEFAULT_ACTIVE_MINUS1 0x00002002
+#define AVC_PPSERR_NUM_REF_IDX_L1_DEFAULT_ACTIVE_MINUS1 0x00002003
+#define AVC_PPSERR_INIT_QP_MINUS26 0x00002004
+#define AVC_PPSERR_PPS_CB_QP_OFFSET 0x00002006
+#define AVC_PPSERR_PPS_CR_QP_OFFSET 0x00002007
+#define AVC_PPSERR_SCALING_LIST 0x0000200E
+#define AVC_PPSERR_MORE_RBSP_DATA_ERROR 0x00002012
+#define AVC_PPSERR_PPS_PIC_PARAMETER_SET_ID_RANGE_OUT 0x00002013
+#define AVC_PPSERR_PPS_SEQ_PARAMETER_SET_ID_RANGE_OUT 0x00002014
+#define AVC_PPSERR_NUM_REF_IDX_L0_DEFAULT_ACTIVE_MINUS1_RANGE_OUT 0x00002015
+#define AVC_PPSERR_NUM_REF_IDX_L1_DEFAULT_ACTIVE_MINUS1_RANGE_OUT 0x00002016
+#define AVC_PPSERR_PPS_CB_QP_OFFSET_RANGE_OUT 0x00002017
+#define AVC_PPSERR_PPS_CR_QP_OFFSET_RANGE_OUT 0x00002018
+#define AVC_SHERR_SLICE_PIC_PARAMETER_SET_ID 0x00003000
+#define AVC_SHERR_ACTIVATE_PPS 0x00003001
+#define AVC_SHERR_ACTIVATE_SPS 0x00003002
+#define AVC_SHERR_SLICE_TYPE 0x00003003
+#define AVC_SHERR_FIRST_MB_IN_SLICE 0x00003004
+#define AVC_SHERR_RPLM 0x00003006
+#define AVC_SHERR_LT_IDX_SPS_IS_OUT_OF_RANGE 0x0000300A
+#define AVC_SHERR_NUM_REF_IDX_L0_ACTIVE_MINUS1 0x0000300C
+#define AVC_SHERR_NUM_REF_IDX_L1_ACTIVE_MINUS1 0x0000300D
+#define AVC_SHERR_PRED_WEIGHT_TABLE 0x0000300F
+#define AVC_SHERR_SLICE_QP_DELTA 0x00003011
+#define AVC_SHERR_SLICE_BETA_OFFSET_DIV2 0x00003015
+#define AVC_SHERR_SLICE_TC_OFFSET_DIV2 0x00003016
+#define AVC_SHERR_DISABLE_DEBLOCK_FILTER_IDC 0x00003017
+#define AVC_SPECERR_OVER_PICTURE_WIDTH_SIZE 0x00004000
+#define AVC_SPECERR_OVER_PICTURE_HEIGHT_SIZE 0x00004001
+#define AVC_SPECERR_OVER_CHROMA_FORMAT 0x00004002
+#define AVC_SPECERR_OVER_BIT_DEPTH 0x00004003
+#define AVC_SPECERR_OVER_BUFFER_OVER_FLOW 0x00004004
+#define AVC_SPECERR_OVER_WRONG_BUFFER_ACCESS 0x00004005
+#define AVC_ETCERR_INIT_SEQ_SPS_NOT_FOUND 0x00005000
+#define AVC_ETCERR_DEC_PIC_VCL_NOT_FOUND 0x00005001
+#define AVC_ETCERR_NO_VALID_SLICE_IN_AU 0x00005002
+#define AVC_ETCERR_ASO 0x00005004
+#define AVC_ETCERR_FMO 0x00005005
+#define AVC_ETCERR_INPLACE_V 0x0000500F
+
+/* WAVE6 WARNING ON DECODER (WARN_INFO) */
+#define HEVC_SPSWARN_MAX_SUB_LAYERS_MINUS1 0x00000001
+#define HEVC_SPSWARN_GENERAL_RESERVED_ZERO_44BITS 0x00000002
+#define HEVC_SPSWARN_RESERVED_ZERO_2BITS 0x00000004
+#define HEVC_SPSWARN_SUB_LAYER_RESERVED_ZERO_44BITS 0x00000008
+#define HEVC_SPSWARN_GENERAL_LEVEL_IDC 0x00000010
+#define HEVC_SPSWARN_SPS_MAX_DEC_PIC_BUFFERING_VALUE_OVER 0x00000020
+#define HEVC_SPSWARN_RBSP_TRAILING_BITS 0x00000040
+#define HEVC_SPSWARN_ST_RPS_UE_ERROR 0x00000080
+#define HEVC_SPSWARN_EXTENSION_FLAG 0x01000000
+#define HEVC_SPSWARN_REPLACED_WITH_PREV_SPS 0x02000000
+#define HEVC_PPSWARN_RBSP_TRAILING_BITS 0x00000100
+#define HEVC_PPSWARN_REPLACED_WITH_PREV_PPS 0x00000200
+#define HEVC_SHWARN_FIRST_SLICE_SEGMENT_IN_PIC_FLAG 0x00001000
+#define HEVC_SHWARN_NO_OUTPUT_OF_PRIOR_PICS_FLAG 0x00002000
+#define HEVC_SHWARN_PIC_OUTPUT_FLAG 0x00004000
+#define HEVC_SHWARN_DUPLICATED_SLICE_SEGMENT 0x00008000
+#define HEVC_ETCWARN_INIT_SEQ_VCL_NOT_FOUND 0x00010000
+#define HEVC_ETCWARN_MISSING_REFERENCE_PICTURE 0x00020000
+#define HEVC_ETCWARN_WRONG_TEMPORAL_ID 0x00040000
+#define HEVC_ETCWARN_ERROR_PICTURE_IS_REFERENCED 0x00080000
+#define HEVC_SPECWARN_OVER_PROFILE 0x00100000
+#define HEVC_SPECWARN_OVER_LEVEL 0x00200000
+#define HEVC_PRESWARN_PARSING_ERR 0x04000000
+#define HEVC_PRESWARN_MVD_OUT_OF_RANGE 0x08000000
+#define HEVC_PRESWARN_CU_QP_DELTA_VAL_OUT_OF_RANGE 0x09000000
+#define HEVC_PRESWARN_COEFF_LEVEL_REMAINING_OUT_OF_RANGE 0x0A000000
+#define HEVC_PRESWARN_PCM_ERR 0x0B000000
+#define HEVC_PRESWARN_OVERCONSUME 0x0C000000
+#define HEVC_PRESWARN_END_OF_SUBSET_ONE_BIT_ERR 0x10000000
+#define HEVC_PRESWARN_END_OF_SLICE_SEGMENT_FLAG 0x20000000
+
+#define AVC_SPSWARN_RESERVED_ZERO_2BITS 0x00000004
+#define AVC_SPSWARN_GENERAL_LEVEL_IDC 0x00000010
+#define AVC_SPSWARN_RBSP_TRAILING_BITS 0x00000040
+#define AVC_PPSWARN_RBSP_TRAILING_BITS 0x00000100
+#define AVC_SHWARN_NO_OUTPUT_OF_PRIOR_PICS_FLAG 0x00002000
+#define AVC_ETCWARN_INIT_SEQ_VCL_NOT_FOUND 0x00010000
+#define AVC_ETCWARN_MISSING_REFERENCE_PICTURE 0x00020000
+#define AVC_ETCWARN_ERROR_PICTURE_IS_REFERENCED 0x00080000
+#define AVC_SPECWARN_OVER_PROFILE 0x00100000
+#define AVC_SPECWARN_OVER_LEVEL 0x00200000
+#define AVC_PRESWARN_MVD_RANGE_OUT 0x00400000
+#define AVC_PRESWARN_MB_QPD_RANGE_OUT 0x00500000
+#define AVC_PRESWARN_COEFF_RANGE_OUT 0x00600000
+#define AVC_PRESWARN_MV_RANGE_OUT 0x00700000
+#define AVC_PRESWARN_MB_SKIP_RUN_RANGE_OUT 0x00800000
+#define AVC_PRESWARN_MB_TYPE_RANGE_OUT 0x00900000
+#define AVC_PRESWARN_SUB_MB_TYPE_RANGE_OUT 0x00A00000
+#define AVC_PRESWARN_CBP_RANGE_OUT 0x00B00000
+#define AVC_PRESWARN_INTRA_CHROMA_PRED_MODE_RANGE_OUT 0x00C00000
+#define AVC_PRESWARN_REF_IDX_RANGE_OUT 0x00D00000
+#define AVC_PRESWARN_COEFF_TOKEN_RANGE_OUT 0x00E00000
+#define AVC_PRESWARN_TOTAL_ZERO_RANGE_OUT 0x00F00000
+#define AVC_PRESWARN_RUN_BEFORE_RANGE_OUT 0x01000000
+#define AVC_PRESWARN_OVERCONSUME 0x01100000
+#define AVC_PRESWARN_MISSING_SLICE 0x01200000
+
+#endif /* __WAVE6_VPUERROR_H__ */
--
2.41.0.windows.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v8 05/11] media: chips-media: wave6: Add Wave6 m2m decoder
2026-11-19 8:59 [PATCH v8 00/11] Add support for Wave6 video codec driver Nas Chung
` (3 preceding siblings ...)
2026-11-19 8:59 ` [PATCH v8 04/11] media: chips-media: wave6: Add Wave6 decoder firmware interface Nas Chung
@ 2026-11-19 8:59 ` Nas Chung
2026-11-19 9:31 ` sashiko-bot
2026-11-19 9:00 ` [PATCH v8 06/11] media: chips-media: wave6: Add Wave6 encoder firmware interface Nas Chung
` (5 subsequent siblings)
10 siblings, 1 reply; 23+ messages in thread
From: Nas Chung @ 2026-11-19 8:59 UTC (permalink / raw)
To: mchehab, hverkuil, robh, krzk+dt, conor+dt, Frank.Li, s.hauer
Cc: linux-media, devicetree, linux-kernel, imx, linux-arm-kernel,
festevam, kernel, nicolas.dufresne, ming.qian, marek.vasut,
jackson.lee, lafley.kim, Nas Chung
Add the V4L2 m2m stateful decoder for the Chips&Media Wave6 video codec
IP.
The decoder supports the following features and formats:
- Supported codecs: AVC, HEVC (8-bit streams only)
- Supported raw (CAPTURE) formats:
* V4L2_PIX_FMT_YUV420/NV12/NV21
* V4L2_PIX_FMT_YUV420M/NV12M/NV21M
- Supports scale down
- Supports dynamic resolution change
While other versions of the Wave6 IP may support VP9 and AV1 decoding,
those formats are not implemented or validated in this driver at this
time.
Tested-by: Ming Qian <ming.qian@oss.nxp.com>
Tested-by: Marek Vasut <marek.vasut@mailbox.org>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
---
.../media/platform/chips-media/wave6/Kconfig | 3 +-
.../media/platform/chips-media/wave6/Makefile | 1 +
.../chips-media/wave6/wave6-vpu-core.c | 21 +
.../chips-media/wave6/wave6-vpu-core.h | 2 +
.../chips-media/wave6/wave6-vpu-dec.c | 1935 +++++++++++++++++
.../platform/chips-media/wave6/wave6-vpuapi.h | 2 +
.../chips-media/wave6/wave6-vpuconfig.h | 10 +
7 files changed, 1973 insertions(+), 1 deletion(-)
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
diff --git a/drivers/media/platform/chips-media/wave6/Kconfig b/drivers/media/platform/chips-media/wave6/Kconfig
index 6d87d26fa9fc..9f2c2f92ee38 100644
--- a/drivers/media/platform/chips-media/wave6/Kconfig
+++ b/drivers/media/platform/chips-media/wave6/Kconfig
@@ -10,6 +10,7 @@ config VIDEO_WAVE6_VPU
select GENERIC_ALLOCATOR
help
Chips&Media Wave6 stateful codec driver.
- The wave6 driver manages shared resources such as firmware memory.
+ It provides decoding capabilities for H.264, HEVC,
+ and other video formats.
To compile this driver as a module, choose M here: the
module will be called wave6.
diff --git a/drivers/media/platform/chips-media/wave6/Makefile b/drivers/media/platform/chips-media/wave6/Makefile
index 08f11724b81c..9bfb8b45d091 100644
--- a/drivers/media/platform/chips-media/wave6/Makefile
+++ b/drivers/media/platform/chips-media/wave6/Makefile
@@ -4,5 +4,6 @@ wave6-objs += wave6-vpu.o \
wave6-vpu-core.o \
wave6-vpu-v4l2.o \
wave6-vpuapi.o \
+ wave6-vpu-dec.o \
wave6-hw.o
obj-$(CONFIG_VIDEO_WAVE6_VPU) += wave6.o
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
index 67c6666579d9..4fd86472b427 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
@@ -181,6 +181,11 @@ static void wave6_vpu_core_free_temp_vbuf(void *data)
wave6_vdi_free_dma(&core->temp_vbuf);
}
+static void wave6_vpu_core_unregister_dec(void *data)
+{
+ wave6_vpu_dec_unregister_device(data);
+}
+
static int wave6_vpu_core_probe(struct platform_device *pdev)
{
struct vpu_core_device *core;
@@ -278,6 +283,22 @@ static int wave6_vpu_core_probe(struct platform_device *pdev)
if (ret)
return ret;
+ if (core->res->codec_types & WAVE6_IS_DEC) {
+ ret = wave6_vpu_dec_register_device(core);
+ if (ret) {
+ dev_err(&pdev->dev,
+ "failed to register video_dev_dec: %d\n", ret);
+ return ret;
+ }
+
+ ret = devm_add_action_or_reset(&pdev->dev, wave6_vpu_core_unregister_dec, core);
+ if (ret)
+ return ret;
+ }
+
+ dev_dbg(&pdev->dev, "Added wave6 driver with caps %s\n",
+ core->res->codec_types & WAVE6_IS_DEC ? "'DECODE'" : "");
+
return 0;
}
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.h b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.h
index 788b1403c1d3..1c473c8ce562 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.h
@@ -92,6 +92,8 @@ const char *wave6_vpu_instance_state_name(enum vpu_instance_state state);
void wave6_vpu_set_instance_state(struct vpu_instance *inst,
enum vpu_instance_state state);
int wave6_vpu_wait_interrupt(struct vpu_instance *inst, unsigned int timeout);
+int wave6_vpu_dec_register_device(struct vpu_core_device *core);
+void wave6_vpu_dec_unregister_device(struct vpu_core_device *core);
void wave6_vpu_device_run_timeout(struct work_struct *work);
void wave6_vpu_finish_job(struct vpu_instance *inst);
int wave6_vpu_init_m2m_dev(struct vpu_core_device *core);
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
new file mode 100644
index 000000000000..54b408922bbc
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
@@ -0,0 +1,1935 @@
+// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause)
+/*
+ * Wave6 series multi-standard codec IP - v4l2 stateful decoder interface
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#include <linux/pm_runtime.h>
+
+#include "wave6-vpu-core.h"
+
+#define VPU_DEC_DEV_NAME "C&M Wave6 VPU decoder"
+#define VPU_DEC_DRV_NAME "wave6-dec"
+
+static const struct vpu_format wave6_vpu_dec_fmt_list[2][6] = {
+ [VPU_FMT_TYPE_CODEC] = {
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_HEVC,
+ .max_width = W6_MAX_DEC_PIC_WIDTH,
+ .min_width = W6_MIN_DEC_PIC_WIDTH,
+ .max_height = W6_MAX_DEC_PIC_HEIGHT,
+ .min_height = W6_MIN_DEC_PIC_HEIGHT,
+ .num_planes = 1,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_H264,
+ .max_width = W6_MAX_DEC_PIC_WIDTH,
+ .min_width = W6_MIN_DEC_PIC_WIDTH,
+ .max_height = W6_MAX_DEC_PIC_HEIGHT,
+ .min_height = W6_MIN_DEC_PIC_HEIGHT,
+ .num_planes = 1,
+ },
+ },
+ [VPU_FMT_TYPE_RAW] = {
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_YUV420,
+ .max_width = W6_MAX_DEC_PIC_WIDTH,
+ .min_width = W6_MIN_DEC_PIC_WIDTH,
+ .max_height = W6_MAX_DEC_PIC_HEIGHT,
+ .min_height = W6_MIN_DEC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_420,
+ .endian = VPU_SOURCE_ENDIAN,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_NV12,
+ .max_width = W6_MAX_DEC_PIC_WIDTH,
+ .min_width = W6_MIN_DEC_PIC_WIDTH,
+ .max_height = W6_MAX_DEC_PIC_HEIGHT,
+ .min_height = W6_MIN_DEC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_420,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_NV21,
+ .max_width = W6_MAX_DEC_PIC_WIDTH,
+ .min_width = W6_MIN_DEC_PIC_WIDTH,
+ .max_height = W6_MAX_DEC_PIC_HEIGHT,
+ .min_height = W6_MIN_DEC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_420,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_CRCB_ORDER,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_YUV420M,
+ .max_width = W6_MAX_DEC_PIC_WIDTH,
+ .min_width = W6_MIN_DEC_PIC_WIDTH,
+ .max_height = W6_MAX_DEC_PIC_HEIGHT,
+ .min_height = W6_MIN_DEC_PIC_HEIGHT,
+ .num_planes = 3,
+ .fb_fmt = FORMAT_420,
+ .endian = VPU_SOURCE_ENDIAN,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_NV12M,
+ .max_width = W6_MAX_DEC_PIC_WIDTH,
+ .min_width = W6_MIN_DEC_PIC_WIDTH,
+ .max_height = W6_MAX_DEC_PIC_HEIGHT,
+ .min_height = W6_MIN_DEC_PIC_HEIGHT,
+ .num_planes = 2,
+ .fb_fmt = FORMAT_420,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
+ .max_width = W6_MAX_DEC_PIC_WIDTH,
+ .min_width = W6_MIN_DEC_PIC_WIDTH,
+ .max_height = W6_MAX_DEC_PIC_HEIGHT,
+ .min_height = W6_MIN_DEC_PIC_HEIGHT,
+ .num_planes = 2,
+ .fb_fmt = FORMAT_420,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_CRCB_ORDER,
+ },
+ }
+};
+
+static const struct vpu_format *wave6_find_vpu_fmt(unsigned int v4l2_pix_fmt,
+ enum vpu_fmt_type type)
+{
+ for (int i = 0; i < ARRAY_SIZE(wave6_vpu_dec_fmt_list[type]); i++) {
+ if (wave6_vpu_dec_fmt_list[type][i].v4l2_pix_fmt == v4l2_pix_fmt)
+ return &wave6_vpu_dec_fmt_list[type][i];
+ }
+
+ return NULL;
+}
+
+static const struct vpu_format *wave6_find_vpu_fmt_by_idx(unsigned int idx,
+ enum vpu_fmt_type type)
+{
+ if (idx >= ARRAY_SIZE(wave6_vpu_dec_fmt_list[type]))
+ return NULL;
+
+ if (!wave6_vpu_dec_fmt_list[type][idx].v4l2_pix_fmt)
+ return NULL;
+
+ return &wave6_vpu_dec_fmt_list[type][idx];
+}
+
+static void wave6_vpu_dec_release_fb(struct vpu_instance *inst)
+{
+ for (int i = 0; i < WAVE6_MAX_FBS; i++) {
+ wave6_vdi_free_dma(&inst->frame_vbuf[i]);
+ memset(&inst->frame_buf[i], 0, sizeof(struct frame_buffer));
+ wave6_vdi_free_dma(&inst->aux_vbuf[AUX_BUF_FBC_Y_TBL][i]);
+ wave6_vdi_free_dma(&inst->aux_vbuf[AUX_BUF_FBC_C_TBL][i]);
+ wave6_vdi_free_dma(&inst->aux_vbuf[AUX_BUF_MV_COL][i]);
+ }
+}
+
+static void wave6_vpu_dec_destroy_instance(struct vpu_instance *inst)
+{
+ u32 fail_res = 0;
+ int ret;
+
+ wave6_vpu_disable_instance(inst);
+
+ ret = wave6_vpu_dec_close(inst, &fail_res);
+ if (ret) {
+ dev_err(inst->dev->dev, "failed destroy instance: %d (%d)\n",
+ ret, fail_res);
+ }
+
+ scoped_guard(spinlock, &inst->dev->inst_lock)
+ list_del_init(&inst->list);
+ wave6_vpu_dec_release_fb(inst);
+ wave6_vpu_set_instance_state(inst, VPU_INST_STATE_NONE);
+
+ pm_runtime_put_sync(inst->dev->dev);
+ wave6_vpu_core_deactivate(inst->dev);
+}
+
+static void wave6_handle_bitstream_buffer(struct vpu_instance *inst,
+ bool stream_end)
+{
+ struct vb2_v4l2_buffer *src_buf = NULL;
+ u32 src_size = 0;
+ int ret;
+
+ if (!stream_end)
+ src_buf = v4l2_m2m_next_src_buf(inst->v4l2_fh.m2m_ctx);
+ if (src_buf) {
+ struct vpu_buffer *vpu_buf = wave6_to_vpu_buf(src_buf);
+ dma_addr_t rd_ptr = wave6_get_dma_addr(src_buf, 0);
+
+ if (vpu_buf->consumed) {
+ dev_dbg(inst->dev->dev, "%s: Already consumed buffer\n",
+ __func__);
+ return;
+ }
+
+ vpu_buf->consumed = true;
+ wave6_vpu_dec_set_rd_ptr(inst, rd_ptr, true);
+
+ src_size = vb2_get_plane_payload(&src_buf->vb2_buf, 0) -
+ src_buf->planes[0].data_offset;
+ }
+
+ if (!src_size) {
+ dma_addr_t rd = 0, wr = 0;
+
+ wave6_vpu_dec_get_bitstream_buffer(inst, &rd, &wr);
+ wave6_vpu_dec_set_rd_ptr(inst, wr, true);
+ }
+
+ ret = wave6_vpu_dec_update_bitstream_buffer(inst, src_size);
+ if (ret) {
+ dev_dbg(inst->dev->dev, "%s: Update bitstream buffer fail %d\n",
+ __func__, ret);
+ return;
+ }
+}
+
+static void wave6_update_pix_fmt_cap(struct v4l2_pix_format_mplane *pix_mp,
+ unsigned int width,
+ unsigned int height,
+ bool new_resolution)
+{
+ unsigned int aligned_width;
+
+ if (new_resolution)
+ pix_mp->plane_fmt[0].bytesperline = 0;
+
+ aligned_width = round_up(width, W6_DEC_BUF_ALIGNMENT);
+ wave6_update_pix_fmt(pix_mp, aligned_width, height);
+}
+
+static int wave6_allocate_aux_buffer(struct vpu_instance *inst,
+ enum aux_buffer_type type,
+ int num)
+{
+ struct aux_buffer_size_info size_info;
+ unsigned int size;
+ int ret;
+
+ size_info.width = inst->src_fmt.width;
+ size_info.height = inst->src_fmt.height;
+ size_info.type = type;
+
+ ret = wave6_vpu_dec_get_aux_buffer_size(inst, size_info, &size);
+ if (ret) {
+ dev_err(inst->dev->dev, "%s: Get size fail (type %d)\n", __func__, type);
+ return ret;
+ }
+
+ num = min_t(u32, num, WAVE6_MAX_FBS);
+ for (int i = 0; i < num; i++) {
+ inst->aux_vbuf[type][i].size = size;
+ ret = wave6_vdi_alloc_dma(inst->dev->dev, &inst->aux_vbuf[type][i]);
+ if (ret) {
+ dev_err(inst->dev->dev, "%s: Alloc fail (type %d)\n", __func__, type);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static int wave6_vpu_dec_prepare_fb(struct vpu_instance *inst)
+{
+ int ret;
+ unsigned int fb_num;
+ unsigned int mv_num;
+ unsigned int fb_stride;
+ unsigned int fb_height;
+ unsigned int luma_size;
+ unsigned int chroma_size;
+ struct dec_seq_info *seq_info = &inst->codec_info->dec_info.seq_info;
+
+ fb_num = seq_info->min_frame_buffer_count;
+ mv_num = seq_info->req_mv_buffer_count;
+
+ fb_stride = ALIGN(inst->src_fmt.width, W6_FBC_BUF_ALIGNMENT);
+ fb_height = ALIGN(inst->src_fmt.height, W6_FBC_BUF_ALIGNMENT);
+
+ luma_size = fb_stride * fb_height;
+ chroma_size = ALIGN(fb_stride / 2, W6_FBC_BUF_ALIGNMENT) * fb_height;
+
+ for (unsigned int i = 0; i < fb_num; i++) {
+ struct frame_buffer *frame = &inst->frame_buf[i];
+ struct vpu_buf *vframe = &inst->frame_vbuf[i];
+
+ vframe->size = luma_size + chroma_size;
+ ret = wave6_vdi_alloc_dma(inst->dev->dev, vframe);
+ if (ret) {
+ dev_err(inst->dev->dev, "alloc FBC buffer fail : %zu\n",
+ vframe->size);
+ goto error;
+ }
+
+ frame->buf_y = vframe->daddr;
+ frame->buf_cb = vframe->daddr + luma_size;
+ frame->buf_cr = (dma_addr_t)-1;
+ frame->stride = fb_stride;
+ frame->height = fb_height;
+ frame->map_type = COMPRESSED_FRAME_MAP;
+ }
+
+ ret = wave6_allocate_aux_buffer(inst, AUX_BUF_FBC_Y_TBL, fb_num);
+ if (ret)
+ goto error;
+
+ ret = wave6_allocate_aux_buffer(inst, AUX_BUF_FBC_C_TBL, fb_num);
+ if (ret)
+ goto error;
+
+ ret = wave6_allocate_aux_buffer(inst, AUX_BUF_MV_COL, mv_num);
+ if (ret)
+ goto error;
+
+ wave6_vpu_set_instance_state(inst, VPU_INST_STATE_PIC_RUN);
+
+ return 0;
+
+error:
+ wave6_vpu_dec_release_fb(inst);
+
+ return ret;
+}
+
+static void wave6_vpu_dec_handle_dst_buffer(struct vpu_instance *inst)
+{
+ struct vb2_v4l2_buffer *dst_buf;
+ struct v4l2_m2m_buffer *v4l2_m2m_buf;
+ struct vpu_buffer *vpu_buf;
+ u32 fb_stride = inst->dst_fmt.plane_fmt[0].bytesperline;
+ u32 luma_size = fb_stride * inst->dst_fmt.height;
+ u32 chroma_size = (fb_stride / 2) * (inst->dst_fmt.height / 2);
+ struct frame_buffer disp_buffer = { };
+ struct dec_seq_info *seq_info = &inst->codec_info->dec_info.seq_info;
+ int consumed_num = wave6_vpu_get_consumed_fb_num(inst);
+ int ret;
+
+ v4l2_m2m_for_each_dst_buf(inst->v4l2_fh.m2m_ctx, v4l2_m2m_buf) {
+ dst_buf = &v4l2_m2m_buf->vb;
+ vpu_buf = wave6_to_vpu_buf(dst_buf);
+
+ if (vpu_buf->consumed)
+ continue;
+
+ if (consumed_num >= WAVE6_MAX_FBS)
+ break;
+
+ if (inst->dst_fmt.num_planes == 1) {
+ disp_buffer.buf_y = wave6_get_dma_addr(dst_buf, 0);
+ disp_buffer.buf_cb = disp_buffer.buf_y + luma_size;
+ disp_buffer.buf_cr = disp_buffer.buf_cb + chroma_size;
+ } else if (inst->dst_fmt.num_planes == 2) {
+ disp_buffer.buf_y = wave6_get_dma_addr(dst_buf, 0);
+ disp_buffer.buf_cb = wave6_get_dma_addr(dst_buf, 1);
+ disp_buffer.buf_cr = disp_buffer.buf_cb + chroma_size;
+ } else if (inst->dst_fmt.num_planes == 3) {
+ disp_buffer.buf_y = wave6_get_dma_addr(dst_buf, 0);
+ disp_buffer.buf_cb = wave6_get_dma_addr(dst_buf, 1);
+ disp_buffer.buf_cr = wave6_get_dma_addr(dst_buf, 2);
+ }
+ disp_buffer.height = inst->src_fmt.height;
+ disp_buffer.stride = fb_stride;
+ disp_buffer.map_type = LINEAR_FRAME_MAP;
+ disp_buffer.luma_bit_depth = seq_info->luma_bit_depth;
+ disp_buffer.chroma_bit_depth = seq_info->chroma_bit_depth;
+ disp_buffer.c_fmt_idc = seq_info->c_fmt_idc;
+
+ ret = wave6_vpu_dec_register_display_buffer(inst, disp_buffer);
+ if (ret) {
+ dev_err(inst->dev->dev, "fail register display buffer %d", ret);
+ break;
+ }
+
+ vpu_buf->consumed = true;
+ consumed_num++;
+ }
+
+ if (!inst->fbc_buf_registered) {
+ ret = wave6_vpu_dec_register_frame_buffer(inst);
+ if (ret)
+ dev_err(inst->dev->dev, "register frame buffer fail %d\n", ret);
+ else
+ inst->fbc_buf_registered = true;
+ }
+}
+
+static enum v4l2_quantization to_v4l2_quantization(u32 video_full_range_flag)
+{
+ switch (video_full_range_flag) {
+ case 0:
+ return V4L2_QUANTIZATION_LIM_RANGE;
+ case 1:
+ return V4L2_QUANTIZATION_FULL_RANGE;
+ default:
+ return V4L2_QUANTIZATION_DEFAULT;
+ }
+}
+
+static enum v4l2_colorspace to_v4l2_colorspace(u32 colour_primaries)
+{
+ switch (colour_primaries) {
+ case 1:
+ return V4L2_COLORSPACE_REC709;
+ case 4:
+ return V4L2_COLORSPACE_470_SYSTEM_M;
+ case 5:
+ return V4L2_COLORSPACE_470_SYSTEM_BG;
+ case 6:
+ return V4L2_COLORSPACE_SMPTE170M;
+ case 7:
+ return V4L2_COLORSPACE_SMPTE240M;
+ case 9:
+ return V4L2_COLORSPACE_BT2020;
+ case 11:
+ return V4L2_COLORSPACE_DCI_P3;
+ default:
+ return V4L2_COLORSPACE_DEFAULT;
+ }
+}
+
+static enum v4l2_xfer_func to_v4l2_xfer_func(u32 transfer_characteristics)
+{
+ switch (transfer_characteristics) {
+ case 1:
+ return V4L2_XFER_FUNC_709;
+ case 6:
+ return V4L2_XFER_FUNC_709;
+ case 7:
+ return V4L2_XFER_FUNC_SMPTE240M;
+ case 8:
+ return V4L2_XFER_FUNC_NONE;
+ case 13:
+ return V4L2_XFER_FUNC_SRGB;
+ case 14:
+ return V4L2_XFER_FUNC_709;
+ case 16:
+ return V4L2_XFER_FUNC_SMPTE2084;
+ default:
+ return V4L2_XFER_FUNC_DEFAULT;
+ }
+}
+
+static enum v4l2_ycbcr_encoding to_v4l2_ycbcr_encoding(u32 matrix_coeffs)
+{
+ switch (matrix_coeffs) {
+ case 1:
+ return V4L2_YCBCR_ENC_709;
+ case 5:
+ return V4L2_YCBCR_ENC_601;
+ case 6:
+ return V4L2_YCBCR_ENC_601;
+ case 7:
+ return V4L2_YCBCR_ENC_SMPTE240M;
+ case 9:
+ return V4L2_YCBCR_ENC_BT2020;
+ case 10:
+ return V4L2_YCBCR_ENC_BT2020_CONST_LUM;
+ default:
+ return V4L2_YCBCR_ENC_DEFAULT;
+ }
+}
+
+static void wave6_update_color_info(struct vpu_instance *inst,
+ struct dec_seq_info *seq_info)
+{
+ struct color_param *color = &seq_info->color;
+
+ if (!color->video_signal_type_present)
+ goto set_default_all;
+
+ inst->quantization = to_v4l2_quantization(color->color_range);
+
+ if (!color->color_description_present)
+ goto set_default_color;
+
+ inst->colorspace = to_v4l2_colorspace(color->color_primaries);
+ inst->xfer_func = to_v4l2_xfer_func(color->transfer_characteristics);
+ inst->ycbcr_enc = to_v4l2_ycbcr_encoding(color->matrix_coefficients);
+
+ return;
+
+set_default_all:
+ inst->quantization = V4L2_QUANTIZATION_DEFAULT;
+set_default_color:
+ inst->colorspace = V4L2_COLORSPACE_DEFAULT;
+ inst->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ inst->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+}
+
+static enum v4l2_mpeg_video_hevc_profile to_v4l2_hevc_profile(u32 profile)
+{
+ switch (profile) {
+ case HEVC_PROFILE_MAIN:
+ return V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN;
+ case HEVC_PROFILE_STILLPICTURE:
+ return V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE;
+ default:
+ return V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN;
+ }
+}
+
+static enum v4l2_mpeg_video_h264_profile to_v4l2_h264_profile(u32 profile)
+{
+ switch (profile) {
+ case H264_PROFILE_BP:
+ return V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE;
+ case H264_PROFILE_MP:
+ return V4L2_MPEG_VIDEO_H264_PROFILE_MAIN;
+ case H264_PROFILE_EXTENDED:
+ return V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED;
+ case H264_PROFILE_HP:
+ return V4L2_MPEG_VIDEO_H264_PROFILE_HIGH;
+ default:
+ return V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE;
+ }
+}
+
+static void wave6_update_v4l2_ctrls(struct vpu_instance *inst,
+ struct dec_seq_info *info)
+{
+ struct v4l2_ctrl *ctrl;
+ u32 min_disp_cnt;
+
+ min_disp_cnt = info->frame_buf_delay + 1;
+ ctrl = v4l2_ctrl_find(&inst->v4l2_ctrl_hdl,
+ V4L2_CID_MIN_BUFFERS_FOR_CAPTURE);
+ if (ctrl)
+ v4l2_ctrl_s_ctrl(ctrl, min_disp_cnt);
+
+ if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_HEVC) {
+ ctrl = v4l2_ctrl_find(&inst->v4l2_ctrl_hdl,
+ V4L2_CID_MPEG_VIDEO_HEVC_PROFILE);
+ if (ctrl)
+ v4l2_ctrl_s_ctrl(ctrl, to_v4l2_hevc_profile(info->profile));
+ } else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_H264) {
+ ctrl = v4l2_ctrl_find(&inst->v4l2_ctrl_hdl,
+ V4L2_CID_MPEG_VIDEO_H264_PROFILE);
+ if (ctrl)
+ v4l2_ctrl_s_ctrl(ctrl, to_v4l2_h264_profile(info->profile));
+ }
+}
+
+static int wave6_vpu_dec_start_decode(struct vpu_instance *inst)
+{
+ struct dec_param pic_param;
+ int ret;
+ u32 fail_res = 0;
+
+ memset(&pic_param, 0, sizeof(struct dec_param));
+
+ ret = wave6_vpu_dec_start_one_frame(inst, &pic_param, &fail_res);
+ if (ret) {
+ struct vb2_v4l2_buffer *src_buf = NULL;
+
+ dev_err(inst->dev->dev, "[%d] %s: fail %d\n", inst->id, __func__, ret);
+ wave6_vpu_set_instance_state(inst, VPU_INST_STATE_ERROR);
+ vb2_queue_error(v4l2_m2m_get_src_vq(inst->v4l2_fh.m2m_ctx));
+ vb2_queue_error(v4l2_m2m_get_dst_vq(inst->v4l2_fh.m2m_ctx));
+
+ src_buf = v4l2_m2m_src_buf_remove(inst->v4l2_fh.m2m_ctx);
+ if (src_buf) {
+ v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
+ inst->sequence++;
+ inst->processed_buf_num++;
+ inst->error_buf_num++;
+ }
+ }
+
+ return ret;
+}
+
+static void wave6_vpu_dec_src_buf_done(struct vpu_instance *inst,
+ struct vb2_v4l2_buffer *src_buf,
+ enum vb2_buffer_state state)
+{
+ scoped_guard(mutex, &inst->lock) {
+ v4l2_m2m_src_buf_remove_by_buf(inst->v4l2_fh.m2m_ctx, src_buf);
+ if (inst->drain_src_buf_num)
+ inst->drain_src_buf_num--;
+ }
+
+ v4l2_m2m_buf_done(src_buf, state);
+}
+
+static void wave6_handle_decoded_frame(struct vpu_instance *inst,
+ struct dec_output_info *info)
+{
+ struct vb2_v4l2_buffer *src_buf;
+ struct vb2_v4l2_buffer *dst_buf;
+ struct vpu_buffer *vpu_buf;
+ enum vb2_buffer_state state;
+
+ state = info->decoding_success ? VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
+
+ src_buf = v4l2_m2m_next_src_buf(inst->v4l2_fh.m2m_ctx);
+ if (!src_buf) {
+ dev_err(inst->dev->dev, "[%d] decoder can't find src buffer\n", inst->id);
+ return;
+ }
+
+ vpu_buf = wave6_to_vpu_buf(src_buf);
+ if (!vpu_buf || !vpu_buf->consumed) {
+ dev_err(inst->dev->dev, "[%d] src buffer is not consumed\n", inst->id);
+ return;
+ }
+
+ dst_buf = wave6_get_dst_buf_by_addr(inst, info->frame_decoded_addr);
+ if (dst_buf) {
+ struct vpu_buffer *dst_vpu_buf = wave6_to_vpu_buf(dst_buf);
+
+ if (wave6_to_vpu_buf(dst_buf)->used) {
+ dev_warn(inst->dev->dev, "[%d] duplication frame buffer\n", inst->id);
+ inst->sequence++;
+ }
+ v4l2_m2m_buf_copy_metadata(src_buf, dst_buf);
+ dst_vpu_buf->used = true;
+ if (state == VB2_BUF_STATE_ERROR)
+ dst_vpu_buf->error = true;
+ }
+
+ if (state == VB2_BUF_STATE_ERROR) {
+ dev_dbg(inst->dev->dev, "[%d] error frame %d\n", inst->id, inst->sequence);
+ inst->error_buf_num++;
+ }
+ wave6_vpu_dec_src_buf_done(inst, src_buf, state);
+ inst->processed_buf_num++;
+}
+
+static void wave6_handle_skipped_frame(struct vpu_instance *inst)
+{
+ struct vb2_v4l2_buffer *src_buf;
+ struct vpu_buffer *vpu_buf;
+
+ src_buf = v4l2_m2m_next_src_buf(inst->v4l2_fh.m2m_ctx);
+ if (!src_buf)
+ return;
+
+ vpu_buf = wave6_to_vpu_buf(src_buf);
+ if (!vpu_buf || !vpu_buf->consumed)
+ return;
+
+ dev_dbg(inst->dev->dev, "[%d] skip frame %d\n", inst->id, inst->sequence);
+
+ inst->sequence++;
+ inst->processed_buf_num++;
+ inst->error_buf_num++;
+ wave6_vpu_dec_src_buf_done(inst, src_buf, VB2_BUF_STATE_ERROR);
+}
+
+static void wave6_handle_display_frame(struct vpu_instance *inst,
+ dma_addr_t addr, enum vb2_buffer_state state)
+{
+ struct vb2_v4l2_buffer *dst_buf;
+ struct vpu_buffer *vpu_buf;
+
+ dst_buf = wave6_get_dst_buf_by_addr(inst, addr);
+ if (!dst_buf)
+ return;
+
+ vpu_buf = wave6_to_vpu_buf(dst_buf);
+ if (!vpu_buf->used) {
+ dev_dbg(inst->dev->dev, "[%d] recycle display buffer\n", inst->id);
+ vpu_buf->consumed = false;
+ return;
+ }
+
+ for (int i = 0; i < inst->dst_fmt.num_planes; i++)
+ vb2_set_plane_payload(&dst_buf->vb2_buf, i,
+ inst->dst_fmt.plane_fmt[i].sizeimage);
+
+ if (vpu_buf->error)
+ state = VB2_BUF_STATE_ERROR;
+ dst_buf->sequence = inst->sequence++;
+ dst_buf->field = V4L2_FIELD_NONE;
+ if (state == VB2_BUF_STATE_ERROR)
+ dev_dbg(inst->dev->dev, "[%d] discard frame %d\n", inst->id, dst_buf->sequence);
+ v4l2_m2m_dst_buf_remove_by_buf(inst->v4l2_fh.m2m_ctx, dst_buf);
+ v4l2_m2m_buf_done(dst_buf, state);
+}
+
+static void wave6_handle_display_frames(struct vpu_instance *inst,
+ struct dec_output_info *info)
+{
+ for (int i = 0; i < info->disp_frame_num; i++)
+ wave6_handle_display_frame(inst,
+ info->disp_frame_addr[i],
+ VB2_BUF_STATE_DONE);
+}
+
+static void wave6_handle_discard_frames(struct vpu_instance *inst,
+ struct dec_output_info *info)
+{
+ for (int i = 0; i < info->release_disp_frame_num; i++)
+ wave6_handle_display_frame(inst,
+ info->release_disp_frame_addr[i],
+ VB2_BUF_STATE_ERROR);
+}
+
+static void wave6_handle_last_frame(struct vpu_instance *inst,
+ struct vb2_v4l2_buffer *dst_buf)
+{
+ lockdep_assert_held(&inst->lock);
+
+ if (!dst_buf) {
+ dst_buf = v4l2_m2m_dst_buf_remove(inst->v4l2_fh.m2m_ctx);
+ if (!dst_buf) {
+ inst->next_dst_buf_last = true;
+ return;
+ }
+ }
+
+ for (int i = 0; i < inst->dst_fmt.num_planes; i++)
+ vb2_set_plane_payload(&dst_buf->vb2_buf, i, 0);
+
+ dst_buf->flags |= V4L2_BUF_FLAG_LAST;
+ dst_buf->field = V4L2_FIELD_NONE;
+ v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
+}
+
+static void wave6_vpu_dec_finish_drain(struct vpu_instance *inst)
+{
+ struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+
+ dev_dbg(inst->dev->dev, "[%d] drain done\n", inst->id);
+
+ v4l2_m2m_set_src_buffered(m2m_ctx, false);
+
+ guard(mutex)(&inst->lock);
+
+ v4l2_m2m_mark_stopped(m2m_ctx);
+ inst->drain = false;
+ wave6_handle_last_frame(inst, NULL);
+}
+
+static void wave6_vpu_dec_cancel_drain(struct vpu_instance *inst)
+{
+ scoped_guard(mutex, &inst->lock) {
+ inst->drain = false;
+ inst->drain_src_buf_num = 0;
+ }
+
+ v4l2_m2m_set_src_buffered(inst->v4l2_fh.m2m_ctx, false);
+}
+
+static void wave6_vpu_dec_retry_one_frame(struct vpu_instance *inst)
+{
+ struct vb2_v4l2_buffer *src_buf;
+ struct vpu_buffer *vpu_buf;
+
+ src_buf = v4l2_m2m_next_src_buf(inst->v4l2_fh.m2m_ctx);
+ if (!src_buf)
+ return;
+
+ vpu_buf = wave6_to_vpu_buf(src_buf);
+ vpu_buf->consumed = false;
+}
+
+static void wave6_vpu_dec_handle_source_change(struct vpu_instance *inst)
+{
+ static const struct v4l2_event vpu_event_src_ch = {
+ .type = V4L2_EVENT_SOURCE_CHANGE,
+ .u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION,
+ };
+ struct dec_seq_info *info = &inst->codec_info->dec_info.seq_info;
+
+ dev_dbg(inst->dev->dev, "pic size %dx%d profile %d, min_fb_cnt : %d | min_disp_cnt : %d\n",
+ info->pic_width, info->pic_height,
+ info->profile, info->min_frame_buffer_count, info->frame_buf_delay);
+
+ wave6_vpu_dec_retry_one_frame(inst);
+ wave6_vpu_dec_reset_frame_buffer_info(inst);
+ inst->fbc_buf_registered = false;
+
+ wave6_vpu_set_instance_state(inst, VPU_INST_STATE_INIT_SEQ);
+
+ inst->scaler_info.enable = false;
+ inst->crop.left = info->pic_crop_rect.left;
+ inst->crop.top = info->pic_crop_rect.top;
+ inst->crop.width = info->pic_crop_rect.right - inst->crop.left;
+ inst->crop.height = info->pic_crop_rect.bottom - inst->crop.top;
+
+ wave6_update_v4l2_ctrls(inst, info);
+ wave6_update_color_info(inst, info);
+ wave6_update_pix_fmt(&inst->src_fmt, info->pic_width, info->pic_height);
+ wave6_update_pix_fmt_cap(&inst->dst_fmt,
+ info->pic_width, info->pic_height,
+ true);
+
+ v4l2_event_queue_fh(&inst->v4l2_fh, &vpu_event_src_ch);
+}
+
+static void wave6_vpu_dec_handle_decoding_warn_error(struct vpu_instance *inst,
+ struct dec_output_info *info)
+{
+ if (info->warn_info)
+ dev_dbg(inst->dev->dev, "[%d] decoding %d warning 0x%x\n",
+ inst->id, inst->processed_buf_num, info->warn_info);
+
+ if (info->error_reason)
+ dev_dbg(inst->dev->dev, "[%d] decoding %d error 0x%x\n",
+ inst->id, inst->processed_buf_num, info->error_reason);
+}
+
+static void wave6_vpu_dec_finish_decode(struct vpu_instance *inst, bool error)
+{
+ struct dec_output_info info;
+ int ret;
+
+ memset(&info, 0, sizeof(info));
+
+ ret = wave6_vpu_dec_get_output_info(inst, &info);
+ if (ret)
+ goto finish_decode;
+
+ dev_dbg(inst->dev->dev, "dec %d dis %d noti_flag %d stream_end %d\n",
+ info.frame_decoded, info.frame_display,
+ info.notification_flags, info.stream_end);
+
+ if (info.notification_flags & DEC_NOTI_FLAG_NO_FB) {
+ wave6_vpu_dec_retry_one_frame(inst);
+ goto finish_decode;
+ }
+
+ if (info.notification_flags & DEC_NOTI_FLAG_SEQ_CHANGE) {
+ if (info.frame_display)
+ wave6_handle_display_frames(inst, &info);
+
+ if (info.release_disp_frame_num)
+ wave6_handle_discard_frames(inst, &info);
+
+ wave6_vpu_dec_handle_source_change(inst);
+
+ scoped_guard(mutex, &inst->lock)
+ wave6_handle_last_frame(inst, NULL);
+
+ goto finish_decode;
+ }
+
+ wave6_vpu_dec_handle_decoding_warn_error(inst, &info);
+
+ if (info.frame_decoded)
+ wave6_handle_decoded_frame(inst, &info);
+ else
+ wave6_handle_skipped_frame(inst);
+
+ if (info.frame_display)
+ wave6_handle_display_frames(inst, &info);
+
+ if (info.release_disp_frame_num)
+ wave6_handle_discard_frames(inst, &info);
+
+ if (info.stream_end)
+ wave6_vpu_dec_finish_drain(inst);
+
+finish_decode:
+ wave6_vpu_finish_job(inst);
+}
+
+static int wave6_vpu_dec_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
+{
+ strscpy(cap->driver, VPU_DEC_DRV_NAME, sizeof(cap->driver));
+ strscpy(cap->card, VPU_DEC_DRV_NAME, sizeof(cap->card));
+ strscpy(cap->bus_info, "platform:" VPU_DEC_DRV_NAME, sizeof(cap->bus_info));
+
+ return 0;
+}
+
+static int wave6_vpu_dec_enum_framesizes(struct file *f, void *fh, struct v4l2_frmsizeenum *fsize)
+{
+ const struct vpu_format *vpu_fmt;
+
+ if (fsize->index)
+ return -EINVAL;
+
+ vpu_fmt = wave6_find_vpu_fmt(fsize->pixel_format, VPU_FMT_TYPE_CODEC);
+ if (!vpu_fmt) {
+ vpu_fmt = wave6_find_vpu_fmt(fsize->pixel_format, VPU_FMT_TYPE_RAW);
+ if (!vpu_fmt)
+ return -EINVAL;
+ }
+
+ fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+ fsize->stepwise.min_width = vpu_fmt->min_width;
+ fsize->stepwise.max_width = vpu_fmt->max_width;
+ fsize->stepwise.step_width = W6_DEC_PIC_SIZE_STEP;
+ fsize->stepwise.min_height = vpu_fmt->min_height;
+ fsize->stepwise.max_height = vpu_fmt->max_height;
+ fsize->stepwise.step_height = W6_DEC_PIC_SIZE_STEP;
+
+ return 0;
+}
+
+static int wave6_vpu_dec_enum_fmt_cap(struct file *file, void *fh, struct v4l2_fmtdesc *f)
+{
+ const struct vpu_format *vpu_fmt;
+
+ vpu_fmt = wave6_find_vpu_fmt_by_idx(f->index, VPU_FMT_TYPE_RAW);
+ if (!vpu_fmt)
+ return -EINVAL;
+
+ f->pixelformat = vpu_fmt->v4l2_pix_fmt;
+ f->flags = 0;
+
+ return 0;
+}
+
+static int wave6_vpu_dec_try_fmt_cap(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+ const struct vpu_format *vpu_fmt;
+ int width, height;
+
+ dev_dbg(inst->dev->dev, "%s: 4cc %d w %d h %d plane %d colorspace %d\n",
+ __func__, pix_mp->pixelformat, pix_mp->width, pix_mp->height,
+ pix_mp->num_planes, pix_mp->colorspace);
+
+ if (!V4L2_TYPE_IS_CAPTURE(f->type))
+ return -EINVAL;
+
+ vpu_fmt = wave6_find_vpu_fmt(pix_mp->pixelformat, VPU_FMT_TYPE_RAW);
+ if (!vpu_fmt) {
+ width = inst->dst_fmt.width;
+ height = inst->dst_fmt.height;
+ pix_mp->pixelformat = inst->dst_fmt.pixelformat;
+ pix_mp->num_planes = inst->dst_fmt.num_planes;
+ } else {
+ width = clamp(pix_mp->width,
+ vpu_fmt->min_width,
+ round_up(inst->src_fmt.width, W6_DEC_BUF_ALIGNMENT));
+ height = clamp(pix_mp->height,
+ vpu_fmt->min_height, inst->src_fmt.height);
+ pix_mp->pixelformat = vpu_fmt->v4l2_pix_fmt;
+ pix_mp->num_planes = vpu_fmt->num_planes;
+ }
+
+ if (inst->state >= VPU_INST_STATE_INIT_SEQ) {
+ width = inst->dst_fmt.width;
+ height = inst->dst_fmt.height;
+ }
+
+ wave6_update_pix_fmt_cap(pix_mp, width, height, false);
+ pix_mp->colorspace = inst->colorspace;
+ pix_mp->ycbcr_enc = inst->ycbcr_enc;
+ pix_mp->quantization = inst->quantization;
+ pix_mp->xfer_func = inst->xfer_func;
+
+ return 0;
+}
+
+static int wave6_vpu_dec_s_fmt_cap(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+ const struct vpu_format *vpu_fmt;
+ struct vb2_queue *q;
+ int ret;
+
+ dev_dbg(inst->dev->dev, "%s: 4cc %d w %d h %d plane %d colorspace %d\n",
+ __func__, pix_mp->pixelformat, pix_mp->width, pix_mp->height,
+ pix_mp->num_planes, pix_mp->colorspace);
+
+ q = v4l2_m2m_get_vq(inst->v4l2_fh.m2m_ctx, f->type);
+ if (vb2_is_busy(q))
+ return -EBUSY;
+
+ ret = wave6_vpu_dec_try_fmt_cap(file, fh, f);
+ if (ret)
+ return ret;
+
+ vpu_fmt = wave6_find_vpu_fmt(pix_mp->pixelformat, VPU_FMT_TYPE_RAW);
+ if (!vpu_fmt)
+ return -EINVAL;
+
+ inst->dst_fmt.width = pix_mp->width;
+ inst->dst_fmt.height = pix_mp->height;
+ inst->dst_fmt.pixelformat = pix_mp->pixelformat;
+ inst->dst_fmt.field = pix_mp->field;
+ inst->dst_fmt.flags = pix_mp->flags;
+ inst->dst_fmt.num_planes = pix_mp->num_planes;
+ for (int i = 0; i < inst->dst_fmt.num_planes; i++) {
+ inst->dst_fmt.plane_fmt[i].bytesperline = pix_mp->plane_fmt[i].bytesperline;
+ inst->dst_fmt.plane_fmt[i].sizeimage = pix_mp->plane_fmt[i].sizeimage;
+ }
+
+ inst->cbcr_interleave = !!(vpu_fmt->flags & VPU_FMT_FLAG_CBCR_INTERLEAVED);
+ inst->nv21 = !!(vpu_fmt->flags & VPU_FMT_FLAG_CRCB_ORDER);
+
+ return 0;
+}
+
+static int wave6_vpu_dec_g_fmt_cap(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+
+ pix_mp->width = inst->dst_fmt.width;
+ pix_mp->height = inst->dst_fmt.height;
+ pix_mp->pixelformat = inst->dst_fmt.pixelformat;
+ pix_mp->field = inst->dst_fmt.field;
+ pix_mp->flags = inst->dst_fmt.flags;
+ pix_mp->num_planes = inst->dst_fmt.num_planes;
+ for (int i = 0; i < pix_mp->num_planes; i++) {
+ pix_mp->plane_fmt[i].bytesperline = inst->dst_fmt.plane_fmt[i].bytesperline;
+ pix_mp->plane_fmt[i].sizeimage = inst->dst_fmt.plane_fmt[i].sizeimage;
+ }
+
+ pix_mp->colorspace = inst->colorspace;
+ pix_mp->ycbcr_enc = inst->ycbcr_enc;
+ pix_mp->quantization = inst->quantization;
+ pix_mp->xfer_func = inst->xfer_func;
+
+ return 0;
+}
+
+static int wave6_vpu_dec_enum_fmt_out(struct file *file, void *fh, struct v4l2_fmtdesc *f)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ const struct vpu_format *vpu_fmt;
+
+ dev_dbg(inst->dev->dev, "%s: index %d\n", __func__, f->index);
+
+ vpu_fmt = wave6_find_vpu_fmt_by_idx(f->index, VPU_FMT_TYPE_CODEC);
+ if (!vpu_fmt)
+ return -EINVAL;
+
+ f->pixelformat = vpu_fmt->v4l2_pix_fmt;
+ f->flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED;
+
+ return 0;
+}
+
+static int wave6_vpu_dec_try_fmt_out(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+ const struct vpu_format *vpu_fmt;
+ unsigned int width, height;
+
+ dev_dbg(inst->dev->dev, "%s: 4cc %d w %d h %d plane %d colorspace %d\n",
+ __func__, pix_mp->pixelformat, pix_mp->width, pix_mp->height,
+ pix_mp->num_planes, pix_mp->colorspace);
+
+ if (!V4L2_TYPE_IS_OUTPUT(f->type))
+ return -EINVAL;
+
+ vpu_fmt = wave6_find_vpu_fmt(pix_mp->pixelformat, VPU_FMT_TYPE_CODEC);
+ if (!vpu_fmt) {
+ width = inst->src_fmt.width;
+ height = inst->src_fmt.height;
+ pix_mp->pixelformat = inst->src_fmt.pixelformat;
+ pix_mp->num_planes = inst->src_fmt.num_planes;
+ } else {
+ width = clamp(pix_mp->width,
+ vpu_fmt->min_width, vpu_fmt->max_width);
+ height = clamp(pix_mp->height,
+ vpu_fmt->min_height, vpu_fmt->max_height);
+ pix_mp->pixelformat = vpu_fmt->v4l2_pix_fmt;
+ pix_mp->num_planes = vpu_fmt->num_planes;
+ }
+
+ wave6_update_pix_fmt(pix_mp, width, height);
+ pix_mp->colorspace = inst->colorspace;
+ pix_mp->ycbcr_enc = inst->ycbcr_enc;
+ pix_mp->quantization = inst->quantization;
+ pix_mp->xfer_func = inst->xfer_func;
+
+ return 0;
+}
+
+static int wave6_vpu_dec_s_fmt_out(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ struct v4l2_pix_format_mplane in_pix_mp = f->fmt.pix_mp;
+ struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+ struct vb2_queue *q;
+ int ret;
+
+ dev_dbg(inst->dev->dev, "%s: 4cc %d w %d h %d plane %d colorspace %d\n",
+ __func__, pix_mp->pixelformat, pix_mp->width, pix_mp->height,
+ pix_mp->num_planes, pix_mp->colorspace);
+
+ q = v4l2_m2m_get_vq(inst->v4l2_fh.m2m_ctx, f->type);
+ if (vb2_is_busy(q))
+ return -EBUSY;
+
+ ret = wave6_vpu_dec_try_fmt_out(file, fh, f);
+ if (ret)
+ return ret;
+
+ pix_mp->colorspace = in_pix_mp.colorspace;
+ pix_mp->ycbcr_enc = in_pix_mp.ycbcr_enc;
+ pix_mp->quantization = in_pix_mp.quantization;
+ pix_mp->xfer_func = in_pix_mp.xfer_func;
+
+ inst->src_fmt.width = pix_mp->width;
+ inst->src_fmt.height = pix_mp->height;
+ inst->src_fmt.pixelformat = pix_mp->pixelformat;
+ inst->src_fmt.field = pix_mp->field;
+ inst->src_fmt.flags = pix_mp->flags;
+ inst->src_fmt.num_planes = pix_mp->num_planes;
+ for (int i = 0; i < inst->src_fmt.num_planes; i++) {
+ inst->src_fmt.plane_fmt[i].bytesperline = pix_mp->plane_fmt[i].bytesperline;
+ inst->src_fmt.plane_fmt[i].sizeimage = pix_mp->plane_fmt[i].sizeimage;
+ }
+
+ inst->colorspace = pix_mp->colorspace;
+ inst->ycbcr_enc = pix_mp->ycbcr_enc;
+ inst->quantization = pix_mp->quantization;
+ inst->xfer_func = pix_mp->xfer_func;
+
+ wave6_update_pix_fmt_cap(&inst->dst_fmt,
+ pix_mp->width, pix_mp->height,
+ true);
+
+ return 0;
+}
+
+static int wave6_vpu_dec_g_fmt_out(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+
+ pix_mp->width = inst->src_fmt.width;
+ pix_mp->height = inst->src_fmt.height;
+ pix_mp->pixelformat = inst->src_fmt.pixelformat;
+ pix_mp->field = inst->src_fmt.field;
+ pix_mp->flags = inst->src_fmt.flags;
+ pix_mp->num_planes = inst->src_fmt.num_planes;
+ for (int i = 0; i < pix_mp->num_planes; i++) {
+ pix_mp->plane_fmt[i].bytesperline = inst->src_fmt.plane_fmt[i].bytesperline;
+ pix_mp->plane_fmt[i].sizeimage = inst->src_fmt.plane_fmt[i].sizeimage;
+ }
+
+ pix_mp->colorspace = inst->colorspace;
+ pix_mp->ycbcr_enc = inst->ycbcr_enc;
+ pix_mp->quantization = inst->quantization;
+ pix_mp->xfer_func = inst->xfer_func;
+
+ return 0;
+}
+
+static int wave6_vpu_dec_g_selection(struct file *file, void *fh, struct v4l2_selection *s)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+
+ dev_dbg(inst->dev->dev, "%s: type %d target %d\n",
+ __func__, s->type, s->target);
+
+ if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE &&
+ s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+ return -EINVAL;
+
+ switch (s->target) {
+ case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+ s->r.left = 0;
+ s->r.top = 0;
+ s->r.width = inst->dst_fmt.width;
+ s->r.height = inst->dst_fmt.height;
+ break;
+ case V4L2_SEL_TGT_COMPOSE_PADDED:
+ case V4L2_SEL_TGT_COMPOSE:
+ s->r.left = 0;
+ s->r.top = 0;
+ if (inst->scaler_info.enable) {
+ s->r.width = inst->scaler_info.width;
+ s->r.height = inst->scaler_info.height;
+ } else if (inst->crop.width && inst->crop.height) {
+ s->r = inst->crop;
+ } else {
+ s->r.width = inst->src_fmt.width;
+ s->r.height = inst->src_fmt.height;
+ }
+ break;
+ case V4L2_SEL_TGT_CROP:
+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ case V4L2_SEL_TGT_COMPOSE_DEFAULT:
+ s->r.left = 0;
+ s->r.top = 0;
+ s->r.width = inst->src_fmt.width;
+ s->r.height = inst->src_fmt.height;
+ if (inst->crop.width && inst->crop.height)
+ s->r = inst->crop;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int wave6_vpu_dec_s_selection(struct file *file, void *fh, struct v4l2_selection *s)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ int step = 4;
+ int scale_width, scale_height;
+ int min_scale_width, min_scale_height;
+
+ if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE &&
+ s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+ return -EINVAL;
+
+ if (s->target != V4L2_SEL_TGT_COMPOSE)
+ return -EINVAL;
+
+ if (!(s->flags & (V4L2_SEL_FLAG_GE | V4L2_SEL_FLAG_LE)))
+ s->flags |= V4L2_SEL_FLAG_LE;
+
+ scale_width = clamp(s->r.width, W6_MIN_DEC_PIC_WIDTH,
+ inst->src_fmt.width);
+ scale_height = clamp(s->r.height, W6_MIN_DEC_PIC_HEIGHT,
+ inst->src_fmt.height);
+ if (s->flags & V4L2_SEL_FLAG_GE) {
+ scale_width = round_up(scale_width, step);
+ scale_height = round_up(scale_height, step);
+ }
+ if (s->flags & V4L2_SEL_FLAG_LE) {
+ scale_width = round_down(scale_width, step);
+ scale_height = round_down(scale_height, step);
+ }
+
+ inst->scaler_info.enable = scale_width < inst->src_fmt.width ||
+ scale_height < inst->src_fmt.height;
+
+ if (inst->scaler_info.enable) {
+ min_scale_width = ALIGN((inst->src_fmt.width / 8), step);
+ min_scale_height = ALIGN((inst->src_fmt.height / 8), step);
+
+ if (scale_width < W6_MIN_DEC_PIC_WIDTH)
+ scale_width = W6_MIN_DEC_PIC_WIDTH;
+ if (scale_width < min_scale_width)
+ scale_width = min_scale_width;
+ if (scale_height < W6_MIN_DEC_PIC_HEIGHT)
+ scale_height = W6_MIN_DEC_PIC_HEIGHT;
+ if (scale_height < min_scale_height)
+ scale_height = min_scale_height;
+
+ inst->scaler_info.width = scale_width;
+ inst->scaler_info.height = scale_height;
+ }
+
+ s->r.left = 0;
+ s->r.top = 0;
+ s->r.width = scale_width;
+ s->r.height = scale_height;
+
+ return 0;
+}
+
+static int wave6_vpu_dec_cmd_stop(struct vpu_instance *inst)
+{
+ struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+
+ scoped_guard(mutex, &inst->lock) {
+ if (inst->drain)
+ return -EBUSY;
+
+ if (v4l2_m2m_has_stopped(m2m_ctx))
+ return 0;
+
+ inst->drain_src_buf_num = v4l2_m2m_num_src_bufs_ready(m2m_ctx);
+
+ if (!inst->drain_src_buf_num &&
+ !vb2_is_streaming(v4l2_m2m_get_dst_vq(m2m_ctx)))
+ return 0;
+
+ inst->drain = true;
+ }
+
+ v4l2_m2m_set_src_buffered(m2m_ctx, true);
+ v4l2_m2m_try_schedule(m2m_ctx);
+
+ return 0;
+}
+
+static int wave6_vpu_dec_cmd_start(struct vpu_instance *inst)
+{
+ struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+ struct v4l2_m2m_buffer *buf;
+ int ret;
+
+ scoped_guard(mutex, &inst->lock) {
+ if (inst->next_dst_buf_last ||
+ (inst->drain && inst->state != VPU_INST_STATE_INIT_SEQ))
+ return -EBUSY;
+ }
+
+ if (inst->state == VPU_INST_STATE_INIT_SEQ &&
+ vb2_is_streaming(v4l2_m2m_get_dst_vq(m2m_ctx))) {
+ v4l2_m2m_for_each_dst_buf(m2m_ctx, buf) {
+ for (int i = 0; i < inst->dst_fmt.num_planes; i++) {
+ if (vb2_plane_size(&buf->vb.vb2_buf, i) <
+ inst->dst_fmt.plane_fmt[i].sizeimage)
+ return -EINVAL;
+ }
+ }
+
+ ret = wave6_vpu_dec_prepare_fb(inst);
+ if (ret)
+ return ret;
+ }
+
+ v4l2_m2m_clear_state(m2m_ctx);
+ vb2_clear_last_buffer_dequeued(v4l2_m2m_get_dst_vq(m2m_ctx));
+ v4l2_m2m_try_schedule(m2m_ctx);
+
+ return 0;
+}
+
+static int wave6_vpu_dec_decoder_cmd(struct file *file, void *fh, struct v4l2_decoder_cmd *dc)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ int ret;
+
+ dev_dbg(inst->dev->dev, "%s: cmd %d\n", __func__, dc->cmd);
+
+ ret = v4l2_m2m_ioctl_try_decoder_cmd(file, fh, dc);
+ if (ret)
+ return ret;
+
+ if (!vb2_is_streaming(v4l2_m2m_get_src_vq(inst->v4l2_fh.m2m_ctx)))
+ return 0;
+
+ switch (dc->cmd) {
+ case V4L2_DEC_CMD_STOP:
+ return wave6_vpu_dec_cmd_stop(inst);
+ case V4L2_DEC_CMD_START:
+ return wave6_vpu_dec_cmd_start(inst);
+ default:
+ return -EINVAL;
+ }
+}
+
+static const struct v4l2_ioctl_ops wave6_vpu_dec_ioctl_ops = {
+ .vidioc_querycap = wave6_vpu_dec_querycap,
+ .vidioc_enum_framesizes = wave6_vpu_dec_enum_framesizes,
+
+ .vidioc_enum_fmt_vid_cap = wave6_vpu_dec_enum_fmt_cap,
+ .vidioc_s_fmt_vid_cap_mplane = wave6_vpu_dec_s_fmt_cap,
+ .vidioc_g_fmt_vid_cap_mplane = wave6_vpu_dec_g_fmt_cap,
+ .vidioc_try_fmt_vid_cap_mplane = wave6_vpu_dec_try_fmt_cap,
+
+ .vidioc_enum_fmt_vid_out = wave6_vpu_dec_enum_fmt_out,
+ .vidioc_s_fmt_vid_out_mplane = wave6_vpu_dec_s_fmt_out,
+ .vidioc_g_fmt_vid_out_mplane = wave6_vpu_dec_g_fmt_out,
+ .vidioc_try_fmt_vid_out_mplane = wave6_vpu_dec_try_fmt_out,
+
+ .vidioc_g_selection = wave6_vpu_dec_g_selection,
+ .vidioc_s_selection = wave6_vpu_dec_s_selection,
+
+ .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs,
+ .vidioc_querybuf = v4l2_m2m_ioctl_querybuf,
+ .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
+ .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
+ .vidioc_qbuf = v4l2_m2m_ioctl_qbuf,
+ .vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
+ .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
+ .vidioc_streamon = v4l2_m2m_ioctl_streamon,
+ .vidioc_streamoff = v4l2_m2m_ioctl_streamoff,
+
+ .vidioc_try_decoder_cmd = v4l2_m2m_ioctl_try_decoder_cmd,
+ .vidioc_decoder_cmd = wave6_vpu_dec_decoder_cmd,
+
+ .vidioc_subscribe_event = wave6_vpu_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static int wave6_vpu_dec_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct vpu_instance *inst = wave6_ctrl_to_vpu_inst(ctrl);
+
+ dev_dbg(inst->dev->dev, "%s: name %s value %d\n",
+ __func__, ctrl->name, ctrl->val);
+
+ switch (ctrl->id) {
+ case V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY_ENABLE:
+ inst->disp_mode = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY:
+ case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
+ case V4L2_CID_MPEG_VIDEO_HEVC_PROFILE:
+ case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static const struct v4l2_ctrl_ops wave6_vpu_dec_ctrl_ops = {
+ .s_ctrl = wave6_vpu_dec_s_ctrl,
+};
+
+static void wave6_set_dec_openparam(struct dec_open_param *open_param,
+ struct vpu_instance *inst)
+{
+ open_param->inst_buffer.temp_base = inst->dev->temp_vbuf.daddr;
+ open_param->inst_buffer.temp_size = inst->dev->temp_vbuf.size;
+ open_param->disp_mode = inst->disp_mode;
+}
+
+static int wave6_vpu_dec_create_instance(struct vpu_instance *inst)
+{
+ int ret;
+ struct dec_open_param open_param;
+
+ memset(&open_param, 0, sizeof(struct dec_open_param));
+
+ wave6_vpu_core_activate(inst->dev);
+ ret = pm_runtime_resume_and_get(inst->dev->dev);
+ if (ret) {
+ dev_err(inst->dev->dev, "runtime_resume failed %d\n", ret);
+ goto error_deactivate;
+ }
+
+ inst->std = wave6_to_codec_std(inst->type, inst->src_fmt.pixelformat);
+ if (inst->std == STD_UNKNOWN) {
+ dev_err(inst->dev->dev, "unsupported pixelformat: %.4s\n",
+ (char *)&inst->src_fmt.pixelformat);
+ ret = -EINVAL;
+ goto error_pm;
+ }
+
+ wave6_set_dec_openparam(&open_param, inst);
+
+ ret = wave6_vpu_dec_open(inst, &open_param);
+ if (ret) {
+ dev_err(inst->dev->dev, "failed create instance : %d\n", ret);
+ goto error_pm;
+ }
+
+ dev_dbg(inst->dev->dev, "[%d] decoder\n", inst->id);
+
+ scoped_guard(spinlock, &inst->dev->inst_lock)
+ list_add_tail(&inst->list, &inst->dev->instances);
+ wave6_vpu_enable_instance(inst);
+ wave6_vpu_set_instance_state(inst, VPU_INST_STATE_OPEN);
+
+ return 0;
+
+error_pm:
+ pm_runtime_put_sync(inst->dev->dev);
+error_deactivate:
+ wave6_vpu_core_deactivate(inst->dev);
+
+ return ret;
+}
+
+static int wave6_vpu_dec_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
+ unsigned int *num_planes, unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct vpu_instance *inst = vb2_get_drv_priv(q);
+ struct v4l2_pix_format_mplane inst_format =
+ (V4L2_TYPE_IS_OUTPUT(q->type)) ? inst->src_fmt : inst->dst_fmt;
+
+ dev_dbg(inst->dev->dev, "%s: num_buffers %d num_planes %d type %d\n",
+ __func__, *num_buffers, *num_planes, q->type);
+
+ if (*num_planes) {
+ if (inst_format.num_planes != *num_planes)
+ return -EINVAL;
+
+ for (unsigned int i = 0; i < *num_planes; i++) {
+ if (sizes[i] < inst_format.plane_fmt[i].sizeimage)
+ return -EINVAL;
+ }
+ } else {
+ *num_planes = inst_format.num_planes;
+ for (unsigned int i = 0; i < *num_planes; i++) {
+ sizes[i] = inst_format.plane_fmt[i].sizeimage;
+ dev_dbg(inst->dev->dev, "size[%d] : %d\n", i, sizes[i]);
+ }
+
+ if (V4L2_TYPE_IS_CAPTURE(q->type)) {
+ struct v4l2_ctrl *ctrl;
+ unsigned int min_disp_cnt = 0;
+
+ ctrl = v4l2_ctrl_find(&inst->v4l2_ctrl_hdl,
+ V4L2_CID_MIN_BUFFERS_FOR_CAPTURE);
+ if (ctrl)
+ min_disp_cnt = v4l2_ctrl_g_ctrl(ctrl);
+
+ *num_buffers = max(*num_buffers, min_disp_cnt);
+ }
+ }
+
+ if (V4L2_TYPE_IS_OUTPUT(q->type) && inst->state == VPU_INST_STATE_SEEK)
+ wave6_vpu_dec_destroy_instance(inst);
+
+ return 0;
+}
+
+static int wave6_vpu_dec_seek_header(struct vpu_instance *inst)
+{
+ struct dec_seq_info seq_info;
+ int ret;
+
+ memset(&seq_info, 0, sizeof(struct dec_seq_info));
+
+ ret = wave6_vpu_dec_issue_seq_init(inst);
+ if (ret) {
+ dev_err(inst->dev->dev, "failed wave6_vpu_dec_issue_seq_init %d\n", ret);
+ return ret;
+ }
+
+ ret = wave6_vpu_wait_interrupt(inst, W6_VPU_TIMEOUT);
+ if (ret) {
+ dev_err(inst->dev->dev, "failed to call vpu_wait_interrupt()\n");
+ return ret;
+ }
+
+ ret = wave6_vpu_dec_complete_seq_init(inst, &seq_info);
+ if (ret) {
+ dev_err(inst->dev->dev, "vpu_dec_complete_seq_init: %d, reason : 0x%x\n",
+ ret, seq_info.err_reason);
+ if ((seq_info.err_reason & WAVE6_SYSERR_NOT_SUPPORT) ||
+ (seq_info.err_reason & WAVE6_SYSERR_NOT_SUPPORT_PROFILE)) {
+ ret = -EINVAL;
+ } else if (seq_info.err_reason == HEVC_ETCERR_INIT_SEQ_SPS_NOT_FOUND ||
+ seq_info.err_reason == AVC_ETCERR_INIT_SEQ_SPS_NOT_FOUND) {
+ wave6_handle_skipped_frame(inst);
+ ret = 0;
+ }
+ } else {
+ wave6_vpu_dec_handle_source_change(inst);
+ inst->v4l2_fh.m2m_ctx->ignore_cap_streaming = false;
+ v4l2_m2m_set_dst_buffered(inst->v4l2_fh.m2m_ctx, false);
+ if (vb2_is_streaming(v4l2_m2m_get_dst_vq(inst->v4l2_fh.m2m_ctx))) {
+ scoped_guard(mutex, &inst->lock)
+ wave6_handle_last_frame(inst, NULL);
+ }
+ }
+
+ return ret;
+}
+
+static int wave6_vpu_dec_prepare_decode(struct vpu_instance *inst)
+{
+ bool stream_end;
+
+ scoped_guard(mutex, &inst->lock)
+ stream_end = inst->drain && !inst->drain_src_buf_num;
+
+ if (stream_end && !inst->fbc_buf_registered) {
+ wave6_vpu_dec_finish_drain(inst);
+ return -EAGAIN;
+ }
+
+ wave6_handle_bitstream_buffer(inst, stream_end);
+
+ if (inst->state < VPU_INST_STATE_PIC_RUN) {
+ if (inst->state == VPU_INST_STATE_OPEN &&
+ wave6_vpu_dec_seek_header(inst)) {
+ vb2_queue_error(v4l2_m2m_get_src_vq(inst->v4l2_fh.m2m_ctx));
+ vb2_queue_error(v4l2_m2m_get_dst_vq(inst->v4l2_fh.m2m_ctx));
+ }
+
+ return -EAGAIN;
+ }
+
+ if (!stream_end)
+ wave6_vpu_dec_handle_dst_buffer(inst);
+
+ return 0;
+}
+
+static void wave6_vpu_dec_buf_queue_src(struct vb2_buffer *vb)
+{
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct vpu_instance *inst = vb2_get_drv_priv(vb->vb2_queue);
+
+ dev_dbg(inst->dev->dev, "type %4d index %4d size[0] %4ld size[1] : %4ld | size[2] : %4ld\n",
+ vb->type, vb->index, vb2_plane_size(&vbuf->vb2_buf, 0),
+ vb2_plane_size(&vbuf->vb2_buf, 1), vb2_plane_size(&vbuf->vb2_buf, 2));
+
+ vbuf->sequence = inst->queued_src_buf_num++;
+
+ v4l2_m2m_buf_queue(inst->v4l2_fh.m2m_ctx, vbuf);
+}
+
+static void wave6_vpu_dec_buf_queue_dst(struct vb2_buffer *vb)
+{
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct vpu_instance *inst = vb2_get_drv_priv(vb->vb2_queue);
+
+ dev_dbg(inst->dev->dev, "type %4d index %4d size[0] %4ld size[1] : %4ld | size[2] : %4ld\n",
+ vb->type, vb->index, vb2_plane_size(&vbuf->vb2_buf, 0),
+ vb2_plane_size(&vbuf->vb2_buf, 1), vb2_plane_size(&vbuf->vb2_buf, 2));
+
+ guard(mutex)(&inst->lock);
+
+ inst->queued_dst_buf_num++;
+ if (inst->next_dst_buf_last) {
+ wave6_handle_last_frame(inst, vbuf);
+ inst->next_dst_buf_last = false;
+ } else {
+ v4l2_m2m_buf_queue(inst->v4l2_fh.m2m_ctx, vbuf);
+ }
+}
+
+static void wave6_vpu_dec_buf_queue(struct vb2_buffer *vb)
+{
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct vpu_buffer *vpu_buf = wave6_to_vpu_buf(vbuf);
+
+ vpu_buf->consumed = false;
+ vpu_buf->used = false;
+ vpu_buf->error = false;
+ if (V4L2_TYPE_IS_OUTPUT(vb->type))
+ wave6_vpu_dec_buf_queue_src(vb);
+ else
+ wave6_vpu_dec_buf_queue_dst(vb);
+}
+
+static int wave6_vpu_dec_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+ struct vpu_instance *inst = vb2_get_drv_priv(q);
+ struct v4l2_pix_format_mplane *fmt;
+ int ret = 0;
+
+ if (V4L2_TYPE_IS_OUTPUT(q->type)) {
+ fmt = &inst->src_fmt;
+ if (inst->state == VPU_INST_STATE_NONE) {
+ ret = wave6_vpu_dec_create_instance(inst);
+ if (!ret) {
+ inst->v4l2_fh.m2m_ctx->ignore_cap_streaming = true;
+ v4l2_m2m_set_dst_buffered(inst->v4l2_fh.m2m_ctx, true);
+ }
+ }
+ if (inst->state == VPU_INST_STATE_SEEK)
+ wave6_vpu_set_instance_state(inst, inst->state_in_seek);
+ } else {
+ fmt = &inst->dst_fmt;
+ if (inst->state == VPU_INST_STATE_INIT_SEQ) {
+ ret = wave6_vpu_dec_prepare_fb(inst);
+ if (ret)
+ goto exit;
+ }
+ }
+
+exit:
+ if (ret)
+ wave6_vpu_return_buffers(inst, q->type, VB2_BUF_STATE_QUEUED);
+
+ dev_dbg(inst->dev->dev, "[%d] %s %c%c%c%c %dx%d, %d buffers, ret = %d\n",
+ inst->id, V4L2_TYPE_IS_OUTPUT(q->type) ? "output" : "capture",
+ fmt->pixelformat,
+ fmt->pixelformat >> 8,
+ fmt->pixelformat >> 16,
+ fmt->pixelformat >> 24,
+ fmt->width, fmt->height, vb2_get_num_buffers(q), ret);
+
+ return ret;
+}
+
+static void wave6_vpu_dec_stop_streaming(struct vb2_queue *q)
+{
+ struct vpu_instance *inst = vb2_get_drv_priv(q);
+ struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+
+ dev_dbg(inst->dev->dev, "[%d] %s, input %d, decode %d error %d\n",
+ inst->id, V4L2_TYPE_IS_OUTPUT(q->type) ? "output" : "capture",
+ inst->queued_src_buf_num, inst->processed_buf_num, inst->error_buf_num);
+
+ if (inst->state == VPU_INST_STATE_NONE)
+ goto exit;
+
+ if (V4L2_TYPE_IS_OUTPUT(q->type)) {
+ inst->queued_src_buf_num = 0;
+ inst->processed_buf_num = 0;
+ inst->error_buf_num = 0;
+ inst->state_in_seek = inst->state;
+ wave6_vpu_dec_cancel_drain(inst);
+ wave6_vpu_set_instance_state(inst, VPU_INST_STATE_SEEK);
+ inst->sequence = 0;
+ } else {
+ if (inst->state != VPU_INST_STATE_INIT_SEQ)
+ wave6_vpu_dec_cancel_drain(inst);
+
+ scoped_guard(mutex, &inst->lock)
+ inst->next_dst_buf_last = false;
+ inst->queued_dst_buf_num = 0;
+ inst->sequence = 0;
+
+ if (!v4l2_m2m_has_stopped(m2m_ctx))
+ wave6_vpu_dec_flush_instance(inst);
+ v4l2_m2m_clear_state(m2m_ctx);
+ }
+
+exit:
+ wave6_vpu_return_buffers(inst, q->type, VB2_BUF_STATE_ERROR);
+}
+
+static int wave6_vpu_dec_buf_init(struct vb2_buffer *vb)
+{
+ struct vpu_instance *inst = vb2_get_drv_priv(vb->vb2_queue);
+ struct dec_seq_info *seq_info;
+
+ if (V4L2_TYPE_IS_OUTPUT(vb->type))
+ return 0;
+
+ if (inst->state == VPU_INST_STATE_NONE)
+ return 0;
+
+ seq_info = &inst->codec_info->dec_info.seq_info;
+ if (seq_info->c_fmt_idc != C_FMT_IDC_YUV400)
+ return 0;
+
+ for (int i = 0; i < inst->dst_fmt.num_planes; i++) {
+ void *vaddr = vb2_plane_vaddr(vb, i);
+
+ if (vaddr)
+ memset(vaddr, 0x80, vb2_plane_size(vb, i));
+ }
+
+ return 0;
+}
+
+static int wave6_vpu_dec_buf_prepare(struct vb2_buffer *vb)
+{
+ struct vpu_instance *inst = vb2_get_drv_priv(vb->vb2_queue);
+
+ if (V4L2_TYPE_IS_OUTPUT(vb->type))
+ return 0;
+
+ for (unsigned int i = 0; i < vb->num_planes; i++) {
+ unsigned int size = inst->dst_fmt.plane_fmt[i].sizeimage;
+
+ if (vb2_plane_size(vb, i) < size) {
+ dev_dbg(inst->dev->dev, "plane %u: size %u length %lu\n",
+ i, size, vb2_plane_size(vb, i));
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static const struct vb2_ops wave6_vpu_dec_vb2_ops = {
+ .queue_setup = wave6_vpu_dec_queue_setup,
+ .buf_prepare = wave6_vpu_dec_buf_prepare,
+ .buf_queue = wave6_vpu_dec_buf_queue,
+ .start_streaming = wave6_vpu_dec_start_streaming,
+ .stop_streaming = wave6_vpu_dec_stop_streaming,
+ .buf_init = wave6_vpu_dec_buf_init,
+};
+
+static void wave6_set_default_format(struct v4l2_pix_format_mplane *src_fmt,
+ struct v4l2_pix_format_mplane *dst_fmt)
+{
+ const struct vpu_format *vpu_fmt;
+
+ vpu_fmt = wave6_find_vpu_fmt_by_idx(0, VPU_FMT_TYPE_CODEC);
+ if (vpu_fmt) {
+ src_fmt->pixelformat = vpu_fmt->v4l2_pix_fmt;
+ src_fmt->num_planes = vpu_fmt->num_planes;
+ wave6_update_pix_fmt(src_fmt,
+ W6_DEF_DEC_PIC_WIDTH, W6_DEF_DEC_PIC_HEIGHT);
+ }
+
+ vpu_fmt = wave6_find_vpu_fmt_by_idx(0, VPU_FMT_TYPE_RAW);
+ if (vpu_fmt) {
+ dst_fmt->pixelformat = vpu_fmt->v4l2_pix_fmt;
+ dst_fmt->num_planes = vpu_fmt->num_planes;
+ wave6_update_pix_fmt_cap(dst_fmt,
+ W6_DEF_DEC_PIC_WIDTH, W6_DEF_DEC_PIC_HEIGHT,
+ true);
+ }
+}
+
+static int wave6_vpu_dec_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
+{
+ struct vpu_instance *inst = priv;
+ int ret;
+
+ src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+ src_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ src_vq->mem_ops = &vb2_dma_contig_memops;
+ src_vq->ops = &wave6_vpu_dec_vb2_ops;
+ src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ src_vq->buf_struct_size = sizeof(struct vpu_buffer);
+ src_vq->drv_priv = inst;
+ src_vq->lock = &inst->dev->dev_lock;
+ src_vq->dev = inst->dev->v4l2_dev.dev;
+ ret = vb2_queue_init(src_vq);
+ if (ret)
+ return ret;
+
+ dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ dst_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ dst_vq->mem_ops = &vb2_dma_contig_memops;
+ dst_vq->ops = &wave6_vpu_dec_vb2_ops;
+ dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ dst_vq->buf_struct_size = sizeof(struct vpu_buffer);
+ dst_vq->drv_priv = inst;
+ dst_vq->lock = &inst->dev->dev_lock;
+ dst_vq->dev = inst->dev->v4l2_dev.dev;
+ ret = vb2_queue_init(dst_vq);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static const struct vpu_instance_ops wave6_vpu_dec_inst_ops = {
+ .prepare_process = wave6_vpu_dec_prepare_decode,
+ .start_process = wave6_vpu_dec_start_decode,
+ .finish_process = wave6_vpu_dec_finish_decode,
+};
+
+static int wave6_vpu_open_dec(struct file *filp)
+{
+ struct video_device *vdev = video_devdata(filp);
+ struct vpu_core_device *core = video_drvdata(filp);
+ struct vpu_instance *inst = NULL;
+ int ret;
+
+ inst = kzalloc_obj(*inst);
+ if (!inst)
+ return -ENOMEM;
+
+ init_completion(&inst->irq_done);
+ INIT_DELAYED_WORK(&inst->task_timer, wave6_vpu_device_run_timeout);
+ mutex_init(&inst->lock);
+ inst->dev = core;
+ inst->type = VPU_INST_TYPE_DEC;
+ inst->ops = &wave6_vpu_dec_inst_ops;
+
+ v4l2_fh_init(&inst->v4l2_fh, vdev);
+ filp->private_data = &inst->v4l2_fh;
+ v4l2_fh_add(&inst->v4l2_fh, filp);
+
+ inst->v4l2_fh.m2m_ctx =
+ v4l2_m2m_ctx_init(core->m2m_dev, inst, wave6_vpu_dec_queue_init);
+ if (IS_ERR(inst->v4l2_fh.m2m_ctx)) {
+ ret = PTR_ERR(inst->v4l2_fh.m2m_ctx);
+ goto free_inst;
+ }
+
+ v4l2_ctrl_handler_init(&inst->v4l2_ctrl_hdl, 10);
+ v4l2_ctrl_new_std(&inst->v4l2_ctrl_hdl, &wave6_vpu_dec_ctrl_ops,
+ V4L2_CID_MIN_BUFFERS_FOR_CAPTURE,
+ 1, WAVE6_MAX_FBS, 1, 1);
+ v4l2_ctrl_new_std(&inst->v4l2_ctrl_hdl, &wave6_vpu_dec_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY,
+ 0, 0, 1, 0);
+ v4l2_ctrl_new_std(&inst->v4l2_ctrl_hdl, &wave6_vpu_dec_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY_ENABLE,
+ 0, 1, 1, 0);
+ v4l2_ctrl_new_std_menu(&inst->v4l2_ctrl_hdl, &wave6_vpu_dec_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
+ V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE,
+ 0, V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN);
+ v4l2_ctrl_new_std_menu(&inst->v4l2_ctrl_hdl, &wave6_vpu_dec_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+ V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, 0,
+ V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE);
+
+ if (inst->v4l2_ctrl_hdl.error) {
+ ret = -ENODEV;
+ goto err_m2m_release;
+ }
+
+ inst->v4l2_fh.ctrl_handler = &inst->v4l2_ctrl_hdl;
+ ret = v4l2_ctrl_handler_setup(&inst->v4l2_ctrl_hdl);
+ if (ret)
+ goto err_m2m_release;
+
+ wave6_set_default_format(&inst->src_fmt, &inst->dst_fmt);
+ inst->colorspace = V4L2_COLORSPACE_DEFAULT;
+ inst->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ inst->quantization = V4L2_QUANTIZATION_DEFAULT;
+ inst->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+
+ return 0;
+
+err_m2m_release:
+ v4l2_m2m_ctx_release(inst->v4l2_fh.m2m_ctx);
+ v4l2_ctrl_handler_free(&inst->v4l2_ctrl_hdl);
+free_inst:
+ v4l2_fh_del(&inst->v4l2_fh, filp);
+ v4l2_fh_exit(&inst->v4l2_fh);
+ mutex_destroy(&inst->lock);
+ kfree(inst);
+
+ return ret;
+}
+
+static int wave6_vpu_dec_release(struct file *filp)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(filp);
+
+ v4l2_m2m_ctx_release(inst->v4l2_fh.m2m_ctx);
+
+ if (inst->state != VPU_INST_STATE_NONE)
+ wave6_vpu_dec_destroy_instance(inst);
+
+ v4l2_ctrl_handler_free(&inst->v4l2_ctrl_hdl);
+ v4l2_fh_del(&inst->v4l2_fh, filp);
+ v4l2_fh_exit(&inst->v4l2_fh);
+ mutex_destroy(&inst->lock);
+ kfree(inst);
+
+ return 0;
+}
+
+static const struct v4l2_file_operations wave6_vpu_dec_fops = {
+ .owner = THIS_MODULE,
+ .open = wave6_vpu_open_dec,
+ .release = wave6_vpu_dec_release,
+ .unlocked_ioctl = video_ioctl2,
+ .poll = v4l2_m2m_fop_poll,
+ .mmap = v4l2_m2m_fop_mmap,
+};
+
+int wave6_vpu_dec_register_device(struct vpu_core_device *core)
+{
+ struct video_device *vdev_dec;
+ int ret;
+
+ vdev_dec = video_device_alloc();
+ if (!vdev_dec)
+ return -ENOMEM;
+
+ strscpy(vdev_dec->name, VPU_DEC_DEV_NAME, sizeof(vdev_dec->name));
+ vdev_dec->fops = &wave6_vpu_dec_fops;
+ vdev_dec->ioctl_ops = &wave6_vpu_dec_ioctl_ops;
+ vdev_dec->release = video_device_release;
+ vdev_dec->v4l2_dev = &core->v4l2_dev;
+ vdev_dec->vfl_dir = VFL_DIR_M2M;
+ vdev_dec->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
+ vdev_dec->lock = &core->dev_lock;
+ video_set_drvdata(vdev_dec, core);
+
+ ret = video_register_device(vdev_dec, VFL_TYPE_VIDEO, -1);
+ if (ret) {
+ video_device_release(vdev_dec);
+ return ret;
+ }
+
+ core->video_dev_dec = vdev_dec;
+
+ return 0;
+}
+
+void wave6_vpu_dec_unregister_device(struct vpu_core_device *core)
+{
+ video_unregister_device(core->video_dev_dec);
+}
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
index 7d6a734f64b4..d8d7b7916d88 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
@@ -376,6 +376,7 @@ struct vpu_irq {
* @m2m_dev: V4L2 mem2mem device
* @instances: List of VPU instances
* @inst_lock: Spinlock protecting instance list
+ * @video_dev_dec: Video device node for decoder
* @dev_lock: Mutex protecting video_device, src, dst vb2_queue
* @hw_lock: Mutex protecting register access
* @attr: Hardware attributes retrieved after boot
@@ -397,6 +398,7 @@ struct vpu_core_device {
struct v4l2_m2m_dev *m2m_dev;
struct list_head instances;
spinlock_t inst_lock; /* Protects instance list */
+ struct video_device *video_dev_dec;
struct mutex dev_lock; /* Protects video_device, src, dst vb2_queue */
struct mutex hw_lock; /* Protects register access */
struct vpu_attr attr;
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h b/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
index d1b53f509dfa..ef261b5d328c 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
@@ -20,6 +20,15 @@
#define W6_MAX_PIC_STRIDE (4096 * 4)
#define W6_PIC_STRIDE_ALIGNMENT 32
+#define W6_FBC_BUF_ALIGNMENT 32
+#define W6_DEC_BUF_ALIGNMENT 32
+#define W6_DEF_DEC_PIC_WIDTH 720
+#define W6_DEF_DEC_PIC_HEIGHT 480
+#define W6_MIN_DEC_PIC_WIDTH 64
+#define W6_MIN_DEC_PIC_HEIGHT 64
+#define W6_MAX_DEC_PIC_WIDTH 4096
+#define W6_MAX_DEC_PIC_HEIGHT 4096
+#define W6_DEC_PIC_SIZE_STEP 1
#define W6_VPU_POLL_DELAY_US 10
#define W6_VPU_POLL_TIMEOUT 300000
@@ -29,6 +38,7 @@
#define HOST_ENDIAN VDI_128BIT_LITTLE_ENDIAN
#define VPU_STREAM_ENDIAN HOST_ENDIAN
+#define VPU_SOURCE_ENDIAN HOST_ENDIAN
#define W6_MAX_REMAP_PAGE_SIZE (1024 * 1024)
#define W6_MAX_CODE_BUF_SIZE (4 * 1024 * 1024)
--
2.41.0.windows.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v8 06/11] media: chips-media: wave6: Add Wave6 encoder firmware interface
2026-11-19 8:59 [PATCH v8 00/11] Add support for Wave6 video codec driver Nas Chung
` (4 preceding siblings ...)
2026-11-19 8:59 ` [PATCH v8 05/11] media: chips-media: wave6: Add Wave6 m2m decoder Nas Chung
@ 2026-11-19 9:00 ` Nas Chung
2026-11-19 9:10 ` sashiko-bot
2026-11-19 9:00 ` [PATCH v8 07/11] media: chips-media: wave6: Add Wave6 m2m encoder Nas Chung
` (4 subsequent siblings)
10 siblings, 1 reply; 23+ messages in thread
From: Nas Chung @ 2026-11-19 9:00 UTC (permalink / raw)
To: mchehab, hverkuil, robh, krzk+dt, conor+dt, Frank.Li, s.hauer
Cc: linux-media, devicetree, linux-kernel, imx, linux-arm-kernel,
festevam, kernel, nicolas.dufresne, ming.qian, marek.vasut,
jackson.lee, lafley.kim, Nas Chung
Add an interface layer to manage hardware register configuration
and communication with the Chips&Media Wave6 video codec IP for
video encoding.
The interface provides low-level helper functions used by the
Wave6 core driver to implement video encoding operations.
It handles command submission to the firmware via MMIO registers,
and waits for a response by polling the firmware busy flag.
The encoder commands cover instance creation and destruction,
parameter setting and change, frame buffer registration, and picture
encoding.
Parameter setting and picture encoding complete asynchronously:
the interface interrupt reports completion and a query command reads
back the result.
Tested-by: Ming Qian <ming.qian@oss.nxp.com>
Tested-by: Marek Vasut <marek.vasut@mailbox.org>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
---
.../platform/chips-media/wave6/wave6-hw.c | 940 +++++++++++++++++-
.../platform/chips-media/wave6/wave6-hw.h | 7 +
.../chips-media/wave6/wave6-regdefine.h | 312 ++++++
.../chips-media/wave6/wave6-vpu-core.c | 3 +-
.../platform/chips-media/wave6/wave6-vpuapi.c | 131 +++
.../platform/chips-media/wave6/wave6-vpuapi.h | 428 ++++++++
.../chips-media/wave6/wave6-vpuconfig.h | 14 +
7 files changed, 1833 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/chips-media/wave6/wave6-hw.c b/drivers/media/platform/chips-media/wave6/wave6-hw.c
index 872a540df021..171391f797cb 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-hw.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-hw.c
@@ -162,7 +162,9 @@ void wave6_vpu_enable_interrupt(struct vpu_core_device *core)
lockdep_assert_held(&core->hw_lock);
- data = BIT(W6_INT_BIT_INIT_SEQ);
+ data = BIT(W6_INT_BIT_ENC_SET_PARAM);
+ data |= BIT(W6_INT_BIT_ENC_PIC);
+ data |= BIT(W6_INT_BIT_INIT_SEQ);
data |= BIT(W6_INT_BIT_DEC_PIC);
data |= BIT(W6_INT_BIT_BSBUF_ERROR);
data |= BIT(W6_INT_BIT_REQ_WORK_BUF);
@@ -932,3 +934,939 @@ int wave6_vpu_dec_flush_instance(struct vpu_instance *inst)
return 0;
}
+
+struct enc_cmd_set_param_reg {
+ u32 enable;
+ u32 src_size;
+ u32 custom_map_endian;
+ u32 sps;
+ u32 pps;
+ u32 gop;
+ u32 intra;
+ u32 conf_win0;
+ u32 conf_win1;
+ u32 rdo;
+ u32 slice;
+ u32 intra_refresh;
+ u32 intra_qp;
+ u32 rc_frame_rate;
+ u32 rc_target_rate;
+ u32 rc;
+ u32 hvs;
+ u32 rc_max_bitrate;
+ u32 rc_vbv_buffer_size;
+ u32 inter_qp;
+ u32 rot;
+ u32 num_units_in_tick;
+ u32 time_scale;
+ u32 num_ticks_poc_diff_one;
+ u32 max_intra_pic_bit;
+ u32 max_inter_pic_bit;
+ u32 bg;
+ u32 non_vcl_param;
+ u32 vui_rbsp_addr;
+ u32 hrd_rbsp_addr;
+ u32 qround_offset;
+ u32 quant1;
+ u32 quant2;
+ u32 custom_gop;
+ u32 custom_gop_pic[MAX_CUSTOM_GOP_NUM];
+ u32 tile_param;
+ u32 custom_lambda[MAX_CUSTOM_LAMBDA_NUM];
+ u32 temp_layer_qp[MAX_NUM_CHANGEABLE_TEMP_LAYER];
+ u32 scaler_size;
+ u32 scaler;
+ u32 color;
+ u32 sar;
+ u32 sar_extended;
+};
+
+struct enc_cmd_change_param_reg {
+ u32 enable;
+ u32 rc_target_rate;
+};
+
+int wave6_vpu_build_up_enc_param(struct device *dev, struct vpu_instance *inst,
+ struct enc_open_param *param)
+{
+ struct enc_info *p_enc_info = &inst->codec_info->enc_info;
+ u32 reg_val;
+ int ret;
+
+ lockdep_assert_held(&inst->dev->hw_lock);
+
+ p_enc_info->seq_info_obtained = false;
+ p_enc_info->sec_axi.use_enc_rdo = true;
+ p_enc_info->sec_axi.use_enc_lf = true;
+ p_enc_info->mir_dir = param->mir_dir;
+ p_enc_info->rot_angle = param->rot_angle;
+
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_TEMP_BASE,
+ param->inst_buffer.temp_base);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_TEMP_SIZE,
+ param->inst_buffer.temp_size);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_AR_TABLE_BASE,
+ param->inst_buffer.ar_base);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_BS_PARAM, VPU_STREAM_ENDIAN);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_SRC_OPT, 0);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_ADDR_EXT, param->ext_addr_vcpu);
+
+ reg_val = CREATE_INST_CORE_INFO_CQ_DEPTH(COMMAND_QUEUE_DEPTH) |
+ CREATE_INST_CORE_INFO_CORE_IDC(SINGLE_CORE_IDC) |
+ CREATE_INST_CORE_INFO_CORE_NUM(SINGLE_CORE);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_CORE_INFO, reg_val);
+
+ reg_val = CREATE_INST_PRIORITY_SECURITY_FLAG(param->is_secure_inst) |
+ CREATE_INST_PRIORITY_VALUE(param->inst_priority);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_PRIORITY, reg_val);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_CREATE_INST_TIMEOUT_CYCLE_COUNT,
+ W6_VPU_TIMEOUT_CYCLE_COUNT);
+
+ wave6_send_command(inst->dev, 0, inst->std, W6_CMD_CREATE_INSTANCE);
+ ret = wave6_wait_vpu_busy(inst->dev, W6_VPU_BUSY_STATUS);
+ if (ret) {
+ dev_err(inst->dev->dev, "%s: timeout\n", __func__);
+ return ret;
+ }
+
+ if (!vpu_read_reg_relaxed(inst->dev, W6_RET_SUCCESS)) {
+ u32 reason_code = vpu_read_reg_relaxed(inst->dev, W6_RET_FAIL_REASON);
+
+ wave6_print_reg_err(inst->dev, reason_code);
+ return -EIO;
+ }
+
+ inst->id = vpu_read_reg_relaxed(inst->dev, W6_RET_INSTANCE_ID);
+
+ return 0;
+}
+
+static int wave6_set_enc_crop_info(u32 codec, struct enc_codec_param *param,
+ u32 rot_mir, int width, int height)
+{
+ int aligned_height = (codec == W_HEVC_ENC) ? ALIGN(height, 32) : ALIGN(height, 16);
+ int aligned_width = (codec == W_HEVC_ENC) ? ALIGN(width, 32) : ALIGN(width, 16);
+ int crop_right, crop_left, crop_top, crop_bot;
+ int pad_right, pad_bot;
+
+ if (codec == W_HEVC_ENC &&
+ (!rot_mir ||
+ rot_mir == (SET_PARAM_ROT_MIRH | SET_PARAM_ROT_MIRV | SET_PARAM_ROT_180)))
+ return 0;
+
+ pad_right = aligned_width - width;
+ pad_bot = aligned_height - height;
+
+ if (param->conf_win.right > 0)
+ crop_right = param->conf_win.right + pad_right;
+ else
+ crop_right = pad_right;
+
+ if (param->conf_win.bottom > 0)
+ crop_bot = param->conf_win.bottom + pad_bot;
+ else
+ crop_bot = pad_bot;
+
+ crop_top = param->conf_win.top;
+ crop_left = param->conf_win.left;
+
+ if (rot_mir == SET_PARAM_ROT_90 ||
+ rot_mir == (SET_PARAM_ROT_MIRH | SET_PARAM_ROT_MIRV | SET_PARAM_ROT_270)) {
+ param->conf_win.top = crop_right;
+ param->conf_win.left = crop_top;
+ param->conf_win.bottom = crop_left;
+ param->conf_win.right = crop_bot;
+ } else if (rot_mir == SET_PARAM_ROT_180 ||
+ rot_mir == (SET_PARAM_ROT_MIRH | SET_PARAM_ROT_MIRV)) {
+ param->conf_win.top = crop_bot;
+ param->conf_win.left = crop_right;
+ param->conf_win.bottom = crop_top;
+ param->conf_win.right = crop_left;
+ } else if (rot_mir == SET_PARAM_ROT_270 ||
+ rot_mir == (SET_PARAM_ROT_MIRH | SET_PARAM_ROT_MIRV | SET_PARAM_ROT_90)) {
+ param->conf_win.top = crop_left;
+ param->conf_win.left = crop_bot;
+ param->conf_win.bottom = crop_right;
+ param->conf_win.right = crop_top;
+ } else if (rot_mir == SET_PARAM_ROT_MIRV ||
+ rot_mir == (SET_PARAM_ROT_MIRH | SET_PARAM_ROT_180)) {
+ param->conf_win.top = crop_bot;
+ param->conf_win.left = crop_left;
+ param->conf_win.bottom = crop_top;
+ param->conf_win.right = crop_right;
+ } else if (rot_mir == SET_PARAM_ROT_MIRH ||
+ rot_mir == (SET_PARAM_ROT_MIRV | SET_PARAM_ROT_180)) {
+ param->conf_win.top = crop_top;
+ param->conf_win.left = crop_right;
+ param->conf_win.bottom = crop_bot;
+ param->conf_win.right = crop_left;
+ } else if (rot_mir == (SET_PARAM_ROT_MIRV | SET_PARAM_ROT_90) ||
+ rot_mir == (SET_PARAM_ROT_MIRH | SET_PARAM_ROT_270)) {
+ param->conf_win.top = crop_left;
+ param->conf_win.left = crop_top;
+ param->conf_win.bottom = crop_right;
+ param->conf_win.right = crop_bot;
+ } else if (rot_mir == (SET_PARAM_ROT_MIRV | SET_PARAM_ROT_270) ||
+ rot_mir == (SET_PARAM_ROT_MIRH | SET_PARAM_ROT_90)) {
+ param->conf_win.top = crop_right;
+ param->conf_win.left = crop_bot;
+ param->conf_win.bottom = crop_left;
+ param->conf_win.right = crop_top;
+ } else {
+ param->conf_win.top = crop_top;
+ param->conf_win.left = crop_left;
+ param->conf_win.bottom = crop_bot;
+ param->conf_win.right = crop_right;
+ }
+
+ return 0;
+}
+
+static void wave6_update_enc_info(struct enc_info *p_enc_info)
+{
+ struct enc_open_param op = p_enc_info->open_param;
+
+ p_enc_info->width = op.pic_width;
+ p_enc_info->height = op.pic_height;
+ p_enc_info->c_fmt_idc = get_chroma_format_idc(op.out_fmt);
+}
+
+static u32 wave6_enc_rot_mir(enum rotation_angle rot_angle,
+ enum mirror_direction mir_dir)
+{
+ u32 rot_mir = 0;
+
+ switch (rot_angle) {
+ case ROT_0:
+ break;
+ case ROT_90:
+ rot_mir |= SET_PARAM_ROT_90;
+ break;
+ case ROT_180:
+ rot_mir |= SET_PARAM_ROT_180;
+ break;
+ case ROT_270:
+ rot_mir |= SET_PARAM_ROT_270;
+ break;
+ }
+
+ switch (mir_dir) {
+ case MIR_NONE:
+ break;
+ case MIR_VER:
+ rot_mir |= SET_PARAM_ROT_MIRV;
+ break;
+ case MIR_HOR:
+ rot_mir |= SET_PARAM_ROT_MIRH;
+ break;
+ case MIR_HOR_VER:
+ rot_mir |= SET_PARAM_ROT_MIRH | SET_PARAM_ROT_MIRV;
+ break;
+ }
+
+ return rot_mir;
+}
+
+static void wave6_gen_set_param_reg_common(struct enc_info *p_enc_info, enum codec_std std,
+ struct enc_cmd_set_param_reg *reg)
+{
+ struct enc_open_param *p_open_param = &p_enc_info->open_param;
+ struct enc_codec_param *p_param = &p_open_param->codec_param;
+ u32 rot_mir;
+
+ rot_mir = wave6_enc_rot_mir(p_enc_info->rot_angle, p_enc_info->mir_dir);
+ wave6_set_enc_crop_info(std, p_param, rot_mir, p_enc_info->width, p_enc_info->height);
+
+ reg->custom_map_endian = VPU_USER_DATA_ENDIAN;
+ if (rot_mir)
+ reg->rot = rot_mir | SET_PARAM_ROT_ENABLE;
+ reg->src_size = SET_PARAM_SRC_SIZE_HEIGHT(p_enc_info->height) |
+ SET_PARAM_SRC_SIZE_WIDTH(p_enc_info->width);
+ reg->conf_win0 = SET_PARAM_CONF_WIN0_BOTTOM(p_param->conf_win.bottom) |
+ SET_PARAM_CONF_WIN0_TOP(p_param->conf_win.top);
+ reg->conf_win1 = SET_PARAM_CONF_WIN1_RIGHT(p_param->conf_win.right) |
+ SET_PARAM_CONF_WIN1_LEFT(p_param->conf_win.left);
+ reg->gop = SET_PARAM_GOP_TEMP_LAYER_CNT(p_param->temp_layer_cnt) |
+ SET_PARAM_GOP_TEMP_LAYER3_QP_ENABLE(p_param->temp_layer[3].change_qp) |
+ SET_PARAM_GOP_TEMP_LAYER2_QP_ENABLE(p_param->temp_layer[2].change_qp) |
+ SET_PARAM_GOP_TEMP_LAYER1_QP_ENABLE(p_param->temp_layer[1].change_qp) |
+ SET_PARAM_GOP_TEMP_LAYER0_QP_ENABLE(p_param->temp_layer[0].change_qp) |
+ SET_PARAM_GOP_PRESET_IDX(p_param->gop_preset_idx);
+ for (int i = 0; i < MAX_NUM_CHANGEABLE_TEMP_LAYER; i++) {
+ reg->temp_layer_qp[i] = SET_PARAM_TEMP_LAYER_QP_B(p_param->temp_layer[i].qp_b) |
+ SET_PARAM_TEMP_LAYER_QP_P(p_param->temp_layer[i].qp_p) |
+ SET_PARAM_TEMP_LAYER_QP_I(p_param->temp_layer[i].qp_i);
+ }
+ reg->intra_refresh = SET_PARAM_INTRA_REFRESH_ARGUMENT(p_param->intra_refresh_arg) |
+ SET_PARAM_INTRA_REFRESH_MODE(p_param->intra_refresh_mode);
+ reg->intra_qp = SET_PARAM_INTRA_QP_MAX(p_param->max_qp_i) |
+ SET_PARAM_INTRA_QP_MIN(p_param->min_qp_i);
+ reg->inter_qp = SET_PARAM_INTER_QP_MAX_B(p_param->max_qp_b) |
+ SET_PARAM_INTER_QP_MIN_B(p_param->min_qp_b) |
+ SET_PARAM_INTER_QP_MAX_P(p_param->max_qp_p) |
+ SET_PARAM_INTER_QP_MIN_P(p_param->min_qp_p);
+ reg->rc_frame_rate = p_param->frame_rate;
+ reg->rc_target_rate = p_param->bitrate;
+ reg->rc_max_bitrate = p_param->max_bitrate;
+ reg->rc_vbv_buffer_size = p_param->cpb_size;
+ reg->rc = SET_PARAM_RC_UPDATE_SPEED(p_param->rc_update_speed) |
+ SET_PARAM_RC_INITIAL_LEVEL(p_param->rc_initial_level) |
+ SET_PARAM_RC_INITIAL_QP(p_param->rc_initial_qp) |
+ SET_PARAM_RC_MODE(p_param->rc_mode) |
+ SET_PARAM_RC_PIC_LEVEL_MAX_DELTA_QP(p_param->pic_rc_max_dqp) |
+ SET_PARAM_RC_VBV_OVERFLOW_DROP_FRAME(p_param->en_skip_frame) |
+ SET_PARAM_RC_CU_LEVEL_ENABLE(p_param->en_cu_level_rate_control) |
+ SET_PARAM_RC_ENABLE(p_param->en_rate_control);
+ reg->hvs = SET_PARAM_HVS_MAX_DELTA_QP(p_param->max_delta_qp) |
+ SET_PARAM_HVS_QP_SCALE(p_param->hvs_qp_scale_div2);
+ reg->num_units_in_tick = p_param->num_units_in_tick;
+ reg->time_scale = p_param->time_scale;
+ reg->num_ticks_poc_diff_one = p_param->num_ticks_poc_diff_one;
+ reg->max_intra_pic_bit = p_param->max_intra_pic_bit;
+ reg->max_inter_pic_bit = p_param->max_inter_pic_bit;
+ reg->bg = SET_PARAM_BG_DELTA_QP(p_param->bg_delta_qp) |
+ SET_PARAM_BG_THRESHOLD_MEAN_DIFF(p_param->bg_th_mean_diff) |
+ SET_PARAM_BG_THRESHOLD_MAX_DIFF(p_param->bg_th_diff) |
+ SET_PARAM_BG_ENABLE(p_param->en_bg_detect);
+ reg->qround_offset = SET_PARAM_QROUND_OFFSET_INTER(p_param->qround_inter) |
+ SET_PARAM_QROUND_OFFSET_INTRA(p_param->qround_intra);
+ reg->custom_gop = p_param->gop_param.size;
+ for (int i = 0; i < p_param->gop_param.size; i++) {
+ struct custom_gop_pic_param pic = p_param->gop_param.pic[i];
+
+ reg->custom_gop_pic[i] = SET_PARAM_CUSTOM_GOP_PIC_TEMP_ID(pic.temporal_id) |
+ SET_PARAM_CUSTOM_GOP_PIC_REF1_POC(pic.ref_poc_l1) |
+ SET_PARAM_CUSTOM_GOP_PIC_REF0_POC(pic.ref_poc_l0) |
+ SET_PARAM_CUSTOM_GOP_PIC_MULTI_REF_P(pic.multi_ref_p) |
+ SET_PARAM_CUSTOM_GOP_PIC_QP(pic.pic_qp) |
+ SET_PARAM_CUSTOM_GOP_PIC_POC_OFFSET(pic.poc_offset) |
+ SET_PARAM_CUSTOM_GOP_PIC_TYPE(pic.pic_type);
+ }
+ for (int i = 0; i < MAX_CUSTOM_LAMBDA_NUM; i++) {
+ reg->custom_lambda[i] = SET_PARAM_CUSTOM_LAMBDA_SSD(p_param->lambda_ssd[i]) |
+ SET_PARAM_CUSTOM_LAMBDA_SAD(p_param->lambda_sad[i]);
+ }
+ reg->scaler_size = SET_PARAM_SCALER_SIZE_HEIGHT(p_enc_info->height) |
+ SET_PARAM_SCALER_SIZE_WIDTH(p_enc_info->width);
+ reg->scaler = SET_PARAM_SCALER_COEF_MODE(p_enc_info->scaler.coef_mode) |
+ SET_PARAM_SCALER_ENABLE(p_enc_info->scaler.enable);
+ reg->color = SET_PARAM_COLOR_RANGE(p_param->color.color_range) |
+ SET_PARAM_COLOR_MATRIX_COEFF(p_param->color.matrix_coefficients) |
+ SET_PARAM_COLOR_TRANS_CHAR(p_param->color.transfer_characteristics) |
+ SET_PARAM_COLOR_PRIMARIES(p_param->color.color_primaries) |
+ SET_PARAM_COLOR_DESCRIPTION_PRESENT_ON;
+ reg->sar = SET_PARAM_SAR_ASPECT_RATIO_IDC(p_param->sar.idc) |
+ SET_PARAM_SAR_ASPECT_RATIO_ENABLE(p_param->sar.enable);
+ reg->sar_extended = SET_PARAM_SAR_EXTENDED_HEIGHT(p_param->sar.height) |
+ SET_PARAM_SAR_EXTENDED_WIDTH(p_param->sar.width);
+}
+
+static void wave6_gen_set_param_reg_hevc(struct enc_info *p_enc_info,
+ struct enc_cmd_set_param_reg *reg)
+{
+ struct enc_open_param *p_open_param = &p_enc_info->open_param;
+ struct enc_codec_param *p_param = &p_open_param->codec_param;
+
+ reg->sps = SET_PARAM_SPS_DEFAULT_SCALING_LIST(p_param->en_scaling_list) |
+ SET_PARAM_SPS_STILL_PICTURE(p_param->en_still_picture) |
+ SET_PARAM_SPS_AUTO_LEVEL_ADJUSTING_ON |
+ SET_PARAM_SPS_STRONG_INTRA_SMOOTHING(p_param->en_intra_smooth) |
+ SET_PARAM_SPS_INTRA_TRANSFORM_SKIP_ON |
+ SET_PARAM_SPS_SAMPLE_ADAPTIVE_OFFSET(p_param->en_sao) |
+ SET_PARAM_SPS_TEMPORAL_MVP(p_param->en_temporal_mvp) |
+ SET_PARAM_SPS_LONGTERM_REFERENCE(p_param->en_longterm) |
+ SET_PARAM_SPS_C_FMT_IDC(p_enc_info->c_fmt_idc) |
+ SET_PARAM_SPS_BIT_DEPTH(p_param->internal_bit_depth) |
+ SET_PARAM_SPS_TIER(p_param->tier) |
+ SET_PARAM_SPS_LEVEL(p_param->level) |
+ SET_PARAM_SPS_PROFILE(p_param->profile);
+ reg->pps = SET_PARAM_PPS_CR_QP_OFFSET(p_param->cr_qp_offset) |
+ SET_PARAM_PPS_CB_QP_OFFSET(p_param->cb_qp_offset) |
+ SET_PARAM_PPS_TC_OFFSET_DIV2(p_param->tc_offset_div2) |
+ SET_PARAM_PPS_BETA_OFFSET_DIV2(p_param->beta_offset_div2) |
+ SET_PARAM_PPS_DEBLOCKING_FILTER(!p_param->en_dbk) |
+ SET_PARAM_PPS_LF_SLICE_BOUNDARY(p_param->en_lf_slice_boundary) |
+ SET_PARAM_PPS_CONST_INTRA_PREDICTION(p_param->en_const_intra_pred);
+ reg->intra = SET_PARAM_INTRA_PERIOD(p_param->intra_period) |
+ SET_PARAM_INTRA_HEADER_MODE(p_param->forced_idr_header) |
+ SET_PARAM_INTRA_QP(p_param->qp) |
+ SET_PARAM_INTRA_REFRESH_TYPE(p_param->decoding_refresh_type);
+ reg->rdo = SET_PARAM_RDO_CUSTOM_LAMBDA(p_param->en_custom_lambda) |
+ SET_PARAM_RDO_ME_SEARCH_CENTER_ON |
+ SET_PARAM_RDO_QROUND_OFFSET(p_param->en_qround_offset) |
+ SET_PARAM_RDO_ADAPTIVE_ROUND_ON |
+ SET_PARAM_RDO_HVS_QP(p_param->en_hvs_qp);
+ reg->slice = SET_PARAM_SLICE_ARGUMENT(p_param->slice_arg) |
+ SET_PARAM_SLICE_MODE(p_param->slice_mode);
+ reg->quant2 = SET_PARAM_QUANT2_LAMBDA_DQP_INTER(p_param->lambda_dqp_inter) |
+ SET_PARAM_QUANT2_LAMBDA_DQP_INTRA(p_param->lambda_dqp_intra);
+ reg->non_vcl_param = SET_PARAM_NON_VCL_PARAM_ENCODE_VUI;
+}
+
+static void wave6_gen_set_param_reg_avc(struct enc_info *p_enc_info,
+ struct enc_cmd_set_param_reg *reg)
+{
+ struct enc_open_param *p_open_param = &p_enc_info->open_param;
+ struct enc_codec_param *p_param = &p_open_param->codec_param;
+
+ reg->sps = SET_PARAM_SPS_DEFAULT_SCALING_LIST(p_param->en_scaling_list) |
+ SET_PARAM_SPS_AUTO_LEVEL_ADJUSTING_ON |
+ SET_PARAM_SPS_LONGTERM_REFERENCE(p_param->en_longterm) |
+ SET_PARAM_SPS_C_FMT_IDC(p_enc_info->c_fmt_idc) |
+ SET_PARAM_SPS_BIT_DEPTH(p_param->internal_bit_depth) |
+ SET_PARAM_SPS_LEVEL(p_param->level) |
+ SET_PARAM_SPS_PROFILE(p_param->profile);
+ reg->pps = SET_PARAM_PPS_ENTROPY_CODING_MODE(p_param->en_cabac) |
+ SET_PARAM_PPS_TRANSFORM8X8(p_param->en_transform8x8) |
+ SET_PARAM_PPS_CR_QP_OFFSET(p_param->cr_qp_offset) |
+ SET_PARAM_PPS_CB_QP_OFFSET(p_param->cb_qp_offset) |
+ SET_PARAM_PPS_TC_OFFSET_DIV2(p_param->tc_offset_div2) |
+ SET_PARAM_PPS_BETA_OFFSET_DIV2(p_param->beta_offset_div2) |
+ SET_PARAM_PPS_DEBLOCKING_FILTER(!p_param->en_dbk) |
+ SET_PARAM_PPS_LF_SLICE_BOUNDARY(p_param->en_lf_slice_boundary) |
+ SET_PARAM_PPS_CONST_INTRA_PREDICTION(p_param->en_const_intra_pred);
+ reg->intra = SET_PARAM_INTRA_HEADER_MODE_AVC(p_param->forced_idr_header) |
+ SET_PARAM_INTRA_IDR_PERIOD_AVC(p_param->idr_period) |
+ SET_PARAM_INTRA_PERIOD_AVC(p_param->intra_period) |
+ SET_PARAM_INTRA_QP_AVC(p_param->qp);
+ reg->rdo = SET_PARAM_RDO_CUSTOM_LAMBDA(p_param->en_custom_lambda) |
+ SET_PARAM_RDO_QROUND_OFFSET(p_param->en_qround_offset) |
+ SET_PARAM_RDO_ADAPTIVE_ROUND_ON |
+ SET_PARAM_RDO_HVS_QP(p_param->en_hvs_qp);
+ reg->slice = SET_PARAM_SLICE_ARGUMENT(p_param->slice_arg) |
+ SET_PARAM_SLICE_MODE(p_param->slice_mode);
+ reg->quant2 = SET_PARAM_QUANT2_LAMBDA_DQP_INTER(p_param->lambda_dqp_inter) |
+ SET_PARAM_QUANT2_LAMBDA_DQP_INTRA(p_param->lambda_dqp_intra);
+ reg->non_vcl_param = SET_PARAM_NON_VCL_PARAM_ENCODE_VUI;
+}
+
+static void wave6_gen_change_param_reg_common(struct vpu_instance *inst,
+ struct enc_info *p_enc_info,
+ struct enc_cmd_change_param_reg *reg)
+{
+ if (p_enc_info->open_param.codec_param.bitrate != inst->enc_ctrls.bitrate) {
+ reg->enable |= SET_PARAM_ENABLE_RC_TARGET_RATE;
+ reg->rc_target_rate = inst->enc_ctrls.bitrate;
+ p_enc_info->open_param.codec_param.bitrate = inst->enc_ctrls.bitrate;
+ }
+}
+
+int wave6_vpu_enc_issue_seq_init(struct vpu_instance *inst)
+{
+ struct enc_cmd_set_param_reg reg = {};
+ struct enc_info *p_enc_info;
+ int ret;
+
+ guard(mutex)(&inst->dev->hw_lock);
+
+ p_enc_info = &inst->codec_info->enc_info;
+
+ wave6_update_enc_info(p_enc_info);
+
+ wave6_gen_set_param_reg_common(p_enc_info, inst->std, ®);
+ if (inst->std == W_HEVC_ENC)
+ wave6_gen_set_param_reg_hevc(p_enc_info, ®);
+ else if (inst->std == W_AVC_ENC)
+ wave6_gen_set_param_reg_avc(p_enc_info, ®);
+
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_OPTION, W6_SET_PARAM_OPT_COMMON);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_ENABLE, reg.enable);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_SRC_SIZE, reg.src_size);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_CUSTOM_MAP_ENDIAN,
+ reg.custom_map_endian);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_SPS, reg.sps);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_PPS, reg.pps);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_GOP, reg.gop);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_INTRA, reg.intra);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_CONF_WIN0, reg.conf_win0);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_CONF_WIN1, reg.conf_win1);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_RDO, reg.rdo);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_SLICE, reg.slice);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_INTRA_REFRESH, reg.intra_refresh);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_INTRA_QP, reg.intra_qp);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_RC_FRAME_RATE, reg.rc_frame_rate);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_RC_TARGET_RATE, reg.rc_target_rate);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_RC, reg.rc);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_HVS, reg.hvs);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_RC_MAX_BITRATE, reg.rc_max_bitrate);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_RC_VBV_BUFFER_SIZE,
+ reg.rc_vbv_buffer_size);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_INTER_QP, reg.inter_qp);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_ROT, reg.rot);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_NUM_UNITS_IN_TICK,
+ reg.num_units_in_tick);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_TIME_SCALE, reg.time_scale);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_NUM_TICKS_POC_DIFF_ONE,
+ reg.num_ticks_poc_diff_one);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_MAX_INTRA_PIC_BIT,
+ reg.max_intra_pic_bit);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_MAX_INTER_PIC_BIT,
+ reg.max_inter_pic_bit);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_BG, reg.bg);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_NON_VCL_PARAM, reg.non_vcl_param);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_VUI_RBSP_ADDR, reg.vui_rbsp_addr);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_HRD_RBSP_ADDR, reg.hrd_rbsp_addr);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_QROUND_OFFSET, reg.qround_offset);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_QUANT1, reg.quant1);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_QUANT2, reg.quant2);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_CUSTOM_GOP, reg.custom_gop);
+ for (int i = 0; i < MAX_CUSTOM_GOP_NUM; i++)
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_CUSTOM_GOP_PIC(i),
+ reg.custom_gop_pic[i]);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_TILE_PARAM, reg.tile_param);
+ for (int i = 0; i < MAX_CUSTOM_LAMBDA_NUM; i++)
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_CUSTOM_LAMBDA(i),
+ reg.custom_lambda[i]);
+ for (int i = 0; i < MAX_NUM_CHANGEABLE_TEMP_LAYER; i++)
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_TEMP_LAYER_QP(i),
+ reg.temp_layer_qp[i]);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_SCALER_SIZE, reg.scaler_size);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_SCALER, reg.scaler);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_COLOR, reg.color);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_SAR, reg.sar);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_SAR_EXTENDED, reg.sar_extended);
+
+ wave6_send_command(inst->dev, inst->id, inst->std, W6_CMD_ENC_SET_PARAM);
+ ret = wave6_wait_vpu_busy(inst->dev, W6_VPU_BUSY_STATUS);
+ if (ret) {
+ dev_err(inst->dev->dev, "%s: timeout\n", __func__);
+ return ret;
+ }
+
+ if (!vpu_read_reg_relaxed(inst->dev, W6_RET_SUCCESS)) {
+ u32 reason_code = vpu_read_reg_relaxed(inst->dev, W6_RET_FAIL_REASON);
+
+ wave6_print_reg_err(inst->dev, reason_code);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+int wave6_vpu_enc_get_seq_info(struct vpu_instance *inst, struct enc_seq_info *info)
+{
+ int ret;
+
+ lockdep_assert_held(&inst->dev->hw_lock);
+
+ ret = wave6_send_query(inst->dev, inst->id, inst->std, W6_QUERY_OPT_GET_RESULT);
+ if (ret)
+ return ret;
+
+ if (vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_ENCODING_SUCCESS) != 1) {
+ info->err_reason = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_ERR_INFO);
+ ret = -EIO;
+ } else {
+ info->warn_info = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_WARN_INFO);
+ }
+
+ info->min_frame_buffer_count = vpu_read_reg_relaxed(inst->dev,
+ W6_RET_ENC_NUM_REQUIRED_FBC_FB);
+ info->min_src_frame_count = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_MIN_SRC_BUF_NUM);
+ info->max_latency_pictures = vpu_read_reg_relaxed(inst->dev,
+ W6_RET_ENC_PIC_MAX_LATENCY_PICTURES);
+ info->req_mv_buffer_count = vpu_read_reg_relaxed(inst->dev,
+ W6_RET_ENC_NUM_REQUIRED_COL_BUF);
+
+ return ret;
+}
+
+int wave6_vpu_enc_issue_seq_change(struct vpu_instance *inst, bool *changed)
+{
+ struct enc_cmd_change_param_reg reg = {};
+ struct enc_info *p_enc_info;
+ int ret;
+
+ guard(mutex)(&inst->dev->hw_lock);
+
+ p_enc_info = &inst->codec_info->enc_info;
+
+ wave6_gen_change_param_reg_common(inst, p_enc_info, ®);
+
+ if (!reg.enable)
+ return 0;
+
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_OPTION,
+ W6_SET_PARAM_OPT_CHANGE_PARAM);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_ENABLE, reg.enable);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_SET_PARAM_RC_TARGET_RATE, reg.rc_target_rate);
+
+ wave6_send_command(inst->dev, inst->id, inst->std, W6_CMD_ENC_SET_PARAM);
+ ret = wave6_wait_vpu_busy(inst->dev, W6_VPU_BUSY_STATUS);
+ if (ret) {
+ dev_warn(inst->dev->dev, "enc set param timed out\n");
+ return ret;
+ }
+
+ if (!vpu_read_reg_relaxed(inst->dev, W6_RET_SUCCESS)) {
+ u32 reason_code = vpu_read_reg_relaxed(inst->dev, W6_RET_FAIL_REASON);
+
+ wave6_print_reg_err(inst->dev, reason_code);
+ return -EIO;
+ }
+
+ *changed = true;
+
+ return 0;
+}
+
+int wave6_vpu_enc_register_frame_buffer(struct vpu_instance *inst)
+{
+ struct enc_info *p_enc_info;
+ struct frame_buffer *fb_arr;
+ dma_addr_t fbc_cr_tbl_addr;
+ u32 fbc_count, mv_count;
+ int ret;
+
+ guard(mutex)(&inst->dev->hw_lock);
+
+ p_enc_info = &inst->codec_info->enc_info;
+
+ fb_arr = &inst->frame_buf[0];
+ fbc_count = p_enc_info->seq_info.min_frame_buffer_count;
+ mv_count = p_enc_info->seq_info.req_mv_buffer_count;
+ if (!fbc_count || fbc_count > MAX_SET_FB_GROUP_SIZE ||
+ !mv_count || mv_count > MAX_SET_FB_GROUP_SIZE)
+ return -EINVAL;
+
+ if (!wave6_check_aux_vbuf(inst, AUX_BUF_FBC_Y_TBL, fbc_count) ||
+ !wave6_check_aux_vbuf(inst, AUX_BUF_FBC_C_TBL, fbc_count) ||
+ !wave6_check_aux_vbuf(inst, AUX_BUF_SUB_SAMPLE, fbc_count) ||
+ !wave6_check_aux_vbuf(inst, AUX_BUF_MV_COL, mv_count))
+ return -EINVAL;
+
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_OPTION,
+ SET_FB_OPTION_END | SET_FB_OPTION_START);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_PIC_INFO,
+ SET_FB_PIC_INFO_STRIDE(fb_arr[0].stride));
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_PIC_SIZE,
+ SET_FB_PIC_SIZE_WIDTH(fb_arr[0].stride) |
+ SET_FB_PIC_SIZE_HEIGHT(fb_arr[0].height));
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_NUM,
+ SET_FB_NUM_FBC_END_IDX(fbc_count - 1) |
+ SET_FB_NUM_MV_COL_END_IDX(mv_count - 1));
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_FBC_STRIDE,
+ SET_FB_FBC_STRIDE_L(fb_arr[0].stride) |
+ SET_FB_FBC_STRIDE_C(fb_arr[0].stride_c));
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_DEFAULT_CDF, 0);
+
+ for (u32 idx = 0; idx < fbc_count; idx++) {
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_FBC_Y(idx), fb_arr[idx].buf_y);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_FBC_C(idx), fb_arr[idx].buf_cb);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_FBC_CR(idx), fb_arr[idx].buf_cr);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_FBC_Y_OFFSET(idx),
+ inst->aux_vbuf[AUX_BUF_FBC_Y_TBL][idx].daddr);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_FBC_C_OFFSET(idx),
+ inst->aux_vbuf[AUX_BUF_FBC_C_TBL][idx].daddr);
+ fbc_cr_tbl_addr = inst->aux_vbuf[AUX_BUF_FBC_C_TBL][idx].daddr +
+ (inst->aux_vbuf[AUX_BUF_FBC_C_TBL][idx].size >> 1);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_FBC_CR_OFFSET(idx),
+ fbc_cr_tbl_addr);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_SUB_SAMPLED(idx),
+ inst->aux_vbuf[AUX_BUF_SUB_SAMPLE][idx].daddr);
+ }
+ for (u32 idx = 0; idx < mv_count; idx++)
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_SET_FB_MV_COL(idx),
+ inst->aux_vbuf[AUX_BUF_MV_COL][idx].daddr);
+
+ wave6_send_command(inst->dev, inst->id, inst->std, W6_CMD_SET_FB);
+ ret = wave6_wait_vpu_busy(inst->dev, W6_VPU_BUSY_STATUS);
+ if (ret) {
+ dev_err(inst->dev->dev, "%s: timeout\n", __func__);
+ return ret;
+ }
+
+ if (!vpu_read_reg_relaxed(inst->dev, W6_RET_SUCCESS))
+ return -EIO;
+
+ return 0;
+}
+
+struct enc_cmd_enc_pic_reg {
+ u32 bs_start;
+ u32 bs_size;
+ u32 bs_option;
+ u32 sec_axi;
+ u32 report;
+ u32 mv_histo0;
+ u32 mv_histo1;
+ u32 custom_map_param;
+ u32 custom_map_addr;
+ u32 src_pic_idx;
+ u32 src_addr_y;
+ u32 src_addr_u;
+ u32 src_addr_v;
+ u32 src_stride;
+ u32 src_fmt;
+ u32 src_axi_sel;
+ u32 code_option;
+ u32 param;
+ u32 longterm_pic;
+ u32 prefix_sei_nal_addr;
+ u32 prefix_sei_info;
+ u32 suffix_sei_nal_addr;
+ u32 suffix_sei_info;
+ u32 timestamp_low;
+ u32 timestamp_high;
+ u32 csc_coeff[MAX_CSC_COEFF_NUM];
+};
+
+static u32 wave6_enc_src_fmt(enum frame_buffer_format fmt)
+{
+ bool is_csc_format = false;
+ bool is_packed = false;
+ bool is_24bit = false;
+ bool is_10bit = false;
+ bool is_ayuv = false;
+
+ switch (fmt) {
+ case FORMAT_YUYV:
+ is_packed = true;
+ break;
+ case FORMAT_420_P10_16BIT_MSB:
+ is_10bit = true;
+ break;
+ case FORMAT_RGB_24BIT_PACKED:
+ is_24bit = true;
+ is_csc_format = true;
+ break;
+ case FORMAT_RGB_32BIT_PACKED:
+ is_csc_format = true;
+ break;
+ case FORMAT_RGB_P10_32BIT_PACKED:
+ is_csc_format = true;
+ is_10bit = true;
+ break;
+ case FORMAT_YUV444_24BIT_PACKED:
+ is_24bit = true;
+ is_ayuv = true;
+ is_csc_format = true;
+ break;
+ case FORMAT_YUV444_24BIT:
+ is_ayuv = true;
+ break;
+ default:
+ break;
+ }
+
+ return ENC_PIC_SRC_FMT_C_FMT_IDC(get_chroma_format_idc(fmt)) |
+ ENC_PIC_SRC_FMT_CSC_24BIT(is_24bit) |
+ ENC_PIC_SRC_FMT_CSC_AYUV(is_ayuv) |
+ ENC_PIC_SRC_FMT_CSC_ENABLE(is_csc_format) |
+ ENC_PIC_SRC_FMT_10BIT_ENABLE(is_10bit) |
+ ENC_PIC_SRC_FMT_YUV422_PACKED(is_packed);
+}
+
+static void wave6_gen_enc_pic_reg(struct enc_info *p_enc_info, bool cbcr_interleave, bool nv21,
+ struct enc_param *opt, struct enc_cmd_enc_pic_reg *reg)
+{
+ struct enc_open_param open = p_enc_info->open_param;
+ struct enc_codec_param param = open.codec_param;
+ u32 stride_c = opt->source_frame->stride_c;
+ u32 c_fmt_idc;
+
+ c_fmt_idc = get_chroma_format_idc(open.src_fmt);
+ if ((c_fmt_idc == C_FMT_IDC_YUV422 || c_fmt_idc == C_FMT_IDC_YUV444) &&
+ p_enc_info->c_fmt_idc == C_FMT_IDC_YUV420)
+ stride_c *= 2;
+
+ reg->src_fmt = wave6_enc_src_fmt(open.src_fmt) |
+ ENC_PIC_SRC_FMT_CSC_FMT_ORDER(opt->csc.fmt_order) |
+ ENC_PIC_SRC_FMT_ENDIAN(open.source_endian) |
+ ENC_PIC_SRC_FMT_CBCR_ORDER(nv21) |
+ ENC_PIC_SRC_FMT_2PLANE(cbcr_interleave);
+ reg->bs_start = opt->pic_stream_buffer_addr;
+ reg->bs_size = opt->pic_stream_buffer_size;
+ reg->sec_axi = ENC_PIC_SEC_AXI_RDO_ENABLE(p_enc_info->sec_axi.use_enc_rdo) |
+ ENC_PIC_SEC_AXI_LF_ENABLE(p_enc_info->sec_axi.use_enc_lf);
+ reg->report = ENC_PIC_REPORT_MV_HISTO_ENABLE(param.en_report_mv_histo);
+ reg->mv_histo0 = ENC_PIC_MV_HISTO0_THRESHOLD0(param.mv_histo_th0) |
+ ENC_PIC_MV_HISTO0_THRESHOLD1(param.mv_histo_th1);
+ reg->mv_histo1 = ENC_PIC_MV_HISTO1_THRESHOLD2(param.mv_histo_th2) |
+ ENC_PIC_MV_HISTO1_THRESHOLD3(param.mv_histo_th3);
+ reg->src_pic_idx = opt->src_idx;
+ reg->src_addr_y = opt->source_frame->buf_y;
+ reg->src_addr_u = opt->source_frame->buf_cb;
+ reg->src_addr_v = opt->source_frame->buf_cr;
+ reg->src_stride = ENC_PIC_SRC_STRIDE_L(opt->source_frame->stride) |
+ ENC_PIC_SRC_STRIDE_C(stride_c);
+ reg->src_axi_sel = ENC_PIC_SRC_AXI_SEL_PRP_PORT;
+ reg->code_option = ENC_PIC_CODE_OPTION_ENCODE_VCL |
+ ENC_PIC_CODE_OPTION_ENCODE_HEADER;
+ reg->param = ENC_PIC_PARAM_INTRA_4X4(param.intra_4x4) |
+ ENC_PIC_PARAM_FORCE_PIC_TYPE(opt->force_pic_type) |
+ ENC_PIC_PARAM_FORCE_PIC_TYPE_ENABLE(opt->force_pic) |
+ ENC_PIC_PARAM_FORCE_PIC_QP_B(opt->force_pic_qp_b) |
+ ENC_PIC_PARAM_FORCE_PIC_QP_P(opt->force_pic_qp_p) |
+ ENC_PIC_PARAM_FORCE_PIC_QP_I(opt->force_pic_qp_i) |
+ ENC_PIC_PARAM_FORCE_PIC_QP_ENABLE(opt->force_pic_qp) |
+ ENC_PIC_PARAM_PIC_SKIP_FLAG(opt->skip_picture);
+ reg->timestamp_low = lower_32_bits(opt->timestamp);
+ reg->timestamp_high = upper_32_bits(opt->timestamp);
+ reg->csc_coeff[0] = ENC_PIC_CSC_COEFF0_RY(opt->csc.coef_ry) |
+ ENC_PIC_CSC_COEFF0_GY(opt->csc.coef_gy) |
+ ENC_PIC_CSC_COEFF0_BY(opt->csc.coef_by);
+ reg->csc_coeff[1] = ENC_PIC_CSC_COEFF1_RCB(opt->csc.coef_rcb) |
+ ENC_PIC_CSC_COEFF1_GCB(opt->csc.coef_gcb) |
+ ENC_PIC_CSC_COEFF1_BCB(opt->csc.coef_bcb);
+ reg->csc_coeff[2] = ENC_PIC_CSC_COEFF2_RCR(opt->csc.coef_rcr) |
+ ENC_PIC_CSC_COEFF2_GCR(opt->csc.coef_gcr) |
+ ENC_PIC_CSC_COEFF2_BCR(opt->csc.coef_bcr);
+ reg->csc_coeff[3] = ENC_PIC_CSC_COEFF3_OFFSET_Y(opt->csc.offset_y) |
+ ENC_PIC_CSC_COEFF3_OFFSET_CB(opt->csc.offset_cb) |
+ ENC_PIC_CSC_COEFF3_OFFSET_CR(opt->csc.offset_cr);
+}
+
+int wave6_vpu_enc_start_one_frame(struct vpu_instance *inst, struct enc_param *param,
+ u32 *fail_res)
+{
+ struct enc_cmd_enc_pic_reg reg = {};
+ struct enc_info *p_enc_info;
+ int ret;
+
+ guard(mutex)(&inst->dev->hw_lock);
+
+ p_enc_info = &inst->codec_info->enc_info;
+
+ wave6_gen_enc_pic_reg(p_enc_info, inst->cbcr_interleave,
+ inst->nv21, param, ®);
+
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_BS_START, reg.bs_start);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_BS_SIZE, reg.bs_size);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_BS_OPTION, reg.bs_option);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_SEC_AXI, reg.sec_axi);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_REPORT, reg.report);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_MV_HISTO0, reg.mv_histo0);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_MV_HISTO1, reg.mv_histo1);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_CUSTOM_MAP_PARAM, reg.custom_map_param);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_CUSTOM_MAP_ADDR, reg.custom_map_addr);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_SRC_PIC_IDX, reg.src_pic_idx);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_SRC_ADDR_Y, reg.src_addr_y);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_SRC_ADDR_U, reg.src_addr_u);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_SRC_ADDR_V, reg.src_addr_v);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_SRC_STRIDE, reg.src_stride);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_SRC_FMT, reg.src_fmt);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_SRC_AXI_SEL, reg.src_axi_sel);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_CODE_OPTION, reg.code_option);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_PARAM, reg.param);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_LONGTERM_PIC, reg.longterm_pic);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_PREFIX_SEI_NAL_ADDR,
+ reg.prefix_sei_nal_addr);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_PREFIX_SEI_INFO, reg.prefix_sei_info);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_SUFFIX_SEI_NAL_ADDR,
+ reg.suffix_sei_nal_addr);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_SUFFIX_SEI_INFO, reg.suffix_sei_info);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_TIMESTAMP_LOW, reg.timestamp_low);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_TIMESTAMP_HIGH, reg.timestamp_high);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_CSC_COEFF0, reg.csc_coeff[0]);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_CSC_COEFF1, reg.csc_coeff[1]);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_CSC_COEFF2, reg.csc_coeff[2]);
+ vpu_write_reg_relaxed(inst->dev, W6_CMD_ENC_PIC_CSC_COEFF3, reg.csc_coeff[3]);
+
+ wave6_send_command(inst->dev, inst->id, inst->std, W6_CMD_ENC_PIC);
+ ret = wave6_wait_vpu_busy(inst->dev, W6_VPU_BUSY_STATUS);
+ if (ret) {
+ dev_err(inst->dev->dev, "%s: timeout\n", __func__);
+ return -ETIMEDOUT;
+ }
+
+ if (!vpu_read_reg_relaxed(inst->dev, W6_RET_SUCCESS)) {
+ *fail_res = vpu_read_reg_relaxed(inst->dev, W6_RET_FAIL_REASON);
+ wave6_print_reg_err(inst->dev, *fail_res);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+int wave6_vpu_enc_get_output_info(struct vpu_instance *inst, struct enc_output_info *info)
+{
+ u32 reg_val;
+ int ret;
+
+ if (WARN_ON(!info))
+ return -EINVAL;
+
+ guard(mutex)(&inst->dev->hw_lock);
+
+ ret = wave6_send_query(inst->dev, inst->id, inst->std, W6_QUERY_OPT_GET_RESULT);
+ if (ret)
+ return ret;
+
+ info->encoding_success = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_ENCODING_SUCCESS);
+ if (!info->encoding_success)
+ info->error_reason = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_ERR_INFO);
+ else
+ info->warn_info = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_WARN_INFO);
+
+ info->enc_pic_cnt = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_PIC_NUM);
+ info->pic_type = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_PIC_TYPE);
+ info->enc_vcl_nut = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_VCL_NUT);
+ info->non_ref_pic = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_PIC_NON_REF_PIC_FLAG);
+ info->num_of_slices = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_PIC_SLICE_NUM);
+ info->pic_skipped = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_PIC_SKIP);
+ info->num_of_intra = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_PIC_NUM_INTRA);
+ info->num_of_merge = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_PIC_NUM_MERGE);
+ info->num_of_skip_block = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_PIC_NUM_SKIP);
+ info->avg_ctu_qp = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_PIC_AVG_CTU_QP);
+ info->enc_pic_byte = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_PIC_BYTE);
+ info->enc_gop_pic_idx = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_GOP_PIC_IDX);
+ info->enc_pic_poc = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_PIC_POC);
+ info->enc_src_idx = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_USED_SRC_IDX);
+ info->wr_ptr = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_WR_PTR);
+ info->rd_ptr = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_RD_PTR);
+ info->bitstream_buffer = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_RD_PTR);
+ info->pic_distortion_low = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_PIC_DIST_LOW);
+ info->pic_distortion_high = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_PIC_DIST_HIGH);
+ info->mv_histo.cnt0 = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_HISTO_CNT0);
+ info->mv_histo.cnt1 = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_HISTO_CNT1);
+ info->mv_histo.cnt2 = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_HISTO_CNT2);
+ info->mv_histo.cnt3 = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_HISTO_CNT3);
+ info->mv_histo.cnt4 = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_HISTO_CNT4);
+ info->fme_sum.lower_x0 = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_SUM_ME0_X_DIR_LOWER);
+ info->fme_sum.higher_x0 = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_SUM_ME0_X_DIR_HIGHER);
+ info->fme_sum.lower_y0 = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_SUM_ME0_Y_DIR_LOWER);
+ info->fme_sum.higher_y0 = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_SUM_ME0_Y_DIR_HIGHER);
+ info->fme_sum.lower_x1 = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_SUM_ME1_X_DIR_LOWER);
+ info->fme_sum.higher_x1 = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_SUM_ME1_X_DIR_HIGHER);
+ info->fme_sum.lower_y1 = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_SUM_ME1_Y_DIR_LOWER);
+ info->fme_sum.higher_y1 = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_SUM_ME1_Y_DIR_HIGHER);
+ info->src_y_addr = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_SRC_Y_ADDR);
+ info->custom_map_addr = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_CUSTOM_MAP_ADDR);
+ info->prefix_sei_nal_addr = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_PREFIX_SEI_NAL_ADDR);
+ info->suffix_sei_nal_addr = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_SUFFIX_SEI_NAL_ADDR);
+
+ info->recon_frame_index = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_PIC_IDX);
+ if (info->recon_frame_index == RECON_IDX_FLAG_HEADER_ONLY)
+ info->bitstream_size = info->enc_pic_byte;
+ else if (info->recon_frame_index < 0)
+ info->bitstream_size = 0;
+ else
+ info->bitstream_size = info->enc_pic_byte;
+
+ reg_val = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_TIMESTAMP_LOW);
+ info->timestamp = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_TIMESTAMP_HIGH);
+ info->timestamp = (info->timestamp << 32) | reg_val;
+
+ return 0;
+}
+
+int wave6_vpu_enc_fini_seq(struct vpu_instance *inst, u32 *fail_res)
+{
+ int ret;
+
+ lockdep_assert_held(&inst->dev->hw_lock);
+
+ wave6_send_command(inst->dev, inst->id, inst->std, W6_CMD_DESTROY_INSTANCE);
+ ret = wave6_wait_vpu_busy(inst->dev, W6_VPU_BUSY_STATUS);
+ if (ret)
+ return -ETIMEDOUT;
+
+ if (!vpu_read_reg_relaxed(inst->dev, W6_RET_SUCCESS)) {
+ *fail_res = vpu_read_reg_relaxed(inst->dev, W6_RET_FAIL_REASON);
+ wave6_print_reg_err(inst->dev, *fail_res);
+ return -EIO;
+ }
+
+ return 0;
+}
diff --git a/drivers/media/platform/chips-media/wave6/wave6-hw.h b/drivers/media/platform/chips-media/wave6/wave6-hw.h
index b163c6bb6724..7adb0a07390e 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-hw.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-hw.h
@@ -32,6 +32,8 @@ enum product_id {
#define WTL_PIXEL_16BIT 1
#define WTL_PIXEL_32BIT 2
+#define MAX_CSC_COEFF_NUM 4
+
#define H265_NAL_UNIT_TYPE_IDR_W_RADL 19
#define H265_NAL_UNIT_TYPE_IDR_N_LP 20
@@ -46,4 +48,9 @@ int wave6_vpu_dec_get_seq_info(struct vpu_instance *inst, struct dec_seq_info *i
int wave6_vpu_dec_fini_seq(struct vpu_instance *inst, u32 *fail_res);
dma_addr_t wave6_vpu_dec_get_rd_ptr(struct vpu_instance *inst);
+int wave6_vpu_build_up_enc_param(struct device *dev, struct vpu_instance *inst,
+ struct enc_open_param *param);
+int wave6_vpu_enc_get_seq_info(struct vpu_instance *inst, struct enc_seq_info *info);
+int wave6_vpu_enc_fini_seq(struct vpu_instance *inst, u32 *fail_res);
+
#endif /* __WAVE6_HW_H__ */
diff --git a/drivers/media/platform/chips-media/wave6/wave6-regdefine.h b/drivers/media/platform/chips-media/wave6/wave6-regdefine.h
index 0651f5c1e652..1038b87b2ff9 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-regdefine.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-regdefine.h
@@ -20,6 +20,8 @@ enum wave6_command {
W6_CMD_INIT_SEQ = 0x0040,
W6_CMD_SET_FB = 0x0080,
W6_CMD_DEC_PIC = 0x0100,
+ W6_CMD_ENC_PIC = 0x0100,
+ W6_CMD_ENC_SET_PARAM = 0x0200,
W6_CMD_DEC_SET_DISP = 0x0400,
W6_CMD_INIT_WORK_BUF = 0x1000,
W6_CMD_QUERY = 0x4000,
@@ -30,6 +32,11 @@ enum wave6_init_seq_option {
W6_INIT_SEQ_OPT_W_THUMBNAIL = 17,
};
+enum wave6_set_param_option {
+ W6_SET_PARAM_OPT_COMMON = 0,
+ W6_SET_PARAM_OPT_CHANGE_PARAM = 1,
+};
+
enum wave6_dec_pic_option {
W6_DEC_PIC_OPT_NORMAL = 0,
W6_DEC_PIC_OPT_W_THUMBNAIL = 16,
@@ -53,6 +60,8 @@ enum wave6_interrupt_bit {
W6_INT_BIT_INIT_SEQ = 6,
W6_INT_BIT_SET_FB = 7,
W6_INT_BIT_DEC_PIC = 8,
+ W6_INT_BIT_ENC_PIC = 8,
+ W6_INT_BIT_ENC_SET_PARAM = 9,
W6_INT_BIT_SET_DISP = 10,
W6_INT_BIT_REQ_WORK_BUF = 12,
W6_INT_BIT_BSBUF_ERROR = 15,
@@ -125,6 +134,7 @@ enum wave6_interrupt_bit {
/* COMMON - CREATE_INSTANCE */
#define W6_CMD_CREATE_INST_BS_PARAM (W6_REG_BASE + 0x310)
+#define W6_CMD_CREATE_INST_SRC_OPT (W6_REG_BASE + 0x314)
#define W6_CMD_CREATE_INST_ADDR_EXT (W6_REG_BASE + 0x318)
#define W6_CMD_CREATE_INST_DISP_MODE (W6_REG_BASE + 0x31C)
#define W6_CMD_CREATE_INST_CORE_INFO (W6_REG_BASE + 0x330)
@@ -136,6 +146,7 @@ enum wave6_interrupt_bit {
#define CREATE_INST_PRIORITY_VALUE(x) FIELD_PREP(GENMASK(4, 0), (x))
#define W6_CMD_CREATE_INST_TEMP_BASE (W6_REG_BASE + 0x348)
#define W6_CMD_CREATE_INST_TEMP_SIZE (W6_REG_BASE + 0x34C)
+#define W6_CMD_CREATE_INST_AR_TABLE_BASE (W6_REG_BASE + 0x358)
#define W6_CMD_CREATE_INST_TIMEOUT_CYCLE_COUNT (W6_REG_BASE + 0x3A8)
/* COMMON - SET_FB */
@@ -147,6 +158,7 @@ enum wave6_interrupt_bit {
#define SET_FB_PIC_INFO_C_FMT_IDC(x) FIELD_PREP(GENMASK(26, 25), (x))
#define SET_FB_PIC_INFO_L_BIT_DEPTH(x) FIELD_PREP(GENMASK(24, 21), (x))
#define SET_FB_PIC_INFO_C_BIT_DEPTH(x) FIELD_PREP(GENMASK(20, 17), (x))
+#define SET_FB_PIC_INFO_STRIDE(x) FIELD_PREP(GENMASK(15, 0), (x))
#define W6_CMD_SET_FB_PIC_SIZE (W6_REG_BASE + 0x304)
#define SET_FB_PIC_SIZE_WIDTH(x) FIELD_PREP(GENMASK(31, 16), (x))
#define SET_FB_PIC_SIZE_HEIGHT(x) FIELD_PREP(GENMASK(15, 0), (x))
@@ -155,6 +167,9 @@ enum wave6_interrupt_bit {
#define SET_FB_NUM_FBC_END_IDX(x) FIELD_PREP(GENMASK(20, 16), (x))
#define SET_FB_NUM_MV_COL_START_IDX(x) FIELD_PREP(GENMASK(9, 5), (x))
#define SET_FB_NUM_MV_COL_END_IDX(x) FIELD_PREP(GENMASK(4, 0), (x))
+#define W6_CMD_SET_FB_FBC_STRIDE (W6_REG_BASE + 0x30C)
+#define SET_FB_FBC_STRIDE_L(x) FIELD_PREP(GENMASK(31, 16), (x))
+#define SET_FB_FBC_STRIDE_C(x) FIELD_PREP(GENMASK(15, 0), (x))
/* W6_CMD_SET_FB_FBC_Y(x)
* Address range: 0x310, 0x328, ..., 0x478 (step = 0x18)
@@ -181,6 +196,10 @@ enum wave6_interrupt_bit {
*/
#define W6_CMD_SET_FB_MV_COL(x) (W6_REG_BASE + 0x320 + ((x) * 0x18))
+/* W6_CMD_SET_FB_SUB_SAMPLED(x)
+ * Address range: 0x324, 0x33C, ..., 0x48C (step = 0x18)
+ */
+#define W6_CMD_SET_FB_SUB_SAMPLED(x) (W6_REG_BASE + 0x324 + ((x) * 0x18))
#define W6_CMD_SET_FB_DEFAULT_CDF (W6_REG_BASE + 0x494)
#define W6_CMD_SET_FB_SEGMAP (W6_REG_BASE + 0x498)
#define W6_CMD_SET_FB_MV_COL_PRE_ENT (W6_REG_BASE + 0x4DC)
@@ -330,4 +349,297 @@ enum wave6_interrupt_bit {
#define W6_RET_DEC_FLUSH_CMD_BUF_STATE_USED_IDC (W6_REG_BASE + 0x580)
#define W6_RET_DEC_FLUSH_CMD_BUF_STATE_USING_IDC (W6_REG_BASE + 0x584)
+/* ENCODER - SET_PARAM */
+#define W6_CMD_ENC_SET_PARAM_OPTION (W6_REG_BASE + 0x204)
+#define W6_CMD_ENC_SET_PARAM_ENABLE (W6_REG_BASE + 0x300)
+#define SET_PARAM_ENABLE_RC_TARGET_RATE BIT(10)
+#define W6_CMD_ENC_SET_PARAM_SRC_SIZE (W6_REG_BASE + 0x304)
+#define SET_PARAM_SRC_SIZE_HEIGHT(x) FIELD_PREP(GENMASK(31, 16), (x))
+#define SET_PARAM_SRC_SIZE_WIDTH(x) FIELD_PREP(GENMASK(15, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_CUSTOM_MAP_ENDIAN (W6_REG_BASE + 0x308)
+#define W6_CMD_ENC_SET_PARAM_SPS (W6_REG_BASE + 0x30C)
+#define SET_PARAM_SPS_DEFAULT_SCALING_LIST(x) FIELD_PREP(GENMASK(31, 31), (x))
+#define SET_PARAM_SPS_STILL_PICTURE(x) FIELD_PREP(GENMASK(30, 30), (x))
+#define SET_PARAM_SPS_AUTO_LEVEL_ADJUSTING_ON BIT(28)
+#define SET_PARAM_SPS_STRONG_INTRA_SMOOTHING(x) FIELD_PREP(GENMASK(27, 27), (x))
+#define SET_PARAM_SPS_INTRA_TRANSFORM_SKIP_ON BIT(25)
+#define SET_PARAM_SPS_SAMPLE_ADAPTIVE_OFFSET(x) FIELD_PREP(GENMASK(24, 24), (x))
+#define SET_PARAM_SPS_TEMPORAL_MVP(x) FIELD_PREP(GENMASK(23, 23), (x))
+#define SET_PARAM_SPS_LONGTERM_REFERENCE(x) FIELD_PREP(GENMASK(21, 21), (x))
+#define SET_PARAM_SPS_C_FMT_IDC(x) FIELD_PREP(GENMASK(20, 19), (x))
+#define SET_PARAM_SPS_BIT_DEPTH(x) FIELD_PREP(GENMASK(18, 14), (x))
+#define SET_PARAM_SPS_TIER(x) FIELD_PREP(GENMASK(13, 12), (x))
+#define SET_PARAM_SPS_LEVEL(x) FIELD_PREP(GENMASK(11, 3), (x))
+#define SET_PARAM_SPS_PROFILE(x) FIELD_PREP(GENMASK(2, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_PPS (W6_REG_BASE + 0x310)
+#define SET_PARAM_PPS_ENTROPY_CODING_MODE(x) FIELD_PREP(GENMASK(30, 30), (x))
+#define SET_PARAM_PPS_TRANSFORM8X8(x) FIELD_PREP(GENMASK(29, 29), (x))
+#define SET_PARAM_PPS_CR_QP_OFFSET(x) FIELD_PREP(GENMASK(23, 19), (x))
+#define SET_PARAM_PPS_CB_QP_OFFSET(x) FIELD_PREP(GENMASK(18, 14), (x))
+#define SET_PARAM_PPS_TC_OFFSET_DIV2(x) FIELD_PREP(GENMASK(13, 10), (x))
+#define SET_PARAM_PPS_BETA_OFFSET_DIV2(x) FIELD_PREP(GENMASK(9, 6), (x))
+#define SET_PARAM_PPS_DEBLOCKING_FILTER(x) FIELD_PREP(GENMASK(5, 5), (x))
+#define SET_PARAM_PPS_LF_SLICE_BOUNDARY(x) FIELD_PREP(GENMASK(2, 2), (x))
+#define SET_PARAM_PPS_CONST_INTRA_PREDICTION(x) FIELD_PREP(GENMASK(1, 1), (x))
+#define W6_CMD_ENC_SET_PARAM_GOP (W6_REG_BASE + 0x314)
+#define SET_PARAM_GOP_TEMP_LAYER_CNT(x) FIELD_PREP(GENMASK(19, 16), (x))
+#define SET_PARAM_GOP_TEMP_LAYER3_QP_ENABLE(x) FIELD_PREP(GENMASK(11, 11), (x))
+#define SET_PARAM_GOP_TEMP_LAYER2_QP_ENABLE(x) FIELD_PREP(GENMASK(10, 10), (x))
+#define SET_PARAM_GOP_TEMP_LAYER1_QP_ENABLE(x) FIELD_PREP(GENMASK(9, 9), (x))
+#define SET_PARAM_GOP_TEMP_LAYER0_QP_ENABLE(x) FIELD_PREP(GENMASK(8, 8), (x))
+#define SET_PARAM_GOP_PRESET_IDX(x) FIELD_PREP(GENMASK(7, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_INTRA (W6_REG_BASE + 0x318)
+#define SET_PARAM_INTRA_PERIOD(x) FIELD_PREP(GENMASK(31, 16), (x))
+#define SET_PARAM_INTRA_HEADER_MODE(x) FIELD_PREP(GENMASK(10, 9), (x))
+#define SET_PARAM_INTRA_QP(x) FIELD_PREP(GENMASK(8, 3), (x))
+#define SET_PARAM_INTRA_REFRESH_TYPE(x) FIELD_PREP(GENMASK(2, 0), (x))
+
+/* W6_CMD_ENC_SET_PARAM_INTRA for AVC specific fields */
+#define SET_PARAM_INTRA_HEADER_MODE_AVC(x) FIELD_PREP(GENMASK(29, 28), (x))
+#define SET_PARAM_INTRA_IDR_PERIOD_AVC(x) FIELD_PREP(GENMASK(27, 17), (x))
+#define SET_PARAM_INTRA_PERIOD_AVC(x) FIELD_PREP(GENMASK(16, 6), (x))
+#define SET_PARAM_INTRA_QP_AVC(x) FIELD_PREP(GENMASK(5, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_CONF_WIN0 (W6_REG_BASE + 0x31C)
+#define SET_PARAM_CONF_WIN0_BOTTOM(x) FIELD_PREP(GENMASK(31, 16), (x))
+#define SET_PARAM_CONF_WIN0_TOP(x) FIELD_PREP(GENMASK(15, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_CONF_WIN1 (W6_REG_BASE + 0x320)
+#define SET_PARAM_CONF_WIN1_RIGHT(x) FIELD_PREP(GENMASK(31, 16), (x))
+#define SET_PARAM_CONF_WIN1_LEFT(x) FIELD_PREP(GENMASK(15, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_RDO (W6_REG_BASE + 0x324)
+#define SET_PARAM_RDO_CUSTOM_LAMBDA(x) FIELD_PREP(GENMASK(22, 22), (x))
+#define SET_PARAM_RDO_ME_SEARCH_CENTER_ON BIT(21)
+#define SET_PARAM_RDO_QROUND_OFFSET(x) FIELD_PREP(GENMASK(17, 17), (x))
+#define SET_PARAM_RDO_ADAPTIVE_ROUND_ON BIT(3)
+#define SET_PARAM_RDO_HVS_QP(x) FIELD_PREP(GENMASK(2, 2), (x))
+#define W6_CMD_ENC_SET_PARAM_SLICE (W6_REG_BASE + 0x328)
+#define SET_PARAM_SLICE_ARGUMENT(x) FIELD_PREP(GENMASK(31, 3), (x))
+#define SET_PARAM_SLICE_MODE(x) FIELD_PREP(GENMASK(2, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_INTRA_REFRESH (W6_REG_BASE + 0x32C)
+#define SET_PARAM_INTRA_REFRESH_ARGUMENT(x) FIELD_PREP(GENMASK(31, 16), (x))
+#define SET_PARAM_INTRA_REFRESH_MODE(x) FIELD_PREP(GENMASK(2, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_INTRA_QP (W6_REG_BASE + 0x330)
+#define SET_PARAM_INTRA_QP_MAX(x) FIELD_PREP(GENMASK(11, 6), (x))
+#define SET_PARAM_INTRA_QP_MIN(x) FIELD_PREP(GENMASK(5, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_RC_FRAME_RATE (W6_REG_BASE + 0x334)
+#define W6_CMD_ENC_SET_PARAM_RC_TARGET_RATE (W6_REG_BASE + 0x338)
+#define W6_CMD_ENC_SET_PARAM_RC (W6_REG_BASE + 0x33C)
+#define SET_PARAM_RC_UPDATE_SPEED(x) FIELD_PREP(GENMASK(31, 24), (x))
+#define SET_PARAM_RC_INITIAL_LEVEL(x) FIELD_PREP(GENMASK(23, 20), (x))
+#define SET_PARAM_RC_INITIAL_QP(x) FIELD_PREP(GENMASK(19, 14), (x))
+#define SET_PARAM_RC_MODE(x) FIELD_PREP(GENMASK(13, 13), (x))
+#define SET_PARAM_RC_PIC_LEVEL_MAX_DELTA_QP(x) FIELD_PREP(GENMASK(12, 7), (x))
+#define SET_PARAM_RC_VBV_OVERFLOW_DROP_FRAME(x) FIELD_PREP(GENMASK(3, 3), (x))
+#define SET_PARAM_RC_CU_LEVEL_ENABLE(x) FIELD_PREP(GENMASK(1, 1), (x))
+#define SET_PARAM_RC_ENABLE(x) FIELD_PREP(GENMASK(0, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_HVS (W6_REG_BASE + 0x340)
+#define SET_PARAM_HVS_MAX_DELTA_QP(x) FIELD_PREP(GENMASK(17, 12), (x))
+#define SET_PARAM_HVS_QP_SCALE(x) FIELD_PREP(GENMASK(3, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_RC_MAX_BITRATE (W6_REG_BASE + 0x344)
+#define W6_CMD_ENC_SET_PARAM_RC_VBV_BUFFER_SIZE (W6_REG_BASE + 0x348)
+#define W6_CMD_ENC_SET_PARAM_INTER_QP (W6_REG_BASE + 0x34C)
+#define SET_PARAM_INTER_QP_MAX_B(x) FIELD_PREP(GENMASK(23, 18), (x))
+#define SET_PARAM_INTER_QP_MIN_B(x) FIELD_PREP(GENMASK(17, 12), (x))
+#define SET_PARAM_INTER_QP_MAX_P(x) FIELD_PREP(GENMASK(11, 6), (x))
+#define SET_PARAM_INTER_QP_MIN_P(x) FIELD_PREP(GENMASK(5, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_ROT (W6_REG_BASE + 0x350)
+#define SET_PARAM_ROT_MIRH BIT(4)
+#define SET_PARAM_ROT_MIRV BIT(3)
+#define SET_PARAM_ROT_270 FIELD_PREP(GENMASK(2, 1), 3)
+#define SET_PARAM_ROT_180 FIELD_PREP(GENMASK(2, 1), 2)
+#define SET_PARAM_ROT_90 FIELD_PREP(GENMASK(2, 1), 1)
+#define SET_PARAM_ROT_ENABLE BIT(0)
+#define W6_CMD_ENC_SET_PARAM_NUM_UNITS_IN_TICK (W6_REG_BASE + 0x354)
+#define W6_CMD_ENC_SET_PARAM_TIME_SCALE (W6_REG_BASE + 0x358)
+#define W6_CMD_ENC_SET_PARAM_NUM_TICKS_POC_DIFF_ONE (W6_REG_BASE + 0x35C)
+#define W6_CMD_ENC_SET_PARAM_MAX_INTRA_PIC_BIT (W6_REG_BASE + 0x360)
+#define W6_CMD_ENC_SET_PARAM_MAX_INTER_PIC_BIT (W6_REG_BASE + 0x364)
+#define W6_CMD_ENC_SET_PARAM_BG (W6_REG_BASE + 0x36C)
+#define SET_PARAM_BG_DELTA_QP(x) FIELD_PREP(GENMASK(29, 24), (x))
+#define SET_PARAM_BG_THRESHOLD_MEAN_DIFF(x) FIELD_PREP(GENMASK(17, 10), (x))
+#define SET_PARAM_BG_THRESHOLD_MAX_DIFF(x) FIELD_PREP(GENMASK(9, 1), (x))
+#define SET_PARAM_BG_ENABLE(x) FIELD_PREP(GENMASK(0, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_NON_VCL_PARAM (W6_REG_BASE + 0x370)
+#define SET_PARAM_NON_VCL_PARAM_ENCODE_VUI BIT(1)
+#define W6_CMD_ENC_SET_PARAM_VUI_RBSP_ADDR (W6_REG_BASE + 0x374)
+#define W6_CMD_ENC_SET_PARAM_HRD_RBSP_ADDR (W6_REG_BASE + 0x378)
+#define W6_CMD_ENC_SET_PARAM_QROUND_OFFSET (W6_REG_BASE + 0x380)
+#define SET_PARAM_QROUND_OFFSET_INTER(x) FIELD_PREP(GENMASK(23, 13), (x))
+#define SET_PARAM_QROUND_OFFSET_INTRA(x) FIELD_PREP(GENMASK(12, 2), (x))
+#define W6_CMD_ENC_SET_PARAM_QUANT1 (W6_REG_BASE + 0x384)
+#define W6_CMD_ENC_SET_PARAM_QUANT2 (W6_REG_BASE + 0x388)
+#define SET_PARAM_QUANT2_LAMBDA_DQP_INTER(x) FIELD_PREP(GENMASK(19, 14), (x))
+#define SET_PARAM_QUANT2_LAMBDA_DQP_INTRA(x) FIELD_PREP(GENMASK(13, 8), (x))
+#define W6_CMD_ENC_SET_PARAM_CUSTOM_GOP (W6_REG_BASE + 0x38C)
+
+/* W6_CMD_ENC_SET_PARAM_CUSTOM_GOP_PIC(x)
+ * Address range: 0x390, 0x394, ..., 0x3AC (step = 0x4)
+ */
+#define W6_CMD_ENC_SET_PARAM_CUSTOM_GOP_PIC(x) (W6_REG_BASE + 0x390 + ((x) * 0x4))
+#define SET_PARAM_CUSTOM_GOP_PIC_TEMP_ID(x) FIELD_PREP(GENMASK(29, 26), (x))
+#define SET_PARAM_CUSTOM_GOP_PIC_REF1_POC(x) FIELD_PREP(GENMASK(25, 20), (x))
+#define SET_PARAM_CUSTOM_GOP_PIC_REF0_POC(x) FIELD_PREP(GENMASK(19, 14), (x))
+#define SET_PARAM_CUSTOM_GOP_PIC_MULTI_REF_P(x) FIELD_PREP(GENMASK(13, 13), (x))
+#define SET_PARAM_CUSTOM_GOP_PIC_QP(x) FIELD_PREP(GENMASK(12, 7), (x))
+#define SET_PARAM_CUSTOM_GOP_PIC_POC_OFFSET(x) FIELD_PREP(GENMASK(6, 2), (x))
+#define SET_PARAM_CUSTOM_GOP_PIC_TYPE(x) FIELD_PREP(GENMASK(1, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_TILE_PARAM (W6_REG_BASE + 0x3D0)
+
+/* W6_CMD_ENC_SET_PARAM_CUSTOM_LAMBDA(x)
+ * Address range: 0x3D4, 0x3D8, ..., 0x4A0 (step = 0x4)
+ */
+#define W6_CMD_ENC_SET_PARAM_CUSTOM_LAMBDA(x) (W6_REG_BASE + 0x3D4 + ((x) * 0x4))
+#define SET_PARAM_CUSTOM_LAMBDA_SSD(x) FIELD_PREP(GENMASK(20, 7), (x))
+#define SET_PARAM_CUSTOM_LAMBDA_SAD(x) FIELD_PREP(GENMASK(6, 0), (x))
+
+/* W6_CMD_ENC_SET_PARAM_TEMP_LAYER_QP(x)
+ * Address range: 0x4A4, 0x4A8, ..., 0x4B0 (step = 0x4)
+ */
+#define W6_CMD_ENC_SET_PARAM_TEMP_LAYER_QP(x) (W6_REG_BASE + 0x4A4 + ((x) * 0x4))
+#define SET_PARAM_TEMP_LAYER_QP_B(x) FIELD_PREP(GENMASK(17, 12), (x))
+#define SET_PARAM_TEMP_LAYER_QP_P(x) FIELD_PREP(GENMASK(11, 6), (x))
+#define SET_PARAM_TEMP_LAYER_QP_I(x) FIELD_PREP(GENMASK(5, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_SCALER_SIZE (W6_REG_BASE + 0x4B4)
+#define SET_PARAM_SCALER_SIZE_HEIGHT(x) FIELD_PREP(GENMASK(31, 16), (x))
+#define SET_PARAM_SCALER_SIZE_WIDTH(x) FIELD_PREP(GENMASK(15, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_SCALER (W6_REG_BASE + 0x4B8)
+#define SET_PARAM_SCALER_COEF_MODE(x) FIELD_PREP(GENMASK(2, 1), (x))
+#define SET_PARAM_SCALER_ENABLE(x) FIELD_PREP(GENMASK(0, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_COLOR (W6_REG_BASE + 0x4F8)
+#define SET_PARAM_COLOR_RANGE(x) FIELD_PREP(GENMASK(25, 25), (x))
+#define SET_PARAM_COLOR_MATRIX_COEFF(x) FIELD_PREP(GENMASK(24, 17), (x))
+#define SET_PARAM_COLOR_TRANS_CHAR(x) FIELD_PREP(GENMASK(16, 9), (x))
+#define SET_PARAM_COLOR_PRIMARIES(x) FIELD_PREP(GENMASK(8, 1), (x))
+#define SET_PARAM_COLOR_DESCRIPTION_PRESENT_ON BIT(0)
+#define W6_CMD_ENC_SET_PARAM_SAR (W6_REG_BASE + 0x4FC)
+#define SET_PARAM_SAR_ASPECT_RATIO_IDC(x) FIELD_PREP(GENMASK(8, 1), (x))
+#define SET_PARAM_SAR_ASPECT_RATIO_ENABLE(x) FIELD_PREP(GENMASK(0, 0), (x))
+#define W6_CMD_ENC_SET_PARAM_SAR_EXTENDED (W6_REG_BASE + 0x500)
+#define SET_PARAM_SAR_EXTENDED_HEIGHT(x) FIELD_PREP(GENMASK(31, 16), (x))
+#define SET_PARAM_SAR_EXTENDED_WIDTH(x) FIELD_PREP(GENMASK(15, 0), (x))
+
+/* ENCODER - ENC_PIC */
+#define W6_CMD_ENC_PIC_BS_START (W6_REG_BASE + 0x300)
+#define W6_CMD_ENC_PIC_BS_SIZE (W6_REG_BASE + 0x304)
+#define W6_CMD_ENC_PIC_BS_OPTION (W6_REG_BASE + 0x308)
+#define W6_CMD_ENC_PIC_SEC_AXI (W6_REG_BASE + 0x30C)
+#define ENC_PIC_SEC_AXI_RDO_ENABLE(x) FIELD_PREP(GENMASK(1, 1), (x))
+#define ENC_PIC_SEC_AXI_LF_ENABLE(x) FIELD_PREP(GENMASK(0, 0), (x))
+#define W6_CMD_ENC_PIC_REPORT (W6_REG_BASE + 0x310)
+#define ENC_PIC_REPORT_MV_HISTO_ENABLE(x) FIELD_PREP(GENMASK(1, 1), (x))
+#define W6_CMD_ENC_PIC_MV_HISTO0 (W6_REG_BASE + 0x318)
+#define ENC_PIC_MV_HISTO0_THRESHOLD0(x) FIELD_PREP(GENMASK(31, 16), (x))
+#define ENC_PIC_MV_HISTO0_THRESHOLD1(x) FIELD_PREP(GENMASK(15, 0), (x))
+#define W6_CMD_ENC_PIC_MV_HISTO1 (W6_REG_BASE + 0x31C)
+#define ENC_PIC_MV_HISTO1_THRESHOLD2(x) FIELD_PREP(GENMASK(31, 16), (x))
+#define ENC_PIC_MV_HISTO1_THRESHOLD3(x) FIELD_PREP(GENMASK(15, 0), (x))
+#define W6_CMD_ENC_PIC_CUSTOM_MAP_PARAM (W6_REG_BASE + 0x320)
+#define W6_CMD_ENC_PIC_CUSTOM_MAP_ADDR (W6_REG_BASE + 0x324)
+#define W6_CMD_ENC_PIC_SRC_PIC_IDX (W6_REG_BASE + 0x32C)
+#define ENC_PIC_SRC_PIC_IDX_END 0xFFFFFFFF
+#define W6_CMD_ENC_PIC_SRC_ADDR_Y (W6_REG_BASE + 0x330)
+#define W6_CMD_ENC_PIC_SRC_ADDR_U (W6_REG_BASE + 0x334)
+#define W6_CMD_ENC_PIC_SRC_ADDR_V (W6_REG_BASE + 0x338)
+#define W6_CMD_ENC_PIC_SRC_STRIDE (W6_REG_BASE + 0x33C)
+#define ENC_PIC_SRC_STRIDE_L(x) FIELD_PREP(GENMASK(31, 16), (x))
+#define ENC_PIC_SRC_STRIDE_C(x) FIELD_PREP(GENMASK(15, 0), (x))
+#define W6_CMD_ENC_PIC_SRC_FMT (W6_REG_BASE + 0x340)
+#define ENC_PIC_SRC_FMT_C_FMT_IDC(x) FIELD_PREP(GENMASK(30, 28), (x))
+#define ENC_PIC_SRC_FMT_CSC_24BIT(x) FIELD_PREP(GENMASK(25, 25), (x))
+#define ENC_PIC_SRC_FMT_CSC_AYUV(x) FIELD_PREP(GENMASK(24, 24), (x))
+#define ENC_PIC_SRC_FMT_CSC_ENABLE(x) FIELD_PREP(GENMASK(20, 20), (x))
+#define ENC_PIC_SRC_FMT_CSC_FMT_ORDER(x) FIELD_PREP(GENMASK(19, 16), (x))
+#define ENC_PIC_SRC_FMT_ENDIAN(x) FIELD_PREP(GENMASK(15, 12), (x))
+#define ENC_PIC_SRC_FMT_10BIT_ORDER(x) FIELD_PREP(GENMASK(6, 6), (x))
+#define ENC_PIC_SRC_FMT_10BIT_3PIXEL_4BYTE(x) FIELD_PREP(GENMASK(5, 5), (x))
+#define ENC_PIC_SRC_FMT_10BIT_ENABLE(x) FIELD_PREP(GENMASK(4, 4), (x))
+#define ENC_PIC_SRC_FMT_YUV422_PACKED_ORDER(x) FIELD_PREP(GENMASK(3, 3), (x))
+#define ENC_PIC_SRC_FMT_CBCR_ORDER(x) FIELD_PREP(GENMASK(2, 2), (x))
+#define ENC_PIC_SRC_FMT_2PLANE(x) FIELD_PREP(GENMASK(1, 1), (x))
+#define ENC_PIC_SRC_FMT_YUV422_PACKED(x) FIELD_PREP(GENMASK(0, 0), (x))
+#define W6_CMD_ENC_PIC_SRC_AXI_SEL (W6_REG_BASE + 0x348)
+#define ENC_PIC_SRC_AXI_SEL_PRIMARY_PORT 1
+#define ENC_PIC_SRC_AXI_SEL_PRP_PORT 0
+#define W6_CMD_ENC_PIC_CODE_OPTION (W6_REG_BASE + 0x34C)
+#define ENC_PIC_CODE_OPTION_ENCODE_VCL BIT(1)
+#define ENC_PIC_CODE_OPTION_ENCODE_HEADER BIT(0)
+#define W6_CMD_ENC_PIC_PARAM (W6_REG_BASE + 0x350)
+#define ENC_PIC_PARAM_INTRA_4X4(x) FIELD_PREP(GENMASK(29, 28), (x))
+#define ENC_PIC_PARAM_FORCE_PIC_TYPE(x) FIELD_PREP(GENMASK(23, 21), (x))
+#define ENC_PIC_PARAM_FORCE_PIC_TYPE_ENABLE(x) FIELD_PREP(GENMASK(20, 20), (x))
+#define ENC_PIC_PARAM_FORCE_PIC_QP_B(x) FIELD_PREP(GENMASK(19, 14), (x))
+#define ENC_PIC_PARAM_FORCE_PIC_QP_P(x) FIELD_PREP(GENMASK(13, 8), (x))
+#define ENC_PIC_PARAM_FORCE_PIC_QP_I(x) FIELD_PREP(GENMASK(7, 2), (x))
+#define ENC_PIC_PARAM_FORCE_PIC_QP_ENABLE(x) FIELD_PREP(GENMASK(1, 1), (x))
+#define ENC_PIC_PARAM_PIC_SKIP_FLAG(x) FIELD_PREP(GENMASK(0, 0), (x))
+#define W6_CMD_ENC_PIC_LONGTERM_PIC (W6_REG_BASE + 0x354)
+#define W6_CMD_ENC_PIC_PREFIX_SEI_NAL_ADDR (W6_REG_BASE + 0x358)
+#define W6_CMD_ENC_PIC_PREFIX_SEI_INFO (W6_REG_BASE + 0x35C)
+#define W6_CMD_ENC_PIC_SUFFIX_SEI_NAL_ADDR (W6_REG_BASE + 0x360)
+#define W6_CMD_ENC_PIC_SUFFIX_SEI_INFO (W6_REG_BASE + 0x364)
+#define W6_CMD_ENC_PIC_CSC_COEFF0 (W6_REG_BASE + 0x374)
+#define ENC_PIC_CSC_COEFF0_RY(x) FIELD_PREP(GENMASK(29, 20), (x))
+#define ENC_PIC_CSC_COEFF0_GY(x) FIELD_PREP(GENMASK(19, 10), (x))
+#define ENC_PIC_CSC_COEFF0_BY(x) FIELD_PREP(GENMASK(9, 0), (x))
+#define W6_CMD_ENC_PIC_CSC_COEFF1 (W6_REG_BASE + 0x378)
+#define ENC_PIC_CSC_COEFF1_RCB(x) FIELD_PREP(GENMASK(29, 20), (x))
+#define ENC_PIC_CSC_COEFF1_GCB(x) FIELD_PREP(GENMASK(19, 10), (x))
+#define ENC_PIC_CSC_COEFF1_BCB(x) FIELD_PREP(GENMASK(9, 0), (x))
+#define W6_CMD_ENC_PIC_CSC_COEFF2 (W6_REG_BASE + 0x37C)
+#define ENC_PIC_CSC_COEFF2_RCR(x) FIELD_PREP(GENMASK(29, 20), (x))
+#define ENC_PIC_CSC_COEFF2_GCR(x) FIELD_PREP(GENMASK(19, 10), (x))
+#define ENC_PIC_CSC_COEFF2_BCR(x) FIELD_PREP(GENMASK(9, 0), (x))
+#define W6_CMD_ENC_PIC_CSC_COEFF3 (W6_REG_BASE + 0x380)
+#define ENC_PIC_CSC_COEFF3_OFFSET_Y(x) FIELD_PREP(GENMASK(29, 20), (x))
+#define ENC_PIC_CSC_COEFF3_OFFSET_CB(x) FIELD_PREP(GENMASK(19, 10), (x))
+#define ENC_PIC_CSC_COEFF3_OFFSET_CR(x) FIELD_PREP(GENMASK(9, 0), (x))
+#define W6_CMD_ENC_PIC_TIMESTAMP_LOW (W6_REG_BASE + 0x448)
+#define W6_CMD_ENC_PIC_TIMESTAMP_HIGH (W6_REG_BASE + 0x44C)
+
+/* ENCODER - QUERY : GET_RESULT */
+#define W6_RET_ENC_RD_PTR (W6_REG_BASE + 0x300)
+#define W6_RET_ENC_WR_PTR (W6_REG_BASE + 0x304)
+#define W6_RET_ENC_NUM_REQUIRED_FBC_FB (W6_REG_BASE + 0x308)
+#define W6_RET_ENC_MIN_SRC_BUF_NUM (W6_REG_BASE + 0x30C)
+#define W6_RET_ENC_PIC_TYPE (W6_REG_BASE + 0x310)
+#define W6_RET_ENC_PIC_POC (W6_REG_BASE + 0x314)
+#define W6_RET_ENC_PIC_IDX (W6_REG_BASE + 0x318)
+#define W6_RET_ENC_PIC_SLICE_NUM (W6_REG_BASE + 0x31C)
+#define W6_RET_ENC_PIC_SKIP (W6_REG_BASE + 0x320)
+#define W6_RET_ENC_PIC_NUM_INTRA (W6_REG_BASE + 0x324)
+#define W6_RET_ENC_PIC_NUM_MERGE (W6_REG_BASE + 0x328)
+#define W6_RET_ENC_PIC_NON_REF_PIC_FLAG (W6_REG_BASE + 0x32C)
+#define W6_RET_ENC_PIC_NUM_SKIP (W6_REG_BASE + 0x330)
+#define W6_RET_ENC_PIC_AVG_CTU_QP (W6_REG_BASE + 0x334)
+#define W6_RET_ENC_PIC_BYTE (W6_REG_BASE + 0x338)
+#define W6_RET_ENC_GOP_PIC_IDX (W6_REG_BASE + 0x33C)
+#define W6_RET_ENC_USED_SRC_IDX (W6_REG_BASE + 0x340)
+#define W6_RET_ENC_PIC_NUM (W6_REG_BASE + 0x344)
+#define W6_RET_ENC_VCL_NUT (W6_REG_BASE + 0x348)
+#define W6_RET_ENC_PIC_DIST_LOW (W6_REG_BASE + 0x350)
+#define W6_RET_ENC_PIC_DIST_HIGH (W6_REG_BASE + 0x354)
+#define W6_RET_ENC_PIC_MAX_LATENCY_PICTURES (W6_REG_BASE + 0x358)
+#define W6_RET_ENC_HISTO_CNT0 (W6_REG_BASE + 0x360)
+#define W6_RET_ENC_HISTO_CNT1 (W6_REG_BASE + 0x364)
+#define W6_RET_ENC_HISTO_CNT2 (W6_REG_BASE + 0x368)
+#define W6_RET_ENC_HISTO_CNT3 (W6_REG_BASE + 0x36C)
+#define W6_RET_ENC_HISTO_CNT4 (W6_REG_BASE + 0x370)
+#define W6_RET_ENC_WARN_INFO (W6_REG_BASE + 0x3AC)
+#define W6_RET_ENC_ERR_INFO (W6_REG_BASE + 0x3B0)
+#define W6_RET_ENC_ENCODING_SUCCESS (W6_REG_BASE + 0x3B4)
+#define W6_RET_ENC_SUM_ME0_X_DIR_LOWER (W6_REG_BASE + 0x3B8)
+#define W6_RET_ENC_SUM_ME0_X_DIR_HIGHER (W6_REG_BASE + 0x3BC)
+#define W6_RET_ENC_SUM_ME0_Y_DIR_LOWER (W6_REG_BASE + 0x3C0)
+#define W6_RET_ENC_SUM_ME0_Y_DIR_HIGHER (W6_REG_BASE + 0x3C4)
+#define W6_RET_ENC_SUM_ME1_X_DIR_LOWER (W6_REG_BASE + 0x3C8)
+#define W6_RET_ENC_SUM_ME1_X_DIR_HIGHER (W6_REG_BASE + 0x3CC)
+#define W6_RET_ENC_SUM_ME1_Y_DIR_LOWER (W6_REG_BASE + 0x3D0)
+#define W6_RET_ENC_SUM_ME1_Y_DIR_HIGHER (W6_REG_BASE + 0x3D4)
+#define W6_RET_ENC_SRC_Y_ADDR (W6_REG_BASE + 0x3E8)
+#define W6_RET_ENC_CUSTOM_MAP_ADDR (W6_REG_BASE + 0x3EC)
+#define W6_RET_ENC_PREFIX_SEI_NAL_ADDR (W6_REG_BASE + 0x3F0)
+#define W6_RET_ENC_SUFFIX_SEI_NAL_ADDR (W6_REG_BASE + 0x3F4)
+#define W6_RET_ENC_NUM_REQUIRED_COL_BUF (W6_REG_BASE + 0x404)
+#define W6_RET_ENC_TIMESTAMP_LOW (W6_REG_BASE + 0x44C)
+#define W6_RET_ENC_TIMESTAMP_HIGH (W6_REG_BASE + 0x450)
+
#endif /* __WAVE6_REGDEFINE_H__ */
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
index 4fd86472b427..2baf51a5709d 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
@@ -92,7 +92,8 @@ static irqreturn_t wave6_vpu_core_irq_thread(int irq, void *dev_id)
if (!inst)
continue;
- if (irq_info.status & BIT(W6_INT_BIT_INIT_SEQ)) {
+ if ((irq_info.status & BIT(W6_INT_BIT_INIT_SEQ)) ||
+ (irq_info.status & BIT(W6_INT_BIT_ENC_SET_PARAM))) {
complete(&inst->irq_done);
continue;
}
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.c b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.c
index c8d1138334b3..b841e3f1c9a1 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.c
@@ -200,3 +200,134 @@ void wave6_vpu_dec_reset_frame_buffer_info(struct vpu_instance *inst)
wave6_vdi_free_dma(&inst->aux_vbuf[AUX_BUF_FBC_C_TBL][i]);
}
}
+
+int wave6_vpu_enc_open(struct vpu_instance *inst, struct enc_open_param *pop)
+{
+ struct enc_info *p_enc_info;
+ int ret;
+ struct vpu_core_device *core = inst->dev;
+
+ guard(mutex)(&core->hw_lock);
+
+ if (!wave6_vpu_is_init(core))
+ return -ENODEV;
+
+ inst->codec_info = kzalloc_obj(*inst->codec_info);
+ if (!inst->codec_info)
+ return -ENOMEM;
+
+ p_enc_info = &inst->codec_info->enc_info;
+ p_enc_info->open_param = *pop;
+
+ ret = wave6_vpu_build_up_enc_param(core->dev, inst, pop);
+ if (ret) {
+ kfree(inst->codec_info);
+ inst->codec_info = NULL;
+ }
+
+ return ret;
+}
+
+int wave6_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)
+{
+ int ret;
+
+ if (WARN_ON(!inst->codec_info))
+ return -EINVAL;
+
+ guard(mutex)(&inst->dev->hw_lock);
+
+ ret = wave6_vpu_enc_fini_seq(inst, fail_res);
+ if (ret)
+ dev_warn(inst->dev->dev, "enc seq end timed out\n");
+
+ kfree(inst->codec_info);
+ inst->codec_info = NULL;
+
+ return ret;
+}
+
+int wave6_vpu_enc_get_aux_buffer_size(struct vpu_instance *inst,
+ struct aux_buffer_size_info info, u32 *size)
+{
+ struct enc_info *p_enc_info = &inst->codec_info->enc_info;
+ int width = info.width;
+ int height = info.height;
+ int buf_size, twice;
+
+ if (info.type == AUX_BUF_FBC_Y_TBL) {
+ switch (inst->std) {
+ case W_HEVC_ENC:
+ case W_AVC_ENC:
+ buf_size = WAVE6_FBC_LUMA_TABLE_SIZE(width, height);
+ break;
+ default:
+ return -EINVAL;
+ }
+ } else if (info.type == AUX_BUF_FBC_C_TBL) {
+ if (p_enc_info->c_fmt_idc == C_FMT_IDC_YUV422)
+ twice = 2;
+ else if (p_enc_info->c_fmt_idc == C_FMT_IDC_YUV444)
+ twice = 4;
+ else
+ twice = 1;
+
+ switch (inst->std) {
+ case W_HEVC_ENC:
+ case W_AVC_ENC:
+ buf_size = WAVE6_FBC_CHROMA_TABLE_SIZE(width, height);
+ break;
+ default:
+ return -EINVAL;
+ }
+ buf_size = buf_size * twice;
+ } else if (info.type == AUX_BUF_MV_COL) {
+ switch (inst->std) {
+ case W_HEVC_ENC:
+ buf_size = WAVE6_ENC_HEVC_MVCOL_BUF_SIZE(width, height);
+ break;
+ case W_AVC_ENC:
+ buf_size = WAVE6_ENC_AVC_MVCOL_BUF_SIZE(width, height);
+ break;
+ default:
+ return -EINVAL;
+ }
+ } else if (info.type == AUX_BUF_SUB_SAMPLE) {
+ switch (inst->std) {
+ case W_HEVC_ENC:
+ case W_AVC_ENC:
+ buf_size = WAVE6_ENC_SUBSAMPLED_SIZE(width, height);
+ break;
+ default:
+ return -EINVAL;
+ }
+ } else {
+ return -EINVAL;
+ }
+
+ *size = buf_size;
+
+ return 0;
+}
+
+int wave6_vpu_enc_complete_seq_init(struct vpu_instance *inst,
+ struct enc_seq_info *info)
+{
+ struct enc_info *p_enc_info = &inst->codec_info->enc_info;
+ int ret;
+
+ guard(mutex)(&inst->dev->hw_lock);
+
+ ret = wave6_vpu_enc_get_seq_info(inst, info);
+ if (ret) {
+ p_enc_info->seq_info_obtained = false;
+ return ret;
+ }
+
+ if (!p_enc_info->seq_info_obtained) {
+ p_enc_info->seq_info_obtained = true;
+ p_enc_info->seq_info = *info;
+ }
+
+ return 0;
+}
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
index d8d7b7916d88..d5ecf381276d 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
@@ -56,6 +56,12 @@ enum vpu_instance_state {
(ALIGN((_w), 256) * ALIGN((_h), 64) / 32)
#define WAVE6_FBC_CHROMA_TABLE_SIZE(_w, _h) \
(ALIGN(((_w) / 2), 256) * ALIGN((_h), 64) / 32)
+#define WAVE6_ENC_AVC_MVCOL_BUF_SIZE(_w, _h) \
+ ((ALIGN((_w), 512) / 512) * (ALIGN((_h), 16) / 16) * 16)
+#define WAVE6_ENC_HEVC_MVCOL_BUF_SIZE(_w, _h) \
+ ((ALIGN((_w), 64) / 64) * (ALIGN((_h), 64) / 64) * 128)
+#define WAVE6_ENC_SUBSAMPLED_SIZE(_w, _h) \
+ (ALIGN(((_w) / 4), 16) * ALIGN(((_h) / 4), 32))
enum codec_std {
W_HEVC_DEC = 0x00,
@@ -78,6 +84,28 @@ enum codec_std {
#define H264_VUI_SAR_IDC_EXTENDED 255
+#define DEC_REFRESH_TYPE_NON_IRAP 0
+#define DEC_REFRESH_TYPE_IDR 2
+
+#define DEFAULT_TEMP_LAYER_CNT 1
+#define DEFAULT_RC_INITIAL_LEVEL 8
+#define DEFAULT_RC_INITIAL_QP -1
+#define DEFAULT_PIC_RC_MAX_DQP 3
+#define DEFAULT_QROUND_INTER 85
+#define DEFAULT_QROUND_INTRA 171
+#define DEFAULT_INTRA_4X4 3
+#define DEFAULT_NUM_TICKS_POC_DIFF 1
+#define DEFAULT_RC_UPDATE_SPEED_CBR 64
+#define DEFAULT_RC_UPDATE_SPEED_VBR 16
+#define MAX_INTRA_INTER_PIC_BIT 1500000000
+#define DEFAULT_FRAME_RATE 30
+#define MIN_FRAME_RATE 1
+#define MAX_FRAME_RATE 960
+#define DEFAULT_CPB_SIZE 10000
+#define MIN_CPB_SIZE 10
+#define MAX_CPB_SIZE 100000
+#define AVC_MIN_SLICE_MAX_MB 4
+
#define SEQ_CHANGE_ENABLE_PROFILE BIT(5)
#define SEQ_CHANGE_ENABLE_SIZE BIT(16)
#define SEQ_CHANGE_ENABLE_BIT_DEPTH BIT(18)
@@ -95,6 +123,11 @@ enum codec_std {
#define DEC_NOTI_FLAG_NO_FB BIT(1)
#define DEC_NOTI_FLAG_SEQ_CHANGE BIT(0)
+#define RECON_IDX_FLAG_ENC_END -1
+#define RECON_IDX_FLAG_ENC_DELAY -2
+#define RECON_IDX_FLAG_HEADER_ONLY -3
+#define RECON_IDX_FLAG_CHANGE_PARAM -4
+
#define FW_VERSION_MAJOR_MASK 0xFF000000
#define FW_VERSION_MINOR_MASK 0x00FF0000
#define FW_VERSION_REL_MASK 0x0000FFFF
@@ -102,6 +135,20 @@ enum codec_std {
#define FW_VERSION_MINOR(x) FIELD_GET(FW_VERSION_MINOR_MASK, (x))
#define FW_VERSION_REL(x) FIELD_GET(FW_VERSION_REL_MASK, (x))
+enum mirror_direction {
+ MIR_NONE,
+ MIR_VER,
+ MIR_HOR,
+ MIR_HOR_VER
+};
+
+enum rotation_angle {
+ ROT_0 = 0,
+ ROT_90 = 90,
+ ROT_180 = 180,
+ ROT_270 = 270
+};
+
enum chroma_format_idc {
C_FMT_IDC_YUV400,
C_FMT_IDC_YUV420,
@@ -110,6 +157,28 @@ enum chroma_format_idc {
C_FMT_IDC_RGB
};
+enum csc_format_order {
+ CSC_FMT_ORDER_RGB = 0,
+ CSC_FMT_ORDER_RBG = 1,
+ CSC_FMT_ORDER_GRB = 2,
+ CSC_FMT_ORDER_GBR = 3,
+ CSC_FMT_ORDER_BGR = 4,
+ CSC_FMT_ORDER_BRG = 5,
+
+ CSC_FMT_ORDER_ARGB = 0,
+ CSC_FMT_ORDER_ARBG = 1,
+ CSC_FMT_ORDER_AGRB = 2,
+ CSC_FMT_ORDER_AGBR = 3,
+ CSC_FMT_ORDER_ABGR = 4,
+ CSC_FMT_ORDER_ABRG = 5,
+ CSC_FMT_ORDER_RGBA = 8,
+ CSC_FMT_ORDER_RBGA = 9,
+ CSC_FMT_ORDER_GRBA = 10,
+ CSC_FMT_ORDER_GBRA = 11,
+ CSC_FMT_ORDER_BGRA = 12,
+ CSC_FMT_ORDER_BRGA = 13,
+};
+
enum frame_buffer_format {
FORMAT_ERR = -1,
@@ -183,6 +252,14 @@ enum pic_type {
PIC_TYPE_MAX
};
+enum enc_force_pic_type {
+ ENC_FORCE_PIC_TYPE_I = 0,
+ ENC_FORCE_PIC_TYPE_P = 1,
+ ENC_FORCE_PIC_TYPE_B = 2,
+ ENC_FORCE_PIC_TYPE_IDR = 3,
+ ENC_FORCE_PIC_TYPE_DISABLED = 4,
+};
+
enum display_mode {
DISP_MODE_DISP_ORDER,
DISP_MODE_DEC_ORDER,
@@ -207,6 +284,12 @@ enum aux_buffer_type {
AUX_BUF_TYPE_MAX,
};
+enum intra_refresh_mode {
+ INTRA_REFRESH_NONE = 0,
+ INTRA_REFRESH_ROW = 1,
+ INTRA_REFRESH_COLUMN = 2,
+};
+
struct vpu_attr {
u32 product_id;
u32 product_code;
@@ -243,9 +326,17 @@ struct vpu_rect {
u32 bottom;
};
+struct sar_info {
+ u32 enable;
+ u32 idc;
+ u32 width;
+ u32 height;
+};
+
struct instance_buffer {
dma_addr_t temp_base;
u32 temp_size;
+ dma_addr_t ar_base;
};
struct color_param {
@@ -260,6 +351,8 @@ struct color_param {
struct sec_axi_info {
bool use_dec_ip;
bool use_dec_lf_row;
+ bool use_enc_rdo;
+ bool use_enc_lf;
};
struct aux_buffer_size_info {
@@ -358,6 +451,320 @@ struct dec_info {
enum frame_buffer_format out_fmt;
};
+#define MAX_CUSTOM_LAMBDA_NUM 52
+#define MAX_NUM_TEMP_LAYER 7
+#define MAX_CUSTOM_GOP_NUM 8
+#define MAX_NUM_CHANGEABLE_TEMP_LAYER 4
+
+struct custom_gop_pic_param {
+ int pic_type;
+ int poc_offset;
+ int pic_qp;
+ bool multi_ref_p;
+ int ref_poc_l0;
+ int ref_poc_l1;
+ int temporal_id;
+};
+
+struct custom_gop_param {
+ int size;
+ struct custom_gop_pic_param pic[MAX_CUSTOM_GOP_NUM];
+};
+
+struct temporal_layer_param {
+ bool change_qp;
+ u32 qp_i;
+ u32 qp_p;
+ u32 qp_b;
+};
+
+struct enc_scaler_info {
+ bool enable;
+ int width;
+ int height;
+ int coef_mode;
+};
+
+struct enc_codec_param {
+ u32 internal_bit_depth;
+ u32 decoding_refresh_type;
+ u32 idr_period;
+ u32 intra_period;
+ u32 gop_preset_idx;
+ u32 frame_rate;
+ u32 bitrate;
+ u32 cpb_size;
+ u32 hvs_qp_scale_div2;
+ u32 max_delta_qp;
+ int rc_initial_qp;
+ u32 rc_update_speed;
+ u32 max_bitrate;
+ u32 rc_mode;
+ u32 rc_initial_level;
+ u32 pic_rc_max_dqp;
+ u32 bg_th_diff;
+ u32 bg_th_mean_diff;
+ int bg_delta_qp;
+ u32 intra_refresh_mode;
+ u32 intra_refresh_arg;
+ int beta_offset_div2;
+ int tc_offset_div2;
+ u32 qp;
+ u32 min_qp_i;
+ u32 max_qp_i;
+ u32 min_qp_p;
+ u32 max_qp_p;
+ u32 min_qp_b;
+ u32 max_qp_b;
+ int cb_qp_offset;
+ int cr_qp_offset;
+ u32 qround_intra;
+ u32 qround_inter;
+ int lambda_dqp_intra;
+ int lambda_dqp_inter;
+ u32 slice_mode;
+ u32 slice_arg;
+ u32 level;
+ u32 tier;
+ u32 profile;
+ struct vpu_rect conf_win;
+ u32 forced_idr_header;
+ u16 lambda_ssd[MAX_CUSTOM_LAMBDA_NUM];
+ u16 lambda_sad[MAX_CUSTOM_LAMBDA_NUM];
+ struct custom_gop_param gop_param;
+ struct temporal_layer_param temp_layer[MAX_NUM_CHANGEABLE_TEMP_LAYER];
+ u32 temp_layer_cnt;
+ u32 mv_histo_th0;
+ u32 mv_histo_th1;
+ u32 mv_histo_th2;
+ u32 mv_histo_th3;
+ u32 num_units_in_tick;
+ u32 time_scale;
+ u32 num_ticks_poc_diff_one;
+ struct color_param color;
+ struct sar_info sar;
+ u32 max_intra_pic_bit;
+ u32 max_inter_pic_bit;
+ u32 intra_4x4;
+
+ u32 en_const_intra_pred: 1;
+ u32 en_longterm: 1;
+ u32 en_rate_control: 1;
+ u32 en_transform8x8: 1;
+ u32 en_hvs_qp: 1;
+ u32 en_bg_detect: 1;
+ u32 en_temporal_mvp: 1;
+ u32 en_cabac: 1;
+ u32 en_dbk: 1;
+ u32 en_sao: 1;
+ u32 en_lf_slice_boundary: 1;
+ u32 en_scaling_list: 1;
+ u32 en_qround_offset: 1;
+ u32 en_still_picture: 1;
+ u32 en_intra_smooth: 1;
+ u32 en_custom_lambda: 1;
+ u32 en_report_mv_histo: 1;
+ u32 en_cu_level_rate_control: 1;
+ u32 en_skip_frame: 1;
+};
+
+struct enc_open_param {
+ int pic_width;
+ int pic_height;
+ struct enc_codec_param codec_param;
+ enum endian_mode source_endian;
+ enum frame_buffer_format src_fmt;
+ enum frame_buffer_format out_fmt;
+ u32 ext_addr_vcpu: 8;
+ u32 is_secure_inst: 1;
+ u32 inst_priority: 5;
+ struct instance_buffer inst_buffer;
+ enum mirror_direction mir_dir;
+ enum rotation_angle rot_angle;
+};
+
+struct enc_seq_info {
+ u32 min_frame_buffer_count: 5;
+ u32 min_src_frame_count;
+ u32 req_mv_buffer_count: 5;
+ int max_latency_pictures;
+ int err_reason;
+ int warn_info;
+};
+
+struct enc_csc_param {
+ u32 fmt_order;
+ s32 coef_ry;
+ s32 coef_gy;
+ s32 coef_by;
+ s32 coef_rcb;
+ s32 coef_gcb;
+ s32 coef_bcb;
+ s32 coef_rcr;
+ s32 coef_gcr;
+ s32 coef_bcr;
+ u32 offset_y;
+ u32 offset_cb;
+ u32 offset_cr;
+};
+
+struct enc_param {
+ struct frame_buffer *source_frame;
+ bool skip_picture;
+ dma_addr_t pic_stream_buffer_addr;
+ int pic_stream_buffer_size;
+ bool force_pic_qp;
+ int force_pic_qp_i;
+ int force_pic_qp_p;
+ int force_pic_qp_b;
+ bool force_pic;
+ int force_pic_type;
+ int src_idx;
+ u32 bitrate;
+ struct enc_csc_param csc;
+ u64 timestamp;
+};
+
+struct enc_report_fme_sum {
+ u32 lower_x0;
+ u32 higher_x0;
+ u32 lower_y0;
+ u32 higher_y0;
+ u32 lower_x1;
+ u32 higher_x1;
+ u32 lower_y1;
+ u32 higher_y1;
+};
+
+struct enc_report_mv_histo {
+ u32 cnt0;
+ u32 cnt1;
+ u32 cnt2;
+ u32 cnt3;
+ u32 cnt4;
+};
+
+struct enc_output_info {
+ dma_addr_t bitstream_buffer;
+ u32 bitstream_size;
+ int pic_type;
+ int num_of_slices;
+ int recon_frame_index;
+ dma_addr_t rd_ptr;
+ dma_addr_t wr_ptr;
+ int pic_skipped;
+ int num_of_intra;
+ int num_of_merge;
+ int num_of_skip_block;
+ int avg_ctu_qp;
+ int enc_pic_byte;
+ int enc_gop_pic_idx;
+ int enc_pic_poc;
+ int enc_src_idx;
+ int enc_vcl_nut;
+ int enc_pic_cnt;
+ int error_reason;
+ int warn_info;
+ u32 pic_distortion_low;
+ u32 pic_distortion_high;
+ bool non_ref_pic;
+ bool encoding_success;
+ struct enc_report_fme_sum fme_sum;
+ struct enc_report_mv_histo mv_histo;
+ u64 timestamp;
+ dma_addr_t src_y_addr;
+ dma_addr_t custom_map_addr;
+ dma_addr_t prefix_sei_nal_addr;
+ dma_addr_t suffix_sei_nal_addr;
+};
+
+enum gop_preset_idx {
+ PRESET_IDX_CUSTOM_GOP = 0,
+ PRESET_IDX_ALL_I = 1,
+ PRESET_IDX_IPP = 2,
+ PRESET_IDX_IBBB = 3,
+ PRESET_IDX_IBPBP = 4,
+ PRESET_IDX_IBBBP = 5,
+ PRESET_IDX_IPPPP = 6,
+ PRESET_IDX_IBBBB = 7,
+ PRESET_IDX_RA_IB = 8,
+ PRESET_IDX_IPP_SINGLE = 9,
+ PRESET_IDX_MAX,
+};
+
+struct enc_info {
+ struct enc_open_param open_param;
+ struct enc_seq_info seq_info;
+ enum mirror_direction mir_dir;
+ enum rotation_angle rot_angle;
+ bool seq_info_obtained;
+ struct sec_axi_info sec_axi;
+ u32 width;
+ u32 height;
+ struct enc_scaler_info scaler;
+ enum chroma_format_idc c_fmt_idc;
+};
+
+struct h264_enc_controls {
+ u32 profile;
+ u32 level;
+ u32 min_qp;
+ u32 max_qp;
+ u32 i_frame_qp;
+ u32 p_frame_qp;
+ u32 b_frame_qp;
+ u32 loop_filter_mode;
+ u32 loop_filter_beta;
+ u32 loop_filter_alpha;
+ u32 _8x8_transform;
+ u32 constrained_intra_prediction;
+ u32 chroma_qp_index_offset;
+ u32 entropy_mode;
+ u32 i_period;
+ u32 vui_sar_enable;
+ u32 vui_sar_idc;
+ u32 vui_ext_sar_width;
+ u32 vui_ext_sar_height;
+ u32 cpb_size;
+};
+
+struct hevc_enc_controls {
+ u32 profile;
+ u32 level;
+ u32 min_qp;
+ u32 max_qp;
+ u32 i_frame_qp;
+ u32 p_frame_qp;
+ u32 b_frame_qp;
+ u32 loop_filter_mode;
+ u32 lf_beta_offset_div2;
+ u32 lf_tc_offset_div2;
+ u32 refresh_type;
+ u32 refresh_period;
+ u32 const_intra_pred;
+ u32 strong_smoothing;
+ u32 tmv_prediction;
+};
+
+struct enc_controls {
+ u32 frame_rate;
+ u32 rotation_angle;
+ u32 mirror_direction;
+ u32 bitrate;
+ u32 bitrate_mode;
+ u32 gop_size;
+ u32 frame_rc_enable;
+ u32 mb_rc_enable;
+ u32 slice_mode;
+ u32 slice_max_mb;
+ u32 prepend_spspps_to_idr;
+ u32 intra_refresh_period;
+ struct h264_enc_controls h264;
+ struct hevc_enc_controls hevc;
+ u32 force_key_frame;
+ u32 frame_skip_mode;
+};
+
/**
* struct vpu_irq - VPU interrupt information
* @status: Value of the VPU interrupt status
@@ -441,6 +848,7 @@ struct vpu_instance_ops {
* @src_fmt: V4L2 pixel format for source
* @dst_fmt: V4L2 pixel format for destination
* @crop: Crop rectangle
+ * @codec_rect: Actual encoding rectangle
* @scaler_info: Decoder scaler information
* @colorspace: V4L2 colorspace
* @xfer_func: V4L2 transfer function
@@ -456,6 +864,7 @@ struct vpu_instance_ops {
* @frame_buf: Metadata for reference frame buffers
* @frame_vbuf: Reference frame buffers
* @aux_vbuf: Auxiliary buffers
+ * @ar_vbuf: Auxiliary AR buffer for encoder
* @fbc_buf_registered: Reference frame buffers registration flag
* @queued_src_buf_num: number of queued source buffers
* @queued_dst_buf_num: number of queued destination buffers
@@ -467,7 +876,9 @@ struct vpu_instance_ops {
* @next_dst_buf_last: Next queued CAPTURE buffer is returned as the last one
* @cbcr_interleave: CbCr interleave format flag
* @nv21: NV21 format flag
+ * @error_recovery: Error recovery flag
* @disp_mode: Display mode for decoder V4L2 control
+ * @enc_ctrls: Encoder V4L2 control and streaming parameters
*/
struct vpu_instance {
struct list_head list;
@@ -489,6 +900,7 @@ struct vpu_instance {
struct v4l2_pix_format_mplane src_fmt;
struct v4l2_pix_format_mplane dst_fmt;
struct v4l2_rect crop;
+ struct v4l2_rect codec_rect;
struct dec_scaler_info scaler_info;
enum v4l2_colorspace colorspace;
enum v4l2_xfer_func xfer_func;
@@ -503,11 +915,13 @@ struct vpu_instance {
enum codec_std std;
u32 id;
union {
+ struct enc_info enc_info;
struct dec_info dec_info;
} *codec_info;
struct frame_buffer frame_buf[WAVE6_MAX_FBS];
struct vpu_buf frame_vbuf[WAVE6_MAX_FBS];
struct vpu_buf aux_vbuf[AUX_BUF_TYPE_MAX][WAVE6_MAX_FBS];
+ struct vpu_buf ar_vbuf;
bool fbc_buf_registered;
u32 queued_src_buf_num;
u32 queued_dst_buf_num;
@@ -519,8 +933,10 @@ struct vpu_instance {
bool next_dst_buf_last;
bool cbcr_interleave;
bool nv21;
+ bool error_recovery;
enum display_mode disp_mode;
+ struct enc_controls enc_ctrls;
};
void wave6_vpu_writel(struct vpu_core_device *core, u32 addr, u32 data);
@@ -548,4 +964,16 @@ void wave6_vpu_dec_get_bitstream_buffer(struct vpu_instance *inst,
int wave6_vpu_dec_update_bitstream_buffer(struct vpu_instance *inst, int size);
int wave6_vpu_dec_flush_instance(struct vpu_instance *inst);
+int wave6_vpu_enc_open(struct vpu_instance *inst, struct enc_open_param *enc_op_param);
+int wave6_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res);
+int wave6_vpu_enc_issue_seq_init(struct vpu_instance *inst);
+int wave6_vpu_enc_issue_seq_change(struct vpu_instance *inst, bool *changed);
+int wave6_vpu_enc_complete_seq_init(struct vpu_instance *inst, struct enc_seq_info *info);
+int wave6_vpu_enc_get_aux_buffer_size(struct vpu_instance *inst,
+ struct aux_buffer_size_info info, u32 *size);
+int wave6_vpu_enc_register_frame_buffer(struct vpu_instance *inst);
+int wave6_vpu_enc_start_one_frame(struct vpu_instance *inst, struct enc_param *param,
+ u32 *fail_res);
+int wave6_vpu_enc_get_output_info(struct vpu_instance *inst, struct enc_output_info *info);
+
#endif /* __WAVE6_VPUAPI_H__ */
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h b/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
index ef261b5d328c..36daea5d02d1 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
@@ -30,6 +30,17 @@
#define W6_MAX_DEC_PIC_HEIGHT 4096
#define W6_DEC_PIC_SIZE_STEP 1
+#define W6_DEF_ENC_PIC_WIDTH 416
+#define W6_DEF_ENC_PIC_HEIGHT 240
+#define W6_MIN_ENC_PIC_WIDTH 256
+#define W6_MIN_ENC_PIC_HEIGHT 128
+#define W6_MAX_ENC_PIC_WIDTH 4096
+#define W6_MAX_ENC_PIC_HEIGHT 4096
+#define W6_ENC_PIC_SIZE_STEP 8
+#define W6_ENC_CROP_X_POS_STEP 32
+#define W6_ENC_CROP_Y_POS_STEP 8
+#define W6_ENC_CROP_STEP 2
+
#define W6_VPU_POLL_DELAY_US 10
#define W6_VPU_POLL_TIMEOUT 300000
#define W6_BOOT_WAIT_TIMEOUT 10000
@@ -38,13 +49,16 @@
#define HOST_ENDIAN VDI_128BIT_LITTLE_ENDIAN
#define VPU_STREAM_ENDIAN HOST_ENDIAN
+#define VPU_USER_DATA_ENDIAN HOST_ENDIAN
#define VPU_SOURCE_ENDIAN HOST_ENDIAN
#define W6_MAX_REMAP_PAGE_SIZE (1024 * 1024)
+#define W6_ARBUF_SIZE (1024)
#define W6_MAX_CODE_BUF_SIZE (4 * 1024 * 1024)
#define W6_CODE_BUF_SIZE (1 * 1024 * 1024)
#define W6_EXTRA_CODE_BUF_SIZE (256 * 1024)
#define W6_TEMPBUF_SIZE (3 * 1024 * 1024)
+#define W627ENC_WORKBUF_SIZE (512 * 1024)
#define W637DEC_WORKBUF_SIZE (2 * 512 * 1024)
#define W637DEC_WORKBUF_SIZE_FOR_CQ (3 * 512 * 1024)
--
2.41.0.windows.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v8 07/11] media: chips-media: wave6: Add Wave6 m2m encoder
2026-11-19 8:59 [PATCH v8 00/11] Add support for Wave6 video codec driver Nas Chung
` (5 preceding siblings ...)
2026-11-19 9:00 ` [PATCH v8 06/11] media: chips-media: wave6: Add Wave6 encoder firmware interface Nas Chung
@ 2026-11-19 9:00 ` Nas Chung
2026-11-19 9:35 ` sashiko-bot
2026-11-19 9:00 ` [PATCH v8 08/11] media: chips-media: wave6: Add trace and debugfs Nas Chung
` (3 subsequent siblings)
10 siblings, 1 reply; 23+ messages in thread
From: Nas Chung @ 2026-11-19 9:00 UTC (permalink / raw)
To: mchehab, hverkuil, robh, krzk+dt, conor+dt, Frank.Li, s.hauer
Cc: linux-media, devicetree, linux-kernel, imx, linux-arm-kernel,
festevam, kernel, nicolas.dufresne, ming.qian, marek.vasut,
jackson.lee, lafley.kim, Nas Chung
Add the V4L2 m2m stateful encoder for the Chips&Media Wave6 video codec
IP.
The encoder supports the following features and formats:
- Supported codecs: AVC, HEVC (8-bit streams only)
- Supported raw (OUTPUT) formats:
* V4L2_PIX_FMT_YUV420/NV12/NV21
* V4L2_PIX_FMT_YUV420M/NV12M/NV21M
* V4L2_PIX_FMT_YUV422P/NV16/NV61
* V4L2_PIX_FMT_YUV422M/NV16M/NV61M
* V4L2_PIX_FMT_YUYV
* V4L2_PIX_FMT_YUV24/NV24/NV42
* V4L2_PIX_FMT_P010
* V4L2_PIX_FMT_RGB24
* V4L2_PIX_FMT_ARGB32/XRGB32/RGBA32/RGBX32
* V4L2_PIX_FMT_ARGB2101010
- Although 10-bit raw formats are accepted (P010, ARGB2101010), the
encoded stream is limited to 8-bit
- Supports crop
- Supports rotation and horizontal/vertical flip, except for YUV 4:4:4
and RGB input
- Supports the V4L2 encoder controls
While other versions of the Wave6 IP may support AV1 encoding, it is
not implemented or validated in this driver at this time.
Tested-by: Ming Qian <ming.qian@oss.nxp.com>
Tested-by: Marek Vasut <marek.vasut@mailbox.org>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
---
.../media/platform/chips-media/wave6/Kconfig | 2 +-
.../media/platform/chips-media/wave6/Makefile | 1 +
.../chips-media/wave6/wave6-vpu-core.c | 20 +-
.../chips-media/wave6/wave6-vpu-core.h | 12 +
.../chips-media/wave6/wave6-vpu-enc.c | 2732 +++++++++++++++++
.../chips-media/wave6/wave6-vpu-v4l2.c | 22 +
.../platform/chips-media/wave6/wave6-vpuapi.h | 2 +
7 files changed, 2789 insertions(+), 2 deletions(-)
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
diff --git a/drivers/media/platform/chips-media/wave6/Kconfig b/drivers/media/platform/chips-media/wave6/Kconfig
index 9f2c2f92ee38..64a11dea18ed 100644
--- a/drivers/media/platform/chips-media/wave6/Kconfig
+++ b/drivers/media/platform/chips-media/wave6/Kconfig
@@ -10,7 +10,7 @@ config VIDEO_WAVE6_VPU
select GENERIC_ALLOCATOR
help
Chips&Media Wave6 stateful codec driver.
- It provides decoding capabilities for H.264, HEVC,
+ It provides encoding and decoding capabilities for H.264, HEVC,
and other video formats.
To compile this driver as a module, choose M here: the
module will be called wave6.
diff --git a/drivers/media/platform/chips-media/wave6/Makefile b/drivers/media/platform/chips-media/wave6/Makefile
index 9bfb8b45d091..53ee91b88bed 100644
--- a/drivers/media/platform/chips-media/wave6/Makefile
+++ b/drivers/media/platform/chips-media/wave6/Makefile
@@ -5,5 +5,6 @@ wave6-objs += wave6-vpu.o \
wave6-vpu-v4l2.o \
wave6-vpuapi.o \
wave6-vpu-dec.o \
+ wave6-vpu-enc.o \
wave6-hw.o
obj-$(CONFIG_VIDEO_WAVE6_VPU) += wave6.o
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
index 2baf51a5709d..aad4023d66a4 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
@@ -187,6 +187,11 @@ static void wave6_vpu_core_unregister_dec(void *data)
wave6_vpu_dec_unregister_device(data);
}
+static void wave6_vpu_core_unregister_enc(void *data)
+{
+ wave6_vpu_enc_unregister_device(data);
+}
+
static int wave6_vpu_core_probe(struct platform_device *pdev)
{
struct vpu_core_device *core;
@@ -296,8 +301,21 @@ static int wave6_vpu_core_probe(struct platform_device *pdev)
if (ret)
return ret;
}
+ if (core->res->codec_types & WAVE6_IS_ENC) {
+ ret = wave6_vpu_enc_register_device(core);
+ if (ret) {
+ dev_err(&pdev->dev,
+ "failed to register video_dev_enc: %d\n", ret);
+ return ret;
+ }
+
+ ret = devm_add_action_or_reset(&pdev->dev, wave6_vpu_core_unregister_enc, core);
+ if (ret)
+ return ret;
+ }
- dev_dbg(&pdev->dev, "Added wave6 driver with caps %s\n",
+ dev_dbg(&pdev->dev, "Added wave6 driver with caps %s %s\n",
+ core->res->codec_types & WAVE6_IS_ENC ? "'ENCODE'" : "",
core->res->codec_types & WAVE6_IS_DEC ? "'DECODE'" : "");
return 0;
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.h b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.h
index 1c473c8ce562..dc0b9867ca37 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.h
@@ -27,6 +27,12 @@ struct vpu_buffer {
bool consumed;
bool used;
bool error;
+ bool force_key_frame;
+ bool force_frame_qp;
+ u32 force_i_frame_qp;
+ u32 force_p_frame_qp;
+ u32 force_b_frame_qp;
+ u32 average_qp;
};
enum vpu_fmt_type {
@@ -36,6 +42,9 @@ enum vpu_fmt_type {
#define VPU_FMT_FLAG_CBCR_INTERLEAVED BIT(0)
#define VPU_FMT_FLAG_CRCB_ORDER BIT(1)
+#define VPU_FMT_FLAG_10BIT BIT(2)
+#define VPU_FMT_FLAG_RGB BIT(3)
+#define VPU_FMT_FLAG_SUPPORT_ROT_MIR BIT(4)
struct vpu_format {
unsigned int v4l2_pix_fmt;
@@ -46,6 +55,7 @@ struct vpu_format {
unsigned int num_planes;
enum frame_buffer_format fb_fmt;
enum endian_mode endian;
+ enum csc_format_order csc_fmt_order;
unsigned int flags;
};
@@ -94,6 +104,8 @@ void wave6_vpu_set_instance_state(struct vpu_instance *inst,
int wave6_vpu_wait_interrupt(struct vpu_instance *inst, unsigned int timeout);
int wave6_vpu_dec_register_device(struct vpu_core_device *core);
void wave6_vpu_dec_unregister_device(struct vpu_core_device *core);
+int wave6_vpu_enc_register_device(struct vpu_core_device *core);
+void wave6_vpu_enc_unregister_device(struct vpu_core_device *core);
void wave6_vpu_device_run_timeout(struct work_struct *work);
void wave6_vpu_finish_job(struct vpu_instance *inst);
int wave6_vpu_init_m2m_dev(struct vpu_core_device *core);
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
new file mode 100644
index 000000000000..7142666d4503
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
@@ -0,0 +1,2732 @@
+// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause)
+/*
+ * Wave6 series multi-standard codec IP - v4l2 stateful encoder interface
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#include <linux/math64.h>
+#include <linux/pm_runtime.h>
+
+#include "wave6-vpu-core.h"
+
+#define VPU_ENC_DEV_NAME "C&M Wave6 VPU encoder"
+#define VPU_ENC_DRV_NAME "wave6-enc"
+
+static const struct vpu_format wave6_vpu_enc_fmt_list[2][23] = {
+ [VPU_FMT_TYPE_CODEC] = {
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_HEVC,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_H264,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ },
+ },
+ [VPU_FMT_TYPE_RAW] = {
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_YUV420,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_420,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_SUPPORT_ROT_MIR,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_NV12,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_420,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_SUPPORT_ROT_MIR,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_NV21,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_420,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_CRCB_ORDER |
+ VPU_FMT_FLAG_SUPPORT_ROT_MIR,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_YUV422P,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_422,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_SUPPORT_ROT_MIR,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_NV16,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_422,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_SUPPORT_ROT_MIR,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_NV61,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_422,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_CRCB_ORDER |
+ VPU_FMT_FLAG_SUPPORT_ROT_MIR,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_YUYV,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_YUYV,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_SUPPORT_ROT_MIR,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_YUV24,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_YUV444_24BIT_PACKED,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_NV24,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_YUV444_24BIT,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_NV42,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_YUV444_24BIT,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_CRCB_ORDER,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_YUV420M,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 3,
+ .fb_fmt = FORMAT_420,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_SUPPORT_ROT_MIR,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_NV12M,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 2,
+ .fb_fmt = FORMAT_420,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_SUPPORT_ROT_MIR,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 2,
+ .fb_fmt = FORMAT_420,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_CRCB_ORDER |
+ VPU_FMT_FLAG_SUPPORT_ROT_MIR,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_YUV422M,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 3,
+ .fb_fmt = FORMAT_422,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_SUPPORT_ROT_MIR,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_NV16M,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 2,
+ .fb_fmt = FORMAT_422,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_SUPPORT_ROT_MIR,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_NV61M,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 2,
+ .fb_fmt = FORMAT_422,
+ .endian = VPU_SOURCE_ENDIAN,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_CRCB_ORDER |
+ VPU_FMT_FLAG_SUPPORT_ROT_MIR,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_P010,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_420_P10_16BIT_MSB,
+ .endian = VDI_128BIT_LE_BYTE_SWAP,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_10BIT |
+ VPU_FMT_FLAG_SUPPORT_ROT_MIR,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_RGB24,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_RGB_24BIT_PACKED,
+ .endian = VPU_SOURCE_ENDIAN,
+ .csc_fmt_order = CSC_FMT_ORDER_RGB,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_RGB,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_ARGB32,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_RGB_32BIT_PACKED,
+ .endian = VPU_SOURCE_ENDIAN,
+ .csc_fmt_order = CSC_FMT_ORDER_ARGB,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_RGB,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_XRGB32,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_RGB_32BIT_PACKED,
+ .endian = VPU_SOURCE_ENDIAN,
+ .csc_fmt_order = CSC_FMT_ORDER_ARGB,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_RGB,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_RGBA32,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_RGB_32BIT_PACKED,
+ .endian = VPU_SOURCE_ENDIAN,
+ .csc_fmt_order = CSC_FMT_ORDER_RGBA,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_RGB,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_RGBX32,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_RGB_32BIT_PACKED,
+ .endian = VPU_SOURCE_ENDIAN,
+ .csc_fmt_order = CSC_FMT_ORDER_RGBA,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_RGB,
+ },
+ {
+ .v4l2_pix_fmt = V4L2_PIX_FMT_ARGB2101010,
+ .max_width = W6_MAX_ENC_PIC_WIDTH,
+ .min_width = W6_MIN_ENC_PIC_WIDTH,
+ .max_height = W6_MAX_ENC_PIC_HEIGHT,
+ .min_height = W6_MIN_ENC_PIC_HEIGHT,
+ .num_planes = 1,
+ .fb_fmt = FORMAT_RGB_P10_32BIT_PACKED,
+ .endian = VDI_128BIT_LE_WORD_BYTE_SWAP,
+ .csc_fmt_order = CSC_FMT_ORDER_ARGB,
+ .flags = VPU_FMT_FLAG_CBCR_INTERLEAVED |
+ VPU_FMT_FLAG_RGB |
+ VPU_FMT_FLAG_10BIT,
+ },
+ }
+};
+
+static const struct vpu_format *wave6_find_vpu_fmt(unsigned int v4l2_pix_fmt,
+ enum vpu_fmt_type type)
+{
+ for (int i = 0; i < ARRAY_SIZE(wave6_vpu_enc_fmt_list[type]); i++) {
+ if (wave6_vpu_enc_fmt_list[type][i].v4l2_pix_fmt == v4l2_pix_fmt)
+ return &wave6_vpu_enc_fmt_list[type][i];
+ }
+
+ return NULL;
+}
+
+static const struct vpu_format *wave6_find_vpu_fmt_by_idx(unsigned int idx,
+ enum vpu_fmt_type type)
+{
+ if (idx >= ARRAY_SIZE(wave6_vpu_enc_fmt_list[type]))
+ return NULL;
+
+ if (!wave6_vpu_enc_fmt_list[type][idx].v4l2_pix_fmt)
+ return NULL;
+
+ return &wave6_vpu_enc_fmt_list[type][idx];
+}
+
+static u32 wave6_cpb_size_msec(u32 cpb_size_kb, u32 bitrate)
+{
+ u64 cpb_size_bit;
+ u64 cpb_size_msec;
+
+ cpb_size_bit = (u64)cpb_size_kb * 1000 * BITS_PER_BYTE;
+ cpb_size_msec = div_u64(cpb_size_bit * 1000, bitrate);
+
+ if (cpb_size_msec < MIN_CPB_SIZE || cpb_size_msec > MAX_CPB_SIZE)
+ cpb_size_msec = DEFAULT_CPB_SIZE;
+
+ return cpb_size_msec;
+}
+
+static void wave6_vpu_enc_release_fb(struct vpu_instance *inst)
+{
+ for (int i = 0; i < WAVE6_MAX_FBS; i++) {
+ wave6_vdi_free_dma(&inst->frame_vbuf[i]);
+ memset(&inst->frame_buf[i], 0, sizeof(struct frame_buffer));
+ wave6_vdi_free_dma(&inst->aux_vbuf[AUX_BUF_FBC_Y_TBL][i]);
+ wave6_vdi_free_dma(&inst->aux_vbuf[AUX_BUF_FBC_C_TBL][i]);
+ wave6_vdi_free_dma(&inst->aux_vbuf[AUX_BUF_MV_COL][i]);
+ wave6_vdi_free_dma(&inst->aux_vbuf[AUX_BUF_SUB_SAMPLE][i]);
+ }
+}
+
+static void wave6_vpu_enc_destroy_instance(struct vpu_instance *inst)
+{
+ u32 fail_res = 0;
+ int ret;
+
+ wave6_vpu_disable_instance(inst);
+
+ ret = wave6_vpu_enc_close(inst, &fail_res);
+ if (ret) {
+ dev_err(inst->dev->dev, "failed destroy instance: %d (%d)\n",
+ ret, fail_res);
+ }
+
+ scoped_guard(spinlock, &inst->dev->inst_lock)
+ list_del_init(&inst->list);
+ wave6_vpu_enc_release_fb(inst);
+ wave6_vdi_free_dma(&inst->ar_vbuf);
+ wave6_vpu_set_instance_state(inst, VPU_INST_STATE_NONE);
+
+ pm_runtime_put_sync(inst->dev->dev);
+ wave6_vpu_core_deactivate(inst->dev);
+}
+
+static struct vb2_v4l2_buffer *wave6_get_valid_src_buf(struct vpu_instance *inst)
+{
+ struct vb2_v4l2_buffer *vb2_v4l2_buf;
+ struct v4l2_m2m_buffer *v4l2_m2m_buf;
+ struct vpu_buffer *vpu_buf = NULL;
+
+ v4l2_m2m_for_each_src_buf(inst->v4l2_fh.m2m_ctx, v4l2_m2m_buf) {
+ vb2_v4l2_buf = &v4l2_m2m_buf->vb;
+ vpu_buf = wave6_to_vpu_buf(vb2_v4l2_buf);
+
+ if (!vpu_buf->consumed) {
+ dev_dbg(inst->dev->dev, "no consumed src idx : %d\n",
+ vb2_v4l2_buf->vb2_buf.index);
+ return vb2_v4l2_buf;
+ }
+ }
+
+ return NULL;
+}
+
+static struct vb2_v4l2_buffer *wave6_get_valid_dst_buf(struct vpu_instance *inst)
+{
+ struct vb2_v4l2_buffer *vb2_v4l2_buf;
+ struct v4l2_m2m_buffer *v4l2_m2m_buf;
+ struct vpu_buffer *vpu_buf;
+
+ v4l2_m2m_for_each_dst_buf(inst->v4l2_fh.m2m_ctx, v4l2_m2m_buf) {
+ vb2_v4l2_buf = &v4l2_m2m_buf->vb;
+ vpu_buf = wave6_to_vpu_buf(vb2_v4l2_buf);
+
+ if (!vpu_buf->consumed) {
+ dev_dbg(inst->dev->dev, "no consumed dst idx : %d\n",
+ vb2_v4l2_buf->vb2_buf.index);
+ return vb2_v4l2_buf;
+ }
+ }
+
+ return NULL;
+}
+
+static void wave6_set_csc(struct vpu_instance *inst, struct enc_param *pic_param)
+{
+ const struct vpu_format *vpu_fmt;
+ bool is_10bit;
+
+ vpu_fmt = wave6_find_vpu_fmt(inst->src_fmt.pixelformat, VPU_FMT_TYPE_RAW);
+ if (!vpu_fmt)
+ return;
+
+ if (!(vpu_fmt->flags & VPU_FMT_FLAG_RGB))
+ return;
+
+ is_10bit = !!(vpu_fmt->flags & VPU_FMT_FLAG_10BIT);
+ pic_param->csc.fmt_order = vpu_fmt->csc_fmt_order;
+
+ if (inst->ycbcr_enc == V4L2_YCBCR_ENC_DEFAULT ||
+ inst->ycbcr_enc == V4L2_YCBCR_ENC_601) {
+ if (inst->quantization == V4L2_QUANTIZATION_FULL_RANGE) {
+ /*
+ * Y 0.299(R) 0.587(G) 0.114(B)
+ * Cb -0.16874(R) -0.33126(G) 0.5(B)
+ * Cr 0.5(R) -0.41869(G) -0.08131(B)
+ */
+ pic_param->csc.coef_ry = 0x099;
+ pic_param->csc.coef_gy = 0x12d;
+ pic_param->csc.coef_by = 0x03a;
+ pic_param->csc.coef_rcb = 0xffffffaa;
+ pic_param->csc.coef_gcb = 0xffffff56;
+ pic_param->csc.coef_bcb = 0x100;
+ pic_param->csc.coef_rcr = 0x100;
+ pic_param->csc.coef_gcr = 0xffffff2a;
+ pic_param->csc.coef_bcr = 0xffffffd6;
+ pic_param->csc.offset_y = 0x0;
+ pic_param->csc.offset_cb = (is_10bit) ? 0x200 : 0x80;
+ pic_param->csc.offset_cr = (is_10bit) ? 0x200 : 0x80;
+ } else {
+ /*
+ * Y 0.258(R) 0.504(G) 0.098(B)
+ * Cb -0.1484(R) -0.2891(G) 0.4375(B)
+ * Cr 0.4375(R) -0.3672(G) -0.0703(B)
+ */
+ pic_param->csc.coef_ry = 0x084;
+ pic_param->csc.coef_gy = 0x102;
+ pic_param->csc.coef_by = 0x032;
+ pic_param->csc.coef_rcb = 0xffffffb4;
+ pic_param->csc.coef_gcb = 0xffffff6c;
+ pic_param->csc.coef_bcb = 0x0e0;
+ pic_param->csc.coef_rcr = 0x0e0;
+ pic_param->csc.coef_gcr = 0xffffff44;
+ pic_param->csc.coef_bcr = 0xffffffdc;
+ pic_param->csc.offset_y = (is_10bit) ? 0x40 : 0x10;
+ pic_param->csc.offset_cb = (is_10bit) ? 0x200 : 0x80;
+ pic_param->csc.offset_cr = (is_10bit) ? 0x200 : 0x80;
+ }
+ } else if (inst->ycbcr_enc == V4L2_YCBCR_ENC_709) {
+ if (inst->quantization == V4L2_QUANTIZATION_FULL_RANGE) {
+ /*
+ * Y 0.2126(R) 0.7152(G) 0.0722(B)
+ * Cb -0.11457(R) -0.38543(G) 0.5(B)
+ * Cr 0.5(R) -0.45415(G) -0.04585(B)
+ */
+ pic_param->csc.coef_ry = 0x06d;
+ pic_param->csc.coef_gy = 0x16e;
+ pic_param->csc.coef_by = 0x025;
+ pic_param->csc.coef_rcb = 0xffffffc5;
+ pic_param->csc.coef_gcb = 0xffffff3b;
+ pic_param->csc.coef_bcb = 0x100;
+ pic_param->csc.coef_rcr = 0x100;
+ pic_param->csc.coef_gcr = 0xffffff17;
+ pic_param->csc.coef_bcr = 0xffffffe9;
+ pic_param->csc.offset_y = 0x0;
+ pic_param->csc.offset_cb = (is_10bit) ? 0x200 : 0x80;
+ pic_param->csc.offset_cr = (is_10bit) ? 0x200 : 0x80;
+ } else {
+ pic_param->csc.coef_ry = 0x05e;
+ pic_param->csc.coef_gy = 0x13b;
+ pic_param->csc.coef_by = 0x020;
+ pic_param->csc.coef_rcb = 0xffffffcc;
+ pic_param->csc.coef_gcb = 0xffffff53;
+ pic_param->csc.coef_bcb = 0x0e1;
+ pic_param->csc.coef_rcr = 0x0e1;
+ pic_param->csc.coef_gcr = 0xffffff34;
+ pic_param->csc.coef_bcr = 0xffffffeb;
+ pic_param->csc.offset_y = (is_10bit) ? 0x40 : 0x10;
+ pic_param->csc.offset_cb = (is_10bit) ? 0x200 : 0x80;
+ pic_param->csc.offset_cr = (is_10bit) ? 0x200 : 0x80;
+ }
+ } else if (inst->ycbcr_enc == V4L2_YCBCR_ENC_BT2020) {
+ if (inst->quantization == V4L2_QUANTIZATION_FULL_RANGE) {
+ /*
+ * Y 0.2627(R) 0.678(G) 0.0593(B)
+ * Cb -0.13963(R) -0.36037(G) 0.5(B)
+ * Cr 0.5(R) -0.45979(G) -0.04021(B)
+ */
+ pic_param->csc.coef_ry = 0x087;
+ pic_param->csc.coef_gy = 0x15b;
+ pic_param->csc.coef_by = 0x01e;
+ pic_param->csc.coef_rcb = 0xffffffb9;
+ pic_param->csc.coef_gcb = 0xffffff47;
+ pic_param->csc.coef_bcb = 0x100;
+ pic_param->csc.coef_rcr = 0x100;
+ pic_param->csc.coef_gcr = 0xffffff15;
+ pic_param->csc.coef_bcr = 0xffffffeb;
+ pic_param->csc.offset_y = 0x0;
+ pic_param->csc.offset_cb = (is_10bit) ? 0x200 : 0x80;
+ pic_param->csc.offset_cr = (is_10bit) ? 0x200 : 0x80;
+ } else {
+ pic_param->csc.coef_ry = 0x074;
+ pic_param->csc.coef_gy = 0x12a;
+ pic_param->csc.coef_by = 0x01a;
+ pic_param->csc.coef_rcb = 0xffffffc1;
+ pic_param->csc.coef_gcb = 0xffffff5e;
+ pic_param->csc.coef_bcb = 0x0e1;
+ pic_param->csc.coef_rcr = 0x0e1;
+ pic_param->csc.coef_gcr = 0xffffff31;
+ pic_param->csc.coef_bcr = 0xffffffee;
+ pic_param->csc.offset_y = (is_10bit) ? 0x40 : 0x10;
+ pic_param->csc.offset_cb = (is_10bit) ? 0x200 : 0x80;
+ pic_param->csc.offset_cr = (is_10bit) ? 0x200 : 0x80;
+ }
+ } else if (inst->ycbcr_enc == V4L2_YCBCR_ENC_SMPTE240M) {
+ if (inst->quantization == V4L2_QUANTIZATION_FULL_RANGE) {
+ /*
+ * Y 0.2122(R) 0.7013(G) 0.0865(B)
+ * Cb -0.1161(R) -0.3839(G) 0.5(B)
+ * Cr 0.5(R) -0.4451(G) -0.0549(B)
+ */
+ pic_param->csc.coef_ry = 0x06d;
+ pic_param->csc.coef_gy = 0x167;
+ pic_param->csc.coef_by = 0x02c;
+ pic_param->csc.coef_rcb = 0xffffffc5;
+ pic_param->csc.coef_gcb = 0xffffff3b;
+ pic_param->csc.coef_bcb = 0x100;
+ pic_param->csc.coef_rcr = 0x100;
+ pic_param->csc.coef_gcr = 0xffffff1c;
+ pic_param->csc.coef_bcr = 0xffffffe4;
+ pic_param->csc.offset_y = 0x0;
+ pic_param->csc.offset_cb = (is_10bit) ? 0x200 : 0x80;
+ pic_param->csc.offset_cr = (is_10bit) ? 0x200 : 0x80;
+ } else {
+ pic_param->csc.coef_ry = 0x05d;
+ pic_param->csc.coef_gy = 0x134;
+ pic_param->csc.coef_by = 0x026;
+ pic_param->csc.coef_rcb = 0xffffffcc;
+ pic_param->csc.coef_gcb = 0xffffff53;
+ pic_param->csc.coef_bcb = 0x0e1;
+ pic_param->csc.coef_rcr = 0x0e1;
+ pic_param->csc.coef_gcr = 0xffffff38;
+ pic_param->csc.coef_bcr = 0xffffffe7;
+ pic_param->csc.offset_y = (is_10bit) ? 0x40 : 0x10;
+ pic_param->csc.offset_cb = (is_10bit) ? 0x200 : 0x80;
+ pic_param->csc.offset_cr = (is_10bit) ? 0x200 : 0x80;
+ }
+ } else if (inst->ycbcr_enc == V4L2_YCBCR_ENC_XV601) {
+ if (inst->quantization == V4L2_QUANTIZATION_LIM_RANGE) {
+ /*
+ * Y 0.2558(R) 0.5021(G) 0.0975(B)
+ * Cb -0.1476(R) -0.2899(G) 0.4375(B)
+ * Cr 0.4375(R) -0.3664(G) -0.0711(B)
+ */
+ pic_param->csc.coef_ry = 0x083;
+ pic_param->csc.coef_gy = 0x101;
+ pic_param->csc.coef_by = 0x032;
+ pic_param->csc.coef_rcb = 0xffffffb4;
+ pic_param->csc.coef_gcb = 0xffffff6c;
+ pic_param->csc.coef_bcb = 0x0e0;
+ pic_param->csc.coef_rcr = 0x0e0;
+ pic_param->csc.coef_gcr = 0xffffff44;
+ pic_param->csc.coef_bcr = 0xffffffdc;
+ pic_param->csc.offset_y = (is_10bit) ? 0x40 : 0x10;
+ pic_param->csc.offset_cb = 0x0;
+ pic_param->csc.offset_cr = 0x0;
+ }
+ } else if (inst->ycbcr_enc == V4L2_YCBCR_ENC_XV709) {
+ if (inst->quantization == V4L2_QUANTIZATION_LIM_RANGE) {
+ /*
+ * Y 0.1819(R) 0.6118(G) 0.0618(B)
+ * Cb -0.1003(R) -0.3372(G) 0.4375(B)
+ * Cr 0.4375(R) -0.3974(G) -0.0401(B)
+ */
+ pic_param->csc.coef_ry = 0x05d;
+ pic_param->csc.coef_gy = 0x139;
+ pic_param->csc.coef_by = 0x020;
+ pic_param->csc.coef_rcb = 0xffffffcd;
+ pic_param->csc.coef_gcb = 0xffffff53;
+ pic_param->csc.coef_bcb = 0x0e0;
+ pic_param->csc.coef_rcr = 0x0e0;
+ pic_param->csc.coef_gcr = 0xffffff35;
+ pic_param->csc.coef_bcr = 0xffffffeb;
+ pic_param->csc.offset_y = (is_10bit) ? 0x40 : 0x10;
+ pic_param->csc.offset_cb = 0x0;
+ pic_param->csc.offset_cr = 0x0;
+ }
+ }
+}
+
+static void wave6_update_crop_info(struct vpu_instance *inst,
+ u32 left, u32 top, u32 width, u32 height)
+{
+ u32 enc_pic_width, enc_pic_height;
+
+ inst->crop.left = left;
+ inst->crop.top = top;
+ inst->crop.width = width;
+ inst->crop.height = height;
+
+ inst->codec_rect.left = round_down(left, W6_ENC_CROP_X_POS_STEP);
+ inst->codec_rect.top = round_down(top, W6_ENC_CROP_Y_POS_STEP);
+
+ enc_pic_width = width + left - inst->codec_rect.left;
+ inst->codec_rect.width = round_up(enc_pic_width, W6_ENC_PIC_SIZE_STEP);
+
+ enc_pic_height = height + top - inst->codec_rect.top;
+ inst->codec_rect.height = round_up(enc_pic_height, W6_ENC_PIC_SIZE_STEP);
+}
+
+static int wave6_allocate_aux_buffer(struct vpu_instance *inst,
+ enum aux_buffer_type type,
+ unsigned int width,
+ unsigned int height,
+ int num)
+{
+ struct aux_buffer_size_info size_info;
+ unsigned int size;
+ int ret;
+
+ size_info.width = width;
+ size_info.height = height;
+ size_info.type = type;
+
+ ret = wave6_vpu_enc_get_aux_buffer_size(inst, size_info, &size);
+ if (ret) {
+ dev_err(inst->dev->dev, "%s: Get size fail (type %d)\n", __func__, type);
+ return ret;
+ }
+
+ for (int i = 0; i < num; i++) {
+ inst->aux_vbuf[type][i].size = size;
+ ret = wave6_vdi_alloc_dma(inst->dev->dev, &inst->aux_vbuf[type][i]);
+ if (ret) {
+ dev_err(inst->dev->dev, "%s: Alloc fail (type %d)\n", __func__, type);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static void wave6_update_frame_buf_addr(struct vpu_instance *inst,
+ struct frame_buffer *frame_buf)
+{
+ const struct v4l2_format_info *fmt_info;
+ u32 stride = inst->src_fmt.plane_fmt[0].bytesperline;
+ u32 offset;
+
+ fmt_info = v4l2_format_info(inst->src_fmt.pixelformat);
+ if (!fmt_info)
+ return;
+
+ offset = inst->codec_rect.top * stride + inst->codec_rect.left * fmt_info->bpp[0];
+ frame_buf->buf_y += offset;
+
+ stride = DIV_ROUND_UP(stride, fmt_info->bpp[0]) * fmt_info->bpp[1];
+ offset = inst->codec_rect.top * stride / fmt_info->vdiv / fmt_info->hdiv
+ + inst->codec_rect.left * fmt_info->bpp[1] / fmt_info->hdiv;
+ frame_buf->buf_cb += offset;
+ frame_buf->buf_cr += offset;
+}
+
+static int wave6_update_seq_param(struct vpu_instance *inst)
+{
+ struct enc_seq_info seq_info;
+ bool changed = false;
+ int ret;
+
+ ret = wave6_vpu_enc_issue_seq_change(inst, &changed);
+ if (ret) {
+ dev_err(inst->dev->dev, "seq change fail %d\n", ret);
+ return ret;
+ }
+
+ if (!changed)
+ return 0;
+
+ ret = wave6_vpu_wait_interrupt(inst, W6_VPU_TIMEOUT);
+ if (ret) {
+ dev_err(inst->dev->dev, "seq change timeout\n");
+ return ret;
+ }
+
+ ret = wave6_vpu_enc_complete_seq_init(inst, &seq_info);
+ if (ret) {
+ dev_err(inst->dev->dev, "seq change error\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+static void wave6_calc_source_frame_info(struct vpu_instance *inst,
+ u32 *stride, u32 *luma_size,
+ u32 *stride_c, u32 *chroma_size)
+{
+ const struct v4l2_format_info *info = v4l2_format_info(inst->src_fmt.pixelformat);
+ u32 height = inst->src_fmt.height;
+
+ *stride = inst->src_fmt.plane_fmt[0].bytesperline;
+ *luma_size = *stride * height;
+
+ if (WARN_ON_ONCE(!info) || info->comp_planes < 2) {
+ *stride_c = 0;
+ *chroma_size = 0;
+ return;
+ }
+
+ *stride_c = DIV_ROUND_UP(*stride, info->bpp[0]);
+ *stride_c = DIV_ROUND_UP(*stride_c, info->hdiv) * info->bpp[1];
+ *chroma_size = *stride_c * DIV_ROUND_UP(height, info->vdiv);
+}
+
+static int wave6_vpu_enc_start_encode(struct vpu_instance *inst)
+{
+ int ret;
+ struct vb2_v4l2_buffer *src_buf = NULL;
+ struct vb2_v4l2_buffer *dst_buf = NULL;
+ struct vpu_buffer *src_vbuf = NULL;
+ struct vpu_buffer *dst_vbuf = NULL;
+ struct frame_buffer frame_buf;
+ struct enc_param pic_param;
+ u32 stride;
+ u32 luma_size;
+ u32 stride_c;
+ u32 chroma_size;
+ u32 fail_res = 0;
+
+ memset(&pic_param, 0, sizeof(struct enc_param));
+ memset(&frame_buf, 0, sizeof(struct frame_buffer));
+
+ wave6_calc_source_frame_info(inst, &stride, &luma_size,
+ &stride_c, &chroma_size);
+
+ src_buf = wave6_get_valid_src_buf(inst);
+ dst_buf = wave6_get_valid_dst_buf(inst);
+
+ if (!src_buf || !dst_buf) {
+ dev_dbg(inst->dev->dev, "no valid src or dst buf\n");
+ return 0;
+ }
+
+ dst_vbuf = wave6_to_vpu_buf(dst_buf);
+ dst_vbuf->consumed = true;
+ dst_vbuf->used = true;
+ pic_param.pic_stream_buffer_addr = wave6_get_dma_addr(dst_buf, 0);
+ pic_param.pic_stream_buffer_size = vb2_plane_size(&dst_buf->vb2_buf, 0);
+
+ src_vbuf = wave6_to_vpu_buf(src_buf);
+ src_vbuf->consumed = true;
+ if (inst->src_fmt.num_planes == 1) {
+ frame_buf.buf_y = wave6_get_dma_addr(src_buf, 0);
+ frame_buf.buf_cb = frame_buf.buf_y + luma_size;
+ frame_buf.buf_cr = frame_buf.buf_cb + chroma_size;
+ } else if (inst->src_fmt.num_planes == 2) {
+ frame_buf.buf_y = wave6_get_dma_addr(src_buf, 0);
+ frame_buf.buf_cb = wave6_get_dma_addr(src_buf, 1);
+ frame_buf.buf_cr = frame_buf.buf_cb + chroma_size;
+ } else if (inst->src_fmt.num_planes == 3) {
+ frame_buf.buf_y = wave6_get_dma_addr(src_buf, 0);
+ frame_buf.buf_cb = wave6_get_dma_addr(src_buf, 1);
+ frame_buf.buf_cr = wave6_get_dma_addr(src_buf, 2);
+ }
+ wave6_update_frame_buf_addr(inst, &frame_buf);
+ frame_buf.stride = stride;
+ frame_buf.stride_c = stride_c;
+ pic_param.source_frame = &frame_buf;
+ pic_param.src_idx = src_buf->vb2_buf.index;
+ if (src_vbuf->force_key_frame || inst->error_recovery) {
+ pic_param.force_pic = true;
+ pic_param.force_pic_type = ENC_FORCE_PIC_TYPE_IDR;
+ inst->error_recovery = false;
+ }
+ if (src_vbuf->force_frame_qp) {
+ pic_param.force_pic_qp = true;
+ pic_param.force_pic_qp_i = src_vbuf->force_i_frame_qp;
+ pic_param.force_pic_qp_p = src_vbuf->force_p_frame_qp;
+ pic_param.force_pic_qp_b = src_vbuf->force_b_frame_qp;
+ }
+ wave6_set_csc(inst, &pic_param);
+
+ ret = wave6_vpu_enc_start_one_frame(inst, &pic_param, &fail_res);
+ if (ret) {
+ dev_err(inst->dev->dev, "[%d] %s: fail %d\n", inst->id, __func__, ret);
+ wave6_vpu_set_instance_state(inst, VPU_INST_STATE_ERROR);
+ vb2_queue_error(v4l2_m2m_get_src_vq(inst->v4l2_fh.m2m_ctx));
+ vb2_queue_error(v4l2_m2m_get_dst_vq(inst->v4l2_fh.m2m_ctx));
+
+ dst_buf = v4l2_m2m_dst_buf_remove(inst->v4l2_fh.m2m_ctx);
+ if (dst_buf) {
+ dst_buf->sequence = inst->sequence;
+ v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
+ }
+
+ src_buf = v4l2_m2m_src_buf_remove(inst->v4l2_fh.m2m_ctx);
+ if (src_buf) {
+ v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
+ inst->sequence++;
+ inst->processed_buf_num++;
+ inst->error_buf_num++;
+ }
+ } else {
+ dev_dbg(inst->dev->dev, "%s: success\n", __func__);
+ }
+
+ return ret;
+}
+
+static void wave6_handle_encoded_frame(struct vpu_instance *inst,
+ struct enc_output_info *info)
+{
+ struct vb2_v4l2_buffer *src_buf;
+ struct vb2_v4l2_buffer *dst_buf;
+ struct vpu_buffer *vpu_buf;
+ struct vpu_buffer *dst_vpu_buf;
+ enum vb2_buffer_state state;
+
+ guard(mutex)(&inst->lock);
+
+ state = info->encoding_success ? VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
+
+ dst_buf = wave6_get_dst_buf_by_addr(inst, info->bitstream_buffer);
+ if (!dst_buf) {
+ dev_err(inst->dev->dev, "[%d] encoder can't find dst buffer\n", inst->id);
+ return;
+ }
+
+ src_buf = v4l2_m2m_src_buf_remove_by_idx(inst->v4l2_fh.m2m_ctx,
+ info->enc_src_idx);
+ if (!src_buf) {
+ dev_err(inst->dev->dev, "[%d] encoder can't find src buffer\n", inst->id);
+ return;
+ }
+
+ vpu_buf = wave6_to_vpu_buf(src_buf);
+ if (!vpu_buf->consumed) {
+ dev_err(inst->dev->dev, "[%d] src buffer is not consumed\n", inst->id);
+ v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
+ return;
+ }
+
+ dst_vpu_buf = wave6_to_vpu_buf(dst_buf);
+
+ dst_vpu_buf->average_qp = info->avg_ctu_qp;
+
+ if (v4l2_m2m_is_last_draining_src_buf(inst->v4l2_fh.m2m_ctx, src_buf)) {
+ dst_buf->flags |= V4L2_BUF_FLAG_LAST;
+ v4l2_m2m_mark_stopped(inst->v4l2_fh.m2m_ctx);
+ }
+
+ v4l2_m2m_buf_copy_metadata(src_buf, dst_buf);
+ v4l2_m2m_buf_done(src_buf, state);
+
+ vb2_set_plane_payload(&dst_buf->vb2_buf, 0, info->bitstream_size);
+ dst_buf->sequence = inst->sequence++;
+ dst_buf->field = V4L2_FIELD_NONE;
+ if (info->pic_type == PIC_TYPE_I)
+ dst_buf->flags |= V4L2_BUF_FLAG_KEYFRAME;
+ else if (info->pic_type == PIC_TYPE_P)
+ dst_buf->flags |= V4L2_BUF_FLAG_PFRAME;
+ else if (info->pic_type == PIC_TYPE_B)
+ dst_buf->flags |= V4L2_BUF_FLAG_BFRAME;
+
+ v4l2_m2m_dst_buf_remove_by_buf(inst->v4l2_fh.m2m_ctx, dst_buf);
+ if (state == VB2_BUF_STATE_ERROR) {
+ dev_dbg(inst->dev->dev, "[%d] error frame %d\n", inst->id, inst->sequence);
+ inst->error_recovery = true;
+ inst->error_buf_num++;
+ }
+ v4l2_m2m_buf_done(dst_buf, state);
+ inst->processed_buf_num++;
+}
+
+static void wave6_vpu_enc_finish_encode(struct vpu_instance *inst, bool error)
+{
+ int ret;
+ struct enc_output_info info;
+
+ if (error) {
+ vb2_queue_error(v4l2_m2m_get_src_vq(inst->v4l2_fh.m2m_ctx));
+ vb2_queue_error(v4l2_m2m_get_dst_vq(inst->v4l2_fh.m2m_ctx));
+
+ wave6_vpu_set_instance_state(inst, VPU_INST_STATE_ERROR);
+
+ goto finish_encode;
+ }
+
+ memset(&info, 0, sizeof(info));
+
+ ret = wave6_vpu_enc_get_output_info(inst, &info);
+ if (ret) {
+ dev_dbg(inst->dev->dev, "vpu_enc_get_output_info fail %d\n", ret);
+ goto finish_encode;
+ }
+
+ if (info.enc_src_idx >= 0 && info.recon_frame_index >= 0)
+ wave6_handle_encoded_frame(inst, &info);
+
+finish_encode:
+ wave6_vpu_finish_job(inst);
+}
+
+static int wave6_vpu_enc_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
+{
+ strscpy(cap->driver, VPU_ENC_DRV_NAME, sizeof(cap->driver));
+ strscpy(cap->card, VPU_ENC_DRV_NAME, sizeof(cap->card));
+ strscpy(cap->bus_info, "platform:" VPU_ENC_DRV_NAME, sizeof(cap->bus_info));
+
+ return 0;
+}
+
+static int wave6_vpu_enc_enum_framesizes(struct file *f, void *fh, struct v4l2_frmsizeenum *fsize)
+{
+ const struct vpu_format *vpu_fmt;
+
+ if (fsize->index)
+ return -EINVAL;
+
+ vpu_fmt = wave6_find_vpu_fmt(fsize->pixel_format, VPU_FMT_TYPE_CODEC);
+ if (!vpu_fmt) {
+ vpu_fmt = wave6_find_vpu_fmt(fsize->pixel_format, VPU_FMT_TYPE_RAW);
+ if (!vpu_fmt)
+ return -EINVAL;
+ }
+
+ fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+ fsize->stepwise.min_width = vpu_fmt->min_width;
+ fsize->stepwise.max_width = vpu_fmt->max_width;
+ fsize->stepwise.step_width = W6_ENC_PIC_SIZE_STEP;
+ fsize->stepwise.min_height = vpu_fmt->min_height;
+ fsize->stepwise.max_height = vpu_fmt->max_height;
+ fsize->stepwise.step_height = W6_ENC_PIC_SIZE_STEP;
+
+ return 0;
+}
+
+static int wave6_vpu_enc_enum_fmt_cap(struct file *file, void *fh, struct v4l2_fmtdesc *f)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ const struct vpu_format *vpu_fmt;
+
+ dev_dbg(inst->dev->dev, "index : %d\n", f->index);
+
+ vpu_fmt = wave6_find_vpu_fmt_by_idx(f->index, VPU_FMT_TYPE_CODEC);
+ if (!vpu_fmt)
+ return -EINVAL;
+
+ f->pixelformat = vpu_fmt->v4l2_pix_fmt;
+ f->flags = 0;
+
+ return 0;
+}
+
+static int wave6_vpu_enc_try_fmt_cap(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+ const struct vpu_format *vpu_fmt;
+ int width, height;
+
+ dev_dbg(inst->dev->dev, "%s: 4cc %d w %d h %d plane %d colorspace %d\n",
+ __func__, pix_mp->pixelformat, pix_mp->width, pix_mp->height,
+ pix_mp->num_planes, pix_mp->colorspace);
+
+ if (!V4L2_TYPE_IS_CAPTURE(f->type))
+ return -EINVAL;
+
+ vpu_fmt = wave6_find_vpu_fmt(pix_mp->pixelformat, VPU_FMT_TYPE_CODEC);
+ if (!vpu_fmt) {
+ width = inst->dst_fmt.width;
+ height = inst->dst_fmt.height;
+ pix_mp->pixelformat = inst->dst_fmt.pixelformat;
+ pix_mp->num_planes = inst->dst_fmt.num_planes;
+ } else {
+ width = pix_mp->width;
+ height = pix_mp->height;
+ pix_mp->pixelformat = vpu_fmt->v4l2_pix_fmt;
+ pix_mp->num_planes = vpu_fmt->num_planes;
+ }
+
+ wave6_update_pix_fmt(pix_mp, width, height);
+ pix_mp->colorspace = inst->colorspace;
+ pix_mp->ycbcr_enc = inst->ycbcr_enc;
+ pix_mp->quantization = inst->quantization;
+ pix_mp->xfer_func = inst->xfer_func;
+
+ return 0;
+}
+
+static int wave6_vpu_enc_s_fmt_cap(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+ struct vb2_queue *q;
+ int ret;
+
+ dev_dbg(inst->dev->dev, "%s: 4cc %d w %d h %d plane %d colorspace %d\n",
+ __func__, pix_mp->pixelformat, pix_mp->width, pix_mp->height,
+ pix_mp->num_planes, pix_mp->colorspace);
+
+ q = v4l2_m2m_get_vq(inst->v4l2_fh.m2m_ctx, f->type);
+ if (vb2_is_busy(q))
+ return -EBUSY;
+
+ ret = wave6_vpu_enc_try_fmt_cap(file, fh, f);
+ if (ret)
+ return ret;
+
+ inst->std = wave6_to_codec_std(inst->type, pix_mp->pixelformat);
+ if (inst->std == STD_UNKNOWN) {
+ dev_err(inst->dev->dev, "unsupported pixelformat: %.4s\n",
+ (char *)&pix_mp->pixelformat);
+ return -EINVAL;
+ }
+
+ inst->dst_fmt.width = pix_mp->width;
+ inst->dst_fmt.height = pix_mp->height;
+ inst->dst_fmt.pixelformat = pix_mp->pixelformat;
+ inst->dst_fmt.field = pix_mp->field;
+ inst->dst_fmt.flags = pix_mp->flags;
+ inst->dst_fmt.num_planes = pix_mp->num_planes;
+ for (int i = 0; i < inst->dst_fmt.num_planes; i++) {
+ inst->dst_fmt.plane_fmt[i].bytesperline = pix_mp->plane_fmt[i].bytesperline;
+ inst->dst_fmt.plane_fmt[i].sizeimage = pix_mp->plane_fmt[i].sizeimage;
+ }
+
+ return 0;
+}
+
+static int wave6_vpu_enc_g_fmt_cap(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+
+ pix_mp->width = inst->dst_fmt.width;
+ pix_mp->height = inst->dst_fmt.height;
+ pix_mp->pixelformat = inst->dst_fmt.pixelformat;
+ pix_mp->field = inst->dst_fmt.field;
+ pix_mp->flags = inst->dst_fmt.flags;
+ pix_mp->num_planes = inst->dst_fmt.num_planes;
+ for (int i = 0; i < pix_mp->num_planes; i++) {
+ pix_mp->plane_fmt[i].bytesperline = inst->dst_fmt.plane_fmt[i].bytesperline;
+ pix_mp->plane_fmt[i].sizeimage = inst->dst_fmt.plane_fmt[i].sizeimage;
+ }
+
+ pix_mp->colorspace = inst->colorspace;
+ pix_mp->ycbcr_enc = inst->ycbcr_enc;
+ pix_mp->quantization = inst->quantization;
+ pix_mp->xfer_func = inst->xfer_func;
+
+ return 0;
+}
+
+static int wave6_vpu_enc_enum_fmt_out(struct file *file, void *fh, struct v4l2_fmtdesc *f)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ const struct vpu_format *vpu_fmt;
+
+ dev_dbg(inst->dev->dev, "%s: index %d\n", __func__, f->index);
+
+ vpu_fmt = wave6_find_vpu_fmt_by_idx(f->index, VPU_FMT_TYPE_RAW);
+ if (!vpu_fmt)
+ return -EINVAL;
+
+ f->pixelformat = vpu_fmt->v4l2_pix_fmt;
+ f->flags = 0;
+
+ return 0;
+}
+
+static int wave6_vpu_enc_try_fmt_out(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+ const struct vpu_format *vpu_fmt;
+ int width, height;
+
+ dev_dbg(inst->dev->dev, "%s: 4cc %d w %d h %d plane %d colorspace %d\n",
+ __func__, pix_mp->pixelformat, pix_mp->width, pix_mp->height,
+ pix_mp->num_planes, pix_mp->colorspace);
+
+ if (!V4L2_TYPE_IS_OUTPUT(f->type))
+ return -EINVAL;
+
+ vpu_fmt = wave6_find_vpu_fmt(pix_mp->pixelformat, VPU_FMT_TYPE_RAW);
+ if (!vpu_fmt) {
+ width = inst->src_fmt.width;
+ height = inst->src_fmt.height;
+ pix_mp->pixelformat = inst->src_fmt.pixelformat;
+ pix_mp->num_planes = inst->src_fmt.num_planes;
+ } else {
+ width = clamp(pix_mp->width,
+ vpu_fmt->min_width, vpu_fmt->max_width);
+ height = clamp(pix_mp->height,
+ vpu_fmt->min_height, vpu_fmt->max_height);
+
+ pix_mp->pixelformat = vpu_fmt->v4l2_pix_fmt;
+ pix_mp->num_planes = vpu_fmt->num_planes;
+ }
+
+ wave6_update_pix_fmt(pix_mp,
+ round_up(width, W6_ENC_PIC_SIZE_STEP),
+ round_up(height, W6_ENC_PIC_SIZE_STEP));
+
+ if (pix_mp->ycbcr_enc == V4L2_YCBCR_ENC_BT2020_CONST_LUM)
+ pix_mp->ycbcr_enc = V4L2_YCBCR_ENC_BT2020;
+ if (pix_mp->ycbcr_enc == V4L2_YCBCR_ENC_XV601 ||
+ pix_mp->ycbcr_enc == V4L2_YCBCR_ENC_XV709) {
+ if (pix_mp->quantization == V4L2_QUANTIZATION_FULL_RANGE)
+ pix_mp->quantization = V4L2_QUANTIZATION_LIM_RANGE;
+ }
+
+ return 0;
+}
+
+static int wave6_vpu_enc_s_fmt_out(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+ const struct vpu_format *vpu_fmt;
+ struct vb2_queue *q;
+ int ret;
+
+ dev_dbg(inst->dev->dev, "%s: 4cc %d w %d h %d plane %d colorspace %d\n",
+ __func__, pix_mp->pixelformat, pix_mp->width, pix_mp->height,
+ pix_mp->num_planes, pix_mp->colorspace);
+
+ q = v4l2_m2m_get_vq(inst->v4l2_fh.m2m_ctx, f->type);
+ if (vb2_is_busy(q))
+ return -EBUSY;
+
+ ret = wave6_vpu_enc_try_fmt_out(file, fh, f);
+ if (ret)
+ return ret;
+
+ vpu_fmt = wave6_find_vpu_fmt(pix_mp->pixelformat, VPU_FMT_TYPE_RAW);
+ if (!vpu_fmt)
+ return -EINVAL;
+
+ inst->src_fmt.width = pix_mp->width;
+ inst->src_fmt.height = pix_mp->height;
+ inst->src_fmt.pixelformat = pix_mp->pixelformat;
+ inst->src_fmt.field = pix_mp->field;
+ inst->src_fmt.flags = pix_mp->flags;
+ inst->src_fmt.num_planes = pix_mp->num_planes;
+ for (int i = 0; i < inst->src_fmt.num_planes; i++) {
+ inst->src_fmt.plane_fmt[i].bytesperline = pix_mp->plane_fmt[i].bytesperline;
+ inst->src_fmt.plane_fmt[i].sizeimage = pix_mp->plane_fmt[i].sizeimage;
+ }
+
+ inst->cbcr_interleave = !!(vpu_fmt->flags & VPU_FMT_FLAG_CBCR_INTERLEAVED);
+ inst->nv21 = !!(vpu_fmt->flags & VPU_FMT_FLAG_CRCB_ORDER);
+ inst->colorspace = pix_mp->colorspace;
+ inst->ycbcr_enc = pix_mp->ycbcr_enc;
+ inst->quantization = pix_mp->quantization;
+ inst->xfer_func = pix_mp->xfer_func;
+
+ wave6_update_pix_fmt(&inst->dst_fmt, pix_mp->width, pix_mp->height);
+ wave6_update_crop_info(inst, 0, 0, pix_mp->width, pix_mp->height);
+
+ return 0;
+}
+
+static int wave6_vpu_enc_g_fmt_out(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+
+ pix_mp->width = inst->src_fmt.width;
+ pix_mp->height = inst->src_fmt.height;
+ pix_mp->pixelformat = inst->src_fmt.pixelformat;
+ pix_mp->field = inst->src_fmt.field;
+ pix_mp->flags = inst->src_fmt.flags;
+ pix_mp->num_planes = inst->src_fmt.num_planes;
+ for (int i = 0; i < pix_mp->num_planes; i++) {
+ pix_mp->plane_fmt[i].bytesperline = inst->src_fmt.plane_fmt[i].bytesperline;
+ pix_mp->plane_fmt[i].sizeimage = inst->src_fmt.plane_fmt[i].sizeimage;
+ }
+
+ pix_mp->colorspace = inst->colorspace;
+ pix_mp->ycbcr_enc = inst->ycbcr_enc;
+ pix_mp->quantization = inst->quantization;
+ pix_mp->xfer_func = inst->xfer_func;
+
+ return 0;
+}
+
+static int wave6_vpu_enc_g_selection(struct file *file, void *fh, struct v4l2_selection *s)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+
+ dev_dbg(inst->dev->dev, "%s: type %d target %d\n",
+ __func__, s->type, s->target);
+
+ if (!V4L2_TYPE_IS_OUTPUT(s->type))
+ return -EINVAL;
+
+ switch (s->target) {
+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ s->r.left = 0;
+ s->r.top = 0;
+ s->r.width = inst->src_fmt.width;
+ s->r.height = inst->src_fmt.height;
+ break;
+ case V4L2_SEL_TGT_CROP:
+ s->r = inst->crop;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int wave6_vpu_enc_s_selection(struct file *file, void *fh, struct v4l2_selection *s)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ u32 max_crop_w, max_crop_h;
+
+ if (!V4L2_TYPE_IS_OUTPUT(s->type))
+ return -EINVAL;
+
+ if (s->target != V4L2_SEL_TGT_CROP)
+ return -EINVAL;
+
+ if (!(s->flags & (V4L2_SEL_FLAG_GE | V4L2_SEL_FLAG_LE)))
+ s->flags |= V4L2_SEL_FLAG_LE;
+
+ if (s->flags & V4L2_SEL_FLAG_GE) {
+ s->r.left = round_up(s->r.left, W6_ENC_CROP_STEP);
+ s->r.top = round_up(s->r.top, W6_ENC_CROP_STEP);
+ s->r.width = round_up(s->r.width, W6_ENC_CROP_STEP);
+ s->r.height = round_up(s->r.height, W6_ENC_CROP_STEP);
+ }
+ if (s->flags & V4L2_SEL_FLAG_LE) {
+ s->r.left = round_down(s->r.left, W6_ENC_CROP_STEP);
+ s->r.top = round_down(s->r.top, W6_ENC_CROP_STEP);
+ s->r.width = round_down(s->r.width, W6_ENC_CROP_STEP);
+ s->r.height = round_down(s->r.height, W6_ENC_CROP_STEP);
+ }
+
+ s->r.left = clamp_t(s32, s->r.left, 0,
+ inst->src_fmt.width - W6_MIN_ENC_PIC_WIDTH);
+ s->r.top = clamp_t(s32, s->r.top, 0,
+ inst->src_fmt.height - W6_MIN_ENC_PIC_HEIGHT);
+
+ max_crop_w = inst->src_fmt.width - s->r.left;
+ max_crop_h = inst->src_fmt.height - s->r.top;
+
+ if (!s->r.width || !s->r.height)
+ return 0;
+ if (max_crop_w < W6_MIN_ENC_PIC_WIDTH)
+ return 0;
+ if (max_crop_h < W6_MIN_ENC_PIC_HEIGHT)
+ return 0;
+
+ s->r.width = clamp(s->r.width, W6_MIN_ENC_PIC_WIDTH, max_crop_w);
+ s->r.height = clamp(s->r.height, W6_MIN_ENC_PIC_HEIGHT, max_crop_h);
+
+ wave6_update_pix_fmt(&inst->dst_fmt, s->r.width, s->r.height);
+ wave6_update_crop_info(inst, s->r.left, s->r.top, s->r.width, s->r.height);
+
+ dev_dbg(inst->dev->dev, "V4L2_SEL_TGT_CROP %dx%dx%dx%d\n",
+ s->r.left, s->r.top, s->r.width, s->r.height);
+
+ return 0;
+}
+
+static int wave6_vpu_enc_encoder_cmd(struct file *file, void *fh, struct v4l2_encoder_cmd *ec)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+ int ret;
+
+ dev_dbg(inst->dev->dev, "%s: cmd %d\n", __func__, ec->cmd);
+
+ ret = v4l2_m2m_ioctl_try_encoder_cmd(file, fh, ec);
+ if (ret)
+ return ret;
+
+ if (!wave6_vpu_both_queues_are_streaming(inst))
+ return 0;
+
+ scoped_guard(mutex, &inst->lock)
+ ret = v4l2_m2m_ioctl_encoder_cmd(file, fh, ec);
+ if (ret)
+ return ret;
+
+ if (ec->cmd == V4L2_ENC_CMD_START) {
+ vb2_clear_last_buffer_dequeued(v4l2_m2m_get_dst_vq(m2m_ctx));
+ v4l2_m2m_try_schedule(m2m_ctx);
+ }
+
+ return 0;
+}
+
+static int wave6_vpu_enc_g_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+
+ dev_dbg(inst->dev->dev, "%s: type %d\n", __func__, a->type);
+
+ if (!V4L2_TYPE_IS_OUTPUT(a->type))
+ return -EINVAL;
+
+ a->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
+ a->parm.output.timeperframe.numerator = 1;
+ a->parm.output.timeperframe.denominator = inst->enc_ctrls.frame_rate;
+
+ dev_dbg(inst->dev->dev, "%s: numerator : %d | denominator : %d\n",
+ __func__,
+ a->parm.output.timeperframe.numerator,
+ a->parm.output.timeperframe.denominator);
+
+ return 0;
+}
+
+static int wave6_vpu_enc_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
+ struct enc_controls *ctrls = &inst->enc_ctrls;
+
+ dev_dbg(inst->dev->dev, "%s: type %d\n", __func__, a->type);
+
+ if (!V4L2_TYPE_IS_OUTPUT(a->type))
+ return -EINVAL;
+
+ a->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
+ if (a->parm.output.timeperframe.denominator &&
+ a->parm.output.timeperframe.numerator) {
+ ctrls->frame_rate = DIV_ROUND_CLOSEST(a->parm.output.timeperframe.denominator,
+ a->parm.output.timeperframe.numerator);
+ ctrls->frame_rate = clamp(ctrls->frame_rate,
+ MIN_FRAME_RATE, MAX_FRAME_RATE);
+ } else {
+ a->parm.output.timeperframe.numerator = 1;
+ a->parm.output.timeperframe.denominator = ctrls->frame_rate;
+ }
+
+ dev_dbg(inst->dev->dev, "%s: numerator : %d | denominator : %d\n",
+ __func__,
+ a->parm.output.timeperframe.numerator,
+ a->parm.output.timeperframe.denominator);
+
+ return 0;
+}
+
+static const struct v4l2_ioctl_ops wave6_vpu_enc_ioctl_ops = {
+ .vidioc_querycap = wave6_vpu_enc_querycap,
+ .vidioc_enum_framesizes = wave6_vpu_enc_enum_framesizes,
+
+ .vidioc_enum_fmt_vid_cap = wave6_vpu_enc_enum_fmt_cap,
+ .vidioc_s_fmt_vid_cap_mplane = wave6_vpu_enc_s_fmt_cap,
+ .vidioc_g_fmt_vid_cap_mplane = wave6_vpu_enc_g_fmt_cap,
+ .vidioc_try_fmt_vid_cap_mplane = wave6_vpu_enc_try_fmt_cap,
+
+ .vidioc_enum_fmt_vid_out = wave6_vpu_enc_enum_fmt_out,
+ .vidioc_s_fmt_vid_out_mplane = wave6_vpu_enc_s_fmt_out,
+ .vidioc_g_fmt_vid_out_mplane = wave6_vpu_enc_g_fmt_out,
+ .vidioc_try_fmt_vid_out_mplane = wave6_vpu_enc_try_fmt_out,
+
+ .vidioc_g_selection = wave6_vpu_enc_g_selection,
+ .vidioc_s_selection = wave6_vpu_enc_s_selection,
+
+ .vidioc_g_parm = wave6_vpu_enc_g_parm,
+ .vidioc_s_parm = wave6_vpu_enc_s_parm,
+
+ .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs,
+ .vidioc_querybuf = v4l2_m2m_ioctl_querybuf,
+ .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
+ .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
+ .vidioc_qbuf = v4l2_m2m_ioctl_qbuf,
+ .vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
+ .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
+ .vidioc_streamon = v4l2_m2m_ioctl_streamon,
+ .vidioc_streamoff = v4l2_m2m_ioctl_streamoff,
+
+ .vidioc_try_encoder_cmd = v4l2_m2m_ioctl_try_encoder_cmd,
+ .vidioc_encoder_cmd = wave6_vpu_enc_encoder_cmd,
+
+ .vidioc_subscribe_event = wave6_vpu_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static int wave6_vpu_enc_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct vpu_instance *inst = wave6_ctrl_to_vpu_inst(ctrl);
+ struct enc_controls *p = &inst->enc_ctrls;
+
+ dev_dbg(inst->dev->dev, "%s: name %s value %d\n",
+ __func__, ctrl->name, ctrl->val);
+
+ switch (ctrl->id) {
+ case V4L2_CID_HFLIP:
+ if (ctrl->val)
+ p->mirror_direction |= MIR_HOR;
+ else
+ p->mirror_direction &= ~MIR_HOR;
+ break;
+ case V4L2_CID_VFLIP:
+ if (ctrl->val)
+ p->mirror_direction |= MIR_VER;
+ else
+ p->mirror_direction &= ~MIR_VER;
+ break;
+ case V4L2_CID_ROTATE:
+ p->rotation_angle = ctrl->val;
+ break;
+ case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
+ break;
+ case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
+ p->gop_size = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MODE:
+ p->slice_mode = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_MB:
+ p->slice_max_mb = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_BITRATE_MODE:
+ p->bitrate_mode = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_BITRATE:
+ p->bitrate = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE:
+ p->frame_rc_enable = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE:
+ p->mb_rc_enable = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:
+ p->force_key_frame = true;
+ break;
+ case V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR:
+ p->prepend_spspps_to_idr = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE:
+ break;
+ case V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD:
+ p->intra_refresh_period = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_FRAME_SKIP_MODE:
+ p->frame_skip_mode = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_HEVC_PROFILE:
+ p->hevc.profile = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_HEVC_LEVEL:
+ p->hevc.level = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_HEVC_MIN_QP:
+ p->hevc.min_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_HEVC_MAX_QP:
+ p->hevc.max_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_QP:
+ p->hevc.i_frame_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_QP:
+ p->hevc.p_frame_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP:
+ p->hevc.b_frame_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE:
+ p->hevc.loop_filter_mode = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_HEVC_LF_BETA_OFFSET_DIV2:
+ p->hevc.lf_beta_offset_div2 = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_HEVC_LF_TC_OFFSET_DIV2:
+ p->hevc.lf_tc_offset_div2 = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_HEVC_REFRESH_TYPE:
+ p->hevc.refresh_type = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_HEVC_REFRESH_PERIOD:
+ p->hevc.refresh_period = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_HEVC_CONST_INTRA_PRED:
+ p->hevc.const_intra_pred = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_HEVC_STRONG_SMOOTHING:
+ p->hevc.strong_smoothing = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_HEVC_TMV_PREDICTION:
+ p->hevc.tmv_prediction = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
+ p->h264.profile = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
+ p->h264.level = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_MIN_QP:
+ p->h264.min_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_MAX_QP:
+ p->h264.max_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP:
+ p->h264.i_frame_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP:
+ p->h264.p_frame_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP:
+ p->h264.b_frame_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_MODE:
+ p->h264.loop_filter_mode = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_BETA:
+ p->h264.loop_filter_beta = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_ALPHA:
+ p->h264.loop_filter_alpha = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_8X8_TRANSFORM:
+ p->h264._8x8_transform = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_CONSTRAINED_INTRA_PREDICTION:
+ p->h264.constrained_intra_prediction = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_CHROMA_QP_INDEX_OFFSET:
+ p->h264.chroma_qp_index_offset = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE:
+ p->h264.entropy_mode = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_I_PERIOD:
+ p->h264.i_period = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE:
+ p->h264.vui_sar_enable = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_IDC:
+ p->h264.vui_sar_idc = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_VUI_EXT_SAR_WIDTH:
+ p->h264.vui_ext_sar_width = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_VUI_EXT_SAR_HEIGHT:
+ p->h264.vui_ext_sar_height = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_CPB_SIZE:
+ p->h264.cpb_size = ctrl->val;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static const struct v4l2_ctrl_ops wave6_vpu_enc_ctrl_ops = {
+ .s_ctrl = wave6_vpu_enc_s_ctrl,
+};
+
+static u32 to_video_full_range_flag(enum v4l2_quantization quantization)
+{
+ switch (quantization) {
+ case V4L2_QUANTIZATION_FULL_RANGE:
+ return 1;
+ case V4L2_QUANTIZATION_LIM_RANGE:
+ default:
+ return 0;
+ }
+}
+
+static u32 to_colour_primaries(enum v4l2_colorspace colorspace)
+{
+ switch (colorspace) {
+ case V4L2_COLORSPACE_SMPTE170M:
+ return 6;
+ case V4L2_COLORSPACE_REC709:
+ case V4L2_COLORSPACE_SRGB:
+ case V4L2_COLORSPACE_JPEG:
+ return 1;
+ case V4L2_COLORSPACE_BT2020:
+ return 9;
+ case V4L2_COLORSPACE_DCI_P3:
+ return 11;
+ case V4L2_COLORSPACE_SMPTE240M:
+ return 7;
+ case V4L2_COLORSPACE_470_SYSTEM_M:
+ return 4;
+ case V4L2_COLORSPACE_470_SYSTEM_BG:
+ return 5;
+ case V4L2_COLORSPACE_RAW:
+ default:
+ return 2;
+ }
+}
+
+static u32 to_transfer_characteristics(enum v4l2_colorspace colorspace,
+ enum v4l2_xfer_func xfer_func)
+{
+ if (xfer_func == V4L2_XFER_FUNC_DEFAULT)
+ xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(colorspace);
+
+ switch (xfer_func) {
+ case V4L2_XFER_FUNC_709:
+ if (colorspace == V4L2_COLORSPACE_SMPTE170M)
+ return 6;
+ else if (colorspace == V4L2_COLORSPACE_BT2020)
+ return 14;
+ else
+ return 1;
+ case V4L2_XFER_FUNC_SRGB:
+ return 13;
+ case V4L2_XFER_FUNC_SMPTE240M:
+ return 7;
+ case V4L2_XFER_FUNC_NONE:
+ return 8;
+ case V4L2_XFER_FUNC_SMPTE2084:
+ return 16;
+ case V4L2_XFER_FUNC_DCI_P3:
+ default:
+ return 2;
+ }
+}
+
+static u32 to_matrix_coeffs(enum v4l2_colorspace colorspace,
+ enum v4l2_ycbcr_encoding ycbcr_enc)
+{
+ if (ycbcr_enc == V4L2_YCBCR_ENC_DEFAULT)
+ ycbcr_enc = V4L2_MAP_YCBCR_ENC_DEFAULT(colorspace);
+
+ switch (ycbcr_enc) {
+ case V4L2_YCBCR_ENC_601:
+ case V4L2_YCBCR_ENC_XV601:
+ if (colorspace == V4L2_COLORSPACE_SMPTE170M)
+ return 6;
+ else
+ return 5;
+ case V4L2_YCBCR_ENC_709:
+ case V4L2_YCBCR_ENC_XV709:
+ return 1;
+ case V4L2_YCBCR_ENC_BT2020:
+ return 9;
+ case V4L2_YCBCR_ENC_BT2020_CONST_LUM:
+ return 10;
+ case V4L2_YCBCR_ENC_SMPTE240M:
+ return 7;
+ default:
+ return 2;
+ }
+}
+
+static void wave6_set_enc_h264_param(struct enc_codec_param *output,
+ struct h264_enc_controls *ctrls)
+{
+ switch (ctrls->profile) {
+ case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
+ case V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE:
+ output->profile = H264_PROFILE_BP;
+ output->internal_bit_depth = 8;
+ break;
+ case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
+ output->profile = H264_PROFILE_MP;
+ output->internal_bit_depth = 8;
+ break;
+ case V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED:
+ output->profile = H264_PROFILE_EXTENDED;
+ output->internal_bit_depth = 8;
+ break;
+ case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
+ output->profile = H264_PROFILE_HP;
+ output->internal_bit_depth = 8;
+ break;
+ default:
+ break;
+ }
+ switch (ctrls->level) {
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
+ output->level = 10;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1B:
+ output->level = 9;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
+ output->level = 11;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
+ output->level = 12;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
+ output->level = 13;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
+ output->level = 20;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
+ output->level = 21;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
+ output->level = 22;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
+ output->level = 30;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
+ output->level = 31;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
+ output->level = 32;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
+ output->level = 40;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
+ output->level = 41;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_4_2:
+ output->level = 42;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_5_0:
+ output->level = 50;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_5_1:
+ output->level = 51;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_5_2:
+ output->level = 52;
+ break;
+ default:
+ break;
+ }
+ output->qp = ctrls->i_frame_qp;
+ output->min_qp_i = ctrls->min_qp;
+ output->max_qp_i = ctrls->max_qp;
+ output->min_qp_p = ctrls->min_qp;
+ output->max_qp_p = ctrls->max_qp;
+ output->min_qp_b = ctrls->min_qp;
+ output->max_qp_b = ctrls->max_qp;
+ switch (ctrls->loop_filter_mode) {
+ case V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_DISABLED:
+ output->en_dbk = 0;
+ output->en_lf_slice_boundary = 0;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_ENABLED:
+ output->en_dbk = 1;
+ output->en_lf_slice_boundary = 1;
+ break;
+ case V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_DISABLED_AT_SLICE_BOUNDARY:
+ output->en_dbk = 1;
+ output->en_lf_slice_boundary = 0;
+ break;
+ default:
+ break;
+ }
+ output->intra_period = ctrls->i_period;
+ output->beta_offset_div2 = ctrls->loop_filter_beta;
+ output->tc_offset_div2 = ctrls->loop_filter_alpha;
+ if (output->profile >= H264_PROFILE_HP)
+ output->en_transform8x8 = ctrls->_8x8_transform;
+ output->en_const_intra_pred = ctrls->constrained_intra_prediction;
+ output->cb_qp_offset = ctrls->chroma_qp_index_offset;
+ output->cr_qp_offset = ctrls->chroma_qp_index_offset;
+ if (output->profile >= H264_PROFILE_MP)
+ output->en_cabac = ctrls->entropy_mode;
+}
+
+static void wave6_set_enc_hevc_param(struct enc_codec_param *output,
+ struct hevc_enc_controls *ctrls)
+{
+ switch (ctrls->profile) {
+ case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN:
+ output->profile = HEVC_PROFILE_MAIN;
+ output->internal_bit_depth = 8;
+ break;
+ case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE:
+ output->profile = HEVC_PROFILE_STILLPICTURE;
+ output->internal_bit_depth = 8;
+ output->en_still_picture = true;
+ break;
+ default:
+ break;
+ }
+ switch (ctrls->level) {
+ case V4L2_MPEG_VIDEO_HEVC_LEVEL_1:
+ output->level = 10 * 3;
+ break;
+ case V4L2_MPEG_VIDEO_HEVC_LEVEL_2:
+ output->level = 20 * 3;
+ break;
+ case V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1:
+ output->level = 21 * 3;
+ break;
+ case V4L2_MPEG_VIDEO_HEVC_LEVEL_3:
+ output->level = 30 * 3;
+ break;
+ case V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1:
+ output->level = 31 * 3;
+ break;
+ case V4L2_MPEG_VIDEO_HEVC_LEVEL_4:
+ output->level = 40 * 3;
+ break;
+ case V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1:
+ output->level = 41 * 3;
+ break;
+ case V4L2_MPEG_VIDEO_HEVC_LEVEL_5:
+ output->level = 50 * 3;
+ break;
+ case V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1:
+ output->level = 51 * 3;
+ break;
+ default:
+ break;
+ }
+ output->qp = ctrls->i_frame_qp;
+ output->min_qp_i = ctrls->min_qp;
+ output->max_qp_i = ctrls->max_qp;
+ output->min_qp_p = ctrls->min_qp;
+ output->max_qp_p = ctrls->max_qp;
+ output->min_qp_b = ctrls->min_qp;
+ output->max_qp_b = ctrls->max_qp;
+ switch (ctrls->loop_filter_mode) {
+ case V4L2_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE_DISABLED:
+ output->en_dbk = 0;
+ output->en_sao = 0;
+ output->en_lf_slice_boundary = 0;
+ break;
+ case V4L2_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE_ENABLED:
+ output->en_dbk = 1;
+ output->en_sao = 1;
+ output->en_lf_slice_boundary = 1;
+ break;
+ case V4L2_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE_DISABLED_AT_SLICE_BOUNDARY:
+ output->en_dbk = 1;
+ output->en_sao = 1;
+ output->en_lf_slice_boundary = 0;
+ break;
+ default:
+ break;
+ }
+ switch (ctrls->refresh_type) {
+ case V4L2_MPEG_VIDEO_HEVC_REFRESH_NONE:
+ output->decoding_refresh_type = DEC_REFRESH_TYPE_NON_IRAP;
+ break;
+ case V4L2_MPEG_VIDEO_HEVC_REFRESH_IDR:
+ output->decoding_refresh_type = DEC_REFRESH_TYPE_IDR;
+ break;
+ default:
+ break;
+ }
+ output->intra_period = ctrls->refresh_period;
+ if (output->idr_period) {
+ output->decoding_refresh_type = DEC_REFRESH_TYPE_IDR;
+ output->intra_period = output->idr_period;
+ output->idr_period = 0;
+ }
+ if (output->profile == HEVC_PROFILE_STILLPICTURE) {
+ output->gop_preset_idx = PRESET_IDX_ALL_I;
+ output->decoding_refresh_type = DEC_REFRESH_TYPE_IDR;
+ output->intra_period = 0;
+ output->idr_period = 0;
+ }
+ output->beta_offset_div2 = ctrls->lf_beta_offset_div2;
+ output->tc_offset_div2 = ctrls->lf_tc_offset_div2;
+ output->en_const_intra_pred = ctrls->const_intra_pred;
+ output->en_intra_smooth = ctrls->strong_smoothing;
+ output->en_temporal_mvp = ctrls->tmv_prediction;
+ output->num_ticks_poc_diff_one = DEFAULT_NUM_TICKS_POC_DIFF;
+ output->intra_4x4 = DEFAULT_INTRA_4X4;
+}
+
+static void wave6_set_enc_open_param(struct enc_open_param *open_param,
+ struct vpu_instance *inst)
+{
+ struct enc_controls *ctrls = &inst->enc_ctrls;
+ struct enc_codec_param *output = &open_param->codec_param;
+ u32 ctu_size = (inst->std == W_AVC_ENC) ? 16 : 64;
+ u32 num_ctu_col = (ALIGN(inst->codec_rect.width, ctu_size) / ctu_size);
+ u32 num_ctu_row = (ALIGN(inst->codec_rect.height, ctu_size) / ctu_size);
+ u64 max_pic_bit = (u64)inst->dst_fmt.plane_fmt[0].sizeimage * 8;
+ const struct vpu_format *vpu_fmt;
+
+ vpu_fmt = wave6_find_vpu_fmt(inst->src_fmt.pixelformat, VPU_FMT_TYPE_RAW);
+ if (!vpu_fmt)
+ return;
+
+ open_param->src_fmt = vpu_fmt->fb_fmt;
+ open_param->source_endian = vpu_fmt->endian;
+ if (vpu_fmt->flags & VPU_FMT_FLAG_SUPPORT_ROT_MIR) {
+ open_param->mir_dir = inst->enc_ctrls.mirror_direction;
+ open_param->rot_angle = inst->enc_ctrls.rotation_angle;
+ }
+ open_param->inst_buffer.temp_base = inst->dev->temp_vbuf.daddr;
+ open_param->inst_buffer.temp_size = inst->dev->temp_vbuf.size;
+ open_param->inst_buffer.ar_base = inst->ar_vbuf.daddr;
+ open_param->pic_width = inst->codec_rect.width;
+ open_param->pic_height = inst->codec_rect.height;
+
+ output->gop_preset_idx = PRESET_IDX_IPP_SINGLE;
+ output->temp_layer_cnt = DEFAULT_TEMP_LAYER_CNT;
+ output->rc_initial_level = DEFAULT_RC_INITIAL_LEVEL;
+ output->pic_rc_max_dqp = DEFAULT_PIC_RC_MAX_DQP;
+ output->rc_initial_qp = DEFAULT_RC_INITIAL_QP;
+ output->qround_inter = DEFAULT_QROUND_INTER;
+ output->qround_intra = DEFAULT_QROUND_INTRA;
+
+ output->frame_rate = ctrls->frame_rate;
+ output->idr_period = ctrls->gop_size;
+ output->rc_mode = ctrls->bitrate_mode;
+ output->rc_update_speed = (ctrls->bitrate_mode) ? DEFAULT_RC_UPDATE_SPEED_CBR :
+ DEFAULT_RC_UPDATE_SPEED_VBR;
+ output->en_rate_control = ctrls->frame_rc_enable;
+ output->en_cu_level_rate_control = ctrls->mb_rc_enable;
+ output->max_intra_pic_bit = min(max_pic_bit, MAX_INTRA_INTER_PIC_BIT);
+ output->max_inter_pic_bit = min(max_pic_bit, MAX_INTRA_INTER_PIC_BIT);
+ output->bitrate = ctrls->bitrate;
+ output->cpb_size = wave6_cpb_size_msec(ctrls->h264.cpb_size, ctrls->bitrate);
+ output->slice_mode = ctrls->slice_mode;
+ output->slice_arg = ctrls->slice_max_mb;
+ if (output->slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) {
+ if (inst->std == W_AVC_ENC)
+ output->slice_arg = clamp(ctrls->slice_max_mb,
+ AVC_MIN_SLICE_MAX_MB,
+ num_ctu_col * num_ctu_row);
+ else
+ output->slice_arg = min(ctrls->slice_max_mb,
+ num_ctu_col * num_ctu_row);
+ }
+ output->forced_idr_header = ctrls->prepend_spspps_to_idr;
+ output->en_skip_frame = (ctrls->frame_skip_mode) ? 1 : 0;
+ if (ctrls->intra_refresh_period) {
+ output->intra_refresh_mode = INTRA_REFRESH_ROW;
+ if (ctrls->intra_refresh_period < num_ctu_row) {
+ output->intra_refresh_arg = (num_ctu_row + ctrls->intra_refresh_period - 1)
+ / ctrls->intra_refresh_period;
+ } else {
+ output->intra_refresh_arg = 1;
+ }
+ }
+ output->sar.enable = ctrls->h264.vui_sar_enable;
+ output->sar.idc = ctrls->h264.vui_sar_idc;
+ if (output->sar.idc == V4L2_MPEG_VIDEO_H264_VUI_SAR_IDC_EXTENDED)
+ output->sar.idc = H264_VUI_SAR_IDC_EXTENDED;
+ output->sar.width = ctrls->h264.vui_ext_sar_width;
+ output->sar.height = ctrls->h264.vui_ext_sar_height;
+ output->color.color_range = to_video_full_range_flag(inst->quantization);
+ output->color.color_primaries = to_colour_primaries(inst->colorspace);
+ output->color.transfer_characteristics = to_transfer_characteristics(inst->colorspace,
+ inst->xfer_func);
+ output->color.matrix_coefficients = to_matrix_coeffs(inst->colorspace, inst->ycbcr_enc);
+ output->conf_win.left = inst->crop.left - inst->codec_rect.left;
+ output->conf_win.top = inst->crop.top - inst->codec_rect.top;
+ output->conf_win.right = inst->codec_rect.width
+ - inst->crop.width - output->conf_win.left;
+ output->conf_win.bottom = inst->codec_rect.height
+ - inst->crop.height - output->conf_win.top;
+
+ switch (inst->std) {
+ case W_AVC_ENC:
+ wave6_set_enc_h264_param(output, &ctrls->h264);
+ break;
+ case W_HEVC_ENC:
+ wave6_set_enc_hevc_param(output, &ctrls->hevc);
+ break;
+ default:
+ break;
+ }
+}
+
+static int wave6_vpu_enc_create_instance(struct vpu_instance *inst)
+{
+ int ret;
+ struct enc_open_param open_param;
+
+ memset(&open_param, 0, sizeof(struct enc_open_param));
+
+ wave6_vpu_core_activate(inst->dev);
+ ret = pm_runtime_resume_and_get(inst->dev->dev);
+ if (ret) {
+ dev_err(inst->dev->dev, "runtime_resume failed %d\n", ret);
+ goto error_deactivate;
+ }
+
+ inst->ar_vbuf.size = ALIGN(W6_ARBUF_SIZE, 4096);
+ ret = wave6_vdi_alloc_dma(inst->dev->dev, &inst->ar_vbuf);
+ if (ret) {
+ dev_err(inst->dev->dev, "alloc ar of size %zu failed\n",
+ inst->ar_vbuf.size);
+ goto error_pm;
+ }
+
+ wave6_set_enc_open_param(&open_param, inst);
+
+ ret = wave6_vpu_enc_open(inst, &open_param);
+ if (ret) {
+ dev_err(inst->dev->dev, "failed create instance : %d\n", ret);
+ goto error_open;
+ }
+
+ dev_dbg(inst->dev->dev, "[%d] encoder\n", inst->id);
+
+ scoped_guard(spinlock, &inst->dev->inst_lock)
+ list_add_tail(&inst->list, &inst->dev->instances);
+ wave6_vpu_enable_instance(inst);
+ wave6_vpu_set_instance_state(inst, VPU_INST_STATE_OPEN);
+
+ return 0;
+
+error_open:
+ wave6_vdi_free_dma(&inst->ar_vbuf);
+error_pm:
+ pm_runtime_put_sync(inst->dev->dev);
+error_deactivate:
+ wave6_vpu_core_deactivate(inst->dev);
+
+ return ret;
+}
+
+static int wave6_vpu_enc_initialize_instance(struct vpu_instance *inst)
+{
+ int ret;
+ struct enc_seq_info seq_info;
+ struct v4l2_ctrl *ctrl;
+
+ ret = wave6_vpu_enc_issue_seq_init(inst);
+ if (ret) {
+ dev_err(inst->dev->dev, "seq init fail %d\n", ret);
+ return ret;
+ }
+
+ ret = wave6_vpu_wait_interrupt(inst, W6_VPU_TIMEOUT);
+ if (ret) {
+ dev_err(inst->dev->dev, "seq init timeout\n");
+ return ret;
+ }
+
+ ret = wave6_vpu_enc_complete_seq_init(inst, &seq_info);
+ if (ret) {
+ dev_err(inst->dev->dev, "seq init error\n");
+ return ret;
+ }
+
+ dev_dbg(inst->dev->dev, "min_fb_cnt : %d | min_src_cnt : %d\n",
+ seq_info.min_frame_buffer_count,
+ seq_info.min_src_frame_count);
+
+ ctrl = v4l2_ctrl_find(&inst->v4l2_ctrl_hdl,
+ V4L2_CID_MIN_BUFFERS_FOR_OUTPUT);
+ if (ctrl)
+ v4l2_ctrl_s_ctrl(ctrl, seq_info.min_src_frame_count);
+
+ wave6_vpu_set_instance_state(inst, VPU_INST_STATE_INIT_SEQ);
+
+ return 0;
+}
+
+static int wave6_vpu_enc_prepare_fb(struct vpu_instance *inst)
+{
+ int ret;
+ unsigned int fb_num;
+ unsigned int mv_num;
+ unsigned int fb_stride, fb_stride_c;
+ unsigned int fb_height;
+ unsigned int luma_size;
+ unsigned int chroma_size;
+ struct enc_seq_info *seq_info = &inst->codec_info->enc_info.seq_info;
+ struct enc_open_param *open_param = &inst->codec_info->enc_info.open_param;
+
+ fb_num = seq_info->min_frame_buffer_count;
+ mv_num = seq_info->req_mv_buffer_count;
+
+ fb_stride = ALIGN(inst->codec_rect.width, W6_FBC_BUF_ALIGNMENT);
+ fb_height = ALIGN(inst->codec_rect.height, W6_FBC_BUF_ALIGNMENT);
+ if (open_param->rot_angle == ROT_90 || open_param->rot_angle == ROT_270) {
+ fb_stride = ALIGN(inst->codec_rect.height, W6_FBC_BUF_ALIGNMENT);
+ fb_height = ALIGN(inst->codec_rect.width, W6_FBC_BUF_ALIGNMENT);
+ }
+ luma_size = fb_stride * fb_height;
+
+ fb_stride_c = ALIGN(fb_stride / 2, W6_FBC_BUF_ALIGNMENT);
+ chroma_size = fb_stride_c * fb_height;
+
+ for (unsigned int i = 0; i < fb_num; i++) {
+ struct frame_buffer *frame = &inst->frame_buf[i];
+ struct vpu_buf *vframe = &inst->frame_vbuf[i];
+
+ vframe->size = luma_size + chroma_size;
+ ret = wave6_vdi_alloc_dma(inst->dev->dev, vframe);
+ if (ret) {
+ dev_err(inst->dev->dev, "alloc FBC buffer fail : %zu\n",
+ vframe->size);
+ goto error;
+ }
+
+ frame->buf_y = vframe->daddr;
+ frame->buf_cb = vframe->daddr + luma_size;
+ frame->buf_cr = (dma_addr_t)-1;
+ frame->stride = fb_stride;
+ frame->stride_c = fb_stride_c;
+ frame->height = fb_height;
+ frame->map_type = COMPRESSED_FRAME_MAP;
+ }
+
+ ret = wave6_allocate_aux_buffer(inst, AUX_BUF_FBC_Y_TBL,
+ fb_stride, fb_height, fb_num);
+ if (ret)
+ goto error;
+
+ ret = wave6_allocate_aux_buffer(inst, AUX_BUF_FBC_C_TBL,
+ fb_stride, fb_height, fb_num);
+ if (ret)
+ goto error;
+
+ ret = wave6_allocate_aux_buffer(inst, AUX_BUF_MV_COL,
+ fb_stride, fb_height, mv_num);
+ if (ret)
+ goto error;
+
+ ret = wave6_allocate_aux_buffer(inst, AUX_BUF_SUB_SAMPLE,
+ fb_stride, fb_height, fb_num);
+ if (ret)
+ goto error;
+
+ ret = wave6_vpu_enc_register_frame_buffer(inst);
+ if (ret) {
+ dev_err(inst->dev->dev, "register frame buffer fail %d\n", ret);
+ goto error;
+ }
+
+ wave6_vpu_set_instance_state(inst, VPU_INST_STATE_PIC_RUN);
+
+ return 0;
+
+error:
+ wave6_vpu_enc_release_fb(inst);
+
+ return ret;
+}
+
+static int wave6_vpu_enc_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
+ unsigned int *num_planes, unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct vpu_instance *inst = vb2_get_drv_priv(q);
+ struct v4l2_pix_format_mplane inst_format =
+ (V4L2_TYPE_IS_OUTPUT(q->type)) ? inst->src_fmt : inst->dst_fmt;
+
+ dev_dbg(inst->dev->dev, "%s: num_buffers %d num_planes %d type %d\n",
+ __func__, *num_buffers, *num_planes, q->type);
+
+ if (*num_planes) {
+ if (inst_format.num_planes != *num_planes)
+ return -EINVAL;
+
+ for (unsigned int i = 0; i < *num_planes; i++) {
+ if (sizes[i] < inst_format.plane_fmt[i].sizeimage)
+ return -EINVAL;
+ }
+ } else {
+ *num_planes = inst_format.num_planes;
+ for (unsigned int i = 0; i < *num_planes; i++) {
+ sizes[i] = inst_format.plane_fmt[i].sizeimage;
+ dev_dbg(inst->dev->dev, "size[%d] : %d\n", i, sizes[i]);
+ }
+
+ if (V4L2_TYPE_IS_OUTPUT(q->type)) {
+ struct v4l2_ctrl *ctrl;
+ unsigned int min_src_frame_count = 0;
+
+ ctrl = v4l2_ctrl_find(&inst->v4l2_ctrl_hdl,
+ V4L2_CID_MIN_BUFFERS_FOR_OUTPUT);
+ if (ctrl)
+ min_src_frame_count = v4l2_ctrl_g_ctrl(ctrl);
+
+ *num_buffers = max(*num_buffers, min_src_frame_count);
+ }
+ }
+
+ return 0;
+}
+
+static int wave6_vpu_enc_buf_prepare(struct vb2_buffer *vb)
+{
+ struct vpu_instance *inst = vb2_get_drv_priv(vb->vb2_queue);
+
+ if (V4L2_TYPE_IS_CAPTURE(vb->type))
+ return 0;
+
+ for (unsigned int i = 0; i < vb->num_planes; i++) {
+ unsigned int offset = vb->planes[i].data_offset;
+ unsigned int size = inst->src_fmt.plane_fmt[i].sizeimage;
+
+ if (offset > vb2_plane_size(vb, i) ||
+ vb2_plane_size(vb, i) - offset < size) {
+ dev_dbg(inst->dev->dev,
+ "plane %u: offset %u size %u length %lu\n",
+ i, offset, size, vb2_plane_size(vb, i));
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static void wave6_vpu_enc_buf_queue(struct vb2_buffer *vb)
+{
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct vpu_instance *inst = vb2_get_drv_priv(vb->vb2_queue);
+ struct vpu_buffer *vpu_buf = wave6_to_vpu_buf(vbuf);
+
+ dev_dbg(inst->dev->dev, "type %4d index %4d size[0] %4ld size[1] : %4ld | size[2] : %4ld\n",
+ vb->type, vb->index, vb2_plane_size(&vbuf->vb2_buf, 0),
+ vb2_plane_size(&vbuf->vb2_buf, 1), vb2_plane_size(&vbuf->vb2_buf, 2));
+
+ vpu_buf->consumed = false;
+ vpu_buf->used = false;
+ if (V4L2_TYPE_IS_OUTPUT(vb->type)) {
+ vbuf->sequence = inst->queued_src_buf_num++;
+
+ vpu_buf->force_key_frame = inst->enc_ctrls.force_key_frame;
+ inst->enc_ctrls.force_key_frame = false;
+ vpu_buf->force_frame_qp = !inst->enc_ctrls.frame_rc_enable;
+ if (vpu_buf->force_frame_qp) {
+ if (inst->std == W_AVC_ENC) {
+ vpu_buf->force_i_frame_qp = inst->enc_ctrls.h264.i_frame_qp;
+ vpu_buf->force_p_frame_qp = inst->enc_ctrls.h264.p_frame_qp;
+ vpu_buf->force_b_frame_qp = inst->enc_ctrls.h264.b_frame_qp;
+ } else if (inst->std == W_HEVC_ENC) {
+ vpu_buf->force_i_frame_qp = inst->enc_ctrls.hevc.i_frame_qp;
+ vpu_buf->force_p_frame_qp = inst->enc_ctrls.hevc.p_frame_qp;
+ vpu_buf->force_b_frame_qp = inst->enc_ctrls.hevc.b_frame_qp;
+ }
+ }
+ } else {
+ inst->queued_dst_buf_num++;
+
+ if (vb2_is_streaming(vb->vb2_queue) &&
+ v4l2_m2m_dst_buf_is_last(inst->v4l2_fh.m2m_ctx)) {
+ vb2_set_plane_payload(vb, 0, 0);
+ vbuf->field = V4L2_FIELD_NONE;
+ v4l2_m2m_last_buffer_done(inst->v4l2_fh.m2m_ctx, vbuf);
+ return;
+ }
+ }
+
+ v4l2_m2m_buf_queue(inst->v4l2_fh.m2m_ctx, vbuf);
+}
+
+static void wave6_vpu_enc_buf_finish(struct vb2_buffer *vb)
+{
+ struct vpu_instance *inst = vb2_get_drv_priv(vb->vb2_queue);
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct vpu_buffer *vpu_buf = wave6_to_vpu_buf(vbuf);
+ struct v4l2_ctrl *ctrl;
+
+ if (V4L2_TYPE_IS_OUTPUT(vb->type))
+ return;
+
+ ctrl = v4l2_ctrl_find(inst->v4l2_fh.ctrl_handler, V4L2_CID_MPEG_VIDEO_AVERAGE_QP);
+ if (ctrl)
+ v4l2_ctrl_s_ctrl(ctrl, vpu_buf->average_qp);
+}
+
+static int wave6_vpu_enc_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+ struct vpu_instance *inst = vb2_get_drv_priv(q);
+ int ret = 0;
+
+ v4l2_m2m_update_start_streaming_state(inst->v4l2_fh.m2m_ctx, q);
+
+ if (inst->state == VPU_INST_STATE_NONE) {
+ ret = wave6_vpu_enc_create_instance(inst);
+ if (ret)
+ goto exit;
+ }
+
+ if (inst->state == VPU_INST_STATE_OPEN) {
+ ret = wave6_vpu_enc_initialize_instance(inst);
+ if (ret) {
+ wave6_vpu_enc_destroy_instance(inst);
+ goto exit;
+ }
+ }
+
+ if (inst->state == VPU_INST_STATE_INIT_SEQ) {
+ ret = wave6_vpu_enc_prepare_fb(inst);
+ if (ret) {
+ wave6_vpu_enc_destroy_instance(inst);
+ goto exit;
+ }
+ }
+
+exit:
+ if (ret)
+ wave6_vpu_return_buffers(inst, q->type, VB2_BUF_STATE_QUEUED);
+
+ return ret;
+}
+
+static void wave6_vpu_enc_stop_streaming(struct vb2_queue *q)
+{
+ struct vpu_instance *inst = vb2_get_drv_priv(q);
+
+ dev_dbg(inst->dev->dev, "[%d] %s, input %d, encode %d\n",
+ inst->id, V4L2_TYPE_IS_OUTPUT(q->type) ? "output" : "capture",
+ inst->queued_src_buf_num, inst->sequence);
+
+ if (inst->state == VPU_INST_STATE_NONE)
+ goto exit;
+
+ if (V4L2_TYPE_IS_OUTPUT(q->type)) {
+ inst->queued_src_buf_num = 0;
+ inst->processed_buf_num = 0;
+ inst->error_buf_num = 0;
+ inst->sequence = 0;
+ } else {
+ inst->queued_dst_buf_num = 0;
+ }
+
+ if (V4L2_TYPE_IS_CAPTURE(q->type) && inst->state != VPU_INST_STATE_NONE)
+ wave6_vpu_enc_destroy_instance(inst);
+
+exit:
+ wave6_vpu_return_buffers(inst, q->type, VB2_BUF_STATE_ERROR);
+ v4l2_m2m_update_stop_streaming_state(inst->v4l2_fh.m2m_ctx, q);
+}
+
+static const struct vb2_ops wave6_vpu_enc_vb2_ops = {
+ .queue_setup = wave6_vpu_enc_queue_setup,
+ .buf_prepare = wave6_vpu_enc_buf_prepare,
+ .buf_queue = wave6_vpu_enc_buf_queue,
+ .buf_finish = wave6_vpu_enc_buf_finish,
+ .start_streaming = wave6_vpu_enc_start_streaming,
+ .stop_streaming = wave6_vpu_enc_stop_streaming,
+};
+
+static void wave6_set_default_format(struct v4l2_pix_format_mplane *src_fmt,
+ struct v4l2_pix_format_mplane *dst_fmt)
+{
+ const struct vpu_format *vpu_fmt;
+
+ vpu_fmt = wave6_find_vpu_fmt_by_idx(0, VPU_FMT_TYPE_RAW);
+ if (vpu_fmt) {
+ src_fmt->pixelformat = vpu_fmt->v4l2_pix_fmt;
+ src_fmt->num_planes = vpu_fmt->num_planes;
+ wave6_update_pix_fmt(src_fmt,
+ W6_DEF_ENC_PIC_WIDTH, W6_DEF_ENC_PIC_HEIGHT);
+ }
+
+ vpu_fmt = wave6_find_vpu_fmt_by_idx(0, VPU_FMT_TYPE_CODEC);
+ if (vpu_fmt) {
+ dst_fmt->pixelformat = vpu_fmt->v4l2_pix_fmt;
+ dst_fmt->num_planes = vpu_fmt->num_planes;
+ wave6_update_pix_fmt(dst_fmt,
+ W6_DEF_ENC_PIC_WIDTH, W6_DEF_ENC_PIC_HEIGHT);
+ }
+}
+
+static int wave6_vpu_enc_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
+{
+ struct vpu_instance *inst = priv;
+ int ret;
+
+ src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+ src_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ src_vq->mem_ops = &vb2_dma_contig_memops;
+ src_vq->ops = &wave6_vpu_enc_vb2_ops;
+ src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ src_vq->buf_struct_size = sizeof(struct vpu_buffer);
+ src_vq->drv_priv = inst;
+ src_vq->lock = &inst->dev->dev_lock;
+ src_vq->dev = inst->dev->v4l2_dev.dev;
+ ret = vb2_queue_init(src_vq);
+ if (ret)
+ return ret;
+
+ dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ dst_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ dst_vq->mem_ops = &vb2_dma_contig_memops;
+ dst_vq->ops = &wave6_vpu_enc_vb2_ops;
+ dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ dst_vq->buf_struct_size = sizeof(struct vpu_buffer);
+ dst_vq->drv_priv = inst;
+ dst_vq->lock = &inst->dev->dev_lock;
+ dst_vq->dev = inst->dev->v4l2_dev.dev;
+ ret = vb2_queue_init(dst_vq);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static const struct vpu_instance_ops wave6_vpu_enc_inst_ops = {
+ .prepare_process = wave6_update_seq_param,
+ .start_process = wave6_vpu_enc_start_encode,
+ .finish_process = wave6_vpu_enc_finish_encode,
+};
+
+static int wave6_vpu_open_enc(struct file *filp)
+{
+ struct video_device *vdev = video_devdata(filp);
+ struct vpu_core_device *core = video_drvdata(filp);
+ struct vpu_instance *inst = NULL;
+ struct v4l2_ctrl_handler *v4l2_ctrl_hdl;
+ int ret;
+
+ inst = kzalloc_obj(*inst);
+ if (!inst)
+ return -ENOMEM;
+ v4l2_ctrl_hdl = &inst->v4l2_ctrl_hdl;
+
+ init_completion(&inst->irq_done);
+ INIT_DELAYED_WORK(&inst->task_timer, wave6_vpu_device_run_timeout);
+ mutex_init(&inst->lock);
+ inst->dev = core;
+ inst->type = VPU_INST_TYPE_ENC;
+ inst->ops = &wave6_vpu_enc_inst_ops;
+
+ v4l2_fh_init(&inst->v4l2_fh, vdev);
+ filp->private_data = &inst->v4l2_fh;
+ v4l2_fh_add(&inst->v4l2_fh, filp);
+
+ inst->v4l2_fh.m2m_ctx =
+ v4l2_m2m_ctx_init(core->m2m_dev, inst, wave6_vpu_enc_queue_init);
+ if (IS_ERR(inst->v4l2_fh.m2m_ctx)) {
+ ret = PTR_ERR(inst->v4l2_fh.m2m_ctx);
+ goto free_inst;
+ }
+
+ v4l2_ctrl_handler_init(v4l2_ctrl_hdl, 50);
+ v4l2_ctrl_new_std_menu(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
+ V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE, 0,
+ V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN);
+ v4l2_ctrl_new_std_menu(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
+ V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1, 0,
+ V4L2_MPEG_VIDEO_HEVC_LEVEL_5);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEVC_MIN_QP,
+ 0, 51, 1, 8);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEVC_MAX_QP,
+ 0, 51, 1, 51);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_QP,
+ 0, 51, 1, 30);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_QP,
+ 0, 51, 1, 30);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP,
+ 0, 51, 1, 30);
+ v4l2_ctrl_new_std_menu(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE,
+ V4L2_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE_DISABLED_AT_SLICE_BOUNDARY, 0,
+ V4L2_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE_ENABLED);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEVC_LF_BETA_OFFSET_DIV2,
+ -6, 6, 1, 0);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEVC_LF_TC_OFFSET_DIV2,
+ -6, 6, 1, 0);
+ v4l2_ctrl_new_std_menu(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEVC_REFRESH_TYPE,
+ V4L2_MPEG_VIDEO_HEVC_REFRESH_IDR,
+ BIT(V4L2_MPEG_VIDEO_HEVC_REFRESH_CRA),
+ V4L2_MPEG_VIDEO_HEVC_REFRESH_IDR);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEVC_REFRESH_PERIOD,
+ 0, 2047, 1, 0);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEVC_CONST_INTRA_PRED,
+ 0, 1, 1, 0);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEVC_STRONG_SMOOTHING,
+ 0, 1, 1, 1);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEVC_TMV_PREDICTION,
+ 0, 1, 1, 1);
+ v4l2_ctrl_new_std_menu(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+ V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, 0,
+ V4L2_MPEG_VIDEO_H264_PROFILE_HIGH);
+ v4l2_ctrl_new_std_menu(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_LEVEL,
+ V4L2_MPEG_VIDEO_H264_LEVEL_5_2, 0,
+ V4L2_MPEG_VIDEO_H264_LEVEL_5_0);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_MIN_QP,
+ 0, 51, 1, 8);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_MAX_QP,
+ 0, 51, 1, 51);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP,
+ 0, 51, 1, 30);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP,
+ 0, 51, 1, 30);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP,
+ 0, 51, 1, 30);
+ v4l2_ctrl_new_std_menu(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_MODE,
+ V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_DISABLED_AT_SLICE_BOUNDARY, 0,
+ V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_ENABLED);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_ALPHA,
+ -6, 6, 1, 0);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_BETA,
+ -6, 6, 1, 0);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_8X8_TRANSFORM,
+ 0, 1, 1, 1);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_CONSTRAINED_INTRA_PREDICTION,
+ 0, 1, 1, 0);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_CHROMA_QP_INDEX_OFFSET,
+ -12, 12, 1, 0);
+ v4l2_ctrl_new_std_menu(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE,
+ V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC, 0,
+ V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_I_PERIOD,
+ 0, 2047, 1, 0);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE, 0, 1, 1, 0);
+ v4l2_ctrl_new_std_menu(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_IDC,
+ V4L2_MPEG_VIDEO_H264_VUI_SAR_IDC_EXTENDED, 0,
+ V4L2_MPEG_VIDEO_H264_VUI_SAR_IDC_UNSPECIFIED);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_VUI_EXT_SAR_WIDTH,
+ 0, 0xFFFF, 1, 0);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_VUI_EXT_SAR_HEIGHT,
+ 0, 0xFFFF, 1, 0);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_HFLIP,
+ 0, 1, 1, 0);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_VFLIP,
+ 0, 1, 1, 0);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_ROTATE,
+ 0, 270, 90, 0);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_CPB_SIZE,
+ 0, 18750000, 1, 0);
+ v4l2_ctrl_new_std_menu(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_BITRATE_MODE,
+ V4L2_MPEG_VIDEO_BITRATE_MODE_CBR, 0,
+ V4L2_MPEG_VIDEO_BITRATE_MODE_CBR);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_BITRATE,
+ 1000, 240000000, 1, 2097152);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE,
+ 0, 1, 1, 1);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE,
+ 0, 1, 1, 1);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_GOP_SIZE,
+ 0, 2047, 1, 30);
+ v4l2_ctrl_new_std_menu(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MODE,
+ V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB, 0,
+ V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_MB,
+ 1, 0x3FFFF, 1, 1);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME,
+ 0, 1, 1, 0);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR,
+ 0, 1, 1, 1);
+ v4l2_ctrl_new_std_menu(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE,
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC,
+ BIT(V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM),
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD,
+ 0, 2160, 1, 0);
+ v4l2_ctrl_new_std_menu(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_FRAME_SKIP_MODE,
+ V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_BUF_LIMIT,
+ BIT(V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_LEVEL_LIMIT),
+ V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_DISABLED);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave6_vpu_enc_ctrl_ops,
+ V4L2_CID_MIN_BUFFERS_FOR_OUTPUT, 1, 32, 1, 1);
+ v4l2_ctrl_new_std(v4l2_ctrl_hdl, NULL,
+ V4L2_CID_MPEG_VIDEO_AVERAGE_QP, 0, 51, 1, 0);
+
+ if (v4l2_ctrl_hdl->error) {
+ ret = -ENODEV;
+ goto err_m2m_release;
+ }
+
+ inst->v4l2_fh.ctrl_handler = v4l2_ctrl_hdl;
+ ret = v4l2_ctrl_handler_setup(v4l2_ctrl_hdl);
+ if (ret)
+ goto err_m2m_release;
+
+ wave6_set_default_format(&inst->src_fmt, &inst->dst_fmt);
+ inst->std = wave6_to_codec_std(inst->type, inst->dst_fmt.pixelformat);
+ wave6_update_crop_info(inst, 0, 0, inst->dst_fmt.width, inst->dst_fmt.height);
+ inst->colorspace = V4L2_COLORSPACE_DEFAULT;
+ inst->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ inst->quantization = V4L2_QUANTIZATION_DEFAULT;
+ inst->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+ inst->enc_ctrls.frame_rate = DEFAULT_FRAME_RATE;
+
+ return 0;
+
+err_m2m_release:
+ v4l2_m2m_ctx_release(inst->v4l2_fh.m2m_ctx);
+ v4l2_ctrl_handler_free(v4l2_ctrl_hdl);
+free_inst:
+ v4l2_fh_del(&inst->v4l2_fh, filp);
+ v4l2_fh_exit(&inst->v4l2_fh);
+ mutex_destroy(&inst->lock);
+ kfree(inst);
+
+ return ret;
+}
+
+static int wave6_vpu_enc_release(struct file *filp)
+{
+ struct vpu_instance *inst = wave6_file_to_vpu_inst(filp);
+
+ v4l2_m2m_ctx_release(inst->v4l2_fh.m2m_ctx);
+
+ if (inst->state != VPU_INST_STATE_NONE)
+ wave6_vpu_enc_destroy_instance(inst);
+
+ v4l2_ctrl_handler_free(&inst->v4l2_ctrl_hdl);
+ v4l2_fh_del(&inst->v4l2_fh, filp);
+ v4l2_fh_exit(&inst->v4l2_fh);
+ mutex_destroy(&inst->lock);
+ kfree(inst);
+
+ return 0;
+}
+
+static const struct v4l2_file_operations wave6_vpu_enc_fops = {
+ .owner = THIS_MODULE,
+ .open = wave6_vpu_open_enc,
+ .release = wave6_vpu_enc_release,
+ .unlocked_ioctl = video_ioctl2,
+ .poll = v4l2_m2m_fop_poll,
+ .mmap = v4l2_m2m_fop_mmap,
+};
+
+int wave6_vpu_enc_register_device(struct vpu_core_device *core)
+{
+ struct video_device *vdev_enc;
+ int ret;
+
+ vdev_enc = video_device_alloc();
+ if (!vdev_enc)
+ return -ENOMEM;
+
+ strscpy(vdev_enc->name, VPU_ENC_DEV_NAME, sizeof(vdev_enc->name));
+ vdev_enc->fops = &wave6_vpu_enc_fops;
+ vdev_enc->ioctl_ops = &wave6_vpu_enc_ioctl_ops;
+ vdev_enc->release = video_device_release;
+ vdev_enc->v4l2_dev = &core->v4l2_dev;
+ vdev_enc->vfl_dir = VFL_DIR_M2M;
+ vdev_enc->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
+ vdev_enc->lock = &core->dev_lock;
+ video_set_drvdata(vdev_enc, core);
+
+ ret = video_register_device(vdev_enc, VFL_TYPE_VIDEO, -1);
+ if (ret) {
+ video_device_release(vdev_enc);
+ return ret;
+ }
+
+ core->video_dev_enc = vdev_enc;
+
+ return 0;
+}
+
+void wave6_vpu_enc_unregister_device(struct vpu_core_device *core)
+{
+ video_unregister_device(core->video_dev_enc);
+}
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
index 75371566de38..cc17f43e5d2c 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
@@ -131,6 +131,12 @@ static bool wave6_vpu_valid_instance_transition(struct vpu_instance *inst,
next == VPU_INST_STATE_INIT_SEQ ||
next == VPU_INST_STATE_SEEK)
return true;
+ } else {
+ /* to NONE: instance destroyed */
+ /* to INIT_SEQ: encoder parameter configured */
+ if (next == VPU_INST_STATE_NONE ||
+ next == VPU_INST_STATE_INIT_SEQ)
+ return true;
}
break;
case VPU_INST_STATE_INIT_SEQ:
@@ -142,6 +148,12 @@ static bool wave6_vpu_valid_instance_transition(struct vpu_instance *inst,
next == VPU_INST_STATE_PIC_RUN ||
next == VPU_INST_STATE_SEEK)
return true;
+ } else {
+ /* to NONE: instance destroyed */
+ /* to PIC_RUN: frame buffers registered */
+ if (next == VPU_INST_STATE_NONE ||
+ next == VPU_INST_STATE_PIC_RUN)
+ return true;
}
break;
case VPU_INST_STATE_PIC_RUN:
@@ -155,6 +167,12 @@ static bool wave6_vpu_valid_instance_transition(struct vpu_instance *inst,
next == VPU_INST_STATE_SEEK ||
next == VPU_INST_STATE_ERROR)
return true;
+ } else {
+ /* to NONE: instance destroyed */
+ /* to ERROR: encode command failed, encode frame failed */
+ if (next == VPU_INST_STATE_NONE ||
+ next == VPU_INST_STATE_ERROR)
+ return true;
}
break;
case VPU_INST_STATE_SEEK:
@@ -177,6 +195,10 @@ static bool wave6_vpu_valid_instance_transition(struct vpu_instance *inst,
if (next == VPU_INST_STATE_NONE ||
next == VPU_INST_STATE_SEEK)
return true;
+ } else {
+ /* to NONE: instance destroyed */
+ if (next == VPU_INST_STATE_NONE)
+ return true;
}
break;
}
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
index d5ecf381276d..57ead590e3f0 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
@@ -784,6 +784,7 @@ struct vpu_irq {
* @instances: List of VPU instances
* @inst_lock: Spinlock protecting instance list
* @video_dev_dec: Video device node for decoder
+ * @video_dev_enc: Video device node for encoder
* @dev_lock: Mutex protecting video_device, src, dst vb2_queue
* @hw_lock: Mutex protecting register access
* @attr: Hardware attributes retrieved after boot
@@ -806,6 +807,7 @@ struct vpu_core_device {
struct list_head instances;
spinlock_t inst_lock; /* Protects instance list */
struct video_device *video_dev_dec;
+ struct video_device *video_dev_enc;
struct mutex dev_lock; /* Protects video_device, src, dst vb2_queue */
struct mutex hw_lock; /* Protects register access */
struct vpu_attr attr;
--
2.41.0.windows.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v8 08/11] media: chips-media: wave6: Add trace and debugfs
2026-11-19 8:59 [PATCH v8 00/11] Add support for Wave6 video codec driver Nas Chung
` (6 preceding siblings ...)
2026-11-19 9:00 ` [PATCH v8 07/11] media: chips-media: wave6: Add Wave6 m2m encoder Nas Chung
@ 2026-11-19 9:00 ` Nas Chung
2026-11-19 9:14 ` sashiko-bot
2026-11-19 9:00 ` [PATCH v8 09/11] media: chips-media: wave6: Add performance profiling Nas Chung
` (2 subsequent siblings)
10 siblings, 1 reply; 23+ messages in thread
From: Nas Chung @ 2026-11-19 9:00 UTC (permalink / raw)
To: mchehab, hverkuil, robh, krzk+dt, conor+dt, Frank.Li, s.hauer
Cc: linux-media, devicetree, linux-kernel, imx, linux-arm-kernel,
festevam, kernel, nicolas.dufresne, ming.qian, marek.vasut,
jackson.lee, lafley.kim, Nas Chung
Add debugfs entries and trace events to provide detailed
debugging information.
Help diagnose issues and improve debugging capabilities for the
Wave6 core driver.
Print a readable reason for the known firmware system errors instead
of only the raw fail reason code.
Tested-by: Ming Qian <ming.qian@oss.nxp.com>
Tested-by: Marek Vasut <marek.vasut@mailbox.org>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
---
.../media/platform/chips-media/wave6/Makefile | 4 +
.../platform/chips-media/wave6/wave6-hw.c | 36 ++-
.../platform/chips-media/wave6/wave6-trace.h | 284 ++++++++++++++++++
.../chips-media/wave6/wave6-vpu-core.c | 14 +
.../chips-media/wave6/wave6-vpu-dbg.c | 148 +++++++++
.../chips-media/wave6/wave6-vpu-dbg.h | 14 +
.../chips-media/wave6/wave6-vpu-dec.c | 25 +-
.../chips-media/wave6/wave6-vpu-enc.c | 27 +-
.../chips-media/wave6/wave6-vpu-v4l2.c | 3 +
.../platform/chips-media/wave6/wave6-vpuapi.h | 5 +
10 files changed, 551 insertions(+), 9 deletions(-)
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-trace.h
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.h
diff --git a/drivers/media/platform/chips-media/wave6/Makefile b/drivers/media/platform/chips-media/wave6/Makefile
index 53ee91b88bed..426af279af6f 100644
--- a/drivers/media/platform/chips-media/wave6/Makefile
+++ b/drivers/media/platform/chips-media/wave6/Makefile
@@ -1,8 +1,12 @@
# SPDX-License-Identifier: GPL-2.0
+# tell define_trace.h where to find the trace header
+CFLAGS_wave6-vpu-core.o := -I$(src)
+
wave6-objs += wave6-vpu.o \
wave6-vpu-core.o \
wave6-vpu-v4l2.o \
+ wave6-vpu-dbg.o \
wave6-vpuapi.o \
wave6-vpu-dec.o \
wave6-vpu-enc.o \
diff --git a/drivers/media/platform/chips-media/wave6/wave6-hw.c b/drivers/media/platform/chips-media/wave6/wave6-hw.c
index 171391f797cb..a53a87745366 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-hw.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-hw.c
@@ -10,6 +10,7 @@
#include "wave6-vpu-core.h"
#include "wave6-hw.h"
#include "wave6-regdefine.h"
+#include "wave6-trace.h"
#define wave6_print_reg_err(core, reason) \
_wave6_print_reg_err((core), (reason), __func__)
@@ -17,6 +18,7 @@
void wave6_vpu_writel(struct vpu_core_device *core, u32 addr, u32 data)
{
wave6_vdi_writel(core->reg_base, addr, data);
+ trace_wave6_vpu_writel(core->dev, addr, data);
}
u32 wave6_vpu_readl(struct vpu_core_device *core, u32 addr)
@@ -24,6 +26,7 @@ u32 wave6_vpu_readl(struct vpu_core_device *core, u32 addr)
u32 data;
data = wave6_vdi_readl(core->reg_base, addr);
+ trace_wave6_vpu_readl(core->dev, addr, data);
return data;
}
@@ -31,6 +34,7 @@ u32 wave6_vpu_readl(struct vpu_core_device *core, u32 addr)
void wave6_vpu_writel_relaxed(struct vpu_core_device *core, u32 addr, u32 data)
{
wave6_vdi_writel_relaxed(core->reg_base, addr, data);
+ trace_wave6_vpu_writel(core->dev, addr, data);
}
u32 wave6_vpu_readl_relaxed(struct vpu_core_device *core, u32 addr)
@@ -38,6 +42,7 @@ u32 wave6_vpu_readl_relaxed(struct vpu_core_device *core, u32 addr)
u32 data;
data = wave6_vdi_readl_relaxed(core->reg_base, addr);
+ trace_wave6_vpu_readl(core->dev, addr, data);
return data;
}
@@ -45,7 +50,34 @@ u32 wave6_vpu_readl_relaxed(struct vpu_core_device *core, u32 addr)
static void _wave6_print_reg_err(struct vpu_core_device *core, u32 fail_reason,
const char *func)
{
- dev_err(core->dev, "%s: failure: 0x%x\n", func, fail_reason);
+ struct device *dev = core->dev;
+
+ switch (fail_reason) {
+ case WAVE6_SYSERR_QUEUEING_FAIL:
+ dev_err(dev, "%s: queueing failure 0x%x\n", func, fail_reason);
+ break;
+ case WAVE6_SYSERR_RESULT_NOT_READY:
+ dev_err(dev, "%s: result not ready 0x%x\n", func, fail_reason);
+ break;
+ case WAVE6_SYSERR_ACCESS_VIOLATION_HW:
+ dev_err(dev, "%s: access violation 0x%x\n", func, fail_reason);
+ break;
+ case WAVE6_SYSERR_WATCHDOG_TIMEOUT:
+ dev_err(dev, "%s: watchdog timeout 0x%x\n", func, fail_reason);
+ break;
+ case WAVE6_SYSERR_BUS_ERROR:
+ dev_err(dev, "%s: bus error 0x%x\n", func, fail_reason);
+ break;
+ case WAVE6_SYSERR_DOUBLE_FAULT:
+ dev_err(dev, "%s: double fault 0x%x\n", func, fail_reason);
+ break;
+ case WAVE6_SYSERR_VPU_STILL_RUNNING:
+ dev_err(dev, "%s: still running 0x%x\n", func, fail_reason);
+ break;
+ default:
+ dev_err(dev, "%s: failure: 0x%x\n", func, fail_reason);
+ break;
+ }
}
static void wave6_dec_set_display_buffer(struct vpu_instance *inst, struct frame_buffer fb)
@@ -218,6 +250,8 @@ static void wave6_send_command(struct vpu_core_device *core, u32 id, u32 std, u3
vpu_write_reg_relaxed(core, W6_VPU_BUSY_STATUS, BUSY_STATUS_SET);
vpu_write_reg_relaxed(core, W6_COMMAND, cmd);
vpu_write_reg(core, W6_VPU_HOST_INT_REQ, HOST_INT_REQ_ON);
+
+ trace_wave6_vpu_send_command(core, id, std, cmd);
}
static int wave6_send_query(struct vpu_core_device *core, u32 id, u32 std,
diff --git a/drivers/media/platform/chips-media/wave6/wave6-trace.h b/drivers/media/platform/chips-media/wave6/wave6-trace.h
new file mode 100644
index 000000000000..430f0847310d
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-trace.h
@@ -0,0 +1,284 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
+/*
+ * Wave6 series multi-standard codec IP - wave6 driver tracer
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM wave6
+
+#if !defined(__WAVE6_TRACE_H__) || defined(TRACE_HEADER_MULTI_READ)
+#define __WAVE6_TRACE_H__
+
+#include <linux/tracepoint.h>
+
+DECLARE_EVENT_CLASS(wave6_vpu_register_access,
+ TP_PROTO(struct device *dev, u32 addr, u32 value),
+ TP_ARGS(dev, addr, value),
+ TP_STRUCT__entry(__string(name, dev_name(dev))
+ __field(u32, addr)
+ __field(u32, value)),
+ TP_fast_assign(__assign_str(name);
+ __entry->addr = addr;
+ __entry->value = value;),
+ TP_printk("%s:0x%03x 0x%08x",
+ __get_str(name), __entry->addr, __entry->value));
+
+DEFINE_EVENT(wave6_vpu_register_access, wave6_vpu_writel,
+ TP_PROTO(struct device *dev, u32 addr, u32 value),
+ TP_ARGS(dev, addr, value));
+DEFINE_EVENT(wave6_vpu_register_access, wave6_vpu_readl,
+ TP_PROTO(struct device *dev, u32 addr, u32 value),
+ TP_ARGS(dev, addr, value));
+
+TRACE_EVENT(wave6_vpu_send_command,
+ TP_PROTO(struct vpu_core_device *core, u32 id, u32 std, u32 cmd),
+ TP_ARGS(core, id, std, cmd),
+ TP_STRUCT__entry(__string(name, dev_name(core->dev))
+ __field(u32, id)
+ __field(u32, std)
+ __field(u32, cmd)),
+ TP_fast_assign(__assign_str(name);
+ __entry->id = id;
+ __entry->std = std;
+ __entry->cmd = cmd;),
+ TP_printk("%s: inst id %d, std 0x%x, cmd 0x%x",
+ __get_str(name), __entry->id,
+ __entry->std, __entry->cmd));
+
+TRACE_EVENT(wave6_vpu_irq,
+ TP_PROTO(struct vpu_core_device *core, u32 irq, u32 idc),
+ TP_ARGS(core, irq, idc),
+ TP_STRUCT__entry(__string(name, dev_name(core->dev))
+ __field(u32, irq)
+ __field(u32, idc)),
+ TP_fast_assign(__assign_str(name);
+ __entry->irq = irq;
+ __entry->idc = idc;),
+ TP_printk("%s: irq 0x%x, idc 0x%x",
+ __get_str(name), __entry->irq, __entry->idc));
+
+TRACE_EVENT(wave6_vpu_set_state,
+ TP_PROTO(struct vpu_instance *inst, u32 state),
+ TP_ARGS(inst, state),
+ TP_STRUCT__entry(__string(name, dev_name(inst->dev->dev))
+ __field(u32, id)
+ __string(cur_state, wave6_vpu_instance_state_name(inst->state))
+ __string(nxt_state, wave6_vpu_instance_state_name(state))),
+ TP_fast_assign(__assign_str(name);
+ __entry->id = inst->id;
+ __assign_str(cur_state);
+ __assign_str(nxt_state);),
+ TP_printk("%s: inst[%d] set state %s -> %s",
+ __get_str(name), __entry->id,
+ __get_str(cur_state), __get_str(nxt_state)));
+
+DECLARE_EVENT_CLASS(wave6_vpu_inst_internal,
+ TP_PROTO(struct vpu_instance *inst, bool is_out),
+ TP_ARGS(inst, is_out),
+ TP_STRUCT__entry(__string(name, dev_name(inst->dev->dev))
+ __field(u32, id)
+ __string(type, is_out ? "output" : "capture")
+ __field(u32, pixelformat)
+ __field(u32, width)
+ __field(u32, height)
+ __field(u32, buf_cnt_src)
+ __field(u32, buf_cnt_dst)
+ __field(u32, processed_cnt)
+ __field(u32, error_cnt)),
+ TP_fast_assign(__assign_str(name);
+ __entry->id = inst->id;
+ __assign_str(type);
+ __entry->pixelformat = is_out ? inst->src_fmt.pixelformat :
+ inst->dst_fmt.pixelformat;
+ __entry->width = is_out ? inst->src_fmt.width :
+ inst->dst_fmt.width;
+ __entry->height = is_out ? inst->src_fmt.height :
+ inst->dst_fmt.height;
+ __entry->buf_cnt_src = inst->queued_src_buf_num;
+ __entry->buf_cnt_dst = inst->queued_dst_buf_num;
+ __entry->processed_cnt = inst->processed_buf_num;
+ __entry->error_cnt = inst->error_buf_num;),
+ TP_printk("%s: inst[%d] %s %c%c%c%c %dx%d, input %d, %d, process %d, error %d",
+ __get_str(name), __entry->id, __get_str(type),
+ __entry->pixelformat,
+ __entry->pixelformat >> 8,
+ __entry->pixelformat >> 16,
+ __entry->pixelformat >> 24,
+ __entry->width, __entry->height,
+ __entry->buf_cnt_src, __entry->buf_cnt_dst,
+ __entry->processed_cnt, __entry->error_cnt));
+
+DEFINE_EVENT(wave6_vpu_inst_internal, wave6_vpu_start_streaming,
+ TP_PROTO(struct vpu_instance *inst, bool is_out),
+ TP_ARGS(inst, is_out));
+
+DEFINE_EVENT(wave6_vpu_inst_internal, wave6_vpu_stop_streaming,
+ TP_PROTO(struct vpu_instance *inst, bool is_out),
+ TP_ARGS(inst, is_out));
+
+TRACE_EVENT(wave6_vpu_dec_pic,
+ TP_PROTO(struct vpu_instance *inst, u32 srcidx, u32 size),
+ TP_ARGS(inst, srcidx, size),
+ TP_STRUCT__entry(__string(name, dev_name(inst->dev->dev))
+ __field(u32, id)
+ __field(u32, srcidx)
+ __field(u32, start)
+ __field(u32, size)),
+ TP_fast_assign(__assign_str(name);
+ __entry->id = inst->id;
+ __entry->srcidx = srcidx;
+ __entry->start = inst->codec_info->dec_info.stream_rd_ptr;
+ __entry->size = size;),
+ TP_printk("%s: inst[%d] src[%2d] %8x, %d",
+ __get_str(name), __entry->id,
+ __entry->srcidx, __entry->start, __entry->size));
+
+TRACE_EVENT(wave6_vpu_source_change,
+ TP_PROTO(struct vpu_instance *inst, struct dec_seq_info *info),
+ TP_ARGS(inst, info),
+ TP_STRUCT__entry(__string(name, dev_name(inst->dev->dev))
+ __field(u32, id)
+ __field(u32, width)
+ __field(u32, height)
+ __field(u32, profile)
+ __field(u32, level)
+ __field(u32, tier)
+ __field(u32, min_fb_cnt)
+ __field(u32, disp_delay)
+ __field(u32, quantization)
+ __field(u32, colorspace)
+ __field(u32, xfer_func)
+ __field(u32, ycbcr_enc)),
+ TP_fast_assign(__assign_str(name);
+ __entry->id = inst->id;
+ __entry->width = info->pic_width;
+ __entry->height = info->pic_height;
+ __entry->profile = info->profile;
+ __entry->level = info->level;
+ __entry->tier = info->tier;
+ __entry->min_fb_cnt = info->min_frame_buffer_count;
+ __entry->disp_delay = info->frame_buf_delay;
+ __entry->quantization = inst->quantization;
+ __entry->colorspace = inst->colorspace;
+ __entry->xfer_func = inst->xfer_func;
+ __entry->ycbcr_enc = inst->ycbcr_enc;),
+ TP_printk("%s: inst[%d] %dx%d profile %d %d %d min_fb %d delay %d color %d %d %d %d",
+ __get_str(name), __entry->id,
+ __entry->width, __entry->height,
+ __entry->profile, __entry->level, __entry->tier,
+ __entry->min_fb_cnt, __entry->disp_delay,
+ __entry->quantization, __entry->colorspace,
+ __entry->xfer_func, __entry->ycbcr_enc));
+
+TRACE_EVENT(wave6_vpu_dec_done,
+ TP_PROTO(struct vpu_instance *inst, struct dec_output_info *info),
+ TP_ARGS(inst, info),
+ TP_STRUCT__entry(__string(name, dev_name(inst->dev->dev))
+ __field(u32, id)
+ __field(u32, dec_flag)
+ __field(u32, dec_poc)
+ __field(u32, disp_flag)
+ __field(u32, disp_cnt)
+ __field(u32, rel_cnt)
+ __field(u32, src_ch)
+ __field(u32, eos)
+ __field(u32, error)
+ __field(u32, warn)),
+ TP_fast_assign(__assign_str(name);
+ __entry->id = inst->id;
+ __entry->dec_flag = info->frame_decoded;
+ __entry->dec_poc = info->decoded_poc;
+ __entry->disp_flag = info->frame_display;
+ __entry->disp_cnt = info->disp_frame_num;
+ __entry->rel_cnt = info->release_disp_frame_num;
+ __entry->src_ch = info->notification_flags & DEC_NOTI_FLAG_SEQ_CHANGE;
+ __entry->eos = info->stream_end;
+ __entry->error = info->error_reason;
+ __entry->warn = info->warn_info;),
+ TP_printk("%s: inst[%d] dec %d %d disp %d(%d) rel %d src_ch %d eos %d error 0x%x 0x%x",
+ __get_str(name), __entry->id,
+ __entry->dec_flag, __entry->dec_poc,
+ __entry->disp_flag, __entry->disp_cnt,
+ __entry->rel_cnt,
+ __entry->src_ch, __entry->eos,
+ __entry->error, __entry->warn));
+
+TRACE_EVENT(wave6_vpu_enc_pic,
+ TP_PROTO(struct vpu_instance *inst, struct enc_param *param),
+ TP_ARGS(inst, param),
+ TP_STRUCT__entry(__string(name, dev_name(inst->dev->dev))
+ __field(u32, id)
+ __field(u32, srcidx)
+ __field(u64, buf_y)
+ __field(u64, buf_cb)
+ __field(u64, buf_cr)
+ __field(u32, stride)
+ __field(u64, buf_strm)
+ __field(u32, size_strm)
+ __field(u32, force_type_enable)
+ __field(u32, force_type)),
+ TP_fast_assign(__assign_str(name);
+ __entry->id = inst->id;
+ __entry->srcidx = param->src_idx;
+ __entry->buf_y = param->source_frame->buf_y;
+ __entry->buf_cb = param->source_frame->buf_cb;
+ __entry->buf_cr = param->source_frame->buf_cr;
+ __entry->stride = param->source_frame->stride;
+ __entry->buf_strm = param->pic_stream_buffer_addr;
+ __entry->size_strm = param->pic_stream_buffer_size;
+ __entry->force_type_enable = param->force_pic;
+ __entry->force_type = param->force_pic_type;),
+ TP_printk("%s: inst[%d] src[%2d] %8llx %8llx %8llx(%d) dst %8llx(%d) force type %d(%d)",
+ __get_str(name), __entry->id, __entry->srcidx,
+ __entry->buf_y, __entry->buf_cb, __entry->buf_cr,
+ __entry->stride, __entry->buf_strm, __entry->size_strm,
+ __entry->force_type_enable, __entry->force_type));
+
+TRACE_EVENT(wave6_vpu_enc_done,
+ TP_PROTO(struct vpu_instance *inst, struct enc_output_info *info),
+ TP_ARGS(inst, info),
+ TP_STRUCT__entry(__string(name, dev_name(inst->dev->dev))
+ __field(u32, id)
+ __field(u32, srcidx)
+ __field(u32, frmidx)
+ __field(u32, size)
+ __field(u32, type)
+ __field(u32, avg_qp)),
+ TP_fast_assign(__assign_str(name);
+ __entry->id = inst->id;
+ __entry->srcidx = info->enc_src_idx;
+ __entry->frmidx = info->recon_frame_index;
+ __entry->size = info->bitstream_size;
+ __entry->type = info->pic_type;
+ __entry->avg_qp = info->avg_ctu_qp;),
+ TP_printk("%s: inst[%d] src %d, frame %d, size %d, type %d, qp %d",
+ __get_str(name), __entry->id,
+ __entry->srcidx, __entry->frmidx,
+ __entry->size, __entry->type, __entry->avg_qp));
+
+TRACE_EVENT(wave6_vpu_s_ctrl,
+ TP_PROTO(struct vpu_instance *inst, struct v4l2_ctrl *ctrl),
+ TP_ARGS(inst, ctrl),
+ TP_STRUCT__entry(__string(name, dev_name(inst->dev->dev))
+ __field(u32, id)
+ __string(ctrl_name, ctrl->name)
+ __field(u32, val)),
+ TP_fast_assign(__assign_str(name);
+ __entry->id = inst->id;
+ __assign_str(ctrl_name);
+ __entry->val = ctrl->val;),
+ TP_printk("%s: inst[%d] %s = %d",
+ __get_str(name), __entry->id,
+ __get_str(ctrl_name), __entry->val));
+
+#endif /* __WAVE6_TRACE_H__ */
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH .
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE wave6-trace
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
index aad4023d66a4..12efa3afebf7 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
@@ -10,6 +10,7 @@
#include <linux/clk.h>
#include <linux/interrupt.h>
#include <linux/pm_runtime.h>
+#include <linux/debugfs.h>
#include <linux/iopoll.h>
#include "wave6-vpu-core.h"
@@ -17,6 +18,11 @@
#include "wave6-vpuconfig.h"
#include "wave6-hw.h"
+#define CREATE_TRACE_POINTS
+#include "wave6-trace.h"
+
+#define WAVE6_VPU_DEBUGFS_DIR "wave6"
+
static irqreturn_t wave6_vpu_core_irq(int irq, void *dev_id)
{
struct vpu_core_device *core = dev_id;
@@ -32,6 +38,8 @@ static irqreturn_t wave6_vpu_core_irq(int irq, void *dev_id)
vpu_write_reg(core, W6_VPU_VINT_REASON_CLEAR, irq_info.status);
vpu_write_reg(core, W6_VPU_VINT_CLEAR, VINT_CLEAR);
+ trace_wave6_vpu_irq(core, irq_info.status, irq_info.inst_idc);
+
if (irq_info.status & BIT(W6_INT_BIT_REQ_WORK_BUF)) {
wave6_vpu_require_work_buffer(core->vpu, core->reg_base);
@@ -285,6 +293,12 @@ static int wave6_vpu_core_probe(struct platform_device *pdev)
if (ret)
return ret;
+ core->debugfs = debugfs_lookup(WAVE6_VPU_DEBUGFS_DIR, NULL);
+ if (!IS_ERR_OR_NULL(core->debugfs))
+ dput(core->debugfs);
+ else
+ core->debugfs = debugfs_create_dir(WAVE6_VPU_DEBUGFS_DIR, NULL);
+
ret = devm_pm_runtime_enable(&pdev->dev);
if (ret)
return ret;
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c
new file mode 100644
index 000000000000..525e047ecebf
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c
@@ -0,0 +1,148 @@
+// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause)
+/*
+ * Wave6 series multi-standard codec IP - debug interface
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#include <linux/types.h>
+#include <linux/debugfs.h>
+#include <linux/math64.h>
+
+#include "wave6-vpu-core.h"
+#include "wave6-vpu-dbg.h"
+
+static int wave6_vpu_dbg_instance(struct seq_file *s, void *data)
+{
+ struct vpu_instance *inst = s->private;
+ struct vb2_queue *vq;
+
+ if (!inst->v4l2_fh.m2m_ctx)
+ return 0;
+
+ seq_printf(s, "[%s]\n",
+ inst->type == VPU_INST_TYPE_DEC ? "Decoder" : "Encoder");
+
+ seq_printf(s, "%s : product 0x%x, fw_ver %d.%d.%d(r%d), hw_ver 0x%x\n",
+ dev_name(inst->dev->dev),
+ inst->dev->attr.product_code,
+ FW_VERSION_MAJOR(inst->dev->attr.fw_version),
+ FW_VERSION_MINOR(inst->dev->attr.fw_version),
+ FW_VERSION_REL(inst->dev->attr.fw_version),
+ inst->dev->attr.fw_revision,
+ inst->dev->attr.hw_version);
+
+ seq_printf(s, "state = %s\n",
+ wave6_vpu_instance_state_name(inst->state));
+
+ vq = v4l2_m2m_get_src_vq(inst->v4l2_fh.m2m_ctx);
+ seq_printf(s, "output (%2d, %2d): fmt = %c%c%c%c %d x %d, %d;\n",
+ vb2_is_streaming(vq),
+ vb2_get_num_buffers(vq),
+ inst->src_fmt.pixelformat,
+ inst->src_fmt.pixelformat >> 8,
+ inst->src_fmt.pixelformat >> 16,
+ inst->src_fmt.pixelformat >> 24,
+ inst->src_fmt.width,
+ inst->src_fmt.height,
+ vq->last_buffer_dequeued);
+
+ vq = v4l2_m2m_get_dst_vq(inst->v4l2_fh.m2m_ctx);
+ seq_printf(s, "capture(%2d, %2d): fmt = %c%c%c%c %d x %d, %d;\n",
+ vb2_is_streaming(vq),
+ vb2_get_num_buffers(vq),
+ inst->dst_fmt.pixelformat,
+ inst->dst_fmt.pixelformat >> 8,
+ inst->dst_fmt.pixelformat >> 16,
+ inst->dst_fmt.pixelformat >> 24,
+ inst->dst_fmt.width,
+ inst->dst_fmt.height,
+ vq->last_buffer_dequeued);
+
+ seq_printf(s, "crop: (%d, %d) %d x %d\n",
+ inst->crop.left,
+ inst->crop.top,
+ inst->crop.width,
+ inst->crop.height);
+
+ if (inst->scaler_info.enable)
+ seq_printf(s, "scale: %d x %d\n",
+ inst->scaler_info.width, inst->scaler_info.height);
+
+ seq_printf(s, "queued src %d, dst %d, process %d, sequence %d, error %d, stopped %d\n",
+ inst->queued_src_buf_num,
+ inst->queued_dst_buf_num,
+ inst->processed_buf_num,
+ inst->sequence,
+ inst->error_buf_num,
+ v4l2_m2m_has_stopped(inst->v4l2_fh.m2m_ctx));
+
+ if (inst->type == VPU_INST_TYPE_DEC) {
+ seq_printf(s, "%s order\n",
+ inst->disp_mode == DISP_MODE_DISP_ORDER ? "display" : "decode");
+ } else {
+ struct enc_info *p_enc_info = &inst->codec_info->enc_info;
+ struct enc_codec_param *param = &p_enc_info->open_param.codec_param;
+
+ seq_printf(s, "profile %d, level %d, tier %d\n",
+ param->profile, param->level, param->tier);
+
+ seq_printf(s, "frame_rate %d, idr_period %d, intra_period %d\n",
+ param->frame_rate, param->idr_period, param->intra_period);
+
+ seq_printf(s, "rc %d, mode %d, bitrate %d\n",
+ param->en_rate_control,
+ param->rc_mode,
+ param->bitrate);
+
+ seq_printf(s, "qp %d, i_qp [%d, %d], p_qp [%d, %d], b_qp [%d, %d]\n",
+ param->qp,
+ param->min_qp_i, param->max_qp_i,
+ param->min_qp_p, param->max_qp_p,
+ param->min_qp_b, param->max_qp_b);
+ }
+
+ return 0;
+}
+
+static int wave6_vpu_dbg_open(struct inode *inode, struct file *filp)
+{
+ return single_open(filp, wave6_vpu_dbg_instance, inode->i_private);
+}
+
+static const struct file_operations wave6_vpu_dbg_fops = {
+ .owner = THIS_MODULE,
+ .open = wave6_vpu_dbg_open,
+ .release = single_release,
+ .read = seq_read,
+ .llseek = seq_lseek,
+};
+
+int wave6_vpu_create_dbgfs_file(struct vpu_instance *inst)
+{
+ char name[64];
+
+ if (WARN_ON(!inst || !inst->dev))
+ return -EINVAL;
+
+ if (IS_ERR_OR_NULL(inst->dev->debugfs))
+ return 0;
+
+ scnprintf(name, sizeof(name), "instance.%d", inst->id);
+ inst->debugfs = debugfs_create_file((const char *)name,
+ 0444,
+ inst->dev->debugfs,
+ inst,
+ &wave6_vpu_dbg_fops);
+
+ return 0;
+}
+
+void wave6_vpu_remove_dbgfs_file(struct vpu_instance *inst)
+{
+ if (!inst || !inst->debugfs)
+ return;
+
+ debugfs_remove(inst->debugfs);
+ inst->debugfs = NULL;
+}
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.h b/drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.h
new file mode 100644
index 000000000000..53a0d8a2f5f1
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
+/*
+ * Wave6 series multi-standard codec IP - debug interface
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ */
+
+#ifndef __WAVE6_VPU_DBG_H__
+#define __WAVE6_VPU_DBG_H__
+
+int wave6_vpu_create_dbgfs_file(struct vpu_instance *inst);
+void wave6_vpu_remove_dbgfs_file(struct vpu_instance *inst);
+
+#endif /* __WAVE6_VPU_DBG_H__ */
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
index 54b408922bbc..7eab283ec9e7 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
@@ -8,6 +8,8 @@
#include <linux/pm_runtime.h>
#include "wave6-vpu-core.h"
+#include "wave6-vpu-dbg.h"
+#include "wave6-trace.h"
#define VPU_DEC_DEV_NAME "C&M Wave6 VPU decoder"
#define VPU_DEC_DRV_NAME "wave6-dec"
@@ -135,13 +137,16 @@ static void wave6_vpu_dec_release_fb(struct vpu_instance *inst)
}
}
-static void wave6_vpu_dec_destroy_instance(struct vpu_instance *inst)
+static void wave6_vpu_dec_destroy_instance(struct vpu_instance *inst, bool remove_dbgfs)
{
u32 fail_res = 0;
int ret;
wave6_vpu_disable_instance(inst);
+ if (remove_dbgfs)
+ wave6_vpu_remove_dbgfs_file(inst);
+
ret = wave6_vpu_dec_close(inst, &fail_res);
if (ret) {
dev_err(inst->dev->dev, "failed destroy instance: %d (%d)\n",
@@ -190,6 +195,8 @@ static void wave6_handle_bitstream_buffer(struct vpu_instance *inst,
wave6_vpu_dec_set_rd_ptr(inst, wr, true);
}
+ trace_wave6_vpu_dec_pic(inst, src_buf ? src_buf->vb2_buf.index : -1, src_size);
+
ret = wave6_vpu_dec_update_bitstream_buffer(inst, src_size);
if (ret) {
dev_dbg(inst->dev->dev, "%s: Update bitstream buffer fail %d\n",
@@ -769,6 +776,8 @@ static void wave6_vpu_dec_handle_source_change(struct vpu_instance *inst)
info->pic_width, info->pic_height,
true);
+ trace_wave6_vpu_source_change(inst, info);
+
v4l2_event_queue_fh(&inst->v4l2_fh, &vpu_event_src_ch);
}
@@ -795,6 +804,8 @@ static void wave6_vpu_dec_finish_decode(struct vpu_instance *inst, bool error)
if (ret)
goto finish_decode;
+ trace_wave6_vpu_dec_done(inst, &info);
+
dev_dbg(inst->dev->dev, "dec %d dis %d noti_flag %d stream_end %d\n",
info.frame_decoded, info.frame_display,
info.notification_flags, info.stream_end);
@@ -1353,6 +1364,8 @@ static int wave6_vpu_dec_s_ctrl(struct v4l2_ctrl *ctrl)
{
struct vpu_instance *inst = wave6_ctrl_to_vpu_inst(ctrl);
+ trace_wave6_vpu_s_ctrl(inst, ctrl);
+
dev_dbg(inst->dev->dev, "%s: name %s value %d\n",
__func__, ctrl->name, ctrl->val);
@@ -1419,6 +1432,7 @@ static int wave6_vpu_dec_create_instance(struct vpu_instance *inst)
scoped_guard(spinlock, &inst->dev->inst_lock)
list_add_tail(&inst->list, &inst->dev->instances);
wave6_vpu_enable_instance(inst);
+ wave6_vpu_create_dbgfs_file(inst);
wave6_vpu_set_instance_state(inst, VPU_INST_STATE_OPEN);
return 0;
@@ -1471,7 +1485,7 @@ static int wave6_vpu_dec_queue_setup(struct vb2_queue *q, unsigned int *num_buff
}
if (V4L2_TYPE_IS_OUTPUT(q->type) && inst->state == VPU_INST_STATE_SEEK)
- wave6_vpu_dec_destroy_instance(inst);
+ wave6_vpu_dec_destroy_instance(inst, true);
return 0;
}
@@ -1604,6 +1618,8 @@ static int wave6_vpu_dec_start_streaming(struct vb2_queue *q, unsigned int count
struct v4l2_pix_format_mplane *fmt;
int ret = 0;
+ trace_wave6_vpu_start_streaming(inst, V4L2_TYPE_IS_OUTPUT(q->type));
+
if (V4L2_TYPE_IS_OUTPUT(q->type)) {
fmt = &inst->src_fmt;
if (inst->state == VPU_INST_STATE_NONE) {
@@ -1644,6 +1660,8 @@ static void wave6_vpu_dec_stop_streaming(struct vb2_queue *q)
struct vpu_instance *inst = vb2_get_drv_priv(q);
struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+ trace_wave6_vpu_stop_streaming(inst, V4L2_TYPE_IS_OUTPUT(q->type));
+
dev_dbg(inst->dev->dev, "[%d] %s, input %d, decode %d error %d\n",
inst->id, V4L2_TYPE_IS_OUTPUT(q->type) ? "output" : "capture",
inst->queued_src_buf_num, inst->processed_buf_num, inst->error_buf_num);
@@ -1876,10 +1894,11 @@ static int wave6_vpu_dec_release(struct file *filp)
{
struct vpu_instance *inst = wave6_file_to_vpu_inst(filp);
+ wave6_vpu_remove_dbgfs_file(inst);
v4l2_m2m_ctx_release(inst->v4l2_fh.m2m_ctx);
if (inst->state != VPU_INST_STATE_NONE)
- wave6_vpu_dec_destroy_instance(inst);
+ wave6_vpu_dec_destroy_instance(inst, false);
v4l2_ctrl_handler_free(&inst->v4l2_ctrl_hdl);
v4l2_fh_del(&inst->v4l2_fh, filp);
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
index 7142666d4503..4a0739033d1a 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
@@ -9,6 +9,8 @@
#include <linux/pm_runtime.h>
#include "wave6-vpu-core.h"
+#include "wave6-vpu-dbg.h"
+#include "wave6-trace.h"
#define VPU_ENC_DEV_NAME "C&M Wave6 VPU encoder"
#define VPU_ENC_DRV_NAME "wave6-enc"
@@ -366,13 +368,16 @@ static void wave6_vpu_enc_release_fb(struct vpu_instance *inst)
}
}
-static void wave6_vpu_enc_destroy_instance(struct vpu_instance *inst)
+static void wave6_vpu_enc_destroy_instance(struct vpu_instance *inst, bool remove_dbgfs)
{
u32 fail_res = 0;
int ret;
wave6_vpu_disable_instance(inst);
+ if (remove_dbgfs)
+ wave6_vpu_remove_dbgfs_file(inst);
+
ret = wave6_vpu_enc_close(inst, &fail_res);
if (ret) {
dev_err(inst->dev->dev, "failed destroy instance: %d (%d)\n",
@@ -817,6 +822,8 @@ static int wave6_vpu_enc_start_encode(struct vpu_instance *inst)
}
wave6_set_csc(inst, &pic_param);
+ trace_wave6_vpu_enc_pic(inst, &pic_param);
+
ret = wave6_vpu_enc_start_one_frame(inst, &pic_param, &fail_res);
if (ret) {
dev_err(inst->dev->dev, "[%d] %s: fail %d\n", inst->id, __func__, ret);
@@ -931,6 +938,8 @@ static void wave6_vpu_enc_finish_encode(struct vpu_instance *inst, bool error)
goto finish_encode;
}
+ trace_wave6_vpu_enc_done(inst, &info);
+
if (info.enc_src_idx >= 0 && info.recon_frame_index >= 0)
wave6_handle_encoded_frame(inst, &info);
@@ -1426,6 +1435,8 @@ static int wave6_vpu_enc_s_ctrl(struct v4l2_ctrl *ctrl)
struct vpu_instance *inst = wave6_ctrl_to_vpu_inst(ctrl);
struct enc_controls *p = &inst->enc_ctrls;
+ trace_wave6_vpu_s_ctrl(inst, ctrl);
+
dev_dbg(inst->dev->dev, "%s: name %s value %d\n",
__func__, ctrl->name, ctrl->val);
@@ -2040,6 +2051,7 @@ static int wave6_vpu_enc_create_instance(struct vpu_instance *inst)
scoped_guard(spinlock, &inst->dev->inst_lock)
list_add_tail(&inst->list, &inst->dev->instances);
wave6_vpu_enable_instance(inst);
+ wave6_vpu_create_dbgfs_file(inst);
wave6_vpu_set_instance_state(inst, VPU_INST_STATE_OPEN);
return 0;
@@ -2304,6 +2316,8 @@ static int wave6_vpu_enc_start_streaming(struct vb2_queue *q, unsigned int count
struct vpu_instance *inst = vb2_get_drv_priv(q);
int ret = 0;
+ trace_wave6_vpu_start_streaming(inst, V4L2_TYPE_IS_OUTPUT(q->type));
+
v4l2_m2m_update_start_streaming_state(inst->v4l2_fh.m2m_ctx, q);
if (inst->state == VPU_INST_STATE_NONE) {
@@ -2315,7 +2329,7 @@ static int wave6_vpu_enc_start_streaming(struct vb2_queue *q, unsigned int count
if (inst->state == VPU_INST_STATE_OPEN) {
ret = wave6_vpu_enc_initialize_instance(inst);
if (ret) {
- wave6_vpu_enc_destroy_instance(inst);
+ wave6_vpu_enc_destroy_instance(inst, true);
goto exit;
}
}
@@ -2323,7 +2337,7 @@ static int wave6_vpu_enc_start_streaming(struct vb2_queue *q, unsigned int count
if (inst->state == VPU_INST_STATE_INIT_SEQ) {
ret = wave6_vpu_enc_prepare_fb(inst);
if (ret) {
- wave6_vpu_enc_destroy_instance(inst);
+ wave6_vpu_enc_destroy_instance(inst, true);
goto exit;
}
}
@@ -2339,6 +2353,8 @@ static void wave6_vpu_enc_stop_streaming(struct vb2_queue *q)
{
struct vpu_instance *inst = vb2_get_drv_priv(q);
+ trace_wave6_vpu_stop_streaming(inst, V4L2_TYPE_IS_OUTPUT(q->type));
+
dev_dbg(inst->dev->dev, "[%d] %s, input %d, encode %d\n",
inst->id, V4L2_TYPE_IS_OUTPUT(q->type) ? "output" : "capture",
inst->queued_src_buf_num, inst->sequence);
@@ -2356,7 +2372,7 @@ static void wave6_vpu_enc_stop_streaming(struct vb2_queue *q)
}
if (V4L2_TYPE_IS_CAPTURE(q->type) && inst->state != VPU_INST_STATE_NONE)
- wave6_vpu_enc_destroy_instance(inst);
+ wave6_vpu_enc_destroy_instance(inst, true);
exit:
wave6_vpu_return_buffers(inst, q->type, VB2_BUF_STATE_ERROR);
@@ -2673,10 +2689,11 @@ static int wave6_vpu_enc_release(struct file *filp)
{
struct vpu_instance *inst = wave6_file_to_vpu_inst(filp);
+ wave6_vpu_remove_dbgfs_file(inst);
v4l2_m2m_ctx_release(inst->v4l2_fh.m2m_ctx);
if (inst->state != VPU_INST_STATE_NONE)
- wave6_vpu_enc_destroy_instance(inst);
+ wave6_vpu_enc_destroy_instance(inst, false);
v4l2_ctrl_handler_free(&inst->v4l2_ctrl_hdl);
v4l2_fh_del(&inst->v4l2_fh, filp);
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
index cc17f43e5d2c..6980b126dee5 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
@@ -7,6 +7,7 @@
#include "wave6-vpu.h"
#include "wave6-vpu-core.h"
+#include "wave6-trace.h"
void wave6_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,
unsigned int width,
@@ -217,6 +218,8 @@ void wave6_vpu_set_instance_state(struct vpu_instance *inst,
if (!wave6_vpu_valid_instance_transition(inst, state))
return;
+ trace_wave6_vpu_set_state(inst, state);
+
dev_dbg(inst->dev->dev, "[%d] %s -> %s\n",
inst->id,
wave6_vpu_instance_state_name(inst->state),
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
index 57ead590e3f0..bc3d7e192904 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
@@ -796,6 +796,7 @@ struct vpu_irq {
* @irq_buffer: Buffer used by @irq_fifo
* @irq: Interrupt number
* @res: Device compatible data
+ * @debugfs: Debugfs entry
* @active_count: Number of decoder/encoder instances using the core
* @vpu_held: Core holds a reference taken by wave6_vpu_get()
*/
@@ -825,6 +826,7 @@ struct vpu_core_device {
int irq;
const struct wave6_vpu_core_resource *res;
+ struct dentry *debugfs;
atomic_t active_count;
bool vpu_held;
};
@@ -881,6 +883,7 @@ struct vpu_instance_ops {
* @error_recovery: Error recovery flag
* @disp_mode: Display mode for decoder V4L2 control
* @enc_ctrls: Encoder V4L2 control and streaming parameters
+ * @debugfs: Debugfs entry
*/
struct vpu_instance {
struct list_head list;
@@ -939,6 +942,8 @@ struct vpu_instance {
enum display_mode disp_mode;
struct enc_controls enc_ctrls;
+
+ struct dentry *debugfs;
};
void wave6_vpu_writel(struct vpu_core_device *core, u32 addr, u32 data);
--
2.41.0.windows.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v8 09/11] media: chips-media: wave6: Add performance profiling
2026-11-19 8:59 [PATCH v8 00/11] Add support for Wave6 video codec driver Nas Chung
` (7 preceding siblings ...)
2026-11-19 9:00 ` [PATCH v8 08/11] media: chips-media: wave6: Add trace and debugfs Nas Chung
@ 2026-11-19 9:00 ` Nas Chung
2026-11-19 9:20 ` sashiko-bot
2026-11-19 9:00 ` [PATCH v8 10/11] media: chips-media: wave6: Add Wave6 thermal cooling device Nas Chung
2026-11-19 9:00 ` [PATCH v8 11/11] arm64: dts: freescale: imx95: Add video codec node Nas Chung
10 siblings, 1 reply; 23+ messages in thread
From: Nas Chung @ 2026-11-19 9:00 UTC (permalink / raw)
To: mchehab, hverkuil, robh, krzk+dt, conor+dt, Frank.Li, s.hauer
Cc: linux-media, devicetree, linux-kernel, imx, linux-arm-kernel,
festevam, kernel, nicolas.dufresne, ming.qian, marek.vasut,
jackson.lee, lafley.kim, Nas Chung
Add per-instance performance profiling to help identify whether a
throughput problem comes from the firmware or from the host side.
Record per-frame timestamps and the firmware-reported processing time,
and report the frame rates and latencies in the per-instance debugfs
file.
Tested-by: Ming Qian <ming.qian@oss.nxp.com>
Tested-by: Marek Vasut <marek.vasut@mailbox.org>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
---
.../platform/chips-media/wave6/wave6-hw.c | 28 ++++++
.../platform/chips-media/wave6/wave6-hw.h | 2 +
.../chips-media/wave6/wave6-regdefine.h | 6 ++
.../chips-media/wave6/wave6-vpu-core.h | 10 ++
.../chips-media/wave6/wave6-vpu-dbg.c | 36 ++++++++
.../chips-media/wave6/wave6-vpu-dec.c | 15 +++
.../chips-media/wave6/wave6-vpu-enc.c | 13 +++
.../chips-media/wave6/wave6-vpu-v4l2.c | 91 +++++++++++++++++++
.../platform/chips-media/wave6/wave6-vpuapi.h | 28 ++++++
9 files changed, 229 insertions(+)
diff --git a/drivers/media/platform/chips-media/wave6/wave6-hw.c b/drivers/media/platform/chips-media/wave6/wave6-hw.c
index a53a87745366..8d590d8e74e4 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-hw.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-hw.c
@@ -872,6 +872,20 @@ int wave6_vpu_dec_get_output_info(struct vpu_instance *inst, struct dec_output_i
info->timestamp = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_TIMESTAMP_HIGH);
info->timestamp = (info->timestamp << 32) | reg_val;
+ info->cycle.host_cmd_s = vpu_read_reg_relaxed(inst->dev, W6_RET_CQ_IN_TICK);
+ info->cycle.host_cmd_e = vpu_read_reg_relaxed(inst->dev, W6_RET_RQ_OUT_TICK);
+ info->cycle.proc_s = vpu_read_reg_relaxed(inst->dev, W6_RET_HW_RUN_TICK);
+ info->cycle.proc_e = vpu_read_reg_relaxed(inst->dev, W6_RET_HW_DONE_TICK);
+ info->cycle.vpu_s = vpu_read_reg_relaxed(inst->dev, W6_RET_FW_RUN_TICK);
+ info->cycle.vpu_e = vpu_read_reg_relaxed(inst->dev, W6_RET_FW_DONE_TICK);
+ info->cycle.frame_cycle = (info->cycle.vpu_e - info->cycle.host_cmd_s) *
+ CYCLE_PER_TICK_VALUE;
+ info->cycle.proc_cycle = (info->cycle.proc_e - info->cycle.proc_s) *
+ CYCLE_PER_TICK_VALUE;
+ info->cycle.vpu_cycle = ((info->cycle.vpu_e - info->cycle.vpu_s) -
+ (info->cycle.proc_e - info->cycle.proc_s)) *
+ CYCLE_PER_TICK_VALUE;
+
if (decoded_idx >= 0 && decoded_idx < WAVE6_MAX_FBS)
p_dec_info->dec_out_info[decoded_idx].decoded_poc = info->decoded_poc;
@@ -1882,6 +1896,20 @@ int wave6_vpu_enc_get_output_info(struct vpu_instance *inst, struct enc_output_i
info->timestamp = vpu_read_reg_relaxed(inst->dev, W6_RET_ENC_TIMESTAMP_HIGH);
info->timestamp = (info->timestamp << 32) | reg_val;
+ info->cycle.host_cmd_s = vpu_read_reg_relaxed(inst->dev, W6_RET_CQ_IN_TICK);
+ info->cycle.host_cmd_e = vpu_read_reg_relaxed(inst->dev, W6_RET_RQ_OUT_TICK);
+ info->cycle.proc_s = vpu_read_reg_relaxed(inst->dev, W6_RET_HW_RUN_TICK);
+ info->cycle.proc_e = vpu_read_reg_relaxed(inst->dev, W6_RET_HW_DONE_TICK);
+ info->cycle.vpu_s = vpu_read_reg_relaxed(inst->dev, W6_RET_FW_RUN_TICK);
+ info->cycle.vpu_e = vpu_read_reg_relaxed(inst->dev, W6_RET_FW_DONE_TICK);
+ info->cycle.frame_cycle = (info->cycle.vpu_e - info->cycle.host_cmd_s) *
+ CYCLE_PER_TICK_VALUE;
+ info->cycle.proc_cycle = (info->cycle.proc_e - info->cycle.proc_s) *
+ CYCLE_PER_TICK_VALUE;
+ info->cycle.vpu_cycle = ((info->cycle.vpu_e - info->cycle.vpu_s) -
+ (info->cycle.proc_e - info->cycle.proc_s)) *
+ CYCLE_PER_TICK_VALUE;
+
return 0;
}
diff --git a/drivers/media/platform/chips-media/wave6/wave6-hw.h b/drivers/media/platform/chips-media/wave6/wave6-hw.h
index 7adb0a07390e..fe2a09ec4c0f 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-hw.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-hw.h
@@ -34,6 +34,8 @@ enum product_id {
#define MAX_CSC_COEFF_NUM 4
+#define CYCLE_PER_TICK_VALUE 256
+
#define H265_NAL_UNIT_TYPE_IDR_W_RADL 19
#define H265_NAL_UNIT_TYPE_IDR_N_LP 20
diff --git a/drivers/media/platform/chips-media/wave6/wave6-regdefine.h b/drivers/media/platform/chips-media/wave6/wave6-regdefine.h
index 1038b87b2ff9..55395d3fa4a4 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-regdefine.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-regdefine.h
@@ -110,6 +110,12 @@ enum wave6_interrupt_bit {
#define W6_RET_INT_INSTANCE_INFO (W6_REG_BASE + 0x21C)
#define INT_INSTANCE_INFO_CLEAR 0
#define W6_RET_INSTANCE_ID (W6_REG_BASE + 0x220)
+#define W6_RET_CQ_IN_TICK (W6_REG_BASE + 0x23C)
+#define W6_RET_FW_RUN_TICK (W6_REG_BASE + 0x240)
+#define W6_RET_HW_RUN_TICK (W6_REG_BASE + 0x244)
+#define W6_RET_HW_DONE_TICK (W6_REG_BASE + 0x248)
+#define W6_RET_FW_DONE_TICK (W6_REG_BASE + 0x24C)
+#define W6_RET_RQ_OUT_TICK (W6_REG_BASE + 0x250)
/* COMMON - QUERY : GET_VPU_INFO */
#define W6_RET_FW_VERSION (W6_REG_BASE + 0x300)
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.h b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.h
index dc0b9867ca37..1c352016a688 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.h
@@ -32,6 +32,11 @@ struct vpu_buffer {
u32 force_i_frame_qp;
u32 force_p_frame_qp;
u32 force_b_frame_qp;
+ ktime_t ts_input;
+ ktime_t ts_start;
+ ktime_t ts_finish;
+ ktime_t ts_output;
+ u64 hw_time;
u32 average_qp;
};
@@ -101,6 +106,7 @@ enum codec_std wave6_to_codec_std(enum vpu_instance_type type, unsigned int v4l2
const char *wave6_vpu_instance_state_name(enum vpu_instance_state state);
void wave6_vpu_set_instance_state(struct vpu_instance *inst,
enum vpu_instance_state state);
+u64 wave6_vpu_cycle_to_ns(struct vpu_core_device *core, u64 cycle);
int wave6_vpu_wait_interrupt(struct vpu_instance *inst, unsigned int timeout);
int wave6_vpu_dec_register_device(struct vpu_core_device *core);
void wave6_vpu_dec_unregister_device(struct vpu_core_device *core);
@@ -108,6 +114,10 @@ int wave6_vpu_enc_register_device(struct vpu_core_device *core);
void wave6_vpu_enc_unregister_device(struct vpu_core_device *core);
void wave6_vpu_device_run_timeout(struct work_struct *work);
void wave6_vpu_finish_job(struct vpu_instance *inst);
+void wave6_vpu_record_performance_timestamps(struct vpu_instance *inst);
+void wave6_vpu_handle_performance(struct vpu_instance *inst,
+ struct vpu_buffer *vpu_buf);
+void wave6_vpu_reset_performance(struct vpu_instance *inst);
int wave6_vpu_init_m2m_dev(struct vpu_core_device *core);
void wave6_vpu_release_m2m_dev(struct vpu_core_device *core);
int wave6_vpu_subscribe_event(struct v4l2_fh *fh,
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c
index 525e047ecebf..be1061b2b39f 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c
@@ -15,7 +15,13 @@
static int wave6_vpu_dbg_instance(struct seq_file *s, void *data)
{
struct vpu_instance *inst = s->private;
+ struct vpu_performance_info *perf = &inst->performance;
struct vb2_queue *vq;
+ s64 tmp;
+ s64 fps;
+ s64 duration;
+ s64 ms[3];
+ s32 ns[3];
if (!inst->v4l2_fh.m2m_ctx)
return 0;
@@ -77,6 +83,36 @@ static int wave6_vpu_dbg_instance(struct seq_file *s, void *data)
inst->error_buf_num,
v4l2_m2m_has_stopped(inst->v4l2_fh.m2m_ctx));
+ seq_puts(s, "fps");
+ tmp = MSEC_PER_SEC * inst->processed_buf_num;
+ if (perf->ts_last > perf->ts_first + NSEC_PER_MSEC) {
+ duration = ktime_ms_delta(perf->ts_last, perf->ts_first);
+ fps = DIV64_U64_ROUND_CLOSEST(tmp, duration);
+ seq_printf(s, " actual: %lld;", fps);
+ }
+ duration = div_u64(perf->total_sw_time, NSEC_PER_MSEC);
+ if (duration > 0) {
+ fps = DIV64_U64_ROUND_CLOSEST(tmp, duration);
+ seq_printf(s, " sw: %lld;", fps);
+ }
+ duration = div_u64(perf->total_hw_time, NSEC_PER_MSEC);
+ if (duration > 0) {
+ fps = DIV64_U64_ROUND_CLOSEST(tmp, duration);
+ seq_printf(s, " hw: %lld", fps);
+ }
+ seq_putc(s, '\n');
+
+ ms[0] = div_s64_rem(perf->latency_first, NSEC_PER_MSEC, &ns[0]);
+ ms[1] = div_s64_rem(perf->latency_max, NSEC_PER_MSEC, &ns[1]);
+ ms[2] = div_s64_rem(ktime_sub(perf->ts_first, perf->ts_start), NSEC_PER_MSEC, &ns[2]);
+ seq_printf(s, "latency(ms) first: %lld.%06d, max %lld.%06d, setup %lld.%06d\n",
+ ms[0], ns[0], ms[1], ns[1], ms[2], ns[2]);
+
+ ms[0] = div_s64_rem(perf->min_process_time, NSEC_PER_MSEC, &ns[0]);
+ ms[1] = div_s64_rem(perf->max_process_time, NSEC_PER_MSEC, &ns[1]);
+ seq_printf(s, "process frame time(ms) min: %lld.%06d, max %lld.%06d\n",
+ ms[0], ns[0], ms[1], ns[1]);
+
if (inst->type == VPU_INST_TYPE_DEC) {
seq_printf(s, "%s order\n",
inst->disp_mode == DISP_MODE_DISP_ORDER ? "display" : "decode");
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
index 7eab283ec9e7..4fa4371545f9 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
@@ -181,6 +181,7 @@ static void wave6_handle_bitstream_buffer(struct vpu_instance *inst,
return;
}
+ vpu_buf->ts_start = ktime_get_raw();
vpu_buf->consumed = true;
wave6_vpu_dec_set_rd_ptr(inst, rd_ptr, true);
@@ -304,6 +305,8 @@ static int wave6_vpu_dec_prepare_fb(struct vpu_instance *inst)
wave6_vpu_set_instance_state(inst, VPU_INST_STATE_PIC_RUN);
+ wave6_vpu_record_performance_timestamps(inst);
+
return 0;
error:
@@ -606,6 +609,10 @@ static void wave6_handle_decoded_frame(struct vpu_instance *inst,
dst_vpu_buf->used = true;
if (state == VB2_BUF_STATE_ERROR)
dst_vpu_buf->error = true;
+ dst_vpu_buf->ts_input = vpu_buf->ts_input;
+ dst_vpu_buf->ts_start = vpu_buf->ts_start;
+ dst_vpu_buf->ts_finish = ktime_get_raw();
+ dst_vpu_buf->hw_time = wave6_vpu_cycle_to_ns(inst->dev, info->cycle.frame_cycle);
}
if (state == VB2_BUF_STATE_ERROR) {
@@ -658,6 +665,9 @@ static void wave6_handle_display_frame(struct vpu_instance *inst,
vb2_set_plane_payload(&dst_buf->vb2_buf, i,
inst->dst_fmt.plane_fmt[i].sizeimage);
+ vpu_buf->ts_output = ktime_get_raw();
+ wave6_vpu_handle_performance(inst, vpu_buf);
+
if (vpu_buf->error)
state = VB2_BUF_STATE_ERROR;
dst_buf->sequence = inst->sequence++;
@@ -1404,6 +1414,8 @@ static int wave6_vpu_dec_create_instance(struct vpu_instance *inst)
memset(&open_param, 0, sizeof(struct dec_open_param));
+ wave6_vpu_record_performance_timestamps(inst);
+
wave6_vpu_core_activate(inst->dev);
ret = pm_runtime_resume_and_get(inst->dev->dev);
if (ret) {
@@ -1567,6 +1579,7 @@ static int wave6_vpu_dec_prepare_decode(struct vpu_instance *inst)
static void wave6_vpu_dec_buf_queue_src(struct vb2_buffer *vb)
{
struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct vpu_buffer *vpu_buf = wave6_to_vpu_buf(vbuf);
struct vpu_instance *inst = vb2_get_drv_priv(vb->vb2_queue);
dev_dbg(inst->dev->dev, "type %4d index %4d size[0] %4ld size[1] : %4ld | size[2] : %4ld\n",
@@ -1574,6 +1587,7 @@ static void wave6_vpu_dec_buf_queue_src(struct vb2_buffer *vb)
vb2_plane_size(&vbuf->vb2_buf, 1), vb2_plane_size(&vbuf->vb2_buf, 2));
vbuf->sequence = inst->queued_src_buf_num++;
+ vpu_buf->ts_input = ktime_get_raw();
v4l2_m2m_buf_queue(inst->v4l2_fh.m2m_ctx, vbuf);
}
@@ -1670,6 +1684,7 @@ static void wave6_vpu_dec_stop_streaming(struct vb2_queue *q)
goto exit;
if (V4L2_TYPE_IS_OUTPUT(q->type)) {
+ wave6_vpu_reset_performance(inst);
inst->queued_src_buf_num = 0;
inst->processed_buf_num = 0;
inst->error_buf_num = 0;
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
index 4a0739033d1a..ca2d4051f9ad 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
@@ -791,6 +791,7 @@ static int wave6_vpu_enc_start_encode(struct vpu_instance *inst)
src_vbuf = wave6_to_vpu_buf(src_buf);
src_vbuf->consumed = true;
+ src_vbuf->ts_start = ktime_get_raw();
if (inst->src_fmt.num_planes == 1) {
frame_buf.buf_y = wave6_get_dma_addr(src_buf, 0);
frame_buf.buf_cb = frame_buf.buf_y + luma_size;
@@ -887,6 +888,12 @@ static void wave6_handle_encoded_frame(struct vpu_instance *inst,
dst_vpu_buf = wave6_to_vpu_buf(dst_buf);
dst_vpu_buf->average_qp = info->avg_ctu_qp;
+ dst_vpu_buf->ts_input = vpu_buf->ts_input;
+ dst_vpu_buf->ts_start = vpu_buf->ts_start;
+ dst_vpu_buf->ts_finish = ktime_get_raw();
+ dst_vpu_buf->hw_time = wave6_vpu_cycle_to_ns(inst->dev, info->cycle.frame_cycle);
+ dst_vpu_buf->ts_output = ktime_get_raw();
+ wave6_vpu_handle_performance(inst, dst_vpu_buf);
if (v4l2_m2m_is_last_draining_src_buf(inst->v4l2_fh.m2m_ctx, src_buf)) {
dst_buf->flags |= V4L2_BUF_FLAG_LAST;
@@ -2023,6 +2030,8 @@ static int wave6_vpu_enc_create_instance(struct vpu_instance *inst)
memset(&open_param, 0, sizeof(struct enc_open_param));
+ wave6_vpu_record_performance_timestamps(inst);
+
wave6_vpu_core_activate(inst->dev);
ret = pm_runtime_resume_and_get(inst->dev->dev);
if (ret) {
@@ -2179,6 +2188,8 @@ static int wave6_vpu_enc_prepare_fb(struct vpu_instance *inst)
wave6_vpu_set_instance_state(inst, VPU_INST_STATE_PIC_RUN);
+ wave6_vpu_record_performance_timestamps(inst);
+
return 0;
error:
@@ -2267,6 +2278,7 @@ static void wave6_vpu_enc_buf_queue(struct vb2_buffer *vb)
if (V4L2_TYPE_IS_OUTPUT(vb->type)) {
vbuf->sequence = inst->queued_src_buf_num++;
+ vpu_buf->ts_input = ktime_get_raw();
vpu_buf->force_key_frame = inst->enc_ctrls.force_key_frame;
inst->enc_ctrls.force_key_frame = false;
vpu_buf->force_frame_qp = !inst->enc_ctrls.frame_rc_enable;
@@ -2363,6 +2375,7 @@ static void wave6_vpu_enc_stop_streaming(struct vb2_queue *q)
goto exit;
if (V4L2_TYPE_IS_OUTPUT(q->type)) {
+ wave6_vpu_reset_performance(inst);
inst->queued_src_buf_num = 0;
inst->processed_buf_num = 0;
inst->error_buf_num = 0;
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
index 6980b126dee5..3ec01ec867df 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
@@ -5,6 +5,9 @@
* Copyright (C) 2026 CHIPS&MEDIA INC
*/
+#include <linux/clk.h>
+#include <linux/math64.h>
+
#include "wave6-vpu.h"
#include "wave6-vpu-core.h"
#include "wave6-trace.h"
@@ -228,6 +231,20 @@ void wave6_vpu_set_instance_state(struct vpu_instance *inst,
inst->state = state;
}
+u64 wave6_vpu_cycle_to_ns(struct vpu_core_device *core, u64 cycle)
+{
+ unsigned long rate;
+
+ if (!core->num_clks || !core->clks)
+ return 0;
+
+ rate = clk_get_rate(core->clks[0].clk);
+ if (!rate)
+ return 0;
+
+ return div64_ul(cycle * NSEC_PER_SEC, rate);
+}
+
int wave6_vpu_wait_interrupt(struct vpu_instance *inst, unsigned int timeout)
{
int ret;
@@ -387,6 +404,80 @@ void wave6_vpu_finish_job(struct vpu_instance *inst)
v4l2_m2m_job_finish(inst->dev->m2m_dev, inst->v4l2_fh.m2m_ctx);
}
+void wave6_vpu_record_performance_timestamps(struct vpu_instance *inst)
+{
+ if (WARN_ON(!inst))
+ return;
+
+ if (inst->state == VPU_INST_STATE_NONE &&
+ !inst->performance.ts_start)
+ inst->performance.ts_start = ktime_get_raw();
+
+ if (inst->state == VPU_INST_STATE_PIC_RUN &&
+ !inst->performance.ts_first)
+ inst->performance.ts_first = ktime_get_raw();
+}
+
+void wave6_vpu_handle_performance(struct vpu_instance *inst, struct vpu_buffer *vpu_buf)
+{
+ s64 latency, time_spent;
+
+ if (WARN_ON(!inst || !vpu_buf))
+ return;
+
+ inst->performance.ts_last = vpu_buf->ts_output;
+
+ latency = vpu_buf->ts_output - vpu_buf->ts_input;
+ time_spent = vpu_buf->ts_finish - vpu_buf->ts_start;
+
+ if (!inst->performance.latency_first)
+ inst->performance.latency_first = latency;
+ inst->performance.latency_max = max_t(s64, latency, inst->performance.latency_max);
+
+ if (!inst->performance.min_process_time)
+ inst->performance.min_process_time = time_spent;
+ else if (inst->performance.min_process_time > time_spent)
+ inst->performance.min_process_time = time_spent;
+
+ if (inst->performance.max_process_time < time_spent)
+ inst->performance.max_process_time = time_spent;
+
+ inst->performance.total_sw_time += time_spent;
+ inst->performance.total_hw_time += vpu_buf->hw_time;
+}
+
+void wave6_vpu_reset_performance(struct vpu_instance *inst)
+{
+ if (WARN_ON(!inst))
+ return;
+
+ if (inst->processed_buf_num) {
+ s64 tmp;
+ s64 fps_act = 0, fps_sw = 0, fps_hw = 0;
+ s64 duration;
+ s64 latency_ms;
+ s32 latency_ns;
+ struct vpu_performance_info *perf = &inst->performance;
+
+ tmp = MSEC_PER_SEC * inst->processed_buf_num;
+ duration = ktime_ms_delta(perf->ts_last, perf->ts_first);
+ if (duration > 0)
+ fps_act = DIV64_U64_ROUND_CLOSEST(tmp, duration);
+ duration = div_u64(perf->total_sw_time, NSEC_PER_MSEC);
+ if (duration > 0)
+ fps_sw = DIV64_U64_ROUND_CLOSEST(tmp, duration);
+ duration = div_u64(perf->total_hw_time, NSEC_PER_MSEC);
+ if (duration)
+ fps_hw = DIV64_U64_ROUND_CLOSEST(tmp, duration);
+ latency_ms = div_s64_rem(perf->latency_first, NSEC_PER_MSEC, &latency_ns);
+ dev_dbg(inst->dev->dev,
+ "[%d] fps actual: %lld, sw: %lld, hw: %lld, latency(ms) %lld.%06d\n",
+ inst->id, fps_act, fps_sw, fps_hw, latency_ms, latency_ns);
+ }
+
+ memset(&inst->performance, 0, sizeof(inst->performance));
+}
+
static const struct v4l2_m2m_ops wave6_vpu_m2m_ops = {
.device_run = wave6_vpu_device_run,
.job_ready = wave6_vpu_job_ready,
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
index bc3d7e192904..00596504c3f5 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
@@ -339,6 +339,18 @@ struct instance_buffer {
dma_addr_t ar_base;
};
+struct report_cycle {
+ u32 host_cmd_s;
+ u32 host_cmd_e;
+ u32 proc_s;
+ u32 proc_e;
+ u32 vpu_s;
+ u32 vpu_e;
+ u32 frame_cycle;
+ u32 proc_cycle;
+ u32 vpu_cycle;
+};
+
struct color_param {
u32 color_range;
u32 matrix_coefficients;
@@ -423,6 +435,7 @@ struct dec_output_info {
int error_reason;
int warn_info;
unsigned int sequence_no;
+ struct report_cycle cycle;
dma_addr_t release_disp_frame_addr[WAVE6_MAX_FBS];
dma_addr_t disp_frame_addr[WAVE6_MAX_FBS];
u64 timestamp;
@@ -671,6 +684,7 @@ struct enc_output_info {
bool encoding_success;
struct enc_report_fme_sum fme_sum;
struct enc_report_mv_histo mv_histo;
+ struct report_cycle cycle;
u64 timestamp;
dma_addr_t src_y_addr;
dma_addr_t custom_map_addr;
@@ -839,6 +853,18 @@ struct vpu_instance_ops {
void (*finish_process)(struct vpu_instance *inst, bool error);
};
+struct vpu_performance_info {
+ ktime_t ts_start;
+ ktime_t ts_first;
+ ktime_t ts_last;
+ s64 latency_first;
+ s64 latency_max;
+ s64 min_process_time;
+ s64 max_process_time;
+ u64 total_sw_time;
+ u64 total_hw_time;
+};
+
/**
* struct vpu_instance - VPU instance structure
* @list: List head for VPU core device instance list
@@ -883,6 +909,7 @@ struct vpu_instance_ops {
* @error_recovery: Error recovery flag
* @disp_mode: Display mode for decoder V4L2 control
* @enc_ctrls: Encoder V4L2 control and streaming parameters
+ * @performance: Performance information
* @debugfs: Debugfs entry
*/
struct vpu_instance {
@@ -943,6 +970,7 @@ struct vpu_instance {
enum display_mode disp_mode;
struct enc_controls enc_ctrls;
+ struct vpu_performance_info performance;
struct dentry *debugfs;
};
--
2.41.0.windows.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v8 10/11] media: chips-media: wave6: Add Wave6 thermal cooling device
2026-11-19 8:59 [PATCH v8 00/11] Add support for Wave6 video codec driver Nas Chung
` (8 preceding siblings ...)
2026-11-19 9:00 ` [PATCH v8 09/11] media: chips-media: wave6: Add performance profiling Nas Chung
@ 2026-11-19 9:00 ` Nas Chung
2026-11-19 9:21 ` sashiko-bot
2026-11-19 9:00 ` [PATCH v8 11/11] arm64: dts: freescale: imx95: Add video codec node Nas Chung
10 siblings, 1 reply; 23+ messages in thread
From: Nas Chung @ 2026-11-19 9:00 UTC (permalink / raw)
To: mchehab, hverkuil, robh, krzk+dt, conor+dt, Frank.Li, s.hauer
Cc: linux-media, devicetree, linux-kernel, imx, linux-arm-kernel,
festevam, kernel, nicolas.dufresne, ming.qian, marek.vasut,
jackson.lee, lafley.kim, Nas Chung
Add a thermal cooling device for the Wave6 VPU, so the thermal
framework can lower the VPU performance state when the SoC gets hot.
The control driver registers the cooling device on its device tree
node. The cooling states are built from the OPP table of the "perf"
power domain.
Tested-by: Ming Qian <ming.qian@oss.nxp.com>
Tested-by: Marek Vasut <marek.vasut@mailbox.org>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
---
.../media/platform/chips-media/wave6/Makefile | 1 +
.../chips-media/wave6/wave6-vpu-thermal.c | 141 ++++++++++++++++++
.../chips-media/wave6/wave6-vpu-thermal.h | 24 +++
.../platform/chips-media/wave6/wave6-vpu.c | 22 +++
.../platform/chips-media/wave6/wave6-vpu.h | 3 +
5 files changed, 191 insertions(+)
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c
create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.h
diff --git a/drivers/media/platform/chips-media/wave6/Makefile b/drivers/media/platform/chips-media/wave6/Makefile
index 426af279af6f..4bdf51c58f46 100644
--- a/drivers/media/platform/chips-media/wave6/Makefile
+++ b/drivers/media/platform/chips-media/wave6/Makefile
@@ -4,6 +4,7 @@
CFLAGS_wave6-vpu-core.o := -I$(src)
wave6-objs += wave6-vpu.o \
+ wave6-vpu-thermal.o \
wave6-vpu-core.o \
wave6-vpu-v4l2.o \
wave6-vpu-dbg.o \
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c
new file mode 100644
index 000000000000..3e21c7edb9e9
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.c
@@ -0,0 +1,141 @@
+// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause)
+/*
+ * Wave6 series multi-standard codec IP - wave6 thermal cooling interface
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ *
+ */
+
+#include <linux/pm_domain.h>
+#include <linux/pm_opp.h>
+#include <linux/units.h>
+
+#include "wave6-vpu-thermal.h"
+
+static int wave6_vpu_thermal_cooling_update(struct vpu_thermal_cooling *thermal,
+ int state)
+{
+ unsigned int level;
+ int ret;
+
+ level = thermal->level_table[state];
+ dev_dbg(thermal->dev, "receive cooling state: %d, new perf level %u\n",
+ state, level);
+
+ ret = dev_pm_genpd_set_performance_state(thermal->dev, level);
+ if (ret) {
+ dev_err(thermal->dev, "failed to set perf to %u, ret = %d\n",
+ level, ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int wave6_vpu_cooling_get_max_state(struct thermal_cooling_device *cdev,
+ unsigned long *state)
+{
+ struct vpu_thermal_cooling *thermal = cdev->devdata;
+
+ *state = thermal->thermal_max;
+
+ return 0;
+}
+
+static int wave6_vpu_cooling_get_cur_state(struct thermal_cooling_device *cdev,
+ unsigned long *state)
+{
+ struct vpu_thermal_cooling *thermal = cdev->devdata;
+
+ *state = thermal->thermal_event;
+
+ return 0;
+}
+
+static int wave6_vpu_cooling_set_cur_state(struct thermal_cooling_device *cdev,
+ unsigned long state)
+{
+ struct vpu_thermal_cooling *thermal = cdev->devdata;
+ int ret;
+
+ ret = wave6_vpu_thermal_cooling_update(thermal, state);
+ if (ret)
+ return ret;
+
+ thermal->thermal_event = state;
+
+ return 0;
+}
+
+static const struct thermal_cooling_device_ops wave6_cooling_ops = {
+ .get_max_state = wave6_vpu_cooling_get_max_state,
+ .get_cur_state = wave6_vpu_cooling_get_cur_state,
+ .set_cur_state = wave6_vpu_cooling_set_cur_state,
+};
+
+int wave6_vpu_cooling_init(struct device *dev, struct vpu_thermal_cooling *thermal)
+{
+ int num_opps;
+ unsigned long freq = ULONG_MAX;
+ int ret = -EINVAL;
+
+ if (WARN_ON(!thermal || !thermal->dev))
+ return -EINVAL;
+
+ num_opps = dev_pm_opp_get_opp_count(thermal->dev);
+ if (num_opps < 0) {
+ dev_err(thermal->dev, "fail to get pm opp count, ret = %d\n", num_opps);
+ return num_opps;
+ }
+ if (num_opps == 0) {
+ dev_err(thermal->dev, "no OPP entries found\n");
+ return -ENODEV;
+ }
+
+ thermal->level_table = devm_kcalloc(dev, num_opps,
+ sizeof(*thermal->level_table),
+ GFP_KERNEL);
+ if (!thermal->level_table) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ for (int i = 0; i < num_opps; i++, freq--) {
+ struct dev_pm_opp *opp;
+ unsigned int level;
+
+ opp = dev_pm_opp_find_freq_floor(thermal->dev, &freq);
+ if (IS_ERR(opp))
+ break;
+
+ level = dev_pm_opp_get_level(opp);
+ dev_pm_opp_put(opp);
+
+ dev_dbg(thermal->dev, "[%d] = %lu, level %u\n", i, freq, level);
+ if (freq < 100 * HZ_PER_MHZ)
+ break;
+
+ thermal->level_table[i] = level;
+ thermal->thermal_max = i;
+ }
+
+ if (!thermal->thermal_max)
+ goto error;
+
+ thermal->thermal_event = 0;
+ thermal->cooling = devm_thermal_of_child_cooling_device_register(dev,
+ dev->of_node,
+ dev_name(thermal->dev),
+ thermal,
+ &wave6_cooling_ops);
+ if (IS_ERR(thermal->cooling)) {
+ dev_err(thermal->dev, "register cooling device failed\n");
+ ret = PTR_ERR(thermal->cooling);
+ goto error;
+ }
+
+ return 0;
+
+error:
+ return ret;
+}
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.h b/drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.h
new file mode 100644
index 000000000000..35377a104aca
--- /dev/null
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-thermal.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
+/*
+ * Wave6 series multi-standard codec IP - wave6 thermal cooling interface
+ *
+ * Copyright (C) 2026 CHIPS&MEDIA INC
+ *
+ */
+
+#ifndef __WAVE6_VPU_THERMAL_H__
+#define __WAVE6_VPU_THERMAL_H__
+
+#include <linux/thermal.h>
+
+struct vpu_thermal_cooling {
+ struct device *dev;
+ int thermal_event;
+ int thermal_max;
+ struct thermal_cooling_device *cooling;
+ unsigned int *level_table;
+};
+
+int wave6_vpu_cooling_init(struct device *dev, struct vpu_thermal_cooling *thermal);
+
+#endif /* __WAVE6_VPU_THERMAL_H__ */
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu.c b/drivers/media/platform/chips-media/wave6/wave6-vpu.c
index 54a6567bc64a..3e8cd67e0d71 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu.c
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu.c
@@ -613,6 +613,21 @@ static int wave6_vpu_attach_pm_domains(struct wave6_vpu_device *vpu)
return 0;
}
+static struct device *wave6_vpu_get_performance_dev(struct wave6_vpu_device *vpu)
+{
+ int index;
+
+ if (!vpu->num_pm_domains)
+ return vpu->dev;
+
+ index = of_property_match_string(vpu->dev->of_node,
+ "power-domain-names", "perf");
+ if (index < 0 || index >= vpu->num_pm_domains)
+ return NULL;
+
+ return vpu->pd_list->pd_devs[index];
+}
+
static int wave6_vpu_probe(struct platform_device *pdev)
{
struct device_node *np;
@@ -696,6 +711,13 @@ static int wave6_vpu_probe(struct platform_device *pdev)
}
}
+ vpu->thermal.dev = wave6_vpu_get_performance_dev(vpu);
+ if (vpu->thermal.dev) {
+ ret = wave6_vpu_cooling_init(vpu->dev, &vpu->thermal);
+ if (ret)
+ dev_err(&pdev->dev, "failed to initialize thermal cooling, %d\n", ret);
+ }
+
wave6_vpu_allocate_work_buffers(vpu);
pm_runtime_enable(&pdev->dev);
diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu.h b/drivers/media/platform/chips-media/wave6/wave6-vpu.h
index 35a1d9f8a003..64e71a8ae597 100644
--- a/drivers/media/platform/chips-media/wave6/wave6-vpu.h
+++ b/drivers/media/platform/chips-media/wave6/wave6-vpu.h
@@ -12,6 +12,7 @@
#include <linux/device.h>
#include "wave6-vpuconfig.h"
+#include "wave6-vpu-thermal.h"
#include "wave6-vdi.h"
#define WAVE6_VPU_PLATFORM_DRIVER_NAME "wave6-vpu"
@@ -93,6 +94,7 @@ struct wave6_vpu_core_resource {
* @work_buffers: Array of work buffers
* @work_buffers_alloc: Number of allocated work buffers
* @work_buffers_avail: Number of available work buffers
+ * @thermal: Thermal cooling device
* @core_count: Number of VPU cores holding a wave6_vpu_get() reference
* @num_pm_domains: Number of attached power domains
* @pd_list: List of attached power domains
@@ -121,6 +123,7 @@ struct wave6_vpu_device {
u32 work_buffers_alloc;
u32 work_buffers_avail;
+ struct vpu_thermal_cooling thermal;
atomic_t core_count;
int num_pm_domains;
--
2.41.0.windows.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v8 11/11] arm64: dts: freescale: imx95: Add video codec node
2026-11-19 8:59 [PATCH v8 00/11] Add support for Wave6 video codec driver Nas Chung
` (9 preceding siblings ...)
2026-11-19 9:00 ` [PATCH v8 10/11] media: chips-media: wave6: Add Wave6 thermal cooling device Nas Chung
@ 2026-11-19 9:00 ` Nas Chung
2026-11-19 9:37 ` sashiko-bot
10 siblings, 1 reply; 23+ messages in thread
From: Nas Chung @ 2026-11-19 9:00 UTC (permalink / raw)
To: mchehab, hverkuil, robh, krzk+dt, conor+dt, Frank.Li, s.hauer
Cc: linux-media, devicetree, linux-kernel, imx, linux-arm-kernel,
festevam, kernel, nicolas.dufresne, ming.qian, marek.vasut,
jackson.lee, lafley.kim, Nas Chung
Add the Chips and Media wave633 video codec node on IMX95 SoCs.
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
---
.../boot/dts/freescale/imx95-19x19-evk.dts | 10 +++++
arch/arm64/boot/dts/freescale/imx95.dtsi | 37 +++++++++++++++++++
2 files changed, 47 insertions(+)
diff --git a/arch/arm64/boot/dts/freescale/imx95-19x19-evk.dts b/arch/arm64/boot/dts/freescale/imx95-19x19-evk.dts
index 47ed0ef67611..a15d5b471708 100644
--- a/arch/arm64/boot/dts/freescale/imx95-19x19-evk.dts
+++ b/arch/arm64/boot/dts/freescale/imx95-19x19-evk.dts
@@ -143,6 +143,11 @@ vdev0buffer: vdev0buffer@88020000 {
reg = <0 0x88020000 0 0x100000>;
no-map;
};
+
+ vpu_boot: memory@a0000000 {
+ reg = <0 0xa0000000 0 0x100000>;
+ no-map;
+ };
};
flexcan1_phy: can-phy0 {
@@ -1225,3 +1230,8 @@ &tpm6 {
pinctrl-0 = <&pinctrl_tpm6>;
status = "okay";
};
+
+&vpu {
+ memory-region = <&vpu_boot>;
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/freescale/imx95.dtsi b/arch/arm64/boot/dts/freescale/imx95.dtsi
index d20c2498b9aa..b6323e527ab7 100644
--- a/arch/arm64/boot/dts/freescale/imx95.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx95.dtsi
@@ -2139,6 +2139,43 @@ vpu_blk_ctrl: clock-controller@4c410000 {
assigned-clock-rates = <133333333>, <667000000>, <500000000>;
};
+ vpu: video-codec@4c4c0000 {
+ compatible = "nxp,imx95-vpu";
+ reg = <0x0 0x4c4c0000 0x0 0x10000>;
+ clocks = <&scmi_clk IMX95_CLK_VPU>,
+ <&vpu_blk_ctrl IMX95_CLK_VPUBLK_WAVE>;
+ clock-names = "core", "vpublk";
+ power-domains = <&scmi_devpd IMX95_PD_VPU>,
+ <&scmi_perf IMX95_PERF_VPU>;
+ power-domain-names = "vpu", "perf";
+ sram = <&sram1>;
+ #cooling-cells = <2>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+ status = "disabled";
+
+ interface@4c480000 {
+ reg = <0x0 0x4c480000 0x0 0x10000>;
+ interrupts = <GIC_SPI 299 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
+ interface@4c490000 {
+ reg = <0x0 0x4c490000 0x0 0x10000>;
+ interrupts = <GIC_SPI 300 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
+ interface@4c4a0000 {
+ reg = <0x0 0x4c4a0000 0x0 0x10000>;
+ interrupts = <GIC_SPI 301 IRQ_TYPE_LEVEL_HIGH>;
+ };
+
+ interface@4c4b0000 {
+ reg = <0x0 0x4c4b0000 0x0 0x10000>;
+ interrupts = <GIC_SPI 302 IRQ_TYPE_LEVEL_HIGH>;
+ };
+ };
+
jpegdec: jpegdec@4c500000 {
compatible = "nxp,imx95-jpgdec", "nxp,imx8qxp-jpgdec";
reg = <0x0 0x4C500000 0x0 0x00050000>;
--
2.41.0.windows.3
^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [PATCH v8 06/11] media: chips-media: wave6: Add Wave6 encoder firmware interface
2026-11-19 9:00 ` [PATCH v8 06/11] media: chips-media: wave6: Add Wave6 encoder firmware interface Nas Chung
@ 2026-11-19 9:10 ` sashiko-bot
0 siblings, 0 replies; 23+ messages in thread
From: sashiko-bot @ 2026-11-19 9:10 UTC (permalink / raw)
To: Nas Chung; +Cc: devicetree, imx, robh, conor+dt, Frank.Li, media-ci
Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] media: wave6: avoid large struct copies on stack in
wave6_gen_enc_pic_reg()
--
commit 0c77c50267143f16a1d2daf6f5d24a657e85b50e
Author: Nas Chung <nas.chung@chipsnmedia.com>
media: chips-media: wave6: Add Wave6 encoder firmware interface
This commit adds the firmware interface and MMIO register configuration
helpers for Wave6 video encoding operations.
> diff --git a/drivers/media/platform/chips-media/wave6/wave6-hw.c b/drivers/media/platform/chips-media/wave6/wave6-hw.c
> index 872a540df0218..171391f797cb8 100644
> --- a/drivers/media/platform/chips-media/wave6/wave6-hw.c
> +++ b/drivers/media/platform/chips-media/wave6/wave6-hw.c
[ ... ]
> +static void wave6_update_enc_info(struct enc_info *p_enc_info)
> +{
> + struct enc_open_param op = p_enc_info->open_param;
> +
> + p_enc_info->width = op.pic_width;
> + p_enc_info->height = op.pic_height;
> + p_enc_info->c_fmt_idc = get_chroma_format_idc(op.out_fmt);
> +}
[ ... ]
> +static void wave6_gen_enc_pic_reg(struct enc_info *p_enc_info, bool cbcr_interleave, bool nv21,
> + struct enc_param *opt, struct enc_cmd_enc_pic_reg *reg)
> +{
> + struct enc_open_param open = p_enc_info->open_param;
> + struct enc_codec_param param = open.codec_param;
> + u32 stride_c = opt->source_frame->stride_c;
> + u32 c_fmt_idc;
[ ... ]
> +int wave6_vpu_enc_start_one_frame(struct vpu_instance *inst, struct enc_param *param,
> + u32 *fail_res)
> +{
> + struct enc_cmd_enc_pic_reg reg = {};
> + struct enc_info *p_enc_info;
> + int ret;
> +
> + guard(mutex)(&inst->dev->hw_lock);
> +
> + p_enc_info = &inst->codec_info->enc_info;
> +
> + wave6_gen_enc_pic_reg(p_enc_info, inst->cbcr_interleave,
> + inst->nv21, param, ®);
[Severity: Medium]
Should wave6_gen_enc_pic_reg() and wave6_update_enc_info() take pointers
to p_enc_info->open_param and its codec_param instead of copying them by
value onto the stack?
struct enc_open_param is around 824 bytes and struct enc_codec_param is
around 772 bytes. When wave6_gen_enc_pic_reg() is inlined into
wave6_vpu_enc_start_one_frame() alongside struct enc_cmd_enc_pic_reg, the
local structures alone consume roughly 1,700 bytes of stack space (and
over 1,220 bytes in wave6_vpu_enc_issue_seq_init() via
wave6_update_enc_info()), exceeding the 1,280-byte CONFIG_FRAME_WARN
threshold on 32-bit builds.
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795075577.git.nas.chung@chipsnmedia.com?part=6
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v8 08/11] media: chips-media: wave6: Add trace and debugfs
2026-11-19 9:00 ` [PATCH v8 08/11] media: chips-media: wave6: Add trace and debugfs Nas Chung
@ 2026-11-19 9:14 ` sashiko-bot
0 siblings, 0 replies; 23+ messages in thread
From: sashiko-bot @ 2026-11-19 9:14 UTC (permalink / raw)
To: Nas Chung; +Cc: devicetree, imx, robh, conor+dt, Frank.Li, media-ci
Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] media: wave6: debugfs directory leak in
wave6_vpu_core_probe() on teardown
--
commit 5ece419bea0b582239e058aa46da23a997a50915
Author: Nas Chung <nas.chung@chipsnmedia.com>
media: chips-media: wave6: Add trace and debugfs
This commit adds tracepoints, per-instance debugfs entries, and readable
firmware error messages to the Wave6 VPU driver.
> diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
> index aad4023d66a44..12efa3afebf7a 100644
> --- a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
> +++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
[ ... ]
> @@ -285,6 +293,12 @@ static int wave6_vpu_core_probe(struct platform_device *pdev)
> if (ret)
> return ret;
>
> + core->debugfs = debugfs_lookup(WAVE6_VPU_DEBUGFS_DIR, NULL);
> + if (!IS_ERR_OR_NULL(core->debugfs))
> + dput(core->debugfs);
> + else
> + core->debugfs = debugfs_create_dir(WAVE6_VPU_DEBUGFS_DIR, NULL);
> +
> ret = devm_pm_runtime_enable(&pdev->dev);
> if (ret)
> return ret;
[Severity: Medium]
Does this leak the wave6 debugfs directory on probe failure or module
unload? Every other resource in wave6_vpu_core_probe() registers a
devm_add_action_or_reset() cleanup callback because wave6_vpu_core_driver
has no .remove callback, and wave6_vpu_remove_dbgfs_file() only removes
the per-instance file, so nothing ever calls debugfs_remove() on
core->debugfs.
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795075577.git.nas.chung@chipsnmedia.com?part=8
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v8 09/11] media: chips-media: wave6: Add performance profiling
2026-11-19 9:00 ` [PATCH v8 09/11] media: chips-media: wave6: Add performance profiling Nas Chung
@ 2026-11-19 9:20 ` sashiko-bot
0 siblings, 0 replies; 23+ messages in thread
From: sashiko-bot @ 2026-11-19 9:20 UTC (permalink / raw)
To: Nas Chung; +Cc: devicetree, imx, robh, conor+dt, Frank.Li, media-ci
Thank you for your contribution! Sashiko AI review found 5 potential
issues to consider:
- [Medium] media: wave6: performance timestamps not re-recorded on
OUTPUT stream restart
- [Medium] media: wave6: skipped performance reset in
wave6_vpu_enc_stop_streaming()
- [Medium] media: wave6: wave6_vpu_cycle_to_ns() queries empty
core->clks instead of vpu
- [Medium] media: wave6: unlocked TOCTOU race and div-by-zero in
wave6_vpu_dbg_instance()
- [Low] media: wave6: negative setup latency in wave6_vpu_dbg_instance()
before PIC_RUN
--
commit bc504ce1eb7a7d90813e57f32d70181940592318
Author: Nas Chung <nas.chung@chipsnmedia.com>
media: chips-media: wave6: Add performance profiling
This commit adds per-instance performance profiling to record per-frame
timestamps and firmware cycle counts, reporting frame rates and latencies
in debugfs.
> diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c
> index 525e047ecebff..be1061b2b39f0 100644
> --- a/drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c
> +++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c
[ ... ]
> @@ -77,6 +83,36 @@ static int wave6_vpu_dbg_instance(struct seq_file *s, void *data)
> inst->error_buf_num,
> v4l2_m2m_has_stopped(inst->v4l2_fh.m2m_ctx));
>
> + seq_puts(s, "fps");
> + tmp = MSEC_PER_SEC * inst->processed_buf_num;
> + if (perf->ts_last > perf->ts_first + NSEC_PER_MSEC) {
> + duration = ktime_ms_delta(perf->ts_last, perf->ts_first);
> + fps = DIV64_U64_ROUND_CLOSEST(tmp, duration);
> + seq_printf(s, " actual: %lld;", fps);
> + }
[Severity: Medium]
Can a concurrent OUTPUT queue stop_streaming() cause a divide-by-zero in
DIV64_U64_ROUND_CLOSEST(tmp, duration)?
wave6_vpu_dbg_instance() reads inst->performance without locking while the
debugfs file is active:
CPU0 CPU1
wave6_vpu_dbg_instance()
if (perf->ts_last >
perf->ts_first + NSEC_PER_MSEC)
wave6_vpu_dec_stop_streaming()
wave6_vpu_reset_performance()
memset(&inst->performance, 0, ...)
duration = ktime_ms_delta(
perf->ts_last, perf->ts_first)
DIV64_U64_ROUND_CLOSEST(
tmp, duration) <-- divide-by-zero when duration == 0
Unlike wave6_vpu_reset_performance(), this path does not check duration > 0
before dividing, and if only perf->ts_last is seen zeroed, duration becomes
negative and is cast to a large u64 divisor.
> + duration = div_u64(perf->total_sw_time, NSEC_PER_MSEC);
> + if (duration > 0) {
> + fps = DIV64_U64_ROUND_CLOSEST(tmp, duration);
> + seq_printf(s, " sw: %lld;", fps);
> + }
> + duration = div_u64(perf->total_hw_time, NSEC_PER_MSEC);
> + if (duration > 0) {
> + fps = DIV64_U64_ROUND_CLOSEST(tmp, duration);
> + seq_printf(s, " hw: %lld", fps);
> + }
> + seq_putc(s, '\n');
> +
> + ms[0] = div_s64_rem(perf->latency_first, NSEC_PER_MSEC, &ns[0]);
> + ms[1] = div_s64_rem(perf->latency_max, NSEC_PER_MSEC, &ns[1]);
> + ms[2] = div_s64_rem(ktime_sub(perf->ts_first, perf->ts_start), NSEC_PER_MSEC, &ns[2]);
> + seq_printf(s, "latency(ms) first: %lld.%06d, max %lld.%06d, setup %lld.%06d\n",
> + ms[0], ns[0], ms[1], ns[1], ms[2], ns[2]);
[Severity: Low]
What happens to the setup latency output if the debugfs file is read before
the instance reaches VPU_INST_STATE_PIC_RUN?
perf->ts_start is set when the instance is created (before
wave6_vpu_create_dbgfs_file() exposes the debugfs entry in
VPU_INST_STATE_OPEN), whereas perf->ts_first remains 0 until
wave6_vpu_dec_prepare_fb() or wave6_vpu_enc_prepare_fb() transitions the
instance to VPU_INST_STATE_PIC_RUN. Reading debugfs in VPU_INST_STATE_OPEN
or VPU_INST_STATE_INIT_SEQ passes -perf->ts_start to div_s64_rem(),
producing a negative quotient ms[2] and a negative remainder ns[2] formatted
with %lld.%06d (e.g. "setup -12345.-678900").
[ ... ]
> diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
> index 7eab283ec9e70..4fa4371545f9b 100644
> --- a/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
> +++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
[ ... ]
> @@ -1670,6 +1684,7 @@ static void wave6_vpu_dec_stop_streaming(struct vb2_queue *q)
> goto exit;
>
> if (V4L2_TYPE_IS_OUTPUT(q->type)) {
> + wave6_vpu_reset_performance(inst);
> inst->queued_src_buf_num = 0;
> inst->processed_buf_num = 0;
> inst->error_buf_num = 0;
[Severity: Medium]
Does zeroing inst->performance on OUTPUT stop_streaming() leave
perf->ts_start and perf->ts_first at 0 when OUTPUT streaming is restarted?
During a decoder seek (VIDIOC_STREAMOFF followed by VIDIOC_STREAMON on the
OUTPUT queue), wave6_vpu_dec_stop_streaming() clears inst->performance and
moves inst->state to VPU_INST_STATE_SEEK. When OUTPUT streaming restarts,
wave6_vpu_dec_start_streaming() restores inst->state directly to
inst->state_in_seek without calling wave6_vpu_record_performance_timestamps():
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c:wave6_vpu_dec_start_streaming() {
...
if (V4L2_TYPE_IS_OUTPUT(q->type)) {
fmt = &inst->src_fmt;
if (inst->state == VPU_INST_STATE_NONE) {
ret = wave6_vpu_dec_create_instance(inst);
...
}
if (inst->state == VPU_INST_STATE_SEEK)
wave6_vpu_set_instance_state(inst, inst->state_in_seek);
}
...
}
The same happens in the encoder when the OUTPUT queue is stopped and
restarted while CAPTURE remains streaming in VPU_INST_STATE_PIC_RUN.
Because perf->ts_first stays 0 while perf->ts_last updates on each decoded
or encoded frame, ktime_ms_delta(perf->ts_last, perf->ts_first) in debugfs
and wave6_vpu_reset_performance() divides by total system uptime since boot.
> diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
> index 4a0739033d1a9..ca2d4051f9adf 100644
> --- a/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
> +++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
[ ... ]
> @@ -2363,6 +2375,7 @@ static void wave6_vpu_enc_stop_streaming(struct vb2_queue *q)
> goto exit;
>
> if (V4L2_TYPE_IS_OUTPUT(q->type)) {
> + wave6_vpu_reset_performance(inst);
> inst->queued_src_buf_num = 0;
> inst->processed_buf_num = 0;
> inst->error_buf_num = 0;
[Severity: Medium]
Is wave6_vpu_reset_performance() skipped whenever the encoder CAPTURE queue
stops streaming before the OUTPUT queue?
In wave6_vpu_enc_stop_streaming(), stopping the CAPTURE queue calls
wave6_vpu_enc_destroy_instance(inst, true), which sets inst->state to
VPU_INST_STATE_NONE. When wave6_vpu_enc_stop_streaming() is then called for
the OUTPUT queue (for example via v4l2_m2m_ctx_release() on file close,
which always stops CAPTURE before OUTPUT, or via VIDIOC_STREAMOFF on CAPTURE
first), the early check:
if (inst->state == VPU_INST_STATE_NONE)
goto exit;
bypasses wave6_vpu_reset_performance(inst) and the counter resets. This
skips logging the performance summary and leaves stale metrics and non-zero
ts_start / ts_first values in inst->performance if streaming is restarted on
the same file descriptor.
> diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
> index 6980b126dee55..3ec01ec867dfc 100644
> --- a/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
> +++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
[ ... ]
> @@ -228,6 +231,20 @@ void wave6_vpu_set_instance_state(struct vpu_instance *inst,
> inst->state = state;
> }
>
> +u64 wave6_vpu_cycle_to_ns(struct vpu_core_device *core, u64 cycle)
> +{
> + unsigned long rate;
> +
> + if (!core->num_clks || !core->clks)
> + return 0;
> +
> + rate = clk_get_rate(core->clks[0].clk);
> + if (!rate)
> + return 0;
> +
> + return div64_ul(cycle * NSEC_PER_SEC, rate);
> +}
[Severity: Medium]
Does wave6_vpu_cycle_to_ns() always return 0 because the clocks belong to
core->vpu rather than core?
In the Device Tree binding (nxp,imx95-vpu.yaml), the clocks property is on
the parent VPU node and is acquired into vpu->clks by wave6_vpu_probe(),
whereas the child interface nodes used to create the wave6-vpu-core devices
have no clocks property. As a result, core->num_clks is 0, per-frame
hw_time and perf->total_hw_time remain 0, and hardware FPS is never
reported.
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795075577.git.nas.chung@chipsnmedia.com?part=9
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v8 10/11] media: chips-media: wave6: Add Wave6 thermal cooling device
2026-11-19 9:00 ` [PATCH v8 10/11] media: chips-media: wave6: Add Wave6 thermal cooling device Nas Chung
@ 2026-11-19 9:21 ` sashiko-bot
0 siblings, 0 replies; 23+ messages in thread
From: sashiko-bot @ 2026-11-19 9:21 UTC (permalink / raw)
To: Nas Chung; +Cc: devicetree, imx, robh, conor+dt, Frank.Li, media-ci
Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] media: chips-media: wave6: UAF of thermal.dev due to devm
teardown order
--
commit 4476163b22eacaa7f1b0467ae8902df8f1de4240
Author: Nas Chung <nas.chung@chipsnmedia.com>
media: chips-media: wave6: Add Wave6 thermal cooling device
This commit adds a thermal cooling device for the Wave6 VPU that adjusts
the "perf" power domain's performance state based on its OPP table.
> diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu.c b/drivers/media/platform/chips-media/wave6/wave6-vpu.c
> index 54a6567bc64a5..3e8cd67e0d718 100644
> --- a/drivers/media/platform/chips-media/wave6/wave6-vpu.c
> +++ b/drivers/media/platform/chips-media/wave6/wave6-vpu.c
> @@ -613,6 +613,21 @@ static int wave6_vpu_attach_pm_domains(struct wave6_vpu_device *vpu)
> return 0;
> }
>
> +static struct device *wave6_vpu_get_performance_dev(struct wave6_vpu_device *vpu)
> +{
> + int index;
> +
> + if (!vpu->num_pm_domains)
> + return vpu->dev;
> +
> + index = of_property_match_string(vpu->dev->of_node,
> + "power-domain-names", "perf");
> + if (index < 0 || index >= vpu->num_pm_domains)
> + return NULL;
> +
> + return vpu->pd_list->pd_devs[index];
> +}
> +
> static int wave6_vpu_probe(struct platform_device *pdev)
> {
> struct device_node *np;
> @@ -696,6 +711,13 @@ static int wave6_vpu_probe(struct platform_device *pdev)
> }
> }
>
> + vpu->thermal.dev = wave6_vpu_get_performance_dev(vpu);
> + if (vpu->thermal.dev) {
> + ret = wave6_vpu_cooling_init(vpu->dev, &vpu->thermal);
> + if (ret)
> + dev_err(&pdev->dev, "failed to initialize thermal cooling, %d\n", ret);
> + }
[Severity: Medium]
Can registering the cooling device via devm on vpu->dev cause a
use-after-free of vpu->thermal.dev during driver teardown?
When multiple power domains are attached via dev_pm_domain_attach_list(),
vpu->thermal.dev points to the virtual domain device in
vpu->pd_list->pd_devs[index], while wave6_vpu_cooling_init() registers
vpu->thermal.cooling via devm_thermal_of_child_cooling_device_register()
on vpu->dev. Because wave6_vpu_remove() (and the wave6_vpu_probe() error
path) calls wave6_vpu_detach_pm_domains() before devres cleans up
vpu->dev, the cooling device remains registered after the virtual domain
device is detached and freed:
CPU0 CPU1
wave6_vpu_remove()
wave6_vpu_detach_pm_domains()
dev_pm_domain_detach_list()
/* unregisters and frees
pd_devs[index] */
__thermal_cdev_update()
wave6_vpu_cooling_set_cur_state()
wave6_vpu_thermal_cooling_update()
dev_pm_genpd_set_performance_state(thermal->dev, ...)
<-- use-after-free
devres_release_all()
thermal_cooling_device_unregister()
Attaching the power domains with devm_pm_domain_attach_list() before
wave6_vpu_cooling_init(), or unregistering the cooling device explicitly
before wave6_vpu_detach_pm_domains(), would keep the teardown order
consistent.
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795075577.git.nas.chung@chipsnmedia.com?part=10
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v8 04/11] media: chips-media: wave6: Add Wave6 decoder firmware interface
2026-11-19 8:59 ` [PATCH v8 04/11] media: chips-media: wave6: Add Wave6 decoder firmware interface Nas Chung
@ 2026-11-19 9:22 ` sashiko-bot
0 siblings, 0 replies; 23+ messages in thread
From: sashiko-bot @ 2026-11-19 9:22 UTC (permalink / raw)
To: Nas Chung; +Cc: devicetree, imx, robh, conor+dt, Frank.Li, media-ci
Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [Low] media: wave6: unset disp_pic_width/height in
wave6_vpu_dec_get_output_info()
- [Low] media: wave6: invalid W6_RET_DEC_BS_RD_PTR read in
wave6_vpu_dec_get_rd_ptr()
- [Low] media: wave6: ctu_size 2-bit bitfield truncates
DEC_PIC_TYPE_CTU_SIZE to 0
--
commit 4430346b0188df2bacf518d95646056af3905289
Author: Nas Chung <nas.chung@chipsnmedia.com>
media: chips-media: wave6: Add Wave6 decoder firmware interface
This commit adds the firmware and register interface layer for the
Chips&Media Wave6 video decoder, covering instance management, sequence
initialization, buffer registration, and picture decoding.
> diff --git a/drivers/media/platform/chips-media/wave6/wave6-hw.c b/drivers/media/platform/chips-media/wave6/wave6-hw.c
> index fcf5be294ef5a..872a540df0218 100644
> --- a/drivers/media/platform/chips-media/wave6/wave6-hw.c
> +++ b/drivers/media/platform/chips-media/wave6/wave6-hw.c
[ ... ]
> @@ -179,3 +276,659 @@ int wave6_vpu_get_version(struct vpu_core_device *core)
[ ... ]
> +int wave6_vpu_dec_get_output_info(struct vpu_instance *inst, struct dec_output_info *info)
> +{
[ ... ]
> + reg_val = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_PIC_SIZE);
> + info->dec_pic_width = DEC_PIC_SIZE_WIDTH(reg_val);
> + info->dec_pic_height = DEC_PIC_SIZE_HEIGHT(reg_val);
> +
> + info->rd_ptr = wave6_vpu_dec_get_rd_ptr(inst);
> + info->wr_ptr = p_dec_info->stream_wr_ptr;
> + info->byte_pos_frame_start = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_AU_START_POS);
> + info->byte_pos_frame_end = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_AU_END_POS);
> + info->stream_end = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_STREAM_END);
> + info->notification_flags = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_NOTIFICATION);
> + info->last_frame_in_au = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_LAST_FRAME_FLAG);
> +
> + reg_val = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_TIMESTAMP_LOW);
> + info->timestamp = vpu_read_reg_relaxed(inst->dev, W6_RET_DEC_TIMESTAMP_HIGH);
> + info->timestamp = (info->timestamp << 32) | reg_val;
> +
> + if (decoded_idx >= 0 && decoded_idx < WAVE6_MAX_FBS)
> + p_dec_info->dec_out_info[decoded_idx].decoded_poc = info->decoded_poc;
> +
> + if (disp_idx >= 0 && disp_idx < WAVE6_MAX_FBS) {
> + info->display_poc = p_dec_info->dec_out_info[disp_idx].decoded_poc;
> + info->disp_pic_width = p_dec_info->dec_out_info[disp_idx].dec_pic_width;
> + info->disp_pic_height = p_dec_info->dec_out_info[disp_idx].dec_pic_height;
> + }
> +
> + info->sequence_no = p_dec_info->seq_info.sequence_no;
> + if (decoded_idx >= 0 && decoded_idx < WAVE6_MAX_FBS)
> + p_dec_info->dec_out_info[decoded_idx] = *info;
[Severity: Low]
When a frame is decoded and displayed in the same step (disp_idx ==
decoded_idx), such as on the initial IDR frame or with zero reorder delay,
does reading disp_pic_width and disp_pic_height from
p_dec_info->dec_out_info[disp_idx] before p_dec_info->dec_out_info[decoded_idx]
is updated leave them set to 0 or stale dimensions from an earlier frame?
> diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.c b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.c
> new file mode 100644
> index 0000000000000..c8d1138334b31
> --- /dev/null
> +++ b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.c
> @@ -0,0 +1,202 @@
[ ... ]
> +int wave6_vpu_dec_complete_seq_init(struct vpu_instance *inst,
> + struct dec_seq_info *info)
> +{
> + struct dec_info *p_dec_info = &inst->codec_info->dec_info;
> + int ret;
> +
> + guard(mutex)(&inst->dev->hw_lock);
> +
> + ret = wave6_vpu_dec_get_seq_info(inst, info);
> + if (!ret)
> + p_dec_info->seq_info_obtained = true;
> +
> + info->rd_ptr = wave6_vpu_dec_get_rd_ptr(inst);
> + info->wr_ptr = p_dec_info->stream_wr_ptr;
> +
> + p_dec_info->seq_info = *info;
> +
> + return ret;
> +}
[ ... ]
> +void wave6_vpu_dec_get_bitstream_buffer(struct vpu_instance *inst,
> + dma_addr_t *p_rd_ptr,
> + dma_addr_t *p_wr_ptr)
> +{
> + struct dec_info *p_dec_info = &inst->codec_info->dec_info;
> + dma_addr_t rd_ptr;
> + dma_addr_t wr_ptr;
> +
> + guard(mutex)(&inst->dev->hw_lock);
> +
> + rd_ptr = wave6_vpu_dec_get_rd_ptr(inst);
> + wr_ptr = p_dec_info->stream_wr_ptr;
> +
> + if (p_rd_ptr)
> + *p_rd_ptr = rd_ptr;
> + if (p_wr_ptr)
> + *p_wr_ptr = wr_ptr;
> +}
[Severity: Low]
Can wave6_vpu_dec_get_rd_ptr() return an unrelated register value when
called outside a W6_QUERY_OPT_GET_RESULT query?
W6_RET_DEC_BS_RD_PTR (offset 0x30C) shares its register offset with
command registers such as W6_CMD_DEC_SET_DISP_Y_BASE and
W6_CMD_DEC_PIC_SEC_AXI, and wave6_vpu_dec_get_rd_ptr() reads it directly
without issuing a query command:
drivers/media/platform/chips-media/wave6/wave6-hw.c:wave6_vpu_dec_get_rd_ptr() {
lockdep_assert_held(&inst->dev->hw_lock);
return vpu_read_reg(inst->dev, W6_RET_DEC_BS_RD_PTR);
}
Calling wave6_vpu_dec_get_rd_ptr() in wave6_vpu_dec_get_bitstream_buffer(),
or in wave6_vpu_dec_complete_seq_init() when wave6_vpu_dec_get_seq_info()
fails (which also overwrites p_dec_info->seq_info on error), reads
whatever value was last written to 0x30C by a previous command.
> diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
> index 62dd4969afafc..7d6a734f64b43 100644
> --- a/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
> +++ b/drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
[ ... ]
> @@ -208,12 +248,116 @@ struct instance_buffer {
[ ... ]
> +struct dec_output_info {
[ ... ]
> + u32 release_disp_frame_num: 5;
> + u32 disp_frame_num: 5;
> + u32 ctu_size: 2;
[Severity: Low]
Does the 2-bit ctu_size bitfield truncate DEC_PIC_TYPE_CTU_SIZE() to 0?
wave6_vpu_dec_get_output_info() assigns info->ctu_size from
DEC_PIC_TYPE_CTU_SIZE(reg_val), which evaluates to (16 << FIELD_GET(GENMASK(11, 10), (x)))
and produces 16, 32, 64, or 128, none of which fit in a 2-bit bitfield.
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795075577.git.nas.chung@chipsnmedia.com?part=4
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v8 02/11] media: chips-media: wave6: Add Wave6 control driver
2026-11-19 8:59 ` [PATCH v8 02/11] media: chips-media: wave6: Add Wave6 control driver Nas Chung
@ 2026-11-19 9:26 ` sashiko-bot
2026-11-19 19:36 ` Frank Li
0 siblings, 1 reply; 23+ messages in thread
From: sashiko-bot @ 2026-11-19 9:26 UTC (permalink / raw)
To: Nas Chung; +Cc: devicetree, imx, robh, conor+dt, Frank.Li, media-ci
Thank you for your contribution! Sashiko AI review found 8 potential
issues to consider:
- [High] media: wave6: vpu->lock held across core device register and
unregister
- [High] media: wave6: data race on work_buffers_avail in
wave6_vpu_require_work_buffer()
- [Medium] media: wave6: stuck PREPARE state on work buf error in
wave6_vpu_init_vpu()
- [Medium] media: wave6: incompatible phys_addr_t pointer in
wave6_vpu_probe()
- [Medium] media: wave6: wave6_vpu_probe() succeeds without required
memory-region
- [Medium] media: wave6: wrong product code value for WAVE677_CODE
- [Low] media: wave6: off-by-one of_irq_get() error check in
wave6_vpu_create_cores()
- [Low] media: wave6: mixed guard(mutex) and goto cleanup in
wave6_vpu_get()
--
commit 600333b152222fe3a820d242ccd857c34cf7d58c
Author: Nas Chung <nas.chung@chipsnmedia.com>
media: chips-media: wave6: Add Wave6 control driver
This commit adds the control driver for the Chips&Media Wave6 video codec IP
to manage firmware loading, shared memory resources, and child core devices.
> diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu.c b/drivers/media/platform/chips-media/wave6/wave6-vpu.c
> new file mode 100644
> index 0000000000000..9160ce53062ce
> --- /dev/null
> +++ b/drivers/media/platform/chips-media/wave6/wave6-vpu.c
[ ... ]
> +static int wave6_vpu_init_vpu(struct wave6_vpu_device *vpu,
> + void __iomem *core_base)
> +{
> + int ret;
> +
> + lockdep_assert_held(&vpu->lock);
> +
> + /* try init directly as firmware is running */
> + if (wave6_vdi_readl(core_base, W6_VPU_VCPU_CUR_PC))
> + goto init_done;
> +
> + wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_PREPARE);
> +
> + wave6_vpu_remap_code_buf(vpu);
> +
> + wave6_vdi_writel(core_base, W6_VPU_BUSY_STATUS, BUSY_STATUS_SET);
> + wave6_vdi_writel(core_base, W6_CMD_INIT_VPU_SEC_AXI_BASE_CORE0,
> + vpu->sram_buf.dma_addr);
> + wave6_vdi_writel(core_base, W6_CMD_INIT_VPU_SEC_AXI_SIZE_CORE0,
> + vpu->sram_buf.size);
> + wave6_vdi_writel(vpu->reg_base, W6_COMMAND_GB, W6_CMD_INIT_VPU);
> + wave6_vdi_writel(vpu->reg_base, W6_VPU_REMAP_CORE_START_GB,
> + REMAP_CORE_START_ON);
> +
> + ret = wave6_vpu_wait_busy(core_base);
> + if (ret) {
> + dev_err(vpu->dev, "init vpu timeout\n");
> + wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_OFF);
> + return -EINVAL;
> + }
> +
> + ret = wave6_vpu_check_result(core_base);
> + if (ret) {
> + dev_err(vpu->dev, "init vpu fail, reason 0x%x\n", ret);
> + wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_OFF);
> + return -EIO;
> + }
> +
> +init_done:
> + ret = wave6_vpu_init_work_buf(vpu, core_base);
> + if (ret)
> + return ret;
> +
> + wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_ON);
> +
> + return 0;
> +}
[Severity: Medium]
Does returning on wave6_vpu_init_work_buf() failure leave vpu->state stuck
in WAVE6_VPU_STATE_PREPARE?
Unlike the timeout and error paths above init_done, this path does not
reset vpu->state to WAVE6_VPU_STATE_OFF. wave6_vpu_get() then powers down
the device and drops core_count to 0, and every later wave6_vpu_get() ->
wave6_vpu_try_boot() sees vpu->state == WAVE6_VPU_STATE_PREPARE and returns
0 without booting the VPU or initializing work buffers.
[ ... ]
> +int wave6_vpu_get(struct wave6_vpu_device *vpu, void __iomem *core_base)
> +{
> + int ret;
> +
> + guard(mutex)(&vpu->lock);
> +
> + /* Only the first core executes boot; others return */
> + if (atomic_inc_return(&vpu->core_count) > 1)
> + return 0;
> +
> + ret = pm_runtime_resume_and_get(vpu->dev);
> + if (ret)
> + goto error_pm;
> +
> + ret = wave6_vpu_try_boot(vpu, core_base);
> + if (ret)
> + goto error_boot;
> +
> + return 0;
> +
> +error_boot:
> + pm_runtime_put_sync(vpu->dev);
> +error_pm:
> + atomic_dec(&vpu->core_count);
> +
> + return ret;
> +}
[Severity: Low]
This isn't a bug, but wave6_vpu_get() mixes guard(mutex) with goto-based
error unwinding in the same function, contrary to the Cleanup API
conventions in include/linux/cleanup.h.
[ ... ]
> +void wave6_vpu_require_work_buffer(struct wave6_vpu_device *vpu,
> + void __iomem *core_base)
> +{
> + struct vpu_buf *vb;
> + u32 size;
> +
> + size = wave6_vdi_readl(core_base, W6_CMD_SET_WORK_BUF_SIZE);
> + if (!size)
> + return;
> +
> + if (WARN_ON(size > W637DEC_WORKBUF_SIZE_FOR_CQ))
> + goto exit;
> +
> + if (WARN_ON(vpu->work_buffers_avail == 0))
> + goto exit;
> +
> + vpu->work_buffers_avail--;
> + vb = &vpu->work_buffers[vpu->work_buffers_avail];
> +
> + wave6_vdi_writel(core_base, W6_CMD_SET_WORK_BUF_ADDR, vb->daddr);
> +
> +exit:
> + wave6_vdi_writel(core_base, W6_CMD_SET_WORK_BUF_SIZE, SET_WORK_BUF_SIZE_ACK);
> +}
[Severity: High]
Can concurrent calls to wave6_vpu_require_work_buffer() across VPU cores
race on vpu->work_buffers_avail?
In commit "media: chips-media: wave6: Add Wave6 core driver", each child
core device calls wave6_vpu_require_work_buffer() directly from its
wave6_vpu_core_irq() hardirq handler without holding any shared lock:
CPU0 (<hardirq> core 0) CPU1 (<hardirq> core 1)
wave6_vpu_core_irq() wave6_vpu_core_irq()
wave6_vpu_require_work_buffer() wave6_vpu_require_work_buffer()
if (work_buffers_avail == 0) if (work_buffers_avail == 0)
work_buffers_avail--; work_buffers_avail--;
vb = &work_buffers[avail]; vb = &work_buffers[avail];
<-- duplicate buffer or underflow
Two cores can be assigned the same work_buffers[] entry, or when
vpu->work_buffers_avail is 1, both cores can pass the zero check and
underflow the u32 counter to 0xffffffff, reading vpu->work_buffers[] out of
bounds.
Also, when jumping to exit on error, should W6_CMD_SET_WORK_BUF_ADDR be
cleared before writing SET_WORK_BUF_SIZE_ACK so the firmware does not
consume a stale address?
> +static int wave6_vpu_create_cores(struct wave6_vpu_device *vpu)
> +{
> + struct device_node *child;
> + int num_cores = 0;
> +
> + for_each_available_child_of_node(vpu->dev->of_node, child) {
> + struct platform_device_info info = {};
> + struct platform_device *pdev;
> + struct resource res[2];
> + int irq;
> +
> + if (num_cores >= W6_VPU_MAX_NUM_CORE) {
> + of_node_put(child);
> + break;
> + }
> +
> + if (of_address_to_resource(child, 0, &res[0])) {
> + dev_warn(vpu->dev, "%pOF: missing reg property\n", child);
> + continue;
> + }
> +
> + irq = of_irq_get(child, 0);
> + if (irq < 0) {
> + dev_warn(vpu->dev, "%pOF: missing interrupts property\n", child);
> + continue;
> + }
> + res[1] = DEFINE_RES_IRQ(irq);
[Severity: Low]
Should the error check for of_irq_get() be "if (irq <= 0)"?
of_irq_get() returns 0 when IRQ mapping fails, which would register the
child platform device with an invalid IRQ 0 resource.
[ ... ]
> +static void wave6_vpu_release(struct wave6_vpu_device *vpu)
> +{
> + guard(mutex)(&vpu->lock);
> +
> + wave6_vpu_destroy_cores(vpu);
> + wave6_vpu_free_work_buffers(vpu);
[ ... ]
> +}
> +
> +static void wave6_vpu_init_firmware(struct wave6_vpu_device *vpu,
> + const struct firmware *fw)
> +{
> + guard(mutex)(&vpu->lock);
> +
> + if (!fw || !fw->data) {
> + dev_err(vpu->dev, "No firmware.\n");
> + return;
> + }
> +
> + if (fw->size + W6_EXTRA_CODE_BUF_SIZE > wave6_vpu_get_code_buf_size(vpu)) {
> + dev_err(vpu->dev, "firmware size (%zd > %zd) is too big\n",
> + fw->size, vpu->code_buf.size);
> + return;
> + }
> +
> + memcpy(vpu->code_buf.vaddr, fw->data, fw->size);
> +
> + if (!wave6_vpu_create_cores(vpu))
> + dev_err(vpu->dev, "Failed to create VPU cores\n");
> +}
[Severity: High]
Does holding vpu->lock across wave6_vpu_create_cores() and
wave6_vpu_destroy_cores() invert the lock hierarchy against child core
device locks and runtime PM transitions?
wave6_vpu_create_cores() -> platform_device_register_full() and
wave6_vpu_destroy_cores() -> platform_device_unregister() acquire the child
core device's device_lock (core_dev->mutex) while vpu->lock is held, and
platform_device_unregister() -> __device_release_driver() also runs
pm_runtime_get_sync(core_dev) and pm_runtime_put_sync(core_dev). In commit
"media: chips-media: wave6: Add Wave6 core driver", the child's runtime PM
and system sleep callbacks call wave6_vpu_get() and wave6_vpu_put(), which
acquire vpu->lock under core_dev->mutex or during a core_dev runtime PM
transition:
CPU0 CPU1
---- ----
lock(&vpu->lock)
lock(&core_dev->mutex)
lock(&core_dev->mutex)
lock(&vpu->lock)
*** DEADLOCK ***
CPU0: wave6_vpu_init_firmware() -> wave6_vpu_create_cores() ->
platform_device_register_full()
CPU1: device_suspend() -> wave6_vpu_core_suspend() ->
pm_runtime_force_suspend() -> wave6_vpu_core_runtime_suspend() ->
wave6_vpu_put()
During teardown, if core_dev is in RPM_RESUMING or RPM_SUSPENDING waiting
for vpu->lock in wave6_vpu_get() or wave6_vpu_put() when
wave6_vpu_release() runs, __device_release_driver()'s
pm_runtime_get_sync(core_dev) will also wait on core_dev->power.wait_queue
while holding vpu->lock.
[ ... ]
> + np = of_parse_phandle(pdev->dev.of_node, "memory-region", 0);
> + if (np) {
> + struct resource mem;
> +
> + ret = of_address_to_resource(np, 0, &mem);
> + of_node_put(np);
> + if (!ret) {
> + vpu->code_buf.phys_addr = mem.start;
> + vpu->code_buf.size = resource_size(&mem);
> + ret = wave6_vpu_init_code_buf(vpu);
> + if (ret)
> + goto error_detach_pm_domains;
> + } else {
> + dev_warn(&pdev->dev, "memory-region is not available.\n");
> + }
> + }
[Severity: Medium]
Should wave6_vpu_probe() fail and jump to error_detach_pm_domains when
"memory-region" is missing or of_address_to_resource() fails?
Without it, vpu->code_buf.size remains 0 while wave6_vpu_probe() continues,
allocates 48 MiB of coherent DMA work buffers and SRAM, and returns 0, only
for wave6_vpu_init_firmware() to unconditionally fail its size check against
wave6_vpu_get_code_buf_size(vpu) and leave those resources allocated without
creating any core devices.
> + vpu->sram_pool = of_gen_pool_get(pdev->dev.of_node, "sram", 0);
> + if (vpu->sram_pool) {
> + vpu->sram_buf.size = vpu->res->sram_size;
> + vpu->sram_buf.vaddr = gen_pool_dma_alloc(vpu->sram_pool,
> + vpu->sram_buf.size,
> + &vpu->sram_buf.phys_addr);
[Severity: Medium]
Does passing &vpu->sram_buf.phys_addr to gen_pool_dma_alloc() fail to build
when phys_addr_t and dma_addr_t have different widths?
struct wave6_vpu_dma_buf declares phys_addr as phys_addr_t, whereas
gen_pool_dma_alloc() takes a dma_addr_t * third argument, which triggers
-Werror=incompatible-pointer-types on configs where
CONFIG_PHYS_ADDR_T_64BIT and CONFIG_ARCH_DMA_ADDR_T_64BIT differ (exposed by
COMPILE_TEST).
> diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h b/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
> new file mode 100644
> index 0000000000000..1d4e016c2a582
> --- /dev/null
> +++ b/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
> @@ -0,0 +1,37 @@
[ ... ]
> +#define WAVE617_CODE 0x6170
> +#define WAVE627_CODE 0x6270
> +#define WAVE633_CODE 0x6330
> +#define WAVE637_CODE 0x6370
> +#define WAVE663_CODE 0x6630
> +#define WAVE677_CODE 0x6670
[Severity: Medium]
Should WAVE677_CODE be 0x6770 instead of 0x6670?
All other Wave6 product codes encode the three-digit model number in bits
[15:4], so wave6_is_product_w_series() in wave6_vpu_try_boot() would reject
a Wave677 reporting 0x6770 with -EINVAL.
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795075577.git.nas.chung@chipsnmedia.com?part=2
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v8 05/11] media: chips-media: wave6: Add Wave6 m2m decoder
2026-11-19 8:59 ` [PATCH v8 05/11] media: chips-media: wave6: Add Wave6 m2m decoder Nas Chung
@ 2026-11-19 9:31 ` sashiko-bot
0 siblings, 0 replies; 23+ messages in thread
From: sashiko-bot @ 2026-11-19 9:31 UTC (permalink / raw)
To: Nas Chung; +Cc: devicetree, imx, robh, conor+dt, Frank.Li, media-ci
Thank you for your contribution! Sashiko AI review found 13 potential
issues to consider:
- [High] media: wave6: infinite job loop on failure in
wave6_vpu_dec_seek_header()
- [High] media: wave6: missing CAPTURE buffer size check after format
update
- [High] media: wave6: use-after-free of m2m_ctx in
wave6_vpu_dec_release()
- [Medium] media: wave6: unhandled query failure in
wave6_vpu_dec_finish_decode()
- [Medium] media: wave6: fbc_buf_registered not reset in
wave6_vpu_dec_release_fb()
- [Medium] media: wave6: zero pixelformat matched in
wave6_find_vpu_fmt()
- [Medium] media: wave6: skipped FBC setup in
wave6_vpu_dec_start_streaming() after seek
- [Medium] media: wave6: duplicate display buffer entries in
p_dec_info->disp_buf[]
- [Medium] media: wave6: infinite job rescheduling loop on
DEC_NOTI_FLAG_NO_FB
- [Medium] media: wave6: race between wave6_vpu_dec_finish_decode() and
timeout work
- [Medium] media: wave6: unlocked cap_q_ctx.rdy_queue traversal races
with queue ops
- [Medium] media: wave6: unlocked state updates and early event in
source change
- [Medium] media: wave6: ignored buffer registration errors in
prepare_decode()
--
commit 2d9ead557a0ff6a4de5fd71e8b36d2fcff95e3a1
Author: Nas Chung <nas.chung@chipsnmedia.com>
media: chips-media: wave6: Add Wave6 m2m decoder
This commit adds the V4L2 m2m stateful decoder driver for the Chips&Media
Wave6 video codec IP, supporting H.264 and HEVC 8-bit decoding, downscaling,
and dynamic resolution changes.
> diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
> new file mode 100644
> index 0000000000000..54b408922bbc2
> --- /dev/null
> +++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
> @@ -0,0 +1,1935 @@
[ ... ]
> +static const struct vpu_format wave6_vpu_dec_fmt_list[2][6] = {
> + [VPU_FMT_TYPE_CODEC] = {
> + {
> + .v4l2_pix_fmt = V4L2_PIX_FMT_HEVC,
> + .max_width = W6_MAX_DEC_PIC_WIDTH,
> + .min_width = W6_MIN_DEC_PIC_WIDTH,
> + .max_height = W6_MAX_DEC_PIC_HEIGHT,
> + .min_height = W6_MIN_DEC_PIC_HEIGHT,
> + .num_planes = 1,
> + },
> + {
> + .v4l2_pix_fmt = V4L2_PIX_FMT_H264,
> + .max_width = W6_MAX_DEC_PIC_WIDTH,
> + .min_width = W6_MIN_DEC_PIC_WIDTH,
> + .max_height = W6_MAX_DEC_PIC_HEIGHT,
> + .min_height = W6_MIN_DEC_PIC_HEIGHT,
> + .num_planes = 1,
> + },
> + },
[ ... ]
> +static const struct vpu_format *wave6_find_vpu_fmt(unsigned int v4l2_pix_fmt,
> + enum vpu_fmt_type type)
> +{
> + for (int i = 0; i < ARRAY_SIZE(wave6_vpu_dec_fmt_list[type]); i++) {
> + if (wave6_vpu_dec_fmt_list[type][i].v4l2_pix_fmt == v4l2_pix_fmt)
> + return &wave6_vpu_dec_fmt_list[type][i];
> + }
> +
> + return NULL;
> +}
[Severity: Medium]
Can wave6_find_vpu_fmt() match an unpopulated entry in
wave6_vpu_dec_fmt_list[VPU_FMT_TYPE_CODEC] when v4l2_pix_fmt is 0?
VPU_FMT_TYPE_CODEC only populates 2 of its 6 slots, leaving slots 2..5
zero-initialized. Passing pixelformat == 0 to VIDIOC_S_FMT on OUTPUT
matches slot 2 instead of falling back to inst->src_fmt, which sets
inst->src_fmt.num_planes and plane sizes to 0 and triggers WARN_ON() in
vb2_core_reqbufs() on a subsequent VIDIOC_REQBUFS.
[ ... ]
> +static void wave6_vpu_dec_release_fb(struct vpu_instance *inst)
> +{
> + for (int i = 0; i < WAVE6_MAX_FBS; i++) {
> + wave6_vdi_free_dma(&inst->frame_vbuf[i]);
> + memset(&inst->frame_buf[i], 0, sizeof(struct frame_buffer));
> + wave6_vdi_free_dma(&inst->aux_vbuf[AUX_BUF_FBC_Y_TBL][i]);
> + wave6_vdi_free_dma(&inst->aux_vbuf[AUX_BUF_FBC_C_TBL][i]);
> + wave6_vdi_free_dma(&inst->aux_vbuf[AUX_BUF_MV_COL][i]);
> + }
> +}
> +
> +static void wave6_vpu_dec_destroy_instance(struct vpu_instance *inst)
> +{
> + u32 fail_res = 0;
> + int ret;
> +
> + wave6_vpu_disable_instance(inst);
> +
> + ret = wave6_vpu_dec_close(inst, &fail_res);
> + if (ret) {
> + dev_err(inst->dev->dev, "failed destroy instance: %d (%d)\n",
> + ret, fail_res);
> + }
> +
> + scoped_guard(spinlock, &inst->dev->inst_lock)
> + list_del_init(&inst->list);
> + wave6_vpu_dec_release_fb(inst);
> + wave6_vpu_set_instance_state(inst, VPU_INST_STATE_NONE);
> +
> + pm_runtime_put_sync(inst->dev->dev);
> + wave6_vpu_core_deactivate(inst->dev);
> +}
[Severity: Medium]
Should wave6_vpu_dec_release_fb() or wave6_vpu_dec_destroy_instance()
reset inst->fbc_buf_registered to false?
When wave6_vpu_dec_queue_setup() destroys the instance in
VPU_INST_STATE_SEEK and a new instance is created on the next OUTPUT
STREAMON, inst->fbc_buf_registered remains true from the destroyed
instance even though all FBC buffers were freed. If V4L2_DEC_CMD_STOP is
then issued before a sequence header is parsed,
wave6_vpu_dec_prepare_decode() skips the !inst->fbc_buf_registered early
drain exit and issues W6_CMD_INIT_SEQ on an empty bitstream.
[ ... ]
> +static void wave6_vpu_dec_handle_dst_buffer(struct vpu_instance *inst)
> +{
> + struct vb2_v4l2_buffer *dst_buf;
> + struct v4l2_m2m_buffer *v4l2_m2m_buf;
> + struct vpu_buffer *vpu_buf;
> + u32 fb_stride = inst->dst_fmt.plane_fmt[0].bytesperline;
> + u32 luma_size = fb_stride * inst->dst_fmt.height;
> + u32 chroma_size = (fb_stride / 2) * (inst->dst_fmt.height / 2);
> + struct frame_buffer disp_buffer = { };
> + struct dec_seq_info *seq_info = &inst->codec_info->dec_info.seq_info;
> + int consumed_num = wave6_vpu_get_consumed_fb_num(inst);
> + int ret;
> +
> + v4l2_m2m_for_each_dst_buf(inst->v4l2_fh.m2m_ctx, v4l2_m2m_buf) {
[Severity: Medium]
What serializes v4l2_m2m_for_each_dst_buf() against concurrent list
modifications on cap_q_ctx.rdy_queue?
v4l2_m2m_for_each_dst_buf() iterates cap_q_ctx.rdy_queue without holding
cap_q_ctx.rdy_spinlock:
kworker CPU0 (VIDIOC_QBUF)
wave6_vpu_device_run()
wave6_vpu_dec_prepare_decode()
wave6_vpu_dec_handle_dst_buffer()
v4l2_m2m_for_each_dst_buf()
wave6_vpu_dec_buf_queue_dst()
guard(mutex)(&inst->lock)
v4l2_m2m_buf_queue()
list_add_tail(&b->list, ...)
/* unlocked list traversal */
Similarly, wave6_vpu_core_irq_thread() traverses cap_q_ctx.rdy_queue
unlocked in wave6_get_dst_buf_by_addr() and calls
v4l2_m2m_dst_buf_remove_by_buf(), which can race with
wave6_vpu_dec_buf_queue_dst() or with v4l2_m2m_for_each_dst_buf() in
wave6_vpu_dec_cmd_start() and corrupt the list traversal.
> + dst_buf = &v4l2_m2m_buf->vb;
> + vpu_buf = wave6_to_vpu_buf(dst_buf);
> +
> + if (vpu_buf->consumed)
> + continue;
> +
> + if (consumed_num >= WAVE6_MAX_FBS)
> + break;
> +
> + if (inst->dst_fmt.num_planes == 1) {
> + disp_buffer.buf_y = wave6_get_dma_addr(dst_buf, 0);
> + disp_buffer.buf_cb = disp_buffer.buf_y + luma_size;
> + disp_buffer.buf_cr = disp_buffer.buf_cb + chroma_size;
> + } else if (inst->dst_fmt.num_planes == 2) {
> + disp_buffer.buf_y = wave6_get_dma_addr(dst_buf, 0);
> + disp_buffer.buf_cb = wave6_get_dma_addr(dst_buf, 1);
> + disp_buffer.buf_cr = disp_buffer.buf_cb + chroma_size;
> + } else if (inst->dst_fmt.num_planes == 3) {
> + disp_buffer.buf_y = wave6_get_dma_addr(dst_buf, 0);
> + disp_buffer.buf_cb = wave6_get_dma_addr(dst_buf, 1);
> + disp_buffer.buf_cr = wave6_get_dma_addr(dst_buf, 2);
> + }
> + disp_buffer.height = inst->src_fmt.height;
> + disp_buffer.stride = fb_stride;
> + disp_buffer.map_type = LINEAR_FRAME_MAP;
> + disp_buffer.luma_bit_depth = seq_info->luma_bit_depth;
> + disp_buffer.chroma_bit_depth = seq_info->chroma_bit_depth;
> + disp_buffer.c_fmt_idc = seq_info->c_fmt_idc;
> +
> + ret = wave6_vpu_dec_register_display_buffer(inst, disp_buffer);
> + if (ret) {
> + dev_err(inst->dev->dev, "fail register display buffer %d", ret);
> + break;
> + }
> +
> + vpu_buf->consumed = true;
> + consumed_num++;
> + }
[Severity: Medium]
Does re-queuing a displayed CAPTURE buffer insert duplicate entries into
p_dec_info->disp_buf[] until all WAVE6_MAX_FBS slots are exhausted?
When a displayed buffer is dequeued and re-queued via VIDIOC_QBUF,
wave6_vpu_dec_buf_queue() resets vpu_buf->consumed = false, so
wave6_vpu_dec_handle_dst_buffer() calls
wave6_vpu_dec_register_display_buffer() again. In commit "media:
chips-media: wave6: Add Wave6 decoder firmware interface",
wave6_dec_set_display_buffer() always writes to the first empty
!disp_buf[i].buf_y slot without checking if fb.buf_y is already present,
and wave6_vpu_dec_get_output_info() only removes buffers on
W6_RET_DEC_RELEASE_IDC, not W6_RET_DEC_DISP_IDC:
drivers/media/platform/chips-media/wave6/wave6-hw.c:wave6_dec_set_display_buffer() {
for (int i = 0; i < WAVE6_MAX_FBS; i++) {
if (!p_dec_info->disp_buf[i].buf_y) {
p_dec_info->disp_buf[i] = fb;
break;
}
}
}
After 31 displayed frames are re-queued, p_dec_info->disp_buf[] is full of
duplicates and cannot record any new buffer address.
> +
> + if (!inst->fbc_buf_registered) {
> + ret = wave6_vpu_dec_register_frame_buffer(inst);
> + if (ret)
> + dev_err(inst->dev->dev, "register frame buffer fail %d\n", ret);
> + else
> + inst->fbc_buf_registered = true;
> + }
> +}
[Severity: Medium]
Should wave6_vpu_dec_handle_dst_buffer() propagate registration errors to
wave6_vpu_dec_prepare_decode()?
If wave6_vpu_dec_register_frame_buffer() or
wave6_vpu_dec_register_display_buffer() fails, the error is logged and
dropped, and wave6_vpu_dec_prepare_decode() still returns 0.
wave6_vpu_device_run() then arms inst->task_timer and calls
wave6_vpu_dec_start_decode() without registered FBC or display buffers.
[ ... ]
> +static void wave6_vpu_dec_handle_source_change(struct vpu_instance *inst)
> +{
> + static const struct v4l2_event vpu_event_src_ch = {
> + .type = V4L2_EVENT_SOURCE_CHANGE,
> + .u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION,
> + };
> + struct dec_seq_info *info = &inst->codec_info->dec_info.seq_info;
> +
> + dev_dbg(inst->dev->dev, "pic size %dx%d profile %d, min_fb_cnt : %d | min_disp_cnt : %d\n",
> + info->pic_width, info->pic_height,
> + info->profile, info->min_frame_buffer_count, info->frame_buf_delay);
> +
> + wave6_vpu_dec_retry_one_frame(inst);
> + wave6_vpu_dec_reset_frame_buffer_info(inst);
> + inst->fbc_buf_registered = false;
> +
> + wave6_vpu_set_instance_state(inst, VPU_INST_STATE_INIT_SEQ);
> +
> + inst->scaler_info.enable = false;
> + inst->crop.left = info->pic_crop_rect.left;
> + inst->crop.top = info->pic_crop_rect.top;
> + inst->crop.width = info->pic_crop_rect.right - inst->crop.left;
> + inst->crop.height = info->pic_crop_rect.bottom - inst->crop.top;
> +
> + wave6_update_v4l2_ctrls(inst, info);
> + wave6_update_color_info(inst, info);
> + wave6_update_pix_fmt(&inst->src_fmt, info->pic_width, info->pic_height);
> + wave6_update_pix_fmt_cap(&inst->dst_fmt,
> + info->pic_width, info->pic_height,
> + true);
> +
> + v4l2_event_queue_fh(&inst->v4l2_fh, &vpu_event_src_ch);
> +}
[Severity: Medium]
Can queuing V4L2_EVENT_SOURCE_CHANGE inside
wave6_vpu_dec_handle_source_change() wake userspace before the caller
finishes marking the last frame?
CPU0 (irq_thread / kworker) CPU1 (userspace)
wave6_vpu_dec_seek_header()
wave6_vpu_dec_handle_source_change()
/* updates formats unlocked */
v4l2_event_queue_fh(...)
/* wakes on SOURCE_CHANGE event */
VIDIOC_STREAMON(CAPTURE)
VIDIOC_QBUF(CAPTURE)
if (vb2_is_streaming(dst_vq))
wave6_handle_last_frame() <-- tags new buffer with LAST
In addition, wave6_vpu_dec_handle_source_change() modifies inst->state,
inst->crop, inst->src_fmt, and inst->dst_fmt without holding
core->dev_lock, so a concurrent VIDIOC_G_FMT or VIDIOC_TRY_FMT can observe
partially updated format fields (such as bytesperline temporarily set to
0). Queuing the event after wave6_handle_last_frame() and serializing the
format update would avoid this race.
[ ... ]
> +static void wave6_vpu_dec_finish_decode(struct vpu_instance *inst, bool error)
> +{
> + struct dec_output_info info;
> + int ret;
> +
> + memset(&info, 0, sizeof(info));
> +
> + ret = wave6_vpu_dec_get_output_info(inst, &info);
> + if (ret)
> + goto finish_decode;
[Severity: Medium]
What synchronizes wave6_vpu_dec_finish_decode() against inst->task_timer
(wave6_vpu_device_run_timeout())?
wave6_vpu_dec_finish_decode() does not cancel inst->task_timer on entry;
wave6_vpu_finish_job() only calls non-synchronous cancel_delayed_work() at
the very end:
CPU0 (irq_thread) CPU1 (kworker)
wave6_vpu_dec_finish_decode() wave6_vpu_device_run_timeout()
wave6_handle_decoded_frame()
src_buf = v4l2_m2m_next_src_buf()
src_buf = v4l2_m2m_src_buf_remove()
v4l2_m2m_buf_done(src_buf, ...)
wave6_vpu_dec_src_buf_done()
v4l2_m2m_src_buf_remove_by_buf()
v4l2_m2m_buf_done(src_buf) <-- double remove and buf_done
If the timeout fires while the completion interrupt is being handled, both
paths remove and complete the same source and destination buffers and both
call v4l2_m2m_job_finish().
[Severity: Medium]
If wave6_vpu_dec_get_output_info() fails here, does jumping to
finish_decode leave the consumed head OUTPUT buffer on the ready queue?
Unlike wave6_vpu_dec_start_decode(), this error path neither removes and
completes the head src_buf with VB2_BUF_STATE_ERROR nor transitions
inst->state to VPU_INST_STATE_ERROR, so v4l2_m2m_job_finish() can schedule
the next job with the already-consumed buffer still at the head of the
queue.
> +
> + dev_dbg(inst->dev->dev, "dec %d dis %d noti_flag %d stream_end %d\n",
> + info.frame_decoded, info.frame_display,
> + info.notification_flags, info.stream_end);
> +
> + if (info.notification_flags & DEC_NOTI_FLAG_NO_FB) {
> + wave6_vpu_dec_retry_one_frame(inst);
> + goto finish_decode;
> + }
[Severity: Medium]
Does DEC_NOTI_FLAG_NO_FB cause an immediate job rescheduling busy loop?
In wave6_vpu_dec_handle_dst_buffer(), queued CAPTURE buffers have
vpu_buf->consumed set to true while vpu_buf->used remains false until
wave6_handle_decoded_frame(). When DEC_NOTI_FLAG_NO_FB jumps to
finish_decode without setting vpu_buf->used, wave6_vpu_check_fb_available()
in commit "media: chips-media: wave6: Add Wave6 core driver" still sees
!vpu_buf->used and reports the job as ready:
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c:wave6_vpu_check_fb_available() {
v4l2_m2m_for_each_dst_buf(inst->v4l2_fh.m2m_ctx, v4l2_m2m_buf) {
vb2_v4l2_buf = &v4l2_m2m_buf->vb;
vpu_buf = wave6_to_vpu_buf(vb2_v4l2_buf);
if (!vpu_buf->used)
return true;
}
}
v4l2_m2m_job_finish() then immediately re-runs the job without waiting for
a new CAPTURE buffer, and wave6_vpu_dec_handle_dst_buffer() skips all
existing CAPTURE buffers because their vpu_buf->consumed is already true.
[ ... ]
> +static int wave6_vpu_dec_s_fmt_out(struct file *file, void *fh, struct v4l2_format *f)
> +{
[ ... ]
> + wave6_update_pix_fmt_cap(&inst->dst_fmt,
> + pix_mp->width, pix_mp->height,
> + true);
> +
> + return 0;
> +}
[Severity: High]
Can updating inst->dst_fmt in wave6_vpu_dec_s_fmt_out() or
wave6_vpu_dec_handle_source_change() allow undersized CAPTURE buffers to
be registered for DMA?
If userspace allocates and queues (or prepares via VIDIOC_PREPARE_BUF)
CAPTURE buffers at the default 720x480 format before CAPTURE STREAMON,
wave6_vpu_dec_buf_prepare() validates their sizes against 720x480 and marks
them prepared. If inst->dst_fmt is then increased (up to 4096x4096) via
VIDIOC_S_FMT on OUTPUT or sequence header parsing, VIDIOC_STREAMON on
CAPTURE calls wave6_vpu_dec_start_streaming() -> wave6_vpu_dec_prepare_fb()
without re-checking the prepared buffers' plane sizes against the new
inst->dst_fmt.plane_fmt[i].sizeimage (unlike wave6_vpu_dec_cmd_start()).
wave6_vpu_dec_handle_dst_buffer() then registers those smaller DMA buffers
with the hardware using the new stride and height, causing out-of-bounds
DMA writes.
[ ... ]
> +static int wave6_vpu_dec_seek_header(struct vpu_instance *inst)
> +{
> + struct dec_seq_info seq_info;
> + int ret;
> +
> + memset(&seq_info, 0, sizeof(struct dec_seq_info));
> +
> + ret = wave6_vpu_dec_issue_seq_init(inst);
> + if (ret) {
> + dev_err(inst->dev->dev, "failed wave6_vpu_dec_issue_seq_init %d\n", ret);
> + return ret;
> + }
> +
> + ret = wave6_vpu_wait_interrupt(inst, W6_VPU_TIMEOUT);
> + if (ret) {
> + dev_err(inst->dev->dev, "failed to call vpu_wait_interrupt()\n");
> + return ret;
> + }
> +
> + ret = wave6_vpu_dec_complete_seq_init(inst, &seq_info);
> + if (ret) {
> + dev_err(inst->dev->dev, "vpu_dec_complete_seq_init: %d, reason : 0x%x\n",
> + ret, seq_info.err_reason);
> + if ((seq_info.err_reason & WAVE6_SYSERR_NOT_SUPPORT) ||
> + (seq_info.err_reason & WAVE6_SYSERR_NOT_SUPPORT_PROFILE)) {
> + ret = -EINVAL;
> + } else if (seq_info.err_reason == HEVC_ETCERR_INIT_SEQ_SPS_NOT_FOUND ||
> + seq_info.err_reason == AVC_ETCERR_INIT_SEQ_SPS_NOT_FOUND) {
> + wave6_handle_skipped_frame(inst);
> + ret = 0;
> + }
> + } else {
> + wave6_vpu_dec_handle_source_change(inst);
> + inst->v4l2_fh.m2m_ctx->ignore_cap_streaming = false;
> + v4l2_m2m_set_dst_buffered(inst->v4l2_fh.m2m_ctx, false);
> + if (vb2_is_streaming(v4l2_m2m_get_dst_vq(inst->v4l2_fh.m2m_ctx))) {
> + scoped_guard(mutex, &inst->lock)
> + wave6_handle_last_frame(inst, NULL);
> + }
> + }
> +
> + return ret;
> +}
> +
> +static int wave6_vpu_dec_prepare_decode(struct vpu_instance *inst)
> +{
> + bool stream_end;
> +
> + scoped_guard(mutex, &inst->lock)
> + stream_end = inst->drain && !inst->drain_src_buf_num;
> +
> + if (stream_end && !inst->fbc_buf_registered) {
> + wave6_vpu_dec_finish_drain(inst);
> + return -EAGAIN;
> + }
> +
> + wave6_handle_bitstream_buffer(inst, stream_end);
> +
> + if (inst->state < VPU_INST_STATE_PIC_RUN) {
> + if (inst->state == VPU_INST_STATE_OPEN &&
> + wave6_vpu_dec_seek_header(inst)) {
> + vb2_queue_error(v4l2_m2m_get_src_vq(inst->v4l2_fh.m2m_ctx));
> + vb2_queue_error(v4l2_m2m_get_dst_vq(inst->v4l2_fh.m2m_ctx));
> + }
> +
> + return -EAGAIN;
> + }
[Severity: High]
Does a failure in wave6_vpu_dec_seek_header() trigger an infinite m2m job
loop?
When wave6_vpu_dec_seek_header() fails with any error other than
SPS_NOT_FOUND (such as WAVE6_SYSERR_NOT_SUPPORT_PROFILE on a 10-bit stream
or a timeout), the consumed head OUTPUT buffer is not removed from the
ready queue and inst->state remains VPU_INST_STATE_OPEN. Calling
vb2_queue_error() only sets q->error without stopping streaming, and
wave6_vpu_job_ready() returns 1 whenever inst->state == VPU_INST_STATE_OPEN.
When wave6_vpu_dec_prepare_decode() returns -EAGAIN,
wave6_vpu_device_run() calls v4l2_m2m_job_finish(), which immediately
re-queues and re-runs the job in an unbounded loop on the same buffer.
[ ... ]
> +static int wave6_vpu_dec_start_streaming(struct vb2_queue *q, unsigned int count)
> +{
> + struct vpu_instance *inst = vb2_get_drv_priv(q);
> + struct v4l2_pix_format_mplane *fmt;
> + int ret = 0;
> +
> + if (V4L2_TYPE_IS_OUTPUT(q->type)) {
> + fmt = &inst->src_fmt;
> + if (inst->state == VPU_INST_STATE_NONE) {
> + ret = wave6_vpu_dec_create_instance(inst);
> + if (!ret) {
> + inst->v4l2_fh.m2m_ctx->ignore_cap_streaming = true;
> + v4l2_m2m_set_dst_buffered(inst->v4l2_fh.m2m_ctx, true);
> + }
> + }
> + if (inst->state == VPU_INST_STATE_SEEK)
> + wave6_vpu_set_instance_state(inst, inst->state_in_seek);
> + } else {
> + fmt = &inst->dst_fmt;
> + if (inst->state == VPU_INST_STATE_INIT_SEQ) {
> + ret = wave6_vpu_dec_prepare_fb(inst);
> + if (ret)
> + goto exit;
> + }
> + }
[Severity: Medium]
Can wave6_vpu_dec_prepare_fb() be skipped if CAPTURE streaming starts while
inst->state is VPU_INST_STATE_SEEK?
If OUTPUT streaming is stopped while in VPU_INST_STATE_INIT_SEQ,
wave6_vpu_dec_stop_streaming() saves inst->state_in_seek =
VPU_INST_STATE_INIT_SEQ and sets inst->state = VPU_INST_STATE_SEEK. If
userspace then starts CAPTURE streaming before restarting OUTPUT streaming,
CAPTURE start_streaming sees inst->state == VPU_INST_STATE_SEEK and skips
wave6_vpu_dec_prepare_fb(), and OUTPUT start_streaming only restores
inst->state to VPU_INST_STATE_INIT_SEQ without calling
wave6_vpu_dec_prepare_fb(), leaving decoding permanently stalled in
VPU_INST_STATE_INIT_SEQ.
Also, in wave6_vpu_dec_try_fmt_cap(), checking inst->state >=
VPU_INST_STATE_INIT_SEQ evaluates to true in VPU_INST_STATE_SEEK even when
inst->state_in_seek was VPU_INST_STATE_OPEN.
[ ... ]
> +static int wave6_vpu_dec_release(struct file *filp)
> +{
> + struct vpu_instance *inst = wave6_file_to_vpu_inst(filp);
> +
> + v4l2_m2m_ctx_release(inst->v4l2_fh.m2m_ctx);
> +
> + if (inst->state != VPU_INST_STATE_NONE)
> + wave6_vpu_dec_destroy_instance(inst);
[Severity: High]
Can releasing inst->v4l2_fh.m2m_ctx before wave6_vpu_dec_destroy_instance()
cause a use-after-free on m2m_ctx?
v4l2_m2m_ctx_release() waits only for TRANS_RUNNING to clear and then frees
m2m_ctx, whereas wave6_vpu_disable_instance() (which sets inst->enable =
false, calls synchronize_irq(), and calls cancel_delayed_work_sync() on
inst->task_timer) only runs afterward inside
wave6_vpu_dec_destroy_instance():
CPU0 CPU1 (irq_thread / kworker)
wave6_vpu_dec_release()
wave6_vpu_device_run_timeout()
v4l2_m2m_job_finish()
/* clears TRANS_RUNNING */
v4l2_m2m_ctx_release()
kfree(m2m_ctx)
wave6_vpu_core_irq_thread()
wave6_vpu_dec_finish_decode()
...
v4l2_m2m_next_src_buf(m2m_ctx)
<-- use-after-free
wave6_vpu_dec_destroy_instance()
wave6_vpu_disable_instance()
A similar race happens if wave6_vpu_finish_job() calls non-synchronous
cancel_delayed_work(&inst->task_timer) and clears TRANS_RUNNING while
wave6_vpu_device_run_timeout() is already running on another CPU. Calling
wave6_vpu_disable_instance() before v4l2_m2m_ctx_release() would ensure the
IRQ thread and timeout work cannot touch m2m_ctx after it is freed.
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795075577.git.nas.chung@chipsnmedia.com?part=5
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v8 03/11] media: chips-media: wave6: Add Wave6 core driver
2026-11-19 8:59 ` [PATCH v8 03/11] media: chips-media: wave6: Add Wave6 core driver Nas Chung
@ 2026-11-19 9:32 ` sashiko-bot
0 siblings, 0 replies; 23+ messages in thread
From: sashiko-bot @ 2026-11-19 9:32 UTC (permalink / raw)
To: Nas Chung; +Cc: devicetree, imx, robh, conor+dt, Frank.Li, media-ci
Thank you for your contribution! Sashiko AI review found 7 potential
issues to consider:
- [High] media: wave6: data race in wave6_vpu_require_work_buffer()
across cores
- [High] media: wave6: dropped interrupts in wave6_vpu_core_irq() and
irq_thread
- [High] media: wave6: unlocked rdy_queue traversal in
wave6_vpu_check_fb_available()
- [Medium] media: wave6: skipped VPU init and active_count race in
runtime_resume
- [Medium] media: wave6: race and missing error state in
wave6_vpu_device_run_timeout()
- [Medium] media: wave6: stale codec sizeimage in wave6_update_pix_fmt()
- [Medium] media: wave6: vpu->lock inversion and clock leak on core
removal
--
commit 203cdebd14b0043756ee7b19b462b06f101581ec
Author: Nas Chung <nas.chung@chipsnmedia.com>
media: chips-media: wave6: Add Wave6 core driver
This commit adds the core platform driver for the Chips&Media Wave6 video
codec IP to manage the per-core interface register regions, interrupts,
runtime PM, and V4L2 mem2mem helpers.
> diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
> new file mode 100644
> index 0000000000000..67c6666579d92
> --- /dev/null
> +++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
> @@ -0,0 +1,392 @@
[ ... ]
> +static irqreturn_t wave6_vpu_core_irq(int irq, void *dev_id)
> +{
> + struct vpu_core_device *core = dev_id;
> + struct vpu_irq irq_info;
> +
> + if (!vpu_read_reg(core, W6_VPU_VPU_INT_STS))
> + return IRQ_NONE;
> +
> + irq_info.status = vpu_read_reg(core, W6_VPU_VINT_REASON);
> + irq_info.inst_idc = vpu_read_reg(core, W6_RET_INT_INSTANCE_INFO);
> +
> + vpu_write_reg(core, W6_RET_INT_INSTANCE_INFO, INT_INSTANCE_INFO_CLEAR);
> + vpu_write_reg(core, W6_VPU_VINT_REASON_CLEAR, irq_info.status);
> + vpu_write_reg(core, W6_VPU_VINT_CLEAR, VINT_CLEAR);
> +
> + if (irq_info.status & BIT(W6_INT_BIT_REQ_WORK_BUF)) {
> + wave6_vpu_require_work_buffer(core->vpu, core->reg_base);
> +
> + return IRQ_HANDLED;
> + }
[Severity: High]
Can concurrent W6_INT_BIT_REQ_WORK_BUF interrupts on different cores race
on core->vpu->work_buffers_avail in wave6_vpu_require_work_buffer()?
Each core has its own interrupt line, but all cores share the parent
core->vpu:
drivers/media/platform/chips-media/wave6/wave6-vpu.c:wave6_vpu_require_work_buffer() {
...
if (WARN_ON(vpu->work_buffers_avail == 0))
goto exit;
vpu->work_buffers_avail--;
vb = &vpu->work_buffers[vpu->work_buffers_avail];
...
}
CPU0 (core 0 hardirq) CPU1 (core 1 hardirq)
wave6_vpu_core_irq() wave6_vpu_core_irq()
wave6_vpu_require_work_buffer() wave6_vpu_require_work_buffer()
work_buffers_avail == 1 work_buffers_avail == 1
vpu->work_buffers_avail-- (0)
vpu->work_buffers_avail-- (U32_MAX)
&vpu->work_buffers[U32_MAX] <-- OOB read
Even when work_buffers_avail > 1, two cores can read the same decremented
index and assign the same DMA work buffer to two codec instances.
[ ... ]
> +static struct vpu_instance *wave6_vpu_core_get_instance(struct vpu_core_device *core,
> + u32 inst_idc)
> +{
> + struct vpu_instance *inst;
> +
> + guard(spinlock)(&core->inst_lock);
> +
> + list_for_each_entry(inst, &core->instances, list) {
> + if ((BIT(inst->id) & inst_idc) && inst->enable)
> + return inst;
> + }
> +
> + return NULL;
> +}
[ ... ]
> +static irqreturn_t wave6_vpu_core_irq_thread(int irq, void *dev_id)
> +{
> + struct vpu_core_device *core = dev_id;
> + struct vpu_instance *inst;
> + struct vpu_irq irq_info;
> +
> + while (kfifo_len(&core->irq_fifo)) {
> + bool error = false;
> +
> + if (!kfifo_out(&core->irq_fifo, &irq_info, sizeof(struct vpu_irq)))
> + break;
> +
> + inst = wave6_vpu_core_get_instance(core, irq_info.inst_idc);
> + if (!inst)
> + continue;
> +
> + if (irq_info.status & BIT(W6_INT_BIT_INIT_SEQ)) {
> + complete(&inst->irq_done);
> + continue;
> + }
> +
> + if (irq_info.status & BIT(W6_INT_BIT_BSBUF_ERROR))
> + error = true;
> +
> + if (inst->ops && inst->ops->finish_process)
> + inst->ops->finish_process(inst, error);
> + }
> +
> + return IRQ_HANDLED;
> +}
[Severity: High]
Can wave6_vpu_core_irq() and wave6_vpu_core_irq_thread() drop pending
interrupts when multiple status or instance bits are set at once?
In wave6_vpu_core_irq(), W6_RET_INT_INSTANCE_INFO and
W6_VPU_VINT_REASON_CLEAR are cleared for the entire irq_info.status mask
before the BIT(W6_INT_BIT_REQ_WORK_BUF) check. If W6_INT_BIT_REQ_WORK_BUF
is set alongside another bit such as W6_INT_BIT_DEC_PIC or
W6_INT_BIT_INIT_SEQ, returning IRQ_HANDLED early without queuing the
remaining bits to core->irq_fifo loses that completion interrupt.
In addition, irq_info.inst_idc is a bitmask of instance IDs, and
W6_RET_INT_INSTANCE_INFO has already been cleared to 0 in the hardirq
handler. Because wave6_vpu_core_irq_thread() calls
wave6_vpu_core_get_instance() only once per FIFO entry and returns the
first matching instance, any other instances with bits set in
irq_info.inst_idc never receive their completion callback and will time
out.
[ ... ]
> +void wave6_vpu_core_activate(struct vpu_core_device *core)
> +{
> + atomic_inc(&core->active_count);
> +}
> +
> +void wave6_vpu_core_deactivate(struct vpu_core_device *core)
> +{
> + atomic_dec(&core->active_count);
> +}
> +
> +static int wave6_vpu_core_wait_activated(struct vpu_core_device *core)
> +{
> + if (!atomic_read(&core->active_count))
> + return 0;
> +
> + return wave6_vpu_core_check_state(core);
> +}
[ ... ]
> + ret = devm_pm_runtime_enable(&pdev->dev);
> + if (ret)
> + return ret;
> +
> + return 0;
> +}
> +
> +static int wave6_vpu_core_runtime_suspend(struct device *dev)
> +{
> + struct vpu_core_device *core = dev_get_drvdata(dev);
> +
> + if (WARN_ON(!core))
> + return -ENODEV;
> +
> + if (core->vpu_held) {
> + wave6_vpu_put(core->vpu, core->reg_base);
> + core->vpu_held = false;
> + }
> +
> + clk_bulk_disable_unprepare(core->num_clks, core->clks);
> +
> + return 0;
> +}
[Severity: Medium]
Can unregistering child core devices under vpu->lock in
wave6_vpu_release() deadlock with wave6_vpu_core_runtime_suspend() or leak
prepared clocks on teardown?
drivers/media/platform/chips-media/wave6/wave6-vpu.c:wave6_vpu_release() {
guard(mutex)(&vpu->lock);
wave6_vpu_destroy_cores(vpu);
...
}
wave6_vpu_destroy_cores() calls platform_device_unregister() while holding
vpu->lock, which waits for child runtime PM transitions and barriers,
whereas wave6_vpu_core_runtime_suspend() and
wave6_vpu_core_runtime_resume() call wave6_vpu_put() and wave6_vpu_get(),
which acquire vpu->lock:
CPU0 CPU1
---- ----
lock(&vpu->lock)
/* runtime PM transition on core */
/* waits for core runtime PM */
lock(&vpu->lock)
*** DEADLOCK ***
CPU0: wave6_vpu_release() -> wave6_vpu_destroy_cores() -> platform_device_unregister()
CPU1: rpm_suspend() -> wave6_vpu_core_runtime_suspend() -> wave6_vpu_put()
Furthermore, wave6_vpu_core_driver has no remove callback or devm action
to suspend a runtime-active core before devm_clk_bulk_get_all() releases
core->clks. If a core is runtime-active at unbind (for example via sysfs
power/control set to "on"), core->clks are never disabled or unprepared
and any held core->vpu_held reference is leaked.
> +
> +static int wave6_vpu_core_runtime_resume(struct device *dev)
> +{
> + struct vpu_core_device *core = dev_get_drvdata(dev);
> + int ret;
> +
> + if (WARN_ON(!core))
> + return -ENODEV;
> +
> + ret = clk_bulk_prepare_enable(core->num_clks, core->clks);
> + if (ret) {
> + dev_err(dev, "failed to enable clocks: %d\n", ret);
> + return ret;
> + }
> +
> + if (atomic_read(&core->active_count)) {
> + ret = wave6_vpu_get(core->vpu, core->reg_base);
> + if (ret)
> + goto error_get_vpu;
> +
> + core->vpu_held = true;
> + }
> +
> + ret = wave6_vpu_core_wait_activated(core);
> + if (ret)
> + goto error_activate;
[Severity: Medium]
What happens if wave6_vpu_core_runtime_resume() runs while
core->active_count is 0?
If the core device is runtime-resumed before wave6_vpu_core_activate() is
called (for instance when runtime PM is forced on via sysfs
power/control), wave6_vpu_core_runtime_resume() skips both wave6_vpu_get()
and wave6_vpu_core_check_state(), leaving core->vpu_held false. When an
instance later calls wave6_vpu_core_activate() followed by
pm_runtime_resume_and_get(), runtime PM sees the device is already
RPM_ACTIVE and returns 0 without running wave6_vpu_core_runtime_resume(),
so the parent VPU is never powered on or initialized and core interrupts
are never enabled.
Also, because wave6_vpu_core_runtime_resume() and
wave6_vpu_core_wait_activated() read core->active_count twice without
synchronization (rather than checking core->vpu_held in
wave6_vpu_core_wait_activated()), an intervening wave6_vpu_core_activate()
can cause wave6_vpu_core_check_state() to poll VPU registers when
wave6_vpu_get() was skipped.
> diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
> new file mode 100644
> index 0000000000000..75371566de385
> --- /dev/null
> +++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
> @@ -0,0 +1,384 @@
> +// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause)
> +/*
> + * Wave6 series multi-standard codec IP - v4l2 driver helper interface
> + *
> + * Copyright (C) 2026 CHIPS&MEDIA INC
> + */
> +
> +#include "wave6-vpu.h"
> +#include "wave6-vpu-core.h"
> +
> +void wave6_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,
> + unsigned int width,
> + unsigned int height)
> +{
> + const struct v4l2_format_info *fmt_info;
> + unsigned int stride_y;
> +
> + pix_mp->width = width;
> + pix_mp->height = height;
> + pix_mp->flags = 0;
> + pix_mp->field = V4L2_FIELD_NONE;
> + memset(pix_mp->reserved, 0, sizeof(pix_mp->reserved));
> +
> + fmt_info = v4l2_format_info(pix_mp->pixelformat);
> + if (!fmt_info) {
> + pix_mp->plane_fmt[0].bytesperline = 0;
> + if (!pix_mp->plane_fmt[0].sizeimage)
> + pix_mp->plane_fmt[0].sizeimage = width * height;
> +
> + return;
> + }
[Severity: Medium]
Does preserving a non-zero pix_mp->plane_fmt[0].sizeimage for compressed
formats leave sizeimage stale when the resolution is increased?
For codec formats (where v4l2_format_info() returns NULL), if
pix_mp->plane_fmt[0].sizeimage was already initialized to a default
resolution's size (or populated by VIDIOC_G_FMT) and wave6_update_pix_fmt()
is then called for a larger resolution without zeroing sizeimage first,
plane_fmt[0].sizeimage remains clamped to the old smaller resolution's
buffer size.
[ ... ]
> +static bool wave6_vpu_check_fb_available(struct vpu_instance *inst)
> +{
> + struct vb2_v4l2_buffer *vb2_v4l2_buf;
> + struct v4l2_m2m_buffer *v4l2_m2m_buf;
> + struct vpu_buffer *vpu_buf;
> +
> + v4l2_m2m_for_each_dst_buf(inst->v4l2_fh.m2m_ctx, v4l2_m2m_buf) {
> + vb2_v4l2_buf = &v4l2_m2m_buf->vb;
> + vpu_buf = wave6_to_vpu_buf(vb2_v4l2_buf);
> +
> + if (!vpu_buf->used)
> + return true;
> + }
> +
> + return false;
> +}
> +
> +static int wave6_vpu_job_ready(void *priv)
> +{
> + struct vpu_instance *inst = priv;
> +
> + dev_dbg(inst->dev->dev, "[%d]%s: state %d\n",
> + inst->id, __func__, inst->state);
> +
> + if (inst->state == VPU_INST_STATE_OPEN)
> + return 1;
> + if (inst->state < VPU_INST_STATE_PIC_RUN)
> + return 0;
> + if (inst->state == VPU_INST_STATE_ERROR)
> + return 0;
> + if (!wave6_vpu_check_fb_available(inst))
> + return 0;
> +
> + return 1;
> +}
[Severity: High]
Isn't m2m_ctx->cap_q_ctx.rdy_queue traversed without holding
cap_q_ctx.rdy_spinlock or core->dev_lock when wave6_vpu_check_fb_available()
is called from wave6_vpu_finish_job()?
v4l2_m2m_for_each_dst_buf() expands to list_for_each_entry() over
m2m_ctx->cap_q_ctx.rdy_queue without taking cap_q_ctx.rdy_spinlock. When
wave6_vpu_finish_job() runs from wave6_vpu_core_irq_thread(), core->dev_lock
is not held, and v4l2_m2m_job_finish() only takes m2m_dev->job_spinlock
before calling wave6_vpu_job_ready(). Meanwhile, VIDIOC_QBUF on another
CPU holds core->dev_lock and cap_q_ctx.rdy_spinlock while mutating
cap_q_ctx.rdy_queue in v4l2_m2m_buf_queue():
CPU0 (irq thread) CPU1 (VIDIOC_QBUF)
wave6_vpu_finish_job() v4l2_m2m_qbuf()
v4l2_m2m_job_finish() v4l2_m2m_buf_queue()
lock(&m2m_dev->job_spinlock) lock(&cap_q_ctx.rdy_spinlock)
wave6_vpu_job_ready() list_add_tail(..., &rdy_queue)
wave6_vpu_check_fb_available()
v4l2_m2m_for_each_dst_buf() <-- concurrent list traversal/mutation
> +
> +void wave6_vpu_device_run_timeout(struct work_struct *work)
> +{
> + struct vpu_instance *inst = container_of(to_delayed_work(work),
> + struct vpu_instance, task_timer);
> + struct vb2_v4l2_buffer *src_buf = NULL;
> + struct vb2_v4l2_buffer *dst_buf = NULL;
> +
> + dev_err(inst->dev->dev, "[%d] sequence %d timeout\n", inst->id, inst->sequence);
> + src_buf = v4l2_m2m_src_buf_remove(inst->v4l2_fh.m2m_ctx);
> + if (src_buf) {
> + v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
> + if (inst->type == VPU_INST_TYPE_DEC)
> + inst->processed_buf_num++;
> + inst->error_buf_num++;
> + }
> +
> + dst_buf = v4l2_m2m_dst_buf_remove(inst->v4l2_fh.m2m_ctx);
> + if (dst_buf)
> + v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
> +
> + vb2_queue_error(v4l2_m2m_get_src_vq(inst->v4l2_fh.m2m_ctx));
> + vb2_queue_error(v4l2_m2m_get_dst_vq(inst->v4l2_fh.m2m_ctx));
> +
> + v4l2_m2m_job_finish(inst->dev->m2m_dev, inst->v4l2_fh.m2m_ctx);
> +}
> +
> +static void wave6_vpu_device_run(void *priv)
> +{
> + struct vpu_instance *inst = priv;
> + int ret;
> +
> + dev_dbg(inst->dev->dev, "[%d]%s: state %d\n",
> + inst->id, __func__, inst->state);
> +
> + ret = inst->ops->prepare_process(inst);
> + if (ret) {
> + v4l2_m2m_job_finish(inst->dev->m2m_dev, inst->v4l2_fh.m2m_ctx);
> + return;
> + }
> +
> + schedule_delayed_work(&inst->task_timer, msecs_to_jiffies(W6_VPU_TIMEOUT));
> +
> + ret = inst->ops->start_process(inst);
> + if (ret)
> + wave6_vpu_finish_job(inst);
> +}
> +
> +void wave6_vpu_finish_job(struct vpu_instance *inst)
> +{
> + cancel_delayed_work(&inst->task_timer);
> + v4l2_m2m_job_finish(inst->dev->m2m_dev, inst->v4l2_fh.m2m_ctx);
> +}
[Severity: Medium]
Can wave6_vpu_device_run_timeout() race with normal job completion in
finish_process() and wave6_vpu_finish_job()?
wave6_vpu_finish_job() uses non-synchronous cancel_delayed_work() and does
not check its return value. If wave6_vpu_device_run_timeout() has already
started running when finish_process() executes on the IRQ thread, nothing
serializes them:
CPU0 (kworker) CPU1 (irq thread)
wave6_vpu_device_run_timeout() wave6_vpu_core_irq_thread()
inst->ops->finish_process()
v4l2_m2m_src_buf_remove() ...
inst->error_buf_num++ wave6_vpu_finish_job()
cancel_delayed_work() -> false
v4l2_m2m_job_finish()
v4l2_m2m_job_finish() <-- double v4l2_m2m_job_finish()
Both contexts concurrently remove buffers, update inst->processed_buf_num
and inst->error_buf_num, and call v4l2_m2m_job_finish().
In addition, wave6_vpu_device_run_timeout() does not transition
inst->state to VPU_INST_STATE_ERROR before calling v4l2_m2m_job_finish().
Because v4l2_m2m_job_finish() immediately checks wave6_vpu_job_ready() to
schedule the next queued job, and neither __v4l2_m2m_try_queue() nor
wave6_vpu_job_ready() checks vq->error, another queued job will be started
on the timed-out instance.
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795075577.git.nas.chung@chipsnmedia.com?part=3
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v8 07/11] media: chips-media: wave6: Add Wave6 m2m encoder
2026-11-19 9:00 ` [PATCH v8 07/11] media: chips-media: wave6: Add Wave6 m2m encoder Nas Chung
@ 2026-11-19 9:35 ` sashiko-bot
0 siblings, 0 replies; 23+ messages in thread
From: sashiko-bot @ 2026-11-19 9:35 UTC (permalink / raw)
To: Nas Chung; +Cc: devicetree, imx, robh, conor+dt, Frank.Li, media-ci
Thank you for your contribution! Sashiko AI review found 12 potential
issues to consider:
- [High] media: wave6: buffer leak and m2m job hang in
wave6_vpu_enc_finish_encode()
- [High] media: wave6: unvalidated CAPTURE size and stale sizeimage in
dst_fmt
- [High] media: wave6: asymmetric instance lifecycle in start and
stop_streaming()
- [High] media: wave6: missing busy check and zero crop return in
s_selection()
- [High] media: wave6: unlocked m2m rdy_queue traversal in encoder
buffer lookup
- [High] media: wave6: race between wave6_vpu_enc_finish_encode() and
timeout work
- [Medium] media: wave6: zero CSC coefficients in wave6_set_csc() for
XV601 and XV709
- [Medium] media: wave6: HEVC refresh controls overwritten in
wave6_set_enc_hevc_param()
- [Medium] media: wave6: skipped CAPTURE validation in
wave6_vpu_enc_buf_prepare()
- [Medium] media: wave6: zero-padded format table match in
wave6_find_vpu_fmt()
- [Medium] media: wave6: wave6_find_vpu_fmt() matches zeroed format
table entries
- [Low] media: wave6: unupdated timeperframe in wave6_vpu_enc_s_parm()
--
commit d914baad0e743b67d2e3cd7e1bd636ba7a5de85f
Author: Nas Chung <nas.chung@chipsnmedia.com>
media: chips-media: wave6: Add Wave6 m2m encoder
This commit adds the V4L2 m2m stateful encoder driver for the Chips&Media
Wave6 video codec IP, supporting H.264 and HEVC encoding.
> diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c b/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
> new file mode 100644
> index 0000000000000..7142666d4503d
> --- /dev/null
> +++ b/drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
> @@ -0,0 +1,2732 @@
[ ... ]
> +static const struct vpu_format wave6_vpu_enc_fmt_list[2][23] = {
> + [VPU_FMT_TYPE_CODEC] = {
> + {
> + .v4l2_pix_fmt = V4L2_PIX_FMT_HEVC,
> + .max_width = W6_MAX_ENC_PIC_WIDTH,
> + .min_width = W6_MIN_ENC_PIC_WIDTH,
> + .max_height = W6_MAX_ENC_PIC_HEIGHT,
> + .min_height = W6_MIN_ENC_PIC_HEIGHT,
> + .num_planes = 1,
> + },
> + {
> + .v4l2_pix_fmt = V4L2_PIX_FMT_H264,
> + .max_width = W6_MAX_ENC_PIC_WIDTH,
> + .min_width = W6_MIN_ENC_PIC_WIDTH,
> + .max_height = W6_MAX_ENC_PIC_HEIGHT,
> + .min_height = W6_MIN_ENC_PIC_HEIGHT,
> + .num_planes = 1,
> + },
> + },
[ ... ]
> +static const struct vpu_format *wave6_find_vpu_fmt(unsigned int v4l2_pix_fmt,
> + enum vpu_fmt_type type)
> +{
> + for (int i = 0; i < ARRAY_SIZE(wave6_vpu_enc_fmt_list[type]); i++) {
> + if (wave6_vpu_enc_fmt_list[type][i].v4l2_pix_fmt == v4l2_pix_fmt)
> + return &wave6_vpu_enc_fmt_list[type][i];
> + }
> +
> + return NULL;
> +}
[Severity: Medium]
Can wave6_find_vpu_fmt() match the unpopulated trailing entries in
wave6_vpu_enc_fmt_list[VPU_FMT_TYPE_CODEC] when v4l2_pix_fmt is 0?
Because wave6_vpu_enc_fmt_list is sized to 23 elements for both sub-arrays
while VPU_FMT_TYPE_CODEC only populates 2 entries, the remaining 21 entries
are zero-initialized. Unlike wave6_find_vpu_fmt_by_idx(), wave6_find_vpu_fmt()
does not reject a zero pixelformat, so passing pixelformat 0 to
VIDIOC_TRY_FMT on the capture queue matches a zeroed entry and sets
pix_mp->pixelformat and pix_mp->num_planes to 0 instead of falling back to
inst->dst_fmt.
[ ... ]
> +static struct vb2_v4l2_buffer *wave6_get_valid_src_buf(struct vpu_instance *inst)
> +{
> + struct vb2_v4l2_buffer *vb2_v4l2_buf;
> + struct v4l2_m2m_buffer *v4l2_m2m_buf;
> + struct vpu_buffer *vpu_buf = NULL;
> +
> + v4l2_m2m_for_each_src_buf(inst->v4l2_fh.m2m_ctx, v4l2_m2m_buf) {
> + vb2_v4l2_buf = &v4l2_m2m_buf->vb;
> + vpu_buf = wave6_to_vpu_buf(vb2_v4l2_buf);
> +
> + if (!vpu_buf->consumed) {
> + dev_dbg(inst->dev->dev, "no consumed src idx : %d\n",
> + vb2_v4l2_buf->vb2_buf.index);
> + return vb2_v4l2_buf;
> + }
> + }
> +
> + return NULL;
> +}
> +
> +static struct vb2_v4l2_buffer *wave6_get_valid_dst_buf(struct vpu_instance *inst)
> +{
> + struct vb2_v4l2_buffer *vb2_v4l2_buf;
> + struct v4l2_m2m_buffer *v4l2_m2m_buf;
> + struct vpu_buffer *vpu_buf;
> +
> + v4l2_m2m_for_each_dst_buf(inst->v4l2_fh.m2m_ctx, v4l2_m2m_buf) {
> + vb2_v4l2_buf = &v4l2_m2m_buf->vb;
> + vpu_buf = wave6_to_vpu_buf(vb2_v4l2_buf);
> +
> + if (!vpu_buf->consumed) {
> + dev_dbg(inst->dev->dev, "no consumed dst idx : %d\n",
> + vb2_v4l2_buf->vb2_buf.index);
> + return vb2_v4l2_buf;
> + }
> + }
> +
> + return NULL;
> +}
[Severity: High]
Does iterating the M2M ready queues in wave6_get_valid_src_buf(),
wave6_get_valid_dst_buf(), and wave6_get_dst_buf_by_addr() race with
v4l2_m2m_buf_queue()?
v4l2_m2m_for_each_src_buf() and v4l2_m2m_for_each_dst_buf() expand to plain
list_for_each_entry() over out_q_ctx.rdy_queue and cap_q_ctx.rdy_queue
without taking q_ctx->rdy_spinlock:
CPU0 (v4l2_m2m_device_run_work) CPU1 (VIDIOC_QBUF)
wave6_vpu_device_run() wave6_vpu_enc_buf_queue()
wave6_vpu_enc_start_encode() v4l2_m2m_buf_queue()
wave6_get_valid_src_buf() spin_lock(&q_ctx->rdy_spinlock)
v4l2_m2m_for_each_src_buf() list_add_tail(&b->list,
&q_ctx->rdy_queue)
/* unlocked list traversal <-- data race */
When wave6_vpu_device_run() runs from the M2M job workqueue, neither
inst->dev->dev_lock nor rdy_spinlock is held. Likewise, when
wave6_handle_encoded_frame() calls wave6_get_dst_buf_by_addr() from the IRQ
thread, it holds only inst->lock while VIDIOC_QBUF mutates rdy_queue under
inst->dev->dev_lock and rdy_spinlock.
> +
> +static void wave6_set_csc(struct vpu_instance *inst, struct enc_param *pic_param)
> +{
> + const struct vpu_format *vpu_fmt;
> + bool is_10bit;
> +
> + vpu_fmt = wave6_find_vpu_fmt(inst->src_fmt.pixelformat, VPU_FMT_TYPE_RAW);
> + if (!vpu_fmt)
> + return;
> +
> + if (!(vpu_fmt->flags & VPU_FMT_FLAG_RGB))
> + return;
> +
> + is_10bit = !!(vpu_fmt->flags & VPU_FMT_FLAG_10BIT);
> + pic_param->csc.fmt_order = vpu_fmt->csc_fmt_order;
> +
> + if (inst->ycbcr_enc == V4L2_YCBCR_ENC_DEFAULT ||
> + inst->ycbcr_enc == V4L2_YCBCR_ENC_601) {
[ ... ]
> + } else if (inst->ycbcr_enc == V4L2_YCBCR_ENC_XV601) {
> + if (inst->quantization == V4L2_QUANTIZATION_LIM_RANGE) {
> + /*
> + * Y 0.2558(R) 0.5021(G) 0.0975(B)
> + * Cb -0.1476(R) -0.2899(G) 0.4375(B)
> + * Cr 0.4375(R) -0.3664(G) -0.0711(B)
> + */
> + pic_param->csc.coef_ry = 0x083;
> + pic_param->csc.coef_gy = 0x101;
> + pic_param->csc.coef_by = 0x032;
> + pic_param->csc.coef_rcb = 0xffffffb4;
> + pic_param->csc.coef_gcb = 0xffffff6c;
> + pic_param->csc.coef_bcb = 0x0e0;
> + pic_param->csc.coef_rcr = 0x0e0;
> + pic_param->csc.coef_gcr = 0xffffff44;
> + pic_param->csc.coef_bcr = 0xffffffdc;
> + pic_param->csc.offset_y = (is_10bit) ? 0x40 : 0x10;
> + pic_param->csc.offset_cb = 0x0;
> + pic_param->csc.offset_cr = 0x0;
> + }
> + } else if (inst->ycbcr_enc == V4L2_YCBCR_ENC_XV709) {
> + if (inst->quantization == V4L2_QUANTIZATION_LIM_RANGE) {
> + /*
> + * Y 0.1819(R) 0.6118(G) 0.0618(B)
> + * Cb -0.1003(R) -0.3372(G) 0.4375(B)
> + * Cr 0.4375(R) -0.3974(G) -0.0401(B)
> + */
> + pic_param->csc.coef_ry = 0x05d;
> + pic_param->csc.coef_gy = 0x139;
> + pic_param->csc.coef_by = 0x020;
> + pic_param->csc.coef_rcb = 0xffffffcd;
> + pic_param->csc.coef_gcb = 0xffffff53;
> + pic_param->csc.coef_bcb = 0x0e0;
> + pic_param->csc.coef_rcr = 0x0e0;
> + pic_param->csc.coef_gcr = 0xffffff35;
> + pic_param->csc.coef_bcr = 0xffffffeb;
> + pic_param->csc.offset_y = (is_10bit) ? 0x40 : 0x10;
> + pic_param->csc.offset_cb = 0x0;
> + pic_param->csc.offset_cr = 0x0;
> + }
> + }
> +}
[Severity: Medium]
Does wave6_set_csc() leave pic_param->csc zeroed when encoding RGB formats
with V4L2_YCBCR_ENC_XV601 or V4L2_YCBCR_ENC_XV709 and default quantization?
inst->quantization defaults to V4L2_QUANTIZATION_DEFAULT, and
wave6_vpu_enc_try_fmt_out() only converts V4L2_QUANTIZATION_FULL_RANGE to
V4L2_QUANTIZATION_LIM_RANGE for XV601 and XV709. Because these two branches
require inst->quantization == V4L2_QUANTIZATION_LIM_RANGE, all CSC
coefficients remain 0 when quantization is V4L2_QUANTIZATION_DEFAULT,
producing all-zero YCbCr output. In addition, V4L2_YCBCR_ENC_DEFAULT is
always treated as BT.601 here instead of being resolved via
V4L2_MAP_YCBCR_ENC_DEFAULT(inst->colorspace) as in to_matrix_coeffs().
[ ... ]
> +static int wave6_vpu_enc_start_encode(struct vpu_instance *inst)
> +{
[ ... ]
> + src_buf = wave6_get_valid_src_buf(inst);
> + dst_buf = wave6_get_valid_dst_buf(inst);
> +
> + if (!src_buf || !dst_buf) {
> + dev_dbg(inst->dev->dev, "no valid src or dst buf\n");
> + return 0;
> + }
> +
> + dst_vbuf = wave6_to_vpu_buf(dst_buf);
> + dst_vbuf->consumed = true;
> + dst_vbuf->used = true;
> + pic_param.pic_stream_buffer_addr = wave6_get_dma_addr(dst_buf, 0);
> + pic_param.pic_stream_buffer_size = vb2_plane_size(&dst_buf->vb2_buf, 0);
> +
> + src_vbuf = wave6_to_vpu_buf(src_buf);
> + src_vbuf->consumed = true;
[ ... ]
> +static void wave6_handle_encoded_frame(struct vpu_instance *inst,
> + struct enc_output_info *info)
> +{
> + struct vb2_v4l2_buffer *src_buf;
> + struct vb2_v4l2_buffer *dst_buf;
> + struct vpu_buffer *vpu_buf;
> + struct vpu_buffer *dst_vpu_buf;
> + enum vb2_buffer_state state;
> +
> + guard(mutex)(&inst->lock);
> +
> + state = info->encoding_success ? VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
> +
> + dst_buf = wave6_get_dst_buf_by_addr(inst, info->bitstream_buffer);
> + if (!dst_buf) {
> + dev_err(inst->dev->dev, "[%d] encoder can't find dst buffer\n", inst->id);
> + return;
> + }
> +
> + src_buf = v4l2_m2m_src_buf_remove_by_idx(inst->v4l2_fh.m2m_ctx,
> + info->enc_src_idx);
> + if (!src_buf) {
> + dev_err(inst->dev->dev, "[%d] encoder can't find src buffer\n", inst->id);
> + return;
> + }
> +
> + vpu_buf = wave6_to_vpu_buf(src_buf);
> + if (!vpu_buf->consumed) {
> + dev_err(inst->dev->dev, "[%d] src buffer is not consumed\n", inst->id);
> + v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
> + return;
> + }
[ ... ]
> + v4l2_m2m_dst_buf_remove_by_buf(inst->v4l2_fh.m2m_ctx, dst_buf);
> + if (state == VB2_BUF_STATE_ERROR) {
> + dev_dbg(inst->dev->dev, "[%d] error frame %d\n", inst->id, inst->sequence);
> + inst->error_recovery = true;
> + inst->error_buf_num++;
> + }
> + v4l2_m2m_buf_done(dst_buf, state);
> + inst->processed_buf_num++;
> +}
[Severity: High]
Can wave6_handle_encoded_frame() race with wave6_vpu_device_run_timeout()
and corrupt the destination queue list or complete dst_buf twice?
wave6_vpu_enc_finish_encode() runs wave6_handle_encoded_frame() before
cancelling inst->task_timer in wave6_vpu_finish_job(), and
wave6_vpu_finish_job() only uses asynchronous cancel_delayed_work() while
wave6_vpu_device_run_timeout() does not take inst->lock:
CPU0 (wave6_vpu_core_irq_thread) CPU1 (wave6_vpu_device_run_timeout)
wave6_vpu_enc_finish_encode()
wave6_handle_encoded_frame()
mutex_lock(&inst->lock)
dst_buf = wave6_get_dst_buf_by_addr()
dst_buf = v4l2_m2m_dst_buf_remove()
v4l2_m2m_buf_done(dst_buf,
VB2_BUF_STATE_ERROR)
v4l2_m2m_dst_buf_remove_by_buf(..., dst_buf)
list_del(&dst_buf->list) <-- double list_del
v4l2_m2m_buf_done(dst_buf, state)
<-- double buffer completion
> +
> +static void wave6_vpu_enc_finish_encode(struct vpu_instance *inst, bool error)
> +{
> + int ret;
> + struct enc_output_info info;
> +
> + if (error) {
> + vb2_queue_error(v4l2_m2m_get_src_vq(inst->v4l2_fh.m2m_ctx));
> + vb2_queue_error(v4l2_m2m_get_dst_vq(inst->v4l2_fh.m2m_ctx));
> +
> + wave6_vpu_set_instance_state(inst, VPU_INST_STATE_ERROR);
> +
> + goto finish_encode;
> + }
> +
> + memset(&info, 0, sizeof(info));
> +
> + ret = wave6_vpu_enc_get_output_info(inst, &info);
> + if (ret) {
> + dev_dbg(inst->dev->dev, "vpu_enc_get_output_info fail %d\n", ret);
> + goto finish_encode;
> + }
> +
> + if (info.enc_src_idx >= 0 && info.recon_frame_index >= 0)
> + wave6_handle_encoded_frame(inst, &info);
> +
> +finish_encode:
> + wave6_vpu_finish_job(inst);
> +}
[Severity: High]
Are the in-flight source and destination buffers stranded on the M2M ready
queues when a frame is skipped or an encode error occurs?
wave6_vpu_enc_start_encode() marks src_vbuf->consumed = true and
dst_vbuf->consumed = true while leaving both buffers on m2m_ctx until
wave6_handle_encoded_frame() removes and completes them. When a frame is
skipped due to V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_BUF_LIMIT (so
info.recon_frame_index < 0), or when error is true or
wave6_vpu_enc_get_output_info() fails, wave6_handle_encoded_frame() is
not called; similarly, its early returns if src_buf is missing or unconsumed
leave dst_buf unremoved. Those buffers remain on the ready queues with
consumed = true and are skipped forever by wave6_get_valid_src_buf() and
wave6_get_valid_dst_buf().
Furthermore, when wave6_vpu_enc_start_encode() subsequently finds no
unconsumed buffer (!src_buf || !dst_buf), it returns 0 without starting a
hardware command, so wave6_vpu_device_run() leaves the M2M job running with
inst->task_timer armed until the 6-second watchdog timeout fires and kills
both queues.
[ ... ]
> +static int wave6_vpu_enc_enum_framesizes(struct file *f, void *fh, struct v4l2_frmsizeenum *fsize)
> +{
> + const struct vpu_format *vpu_fmt;
> +
> + if (fsize->index)
> + return -EINVAL;
> +
> + vpu_fmt = wave6_find_vpu_fmt(fsize->pixel_format, VPU_FMT_TYPE_CODEC);
> + if (!vpu_fmt) {
> + vpu_fmt = wave6_find_vpu_fmt(fsize->pixel_format, VPU_FMT_TYPE_RAW);
> + if (!vpu_fmt)
> + return -EINVAL;
> + }
[Severity: Medium]
Does wave6_vpu_enc_enum_framesizes() succeed with 0x0 stepwise bounds when
fsize->pixel_format is 0?
Because wave6_find_vpu_fmt(0, VPU_FMT_TYPE_CODEC) returns a pointer to the
first zero-padded entry in wave6_vpu_enc_fmt_list[VPU_FMT_TYPE_CODEC],
VIDIOC_ENUM_FRAMESIZES returns 0 with min_width, max_width, min_height, and
max_height all set to 0 instead of returning -EINVAL.
[ ... ]
> +static int wave6_vpu_enc_try_fmt_cap(struct file *file, void *fh, struct v4l2_format *f)
> +{
> + struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
> + struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
> + const struct vpu_format *vpu_fmt;
> + int width, height;
> +
> + dev_dbg(inst->dev->dev, "%s: 4cc %d w %d h %d plane %d colorspace %d\n",
> + __func__, pix_mp->pixelformat, pix_mp->width, pix_mp->height,
> + pix_mp->num_planes, pix_mp->colorspace);
> +
> + if (!V4L2_TYPE_IS_CAPTURE(f->type))
> + return -EINVAL;
> +
> + vpu_fmt = wave6_find_vpu_fmt(pix_mp->pixelformat, VPU_FMT_TYPE_CODEC);
> + if (!vpu_fmt) {
> + width = inst->dst_fmt.width;
> + height = inst->dst_fmt.height;
> + pix_mp->pixelformat = inst->dst_fmt.pixelformat;
> + pix_mp->num_planes = inst->dst_fmt.num_planes;
> + } else {
> + width = pix_mp->width;
> + height = pix_mp->height;
> + pix_mp->pixelformat = vpu_fmt->v4l2_pix_fmt;
> + pix_mp->num_planes = vpu_fmt->num_planes;
> + }
> +
> + wave6_update_pix_fmt(pix_mp, width, height);
[Severity: High]
Can wave6_vpu_enc_try_fmt_cap() set plane_fmt[0].sizeimage to 0, and do
wave6_vpu_enc_s_fmt_out() and wave6_vpu_enc_s_selection() leave
inst->dst_fmt.plane_fmt[0].sizeimage stuck at the initial 416x240 size?
In wave6_update_pix_fmt(), sizeimage for coded formats is only recalculated
when plane_fmt[0].sizeimage is 0:
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c:wave6_update_pix_fmt() {
...
if (!fmt_info) {
pix_mp->plane_fmt[0].bytesperline = 0;
if (!pix_mp->plane_fmt[0].sizeimage)
pix_mp->plane_fmt[0].sizeimage = width * height;
return;
}
...
}
When pix_mp->pixelformat is valid in wave6_vpu_enc_try_fmt_cap(), width and
height are taken directly from userspace without clamping or using
inst->dst_fmt. Passing width = 0 or height = 0 with sizeimage = 0 sets
inst->dst_fmt.plane_fmt[0].sizeimage to 0 in wave6_vpu_enc_s_fmt_cap(),
which triggers WARN_ON(!plane_sizes[i]) in vb2_core_reqbufs().
Conversely, because wave6_vpu_open_enc() initializes inst->dst_fmt to
416x240 (sizeimage = 99840) and neither wave6_vpu_enc_s_fmt_out() nor
wave6_vpu_enc_s_selection() clears inst->dst_fmt.plane_fmt[0].sizeimage
before calling wave6_update_pix_fmt(&inst->dst_fmt, ...), raising the
output resolution leaves capture bitstream buffers undersized at 99840
bytes.
[ ... ]
> +static int wave6_vpu_enc_s_selection(struct file *file, void *fh, struct v4l2_selection *s)
> +{
> + struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
> + u32 max_crop_w, max_crop_h;
> +
> + if (!V4L2_TYPE_IS_OUTPUT(s->type))
> + return -EINVAL;
> +
> + if (s->target != V4L2_SEL_TGT_CROP)
> + return -EINVAL;
> +
> + if (!(s->flags & (V4L2_SEL_FLAG_GE | V4L2_SEL_FLAG_LE)))
> + s->flags |= V4L2_SEL_FLAG_LE;
> +
> + if (s->flags & V4L2_SEL_FLAG_GE) {
> + s->r.left = round_up(s->r.left, W6_ENC_CROP_STEP);
> + s->r.top = round_up(s->r.top, W6_ENC_CROP_STEP);
> + s->r.width = round_up(s->r.width, W6_ENC_CROP_STEP);
> + s->r.height = round_up(s->r.height, W6_ENC_CROP_STEP);
> + }
> + if (s->flags & V4L2_SEL_FLAG_LE) {
> + s->r.left = round_down(s->r.left, W6_ENC_CROP_STEP);
> + s->r.top = round_down(s->r.top, W6_ENC_CROP_STEP);
> + s->r.width = round_down(s->r.width, W6_ENC_CROP_STEP);
> + s->r.height = round_down(s->r.height, W6_ENC_CROP_STEP);
> + }
> +
> + s->r.left = clamp_t(s32, s->r.left, 0,
> + inst->src_fmt.width - W6_MIN_ENC_PIC_WIDTH);
> + s->r.top = clamp_t(s32, s->r.top, 0,
> + inst->src_fmt.height - W6_MIN_ENC_PIC_HEIGHT);
> +
> + max_crop_w = inst->src_fmt.width - s->r.left;
> + max_crop_h = inst->src_fmt.height - s->r.top;
> +
> + if (!s->r.width || !s->r.height)
> + return 0;
> + if (max_crop_w < W6_MIN_ENC_PIC_WIDTH)
> + return 0;
> + if (max_crop_h < W6_MIN_ENC_PIC_HEIGHT)
> + return 0;
> +
> + s->r.width = clamp(s->r.width, W6_MIN_ENC_PIC_WIDTH, max_crop_w);
> + s->r.height = clamp(s->r.height, W6_MIN_ENC_PIC_HEIGHT, max_crop_h);
> +
> + wave6_update_pix_fmt(&inst->dst_fmt, s->r.width, s->r.height);
> + wave6_update_crop_info(inst, s->r.left, s->r.top, s->r.width, s->r.height);
[Severity: High]
Can wave6_vpu_enc_s_selection() modify inst->codec_rect while streaming,
and does it return a 0-sized rectangle when width or height is 0?
wave6_vpu_enc_s_selection() does not check vb2_is_busy() on the queues or
whether inst->state != VPU_INST_STATE_NONE. Because hardware picture
dimensions are configured once at streamon while
wave6_update_frame_buf_addr() adds inst->codec_rect.top and
inst->codec_rect.left to the source frame DMA addresses on every frame,
changing the crop offset mid-stream shifts the DMA base addresses while the
VPU still reads the original frame dimensions past the end of the source
buffer.
In addition, returning 0 early when !s->r.width || !s->r.height bypasses
the clamp() calls right below and reports an invalid 0-sized crop rectangle
back to userspace.
[ ... ]
> +static int wave6_vpu_enc_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
> +{
> + struct vpu_instance *inst = wave6_file_to_vpu_inst(file);
> + struct enc_controls *ctrls = &inst->enc_ctrls;
> +
> + dev_dbg(inst->dev->dev, "%s: type %d\n", __func__, a->type);
> +
> + if (!V4L2_TYPE_IS_OUTPUT(a->type))
> + return -EINVAL;
> +
> + a->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
> + if (a->parm.output.timeperframe.denominator &&
> + a->parm.output.timeperframe.numerator) {
> + ctrls->frame_rate = DIV_ROUND_CLOSEST(a->parm.output.timeperframe.denominator,
> + a->parm.output.timeperframe.numerator);
> + ctrls->frame_rate = clamp(ctrls->frame_rate,
> + MIN_FRAME_RATE, MAX_FRAME_RATE);
> + } else {
> + a->parm.output.timeperframe.numerator = 1;
> + a->parm.output.timeperframe.denominator = ctrls->frame_rate;
> + }
[Severity: Low]
Should wave6_vpu_enc_s_parm() update a->parm.output.timeperframe after
computing and clamping ctrls->frame_rate?
When numerator and denominator are non-zero, the driver rounds and clamps
ctrls->frame_rate but leaves the caller's unadjusted values in
a->parm.output.timeperframe instead of returning 1 / ctrls->frame_rate.
[ ... ]
> +static void wave6_set_enc_hevc_param(struct enc_codec_param *output,
> + struct hevc_enc_controls *ctrls)
> +{
[ ... ]
> + switch (ctrls->refresh_type) {
> + case V4L2_MPEG_VIDEO_HEVC_REFRESH_NONE:
> + output->decoding_refresh_type = DEC_REFRESH_TYPE_NON_IRAP;
> + break;
> + case V4L2_MPEG_VIDEO_HEVC_REFRESH_IDR:
> + output->decoding_refresh_type = DEC_REFRESH_TYPE_IDR;
> + break;
> + default:
> + break;
> + }
> + output->intra_period = ctrls->refresh_period;
> + if (output->idr_period) {
> + output->decoding_refresh_type = DEC_REFRESH_TYPE_IDR;
> + output->intra_period = output->idr_period;
> + output->idr_period = 0;
> + }
[Severity: Medium]
Does checking if (output->idr_period) here overwrite the HEVC refresh type
and period whenever V4L2_CID_MPEG_VIDEO_GOP_SIZE is non-zero?
wave6_set_enc_open_param() sets output->idr_period = ctrls->gop_size
(which defaults to 30) before calling wave6_set_enc_hevc_param(). Checking
if (output->idr_period) instead of if (!output->intra_period) clobbers
ctrls->refresh_period and ctrls->refresh_type unless GOP size is explicitly
set to 0.
[ ... ]
> +static int wave6_vpu_enc_buf_prepare(struct vb2_buffer *vb)
> +{
> + struct vpu_instance *inst = vb2_get_drv_priv(vb->vb2_queue);
> +
> + if (V4L2_TYPE_IS_CAPTURE(vb->type))
> + return 0;
[Severity: Medium]
Should wave6_vpu_enc_buf_prepare() validate the capture plane size against
inst->dst_fmt.plane_fmt[0].sizeimage and reset the plane payload to 0?
Returning early for capture buffers allows undersized imported DMABUFs to
be queued without error, and a re-queued capture buffer that is later
completed as an empty EOS buffer via v4l2_m2m_last_buffer_done() when
V4L2_ENC_CMD_STOP or output streamoff runs retains its stale bytesused
payload from the previous frame.
[ ... ]
> +static int wave6_vpu_enc_start_streaming(struct vb2_queue *q, unsigned int count)
> +{
> + struct vpu_instance *inst = vb2_get_drv_priv(q);
> + int ret = 0;
> +
> + v4l2_m2m_update_start_streaming_state(inst->v4l2_fh.m2m_ctx, q);
> +
> + if (inst->state == VPU_INST_STATE_NONE) {
> + ret = wave6_vpu_enc_create_instance(inst);
> + if (ret)
> + goto exit;
> + }
> +
> + if (inst->state == VPU_INST_STATE_OPEN) {
> + ret = wave6_vpu_enc_initialize_instance(inst);
> + if (ret) {
> + wave6_vpu_enc_destroy_instance(inst);
> + goto exit;
> + }
> + }
> +
> + if (inst->state == VPU_INST_STATE_INIT_SEQ) {
> + ret = wave6_vpu_enc_prepare_fb(inst);
> + if (ret) {
> + wave6_vpu_enc_destroy_instance(inst);
> + goto exit;
> + }
> + }
> +
> +exit:
> + if (ret)
> + wave6_vpu_return_buffers(inst, q->type, VB2_BUF_STATE_QUEUED);
> +
> + return ret;
> +}
> +
> +static void wave6_vpu_enc_stop_streaming(struct vb2_queue *q)
> +{
> + struct vpu_instance *inst = vb2_get_drv_priv(q);
> +
> + dev_dbg(inst->dev->dev, "[%d] %s, input %d, encode %d\n",
> + inst->id, V4L2_TYPE_IS_OUTPUT(q->type) ? "output" : "capture",
> + inst->queued_src_buf_num, inst->sequence);
> +
> + if (inst->state == VPU_INST_STATE_NONE)
> + goto exit;
> +
> + if (V4L2_TYPE_IS_OUTPUT(q->type)) {
> + inst->queued_src_buf_num = 0;
> + inst->processed_buf_num = 0;
> + inst->error_buf_num = 0;
> + inst->sequence = 0;
> + } else {
> + inst->queued_dst_buf_num = 0;
> + }
> +
> + if (V4L2_TYPE_IS_CAPTURE(q->type) && inst->state != VPU_INST_STATE_NONE)
> + wave6_vpu_enc_destroy_instance(inst);
> +
> +exit:
> + wave6_vpu_return_buffers(inst, q->type, VB2_BUF_STATE_ERROR);
> + v4l2_m2m_update_stop_streaming_state(inst->v4l2_fh.m2m_ctx, q);
> +}
[Severity: High]
Can starting streaming on one queue before configuring the other leave the
hardware instance and FBC reference buffers out of sync with the configured
formats?
wave6_vpu_enc_start_streaming() creates the hardware instance and allocates
FBC buffers sized to inst->codec_rect on whichever queue starts streaming
first, while wave6_vpu_enc_s_fmt_out() and wave6_vpu_enc_s_fmt_cap() only
check vb2_is_busy() on their own queue. If userspace starts capture
streaming first (or stops only the output queue, which does not destroy the
instance) and then changes the output resolution or capture codec format,
the second STREAMON sees inst->state == VPU_INST_STATE_PIC_RUN and skips
reinitializing the hardware instance and FBC buffers.
In addition, if capture stops streaming before output,
wave6_vpu_enc_stop_streaming() destroys the instance and sets inst->state
to VPU_INST_STATE_NONE, so stopping the output queue afterwards takes the
early goto exit and skips resetting queued_src_buf_num, processed_buf_num,
error_buf_num, and sequence.
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795075577.git.nas.chung@chipsnmedia.com?part=7
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v8 11/11] arm64: dts: freescale: imx95: Add video codec node
2026-11-19 9:00 ` [PATCH v8 11/11] arm64: dts: freescale: imx95: Add video codec node Nas Chung
@ 2026-11-19 9:37 ` sashiko-bot
0 siblings, 0 replies; 23+ messages in thread
From: sashiko-bot @ 2026-11-19 9:37 UTC (permalink / raw)
To: Nas Chung; +Cc: devicetree, imx, robh, conor+dt, Frank.Li, media-ci
Thank you for your contribution! Sashiko AI review found 8 potential
issues to consider:
- [High] media: chips-media: wave6: UAF of m2m_ctx during release and
job finish
- [High] media: chips-media: wave6: unlocked work_buffers_avail access
in hard IRQ
- [High] media: chips-media: wave6: asymmetric encoder instance
streaming lifecycle
- [High] media: chips-media: wave6: infinite job loop on
wave6_vpu_dec_seek_header()
- [Medium] media: chips-media: wave6: PM domains detached before devm
cooling teardown
- [Medium] media: chips-media: wave6: stale fbc_buf_registered after
destroy_instance
- [Medium] media: chips-media: wave6: wave6_vpu_get() skipped if core
already active
- [Low] media: chips-media: wave6: debugfs dir leaked in
wave6_vpu_core_probe()
--
commit aa070a21021b515d839ff256d8cb72ca0721cb00
Author: Nas Chung <nas.chung@chipsnmedia.com>
arm64: dts: freescale: imx95: Add video codec node
This commit adds the Chips&Media Wave633 video codec node to the i.MX95
SoC device tree and enables it with a reserved boot memory region on the
i.MX95 19x19 EVK board.
> diff --git a/arch/arm64/boot/dts/freescale/imx95.dtsi b/arch/arm64/boot/dts/freescale/imx95.dtsi
> index d20c2498b9aa2..b6323e527ab78 100644
> --- a/arch/arm64/boot/dts/freescale/imx95.dtsi
> +++ b/arch/arm64/boot/dts/freescale/imx95.dtsi
> @@ -2139,6 +2139,43 @@ vpu_blk_ctrl: clock-controller@4c410000 {
> assigned-clock-rates = <133333333>, <667000000>, <500000000>;
> };
>
> + vpu: video-codec@4c4c0000 {
> + compatible = "nxp,imx95-vpu";
> + reg = <0x0 0x4c4c0000 0x0 0x10000>;
> + clocks = <&scmi_clk IMX95_CLK_VPU>,
> + <&vpu_blk_ctrl IMX95_CLK_VPUBLK_WAVE>;
> + clock-names = "core", "vpublk";
> + power-domains = <&scmi_devpd IMX95_PD_VPU>,
> + <&scmi_perf IMX95_PERF_VPU>;
> + power-domain-names = "vpu", "perf";
> + sram = <&sram1>;
> + #cooling-cells = <2>;
[Severity: Medium]
With both "vpu" and "perf" power domains and cooling cells enabled here,
can wave6_vpu_remove() (or probe error unwind in wave6_vpu_probe()) cause a
use-after-free of vpu->thermal.dev in the wave6 driver introduced earlier
in this series (commit "media: chips-media: wave6: Add Wave6 control
driver" and commit "media: chips-media: wave6: Add Wave6 thermal cooling
device")?
wave6_vpu_attach_pm_domains() attaches the domains with the non-managed
dev_pm_domain_attach_list(), and wave6_vpu_get_performance_dev() stores
vpu->pd_list->pd_devs[index] in vpu->thermal.dev. wave6_vpu_cooling_init()
then registers the cooling device via
devm_thermal_of_child_cooling_device_register() on vpu->dev:
drivers/media/platform/chips-media/wave6/wave6-vpu.c:wave6_vpu_remove() {
wait_for_completion(&vpu->fw_done);
wave6_vpu_release(vpu);
pm_runtime_disable(vpu->dev);
wave6_vpu_detach_pm_domains(vpu);
}
Because wave6_vpu_detach_pm_domains() calls dev_pm_domain_detach_list()
before the driver core runs devres_release_all() to unregister the devm
cooling device, vpu->thermal.dev is unregistered and freed while
wave6_vpu_cooling_set_cur_state() can still call
dev_pm_genpd_set_performance_state(thermal->dev, level).
> + #address-cells = <2>;
> + #size-cells = <2>;
> + ranges;
> + status = "disabled";
> +
> + interface@4c480000 {
> + reg = <0x0 0x4c480000 0x0 0x10000>;
> + interrupts = <GIC_SPI 299 IRQ_TYPE_LEVEL_HIGH>;
> + };
> +
> + interface@4c490000 {
> + reg = <0x0 0x4c490000 0x0 0x10000>;
> + interrupts = <GIC_SPI 300 IRQ_TYPE_LEVEL_HIGH>;
> + };
> +
> + interface@4c4a0000 {
> + reg = <0x0 0x4c4a0000 0x0 0x10000>;
> + interrupts = <GIC_SPI 301 IRQ_TYPE_LEVEL_HIGH>;
> + };
> +
> + interface@4c4b0000 {
> + reg = <0x0 0x4c4b0000 0x0 0x10000>;
> + interrupts = <GIC_SPI 302 IRQ_TYPE_LEVEL_HIGH>;
> + };
[Severity: High]
Since the four core interface nodes have separate GIC SPI interrupts, can
concurrent W6_INT_BIT_REQ_WORK_BUF interrupts race on
vpu->work_buffers_avail in wave6_vpu_require_work_buffer() (from commit
"media: chips-media: wave6: Add Wave6 control driver")?
drivers/media/platform/chips-media/wave6/wave6-vpu.c:wave6_vpu_require_work_buffer() {
...
if (WARN_ON(vpu->work_buffers_avail == 0))
goto exit;
vpu->work_buffers_avail--;
vb = &vpu->work_buffers[vpu->work_buffers_avail];
...
}
Each core's hard IRQ handler wave6_vpu_core_irq() calls
wave6_vpu_require_work_buffer(core->vpu, core->reg_base) on the shared
parent vpu without a lock:
CPU0 (<hardirq> SPI 299) CPU1 (<hardirq> SPI 300)
wave6_vpu_core_irq() wave6_vpu_core_irq()
wave6_vpu_require_work_buffer() wave6_vpu_require_work_buffer()
work_buffers_avail == 1 work_buffers_avail == 1
work_buffers_avail-- (0) work_buffers_avail-- (0xffffffff)
vpu->work_buffers[0xffffffff]
<-- out-of-bounds read
Even when work_buffers_avail > 1, two cores racing on the decrement can
receive the same DMA work buffer.
[Severity: High]
In the core, decoder and encoder drivers instantiated for these interface
nodes (commit "media: chips-media: wave6: Add Wave6 core driver", commit
"media: chips-media: wave6: Add Wave6 m2m decoder" and commit "media:
chips-media: wave6: Add Wave6 m2m encoder"), can wave6_vpu_dec_release(),
wave6_vpu_enc_release() and wave6_vpu_finish_job() cause a use-after-free
of inst->v4l2_fh.m2m_ctx?
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c:wave6_vpu_dec_release() {
wave6_vpu_remove_dbgfs_file(inst);
v4l2_m2m_ctx_release(inst->v4l2_fh.m2m_ctx);
if (inst->state != VPU_INST_STATE_NONE)
wave6_vpu_dec_destroy_instance(inst, false);
...
}
drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c:wave6_vpu_finish_job() {
cancel_delayed_work(&inst->task_timer);
v4l2_m2m_job_finish(inst->dev->m2m_dev, inst->v4l2_fh.m2m_ctx);
}
wave6_vpu_dec_stop_streaming() does not destroy the instance (nor does
wave6_vpu_enc_stop_streaming() when only the OUTPUT queue was streaming),
so v4l2_m2m_ctx_release() frees m2m_ctx before destroy_instance() calls
wave6_vpu_disable_instance() to clear inst->enable, synchronize_irq() and
cancel_delayed_work_sync(&inst->task_timer):
CPU0 CPU1
wave6_vpu_dec_release()
v4l2_m2m_ctx_release(m2m_ctx)
kfree(m2m_ctx)
wave6_vpu_core_irq_thread()
inst->ops->finish_process()
...
v4l2_m2m_job_finish(..., m2m_ctx)
<-- use-after-free
wave6_vpu_dec_destroy_instance()
wave6_vpu_disable_instance()
In addition, wave6_vpu_finish_job() uses cancel_delayed_work() rather than
cancel_delayed_work_sync(), allowing an already-running
wave6_vpu_device_run_timeout() work item to access m2m_ctx and call
v4l2_m2m_job_finish() concurrently with frame completion.
[Severity: High]
In commit "media: chips-media: wave6: Add Wave6 m2m encoder", can starting
and stopping only the encoder OUTPUT queue leave the instance in
VPU_INST_STATE_PIC_RUN with stale format parameters and undersized FBC DMA
buffers?
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c:wave6_vpu_enc_start_streaming() {
...
if (inst->state == VPU_INST_STATE_NONE) {
ret = wave6_vpu_enc_create_instance(inst);
if (ret)
goto exit;
}
...
if (inst->state == VPU_INST_STATE_INIT_SEQ) {
ret = wave6_vpu_enc_prepare_fb(inst);
...
}
drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c:wave6_vpu_enc_stop_streaming() {
...
if (V4L2_TYPE_IS_CAPTURE(q->type) && inst->state != VPU_INST_STATE_NONE)
wave6_vpu_enc_destroy_instance(inst, true);
...
}
wave6_vpu_enc_start_streaming() creates the firmware instance and allocates
FBC buffers on the first queue that starts streaming (even if q->type is
OUTPUT), transitioning inst->state to VPU_INST_STATE_PIC_RUN. Because
wave6_vpu_enc_stop_streaming() only destroys the instance when stopping the
CAPTURE queue, stopping OUTPUT leaves inst->state at VPU_INST_STATE_PIC_RUN
while neither queue is streaming; a subsequent VIDIOC_S_FMT to a larger
resolution followed by VIDIOC_STREAMON skips instance recreation and FBC
buffer reallocation.
[Severity: High]
In commit "media: chips-media: wave6: Add Wave6 m2m decoder", does a
failure in wave6_vpu_dec_seek_header() trigger an infinite workqueue loop
in wave6_vpu_device_run()?
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c:wave6_vpu_dec_prepare_decode() {
...
wave6_handle_bitstream_buffer(inst, stream_end);
if (inst->state < VPU_INST_STATE_PIC_RUN) {
if (inst->state == VPU_INST_STATE_OPEN &&
wave6_vpu_dec_seek_header(inst)) {
vb2_queue_error(v4l2_m2m_get_src_vq(inst->v4l2_fh.m2m_ctx));
vb2_queue_error(v4l2_m2m_get_dst_vq(inst->v4l2_fh.m2m_ctx));
}
return -EAGAIN;
}
...
}
When wave6_vpu_dec_seek_header() fails (for example on an unsupported
profile or timeout), the source buffer remains queued on m2m_ctx and
inst->state stays VPU_INST_STATE_OPEN. Because wave6_vpu_job_ready()
unconditionally returns 1 when inst->state == VPU_INST_STATE_OPEN, calling
v4l2_m2m_job_finish() in wave6_vpu_device_run() immediately reschedules
wave6_vpu_device_run() in an endless loop.
[Severity: Medium]
In commit "media: chips-media: wave6: Add Wave6 m2m decoder", does
wave6_vpu_dec_destroy_instance() leave inst->fbc_buf_registered set to true
when an instance is destroyed and recreated during a seek?
drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c:wave6_vpu_dec_destroy_instance() {
...
wave6_vpu_dec_release_fb(inst);
wave6_vpu_set_instance_state(inst, VPU_INST_STATE_NONE);
...
}
When wave6_vpu_dec_queue_setup() calls wave6_vpu_dec_destroy_instance() in
VPU_INST_STATE_SEEK, the FBC buffers are freed and the firmware instance is
closed, but inst->fbc_buf_registered is not reset to false on the reused
struct vpu_instance. If streaming restarts and a drain occurs before new
FBC buffers are registered, wave6_vpu_dec_prepare_decode() sees the stale
inst->fbc_buf_registered flag and skips wave6_vpu_dec_finish_drain().
[Severity: Medium]
In commit "media: chips-media: wave6: Add Wave6 core driver", can
wave6_vpu_get() be skipped if a core device is already runtime-active when
an instance is created?
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c:wave6_vpu_core_runtime_resume() {
...
if (atomic_read(&core->active_count)) {
ret = wave6_vpu_get(core->vpu, core->reg_base);
if (ret)
goto error_get_vpu;
core->vpu_held = true;
}
...
}
If the core device is runtime-resumed while core->active_count is 0 (for
example via sysfs power/control), wave6_vpu_core_runtime_resume() finishes
with core->vpu_held == false. When wave6_vpu_dec_create_instance() or
wave6_vpu_enc_create_instance() later increments active_count and calls
pm_runtime_resume_and_get(inst->dev->dev), runtime PM sees the core device
is already RPM_ACTIVE and returns 0 without calling
wave6_vpu_core_runtime_resume(), leaving the parent VPU device unpowered.
[Severity: Low]
In commit "media: chips-media: wave6: Add trace and debugfs", does
wave6_vpu_core_probe() leak the top-level "wave6" debugfs directory?
drivers/media/platform/chips-media/wave6/wave6-vpu-core.c:wave6_vpu_core_probe() {
...
core->debugfs = debugfs_lookup(WAVE6_VPU_DEBUGFS_DIR, NULL);
if (!IS_ERR_OR_NULL(core->debugfs))
dput(core->debugfs);
else
core->debugfs = debugfs_create_dir(WAVE6_VPU_DEBUGFS_DIR, NULL);
...
}
Nothing removes this directory on probe failure, device removal, or module
unload.
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795075577.git.nas.chung@chipsnmedia.com?part=11
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v8 02/11] media: chips-media: wave6: Add Wave6 control driver
2026-11-19 9:26 ` sashiko-bot
@ 2026-11-19 19:36 ` Frank Li
0 siblings, 0 replies; 23+ messages in thread
From: Frank Li @ 2026-11-19 19:36 UTC (permalink / raw)
To: sashiko-reviews
Cc: Nas Chung, devicetree, imx, robh, conor+dt, Frank.Li, media-ci
On Thu, Nov 19, 2026 at 09:26:25AM +0000, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 8 potential
> issues to consider:
> - [High] media: wave6: vpu->lock held across core device register and
> unregister
> - [High] media: wave6: data race on work_buffers_avail in
> wave6_vpu_require_work_buffer()
> - [Medium] media: wave6: stuck PREPARE state on work buf error in
> wave6_vpu_init_vpu()
> - [Medium] media: wave6: incompatible phys_addr_t pointer in
> wave6_vpu_probe()
> - [Medium] media: wave6: wave6_vpu_probe() succeeds without required
> memory-region
> - [Medium] media: wave6: wrong product code value for WAVE677_CODE
Please fix above issues
Frank
> - [Low] media: wave6: off-by-one of_irq_get() error check in
> wave6_vpu_create_cores()
> - [Low] media: wave6: mixed guard(mutex) and goto cleanup in
> wave6_vpu_get()
> --
>
> commit 600333b152222fe3a820d242ccd857c34cf7d58c
> Author: Nas Chung <nas.chung@chipsnmedia.com>
>
> media: chips-media: wave6: Add Wave6 control driver
>
> This commit adds the control driver for the Chips&Media Wave6 video codec IP
> to manage firmware loading, shared memory resources, and child core devices.
>
> > diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpu.c b/drivers/media/platform/chips-media/wave6/wave6-vpu.c
> > new file mode 100644
> > index 0000000000000..9160ce53062ce
> > --- /dev/null
> > +++ b/drivers/media/platform/chips-media/wave6/wave6-vpu.c
> [ ... ]
> > +static int wave6_vpu_init_vpu(struct wave6_vpu_device *vpu,
> > + void __iomem *core_base)
> > +{
> > + int ret;
> > +
> > + lockdep_assert_held(&vpu->lock);
> > +
> > + /* try init directly as firmware is running */
> > + if (wave6_vdi_readl(core_base, W6_VPU_VCPU_CUR_PC))
> > + goto init_done;
> > +
> > + wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_PREPARE);
> > +
> > + wave6_vpu_remap_code_buf(vpu);
> > +
> > + wave6_vdi_writel(core_base, W6_VPU_BUSY_STATUS, BUSY_STATUS_SET);
> > + wave6_vdi_writel(core_base, W6_CMD_INIT_VPU_SEC_AXI_BASE_CORE0,
> > + vpu->sram_buf.dma_addr);
> > + wave6_vdi_writel(core_base, W6_CMD_INIT_VPU_SEC_AXI_SIZE_CORE0,
> > + vpu->sram_buf.size);
> > + wave6_vdi_writel(vpu->reg_base, W6_COMMAND_GB, W6_CMD_INIT_VPU);
> > + wave6_vdi_writel(vpu->reg_base, W6_VPU_REMAP_CORE_START_GB,
> > + REMAP_CORE_START_ON);
> > +
> > + ret = wave6_vpu_wait_busy(core_base);
> > + if (ret) {
> > + dev_err(vpu->dev, "init vpu timeout\n");
> > + wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_OFF);
> > + return -EINVAL;
> > + }
> > +
> > + ret = wave6_vpu_check_result(core_base);
> > + if (ret) {
> > + dev_err(vpu->dev, "init vpu fail, reason 0x%x\n", ret);
> > + wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_OFF);
> > + return -EIO;
> > + }
> > +
> > +init_done:
> > + ret = wave6_vpu_init_work_buf(vpu, core_base);
> > + if (ret)
> > + return ret;
> > +
> > + wave6_vpu_set_state(vpu, WAVE6_VPU_STATE_ON);
> > +
> > + return 0;
> > +}
>
> [Severity: Medium]
> Does returning on wave6_vpu_init_work_buf() failure leave vpu->state stuck
> in WAVE6_VPU_STATE_PREPARE?
>
> Unlike the timeout and error paths above init_done, this path does not
> reset vpu->state to WAVE6_VPU_STATE_OFF. wave6_vpu_get() then powers down
> the device and drops core_count to 0, and every later wave6_vpu_get() ->
> wave6_vpu_try_boot() sees vpu->state == WAVE6_VPU_STATE_PREPARE and returns
> 0 without booting the VPU or initializing work buffers.
>
> [ ... ]
>
> > +int wave6_vpu_get(struct wave6_vpu_device *vpu, void __iomem *core_base)
> > +{
> > + int ret;
> > +
> > + guard(mutex)(&vpu->lock);
> > +
> > + /* Only the first core executes boot; others return */
> > + if (atomic_inc_return(&vpu->core_count) > 1)
> > + return 0;
> > +
> > + ret = pm_runtime_resume_and_get(vpu->dev);
> > + if (ret)
> > + goto error_pm;
> > +
> > + ret = wave6_vpu_try_boot(vpu, core_base);
> > + if (ret)
> > + goto error_boot;
> > +
> > + return 0;
> > +
> > +error_boot:
> > + pm_runtime_put_sync(vpu->dev);
> > +error_pm:
> > + atomic_dec(&vpu->core_count);
> > +
> > + return ret;
> > +}
>
> [Severity: Low]
> This isn't a bug, but wave6_vpu_get() mixes guard(mutex) with goto-based
> error unwinding in the same function, contrary to the Cleanup API
> conventions in include/linux/cleanup.h.
>
> [ ... ]
>
> > +void wave6_vpu_require_work_buffer(struct wave6_vpu_device *vpu,
> > + void __iomem *core_base)
> > +{
> > + struct vpu_buf *vb;
> > + u32 size;
> > +
> > + size = wave6_vdi_readl(core_base, W6_CMD_SET_WORK_BUF_SIZE);
> > + if (!size)
> > + return;
> > +
> > + if (WARN_ON(size > W637DEC_WORKBUF_SIZE_FOR_CQ))
> > + goto exit;
> > +
> > + if (WARN_ON(vpu->work_buffers_avail == 0))
> > + goto exit;
> > +
> > + vpu->work_buffers_avail--;
> > + vb = &vpu->work_buffers[vpu->work_buffers_avail];
> > +
> > + wave6_vdi_writel(core_base, W6_CMD_SET_WORK_BUF_ADDR, vb->daddr);
> > +
> > +exit:
> > + wave6_vdi_writel(core_base, W6_CMD_SET_WORK_BUF_SIZE, SET_WORK_BUF_SIZE_ACK);
> > +}
>
> [Severity: High]
> Can concurrent calls to wave6_vpu_require_work_buffer() across VPU cores
> race on vpu->work_buffers_avail?
>
> In commit "media: chips-media: wave6: Add Wave6 core driver", each child
> core device calls wave6_vpu_require_work_buffer() directly from its
> wave6_vpu_core_irq() hardirq handler without holding any shared lock:
>
> CPU0 (<hardirq> core 0) CPU1 (<hardirq> core 1)
> wave6_vpu_core_irq() wave6_vpu_core_irq()
> wave6_vpu_require_work_buffer() wave6_vpu_require_work_buffer()
> if (work_buffers_avail == 0) if (work_buffers_avail == 0)
> work_buffers_avail--; work_buffers_avail--;
> vb = &work_buffers[avail]; vb = &work_buffers[avail];
> <-- duplicate buffer or underflow
>
> Two cores can be assigned the same work_buffers[] entry, or when
> vpu->work_buffers_avail is 1, both cores can pass the zero check and
> underflow the u32 counter to 0xffffffff, reading vpu->work_buffers[] out of
> bounds.
>
> Also, when jumping to exit on error, should W6_CMD_SET_WORK_BUF_ADDR be
> cleared before writing SET_WORK_BUF_SIZE_ACK so the firmware does not
> consume a stale address?
>
> > +static int wave6_vpu_create_cores(struct wave6_vpu_device *vpu)
> > +{
> > + struct device_node *child;
> > + int num_cores = 0;
> > +
> > + for_each_available_child_of_node(vpu->dev->of_node, child) {
> > + struct platform_device_info info = {};
> > + struct platform_device *pdev;
> > + struct resource res[2];
> > + int irq;
> > +
> > + if (num_cores >= W6_VPU_MAX_NUM_CORE) {
> > + of_node_put(child);
> > + break;
> > + }
> > +
> > + if (of_address_to_resource(child, 0, &res[0])) {
> > + dev_warn(vpu->dev, "%pOF: missing reg property\n", child);
> > + continue;
> > + }
> > +
> > + irq = of_irq_get(child, 0);
> > + if (irq < 0) {
> > + dev_warn(vpu->dev, "%pOF: missing interrupts property\n", child);
> > + continue;
> > + }
> > + res[1] = DEFINE_RES_IRQ(irq);
>
> [Severity: Low]
> Should the error check for of_irq_get() be "if (irq <= 0)"?
>
> of_irq_get() returns 0 when IRQ mapping fails, which would register the
> child platform device with an invalid IRQ 0 resource.
>
> [ ... ]
>
> > +static void wave6_vpu_release(struct wave6_vpu_device *vpu)
> > +{
> > + guard(mutex)(&vpu->lock);
> > +
> > + wave6_vpu_destroy_cores(vpu);
> > + wave6_vpu_free_work_buffers(vpu);
> [ ... ]
> > +}
> > +
> > +static void wave6_vpu_init_firmware(struct wave6_vpu_device *vpu,
> > + const struct firmware *fw)
> > +{
> > + guard(mutex)(&vpu->lock);
> > +
> > + if (!fw || !fw->data) {
> > + dev_err(vpu->dev, "No firmware.\n");
> > + return;
> > + }
> > +
> > + if (fw->size + W6_EXTRA_CODE_BUF_SIZE > wave6_vpu_get_code_buf_size(vpu)) {
> > + dev_err(vpu->dev, "firmware size (%zd > %zd) is too big\n",
> > + fw->size, vpu->code_buf.size);
> > + return;
> > + }
> > +
> > + memcpy(vpu->code_buf.vaddr, fw->data, fw->size);
> > +
> > + if (!wave6_vpu_create_cores(vpu))
> > + dev_err(vpu->dev, "Failed to create VPU cores\n");
> > +}
>
> [Severity: High]
> Does holding vpu->lock across wave6_vpu_create_cores() and
> wave6_vpu_destroy_cores() invert the lock hierarchy against child core
> device locks and runtime PM transitions?
>
> wave6_vpu_create_cores() -> platform_device_register_full() and
> wave6_vpu_destroy_cores() -> platform_device_unregister() acquire the child
> core device's device_lock (core_dev->mutex) while vpu->lock is held, and
> platform_device_unregister() -> __device_release_driver() also runs
> pm_runtime_get_sync(core_dev) and pm_runtime_put_sync(core_dev). In commit
> "media: chips-media: wave6: Add Wave6 core driver", the child's runtime PM
> and system sleep callbacks call wave6_vpu_get() and wave6_vpu_put(), which
> acquire vpu->lock under core_dev->mutex or during a core_dev runtime PM
> transition:
>
> CPU0 CPU1
> ---- ----
> lock(&vpu->lock)
> lock(&core_dev->mutex)
> lock(&core_dev->mutex)
> lock(&vpu->lock)
>
> *** DEADLOCK ***
>
> CPU0: wave6_vpu_init_firmware() -> wave6_vpu_create_cores() ->
> platform_device_register_full()
> CPU1: device_suspend() -> wave6_vpu_core_suspend() ->
> pm_runtime_force_suspend() -> wave6_vpu_core_runtime_suspend() ->
> wave6_vpu_put()
>
> During teardown, if core_dev is in RPM_RESUMING or RPM_SUSPENDING waiting
> for vpu->lock in wave6_vpu_get() or wave6_vpu_put() when
> wave6_vpu_release() runs, __device_release_driver()'s
> pm_runtime_get_sync(core_dev) will also wait on core_dev->power.wait_queue
> while holding vpu->lock.
>
> [ ... ]
>
> > + np = of_parse_phandle(pdev->dev.of_node, "memory-region", 0);
> > + if (np) {
> > + struct resource mem;
> > +
> > + ret = of_address_to_resource(np, 0, &mem);
> > + of_node_put(np);
> > + if (!ret) {
> > + vpu->code_buf.phys_addr = mem.start;
> > + vpu->code_buf.size = resource_size(&mem);
> > + ret = wave6_vpu_init_code_buf(vpu);
> > + if (ret)
> > + goto error_detach_pm_domains;
> > + } else {
> > + dev_warn(&pdev->dev, "memory-region is not available.\n");
> > + }
> > + }
>
> [Severity: Medium]
> Should wave6_vpu_probe() fail and jump to error_detach_pm_domains when
> "memory-region" is missing or of_address_to_resource() fails?
>
> Without it, vpu->code_buf.size remains 0 while wave6_vpu_probe() continues,
> allocates 48 MiB of coherent DMA work buffers and SRAM, and returns 0, only
> for wave6_vpu_init_firmware() to unconditionally fail its size check against
> wave6_vpu_get_code_buf_size(vpu) and leave those resources allocated without
> creating any core devices.
>
> > + vpu->sram_pool = of_gen_pool_get(pdev->dev.of_node, "sram", 0);
> > + if (vpu->sram_pool) {
> > + vpu->sram_buf.size = vpu->res->sram_size;
> > + vpu->sram_buf.vaddr = gen_pool_dma_alloc(vpu->sram_pool,
> > + vpu->sram_buf.size,
> > + &vpu->sram_buf.phys_addr);
>
> [Severity: Medium]
> Does passing &vpu->sram_buf.phys_addr to gen_pool_dma_alloc() fail to build
> when phys_addr_t and dma_addr_t have different widths?
>
> struct wave6_vpu_dma_buf declares phys_addr as phys_addr_t, whereas
> gen_pool_dma_alloc() takes a dma_addr_t * third argument, which triggers
> -Werror=incompatible-pointer-types on configs where
> CONFIG_PHYS_ADDR_T_64BIT and CONFIG_ARCH_DMA_ADDR_T_64BIT differ (exposed by
> COMPILE_TEST).
>
> > diff --git a/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h b/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
> > new file mode 100644
> > index 0000000000000..1d4e016c2a582
> > --- /dev/null
> > +++ b/drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
> > @@ -0,0 +1,37 @@
> [ ... ]
> > +#define WAVE617_CODE 0x6170
> > +#define WAVE627_CODE 0x6270
> > +#define WAVE633_CODE 0x6330
> > +#define WAVE637_CODE 0x6370
> > +#define WAVE663_CODE 0x6630
> > +#define WAVE677_CODE 0x6670
>
> [Severity: Medium]
> Should WAVE677_CODE be 0x6770 instead of 0x6670?
>
> All other Wave6 product codes encode the three-digit model number in bits
> [15:4], so wave6_is_product_w_series() in wave6_vpu_try_boot() would reject
> a Wave677 reporting 0x6770 with -EINVAL.
>
> --
> Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795075577.git.nas.chung@chipsnmedia.com?part=2
>
^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2026-11-19 19:36 UTC | newest]
Thread overview: 23+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-19 8:59 [PATCH v8 00/11] Add support for Wave6 video codec driver Nas Chung
2026-11-19 8:59 ` [PATCH v8 01/11] dt-bindings: media: nxp: Add Wave6 video codec device Nas Chung
2026-11-19 8:59 ` [PATCH v8 02/11] media: chips-media: wave6: Add Wave6 control driver Nas Chung
2026-11-19 9:26 ` sashiko-bot
2026-11-19 19:36 ` Frank Li
2026-11-19 8:59 ` [PATCH v8 03/11] media: chips-media: wave6: Add Wave6 core driver Nas Chung
2026-11-19 9:32 ` sashiko-bot
2026-11-19 8:59 ` [PATCH v8 04/11] media: chips-media: wave6: Add Wave6 decoder firmware interface Nas Chung
2026-11-19 9:22 ` sashiko-bot
2026-11-19 8:59 ` [PATCH v8 05/11] media: chips-media: wave6: Add Wave6 m2m decoder Nas Chung
2026-11-19 9:31 ` sashiko-bot
2026-11-19 9:00 ` [PATCH v8 06/11] media: chips-media: wave6: Add Wave6 encoder firmware interface Nas Chung
2026-11-19 9:10 ` sashiko-bot
2026-11-19 9:00 ` [PATCH v8 07/11] media: chips-media: wave6: Add Wave6 m2m encoder Nas Chung
2026-11-19 9:35 ` sashiko-bot
2026-11-19 9:00 ` [PATCH v8 08/11] media: chips-media: wave6: Add trace and debugfs Nas Chung
2026-11-19 9:14 ` sashiko-bot
2026-11-19 9:00 ` [PATCH v8 09/11] media: chips-media: wave6: Add performance profiling Nas Chung
2026-11-19 9:20 ` sashiko-bot
2026-11-19 9:00 ` [PATCH v8 10/11] media: chips-media: wave6: Add Wave6 thermal cooling device Nas Chung
2026-11-19 9:21 ` sashiko-bot
2026-11-19 9:00 ` [PATCH v8 11/11] arm64: dts: freescale: imx95: Add video codec node Nas Chung
2026-11-19 9:37 ` sashiko-bot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox