[PATCH v2 0/4] media: rockchip: Add rkvdec2 driver

[PATCH v2 0/4] media: rockchip: Add rkvdec2 driver

[Detlev Casanova]

Add a rkvdec2 driver for newer video decoder found on rk3588 based SoC.

It is also found on other hardware like the RK356x, but not tested yet,
so it will be added later.

Support for yuv 4:2:2 and 10 bits streams will also be added later, when the
https://lore.kernel.org/linux-media/20240618194647.742037-1-jonas@kwiboo.se
patch set is merged.

See the second commit message for more details.

Changes since v1:
- Add support for RCB in SRAM
- Move bindings to rockchip,vdec.yaml
- Add resets bindings
- Add second core and enable them from dtsi file
- Only expose one video device to userspace (but don't support multicore yet)
- Share CABAC table with rkvdec
- Fix iowrite32 call and add preliminary support for arm
- Remove unused code
- Improve registers naming and code style
- Remove DMA_ATTR_ALLOC_SINGLE_PAGES flag

Detlev Casanova (4):
  media: rockchip: Move H264 CABAC table to header file
  media: rockchip: Introduce the rkvdec2 driver
  media: dt-bindings: rockchip: Document RK3588 Video Decoder bindings
  arm64: dts: rockchip: Add rkvdec2 Video Decoder on rk3588(s)

 .../bindings/media/rockchip,vdec.yaml         |   46 +
 arch/arm64/boot/dts/rockchip/rk3588s.dtsi     |   50 +
 drivers/staging/media/Kconfig                 |    1 +
 drivers/staging/media/Makefile                |    1 +
 .../staging/media/rkvdec/rkvdec-h264-cabac.h  |  509 +++++++
 drivers/staging/media/rkvdec/rkvdec-h264.c    |  500 +------
 drivers/staging/media/rkvdec2/Kconfig         |   15 +
 drivers/staging/media/rkvdec2/Makefile        |    3 +
 drivers/staging/media/rkvdec2/TODO            |   12 +
 drivers/staging/media/rkvdec2/rkvdec2-h264.c  |  739 ++++++++++
 drivers/staging/media/rkvdec2/rkvdec2-regs.h  |  345 +++++
 drivers/staging/media/rkvdec2/rkvdec2.c       | 1253 +++++++++++++++++
 drivers/staging/media/rkvdec2/rkvdec2.h       |  130 ++
 13 files changed, 3105 insertions(+), 499 deletions(-)
 create mode 100644 drivers/staging/media/rkvdec/rkvdec-h264-cabac.h
 create mode 100644 drivers/staging/media/rkvdec2/Kconfig
 create mode 100644 drivers/staging/media/rkvdec2/Makefile
 create mode 100644 drivers/staging/media/rkvdec2/TODO
 create mode 100644 drivers/staging/media/rkvdec2/rkvdec2-h264.c
 create mode 100644 drivers/staging/media/rkvdec2/rkvdec2-regs.h
 create mode 100644 drivers/staging/media/rkvdec2/rkvdec2.c
 create mode 100644 drivers/staging/media/rkvdec2/rkvdec2.h

-- 
2.44.2

[PATCH v2 1/4] media: rockchip: Move H264 CABAC table to header file

[Detlev Casanova]

The table will be shared with the rkvdec2 driver in following commits.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
---
 .../staging/media/rkvdec/rkvdec-h264-cabac.h  | 509 ++++++++++++++++++
 drivers/staging/media/rkvdec/rkvdec-h264.c    | 500 +----------------
 2 files changed, 510 insertions(+), 499 deletions(-)
 create mode 100644 drivers/staging/media/rkvdec/rkvdec-h264-cabac.h

diff --git a/drivers/staging/media/rkvdec/rkvdec-h264-cabac.h b/drivers/staging/media/rkvdec/rkvdec-h264-cabac.h
new file mode 100644
index 000000000000..3324c4cb6130
--- /dev/null
+++ b/drivers/staging/media/rkvdec/rkvdec-h264-cabac.h
@@ -0,0 +1,509 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Define the H264 CABAC table common to rkvdec and rkvdec2 drivers.
+ */
+
+#ifndef RKVDEC_H264_CABAC_H_
+#define RKVDEC_H264_CABAC_H_
+
+#define CABAC_ENTRY(ctxidx, idc0_m, idc0_n, idc1_m, idc1_n,	\
+		    idc2_m, idc2_n, intra_m, intra_n)		\
+	[0][(ctxidx)] = {idc0_m, idc0_n},			\
+	[1][(ctxidx)] = {idc1_m, idc1_n},			\
+	[2][(ctxidx)] = {idc2_m, idc2_n},			\
+	[3][(ctxidx)] = {intra_m, intra_n}
+
+/*
+ * Constant CABAC table.
+ * Built from the tables described in section '9.3.1.1 Initialisation process
+ * for context variables' of the H264 spec.
+ */
+static const s8 rkvdec_h264_cabac_table[4][464][2] = {
+	/* Table 9-12 – Values of variables m and n for ctxIdx from 0 to 10 */
+	CABAC_ENTRY(0, 20, -15, 20, -15, 20, -15, 20, -15),
+	CABAC_ENTRY(1, 2, 54, 2, 54, 2, 54, 2, 54),
+	CABAC_ENTRY(2, 3, 74, 3, 74, 3, 74, 3, 74),
+	CABAC_ENTRY(3, 20, -15, 20, -15, 20, -15, 20, -15),
+	CABAC_ENTRY(4, 2, 54, 2, 54, 2, 54, 2, 54),
+	CABAC_ENTRY(5, 3, 74, 3, 74, 3, 74, 3, 74),
+	CABAC_ENTRY(6, -28, 127, -28, 127, -28, 127, -28, 127),
+	CABAC_ENTRY(7, -23, 104, -23, 104, -23, 104, -23, 104),
+	CABAC_ENTRY(8, -6, 53, -6, 53, -6, 53, -6, 53),
+	CABAC_ENTRY(9, -1, 54, -1, 54, -1, 54, -1, 54),
+	CABAC_ENTRY(10, 7, 51, 7, 51, 7, 51, 7, 51),
+
+	/* Table 9-13 – Values of variables m and n for ctxIdx from 11 to 23 */
+	CABAC_ENTRY(11, 23, 33, 22, 25, 29, 16, 0, 0),
+	CABAC_ENTRY(12, 23, 2, 34, 0, 25, 0, 0, 0),
+	CABAC_ENTRY(13, 21, 0, 16, 0, 14, 0, 0, 0),
+	CABAC_ENTRY(14, 1, 9, -2, 9, -10, 51, 0, 0),
+	CABAC_ENTRY(15, 0, 49, 4, 41, -3, 62, 0, 0),
+	CABAC_ENTRY(16, -37, 118, -29, 118, -27, 99, 0, 0),
+	CABAC_ENTRY(17, 5, 57, 2, 65, 26, 16, 0, 0),
+	CABAC_ENTRY(18, -13, 78, -6, 71, -4, 85, 0, 0),
+	CABAC_ENTRY(19, -11, 65, -13, 79, -24, 102, 0, 0),
+	CABAC_ENTRY(20, 1, 62, 5, 52, 5, 57, 0, 0),
+	CABAC_ENTRY(21, 12, 49, 9, 50, 6, 57, 0, 0),
+	CABAC_ENTRY(22, -4, 73, -3, 70, -17, 73, 0, 0),
+	CABAC_ENTRY(23, 17, 50, 10, 54, 14, 57, 0, 0),
+
+	/* Table 9-14 – Values of variables m and n for ctxIdx from 24 to 39 */
+	CABAC_ENTRY(24, 18, 64, 26, 34, 20, 40, 0, 0),
+	CABAC_ENTRY(25, 9, 43, 19, 22, 20, 10, 0, 0),
+	CABAC_ENTRY(26, 29, 0, 40, 0, 29, 0, 0, 0),
+	CABAC_ENTRY(27, 26, 67, 57, 2, 54, 0, 0, 0),
+	CABAC_ENTRY(28, 16, 90, 41, 36, 37, 42, 0, 0),
+	CABAC_ENTRY(29, 9, 104, 26, 69, 12, 97, 0, 0),
+	CABAC_ENTRY(30, -46, 127, -45, 127, -32, 127, 0, 0),
+	CABAC_ENTRY(31, -20, 104, -15, 101, -22, 117, 0, 0),
+	CABAC_ENTRY(32, 1, 67, -4, 76, -2, 74, 0, 0),
+	CABAC_ENTRY(33, -13, 78, -6, 71, -4, 85, 0, 0),
+	CABAC_ENTRY(34, -11, 65, -13, 79, -24, 102, 0, 0),
+	CABAC_ENTRY(35, 1, 62, 5, 52, 5, 57, 0, 0),
+	CABAC_ENTRY(36, -6, 86, 6, 69, -6, 93, 0, 0),
+	CABAC_ENTRY(37, -17, 95, -13, 90, -14, 88, 0, 0),
+	CABAC_ENTRY(38, -6, 61, 0, 52, -6, 44, 0, 0),
+	CABAC_ENTRY(39, 9, 45, 8, 43, 4, 55, 0, 0),
+
+	/* Table 9-15 – Values of variables m and n for ctxIdx from 40 to 53 */
+	CABAC_ENTRY(40, -3, 69, -2, 69, -11, 89, 0, 0),
+	CABAC_ENTRY(41, -6, 81, -5, 82, -15, 103, 0, 0),
+	CABAC_ENTRY(42, -11, 96, -10, 96, -21, 116, 0, 0),
+	CABAC_ENTRY(43, 6, 55, 2, 59, 19, 57, 0, 0),
+	CABAC_ENTRY(44, 7, 67, 2, 75, 20, 58, 0, 0),
+	CABAC_ENTRY(45, -5, 86, -3, 87, 4, 84, 0, 0),
+	CABAC_ENTRY(46, 2, 88, -3, 100, 6, 96, 0, 0),
+	CABAC_ENTRY(47, 0, 58, 1, 56, 1, 63, 0, 0),
+	CABAC_ENTRY(48, -3, 76, -3, 74, -5, 85, 0, 0),
+	CABAC_ENTRY(49, -10, 94, -6, 85, -13, 106, 0, 0),
+	CABAC_ENTRY(50, 5, 54, 0, 59, 5, 63, 0, 0),
+	CABAC_ENTRY(51, 4, 69, -3, 81, 6, 75, 0, 0),
+	CABAC_ENTRY(52, -3, 81, -7, 86, -3, 90, 0, 0),
+	CABAC_ENTRY(53, 0, 88, -5, 95, -1, 101, 0, 0),
+
+	/* Table 9-16 – Values of variables m and n for ctxIdx from 54 to 59 */
+	CABAC_ENTRY(54, -7, 67, -1, 66, 3, 55, 0, 0),
+	CABAC_ENTRY(55, -5, 74, -1, 77, -4, 79, 0, 0),
+	CABAC_ENTRY(56, -4, 74, 1, 70, -2, 75, 0, 0),
+	CABAC_ENTRY(57, -5, 80, -2, 86, -12, 97, 0, 0),
+	CABAC_ENTRY(58, -7, 72, -5, 72, -7, 50, 0, 0),
+	CABAC_ENTRY(59, 1, 58, 0, 61, 1, 60, 0, 0),
+
+	/* Table 9-17 – Values of variables m and n for ctxIdx from 60 to 69 */
+	CABAC_ENTRY(60, 0, 41, 0, 41, 0, 41, 0, 41),
+	CABAC_ENTRY(61, 0, 63, 0, 63, 0, 63, 0, 63),
+	CABAC_ENTRY(62, 0, 63, 0, 63, 0, 63, 0, 63),
+	CABAC_ENTRY(63, 0, 63, 0, 63, 0, 63, 0, 63),
+	CABAC_ENTRY(64, -9, 83, -9, 83, -9, 83, -9, 83),
+	CABAC_ENTRY(65, 4, 86, 4, 86, 4, 86, 4, 86),
+	CABAC_ENTRY(66, 0, 97, 0, 97, 0, 97, 0, 97),
+	CABAC_ENTRY(67, -7, 72, -7, 72, -7, 72, -7, 72),
+	CABAC_ENTRY(68, 13, 41, 13, 41, 13, 41, 13, 41),
+	CABAC_ENTRY(69, 3, 62, 3, 62, 3, 62, 3, 62),
+
+	/* Table 9-18 – Values of variables m and n for ctxIdx from 70 to 104 */
+	CABAC_ENTRY(70, 0, 45, 13, 15, 7, 34, 0, 11),
+	CABAC_ENTRY(71, -4, 78, 7, 51, -9, 88, 1, 55),
+	CABAC_ENTRY(72, -3, 96, 2, 80, -20, 127, 0, 69),
+	CABAC_ENTRY(73, -27, 126, -39, 127, -36, 127, -17, 127),
+	CABAC_ENTRY(74, -28, 98, -18, 91, -17, 91, -13, 102),
+	CABAC_ENTRY(75, -25, 101, -17, 96, -14, 95, 0, 82),
+	CABAC_ENTRY(76, -23, 67, -26, 81, -25, 84, -7, 74),
+	CABAC_ENTRY(77, -28, 82, -35, 98, -25, 86, -21, 107),
+	CABAC_ENTRY(78, -20, 94, -24, 102, -12, 89, -27, 127),
+	CABAC_ENTRY(79, -16, 83, -23, 97, -17, 91, -31, 127),
+	CABAC_ENTRY(80, -22, 110, -27, 119, -31, 127, -24, 127),
+	CABAC_ENTRY(81, -21, 91, -24, 99, -14, 76, -18, 95),
+	CABAC_ENTRY(82, -18, 102, -21, 110, -18, 103, -27, 127),
+	CABAC_ENTRY(83, -13, 93, -18, 102, -13, 90, -21, 114),
+	CABAC_ENTRY(84, -29, 127, -36, 127, -37, 127, -30, 127),
+	CABAC_ENTRY(85, -7, 92, 0, 80, 11, 80, -17, 123),
+	CABAC_ENTRY(86, -5, 89, -5, 89, 5, 76, -12, 115),
+	CABAC_ENTRY(87, -7, 96, -7, 94, 2, 84, -16, 122),
+	CABAC_ENTRY(88, -13, 108, -4, 92, 5, 78, -11, 115),
+	CABAC_ENTRY(89, -3, 46, 0, 39, -6, 55, -12, 63),
+	CABAC_ENTRY(90, -1, 65, 0, 65, 4, 61, -2, 68),
+	CABAC_ENTRY(91, -1, 57, -15, 84, -14, 83, -15, 84),
+	CABAC_ENTRY(92, -9, 93, -35, 127, -37, 127, -13, 104),
+	CABAC_ENTRY(93, -3, 74, -2, 73, -5, 79, -3, 70),
+	CABAC_ENTRY(94, -9, 92, -12, 104, -11, 104, -8, 93),
+	CABAC_ENTRY(95, -8, 87, -9, 91, -11, 91, -10, 90),
+	CABAC_ENTRY(96, -23, 126, -31, 127, -30, 127, -30, 127),
+	CABAC_ENTRY(97, 5, 54, 3, 55, 0, 65, -1, 74),
+	CABAC_ENTRY(98, 6, 60, 7, 56, -2, 79, -6, 97),
+	CABAC_ENTRY(99, 6, 59, 7, 55, 0, 72, -7, 91),
+	CABAC_ENTRY(100, 6, 69, 8, 61, -4, 92, -20, 127),
+	CABAC_ENTRY(101, -1, 48, -3, 53, -6, 56, -4, 56),
+	CABAC_ENTRY(102, 0, 68, 0, 68, 3, 68, -5, 82),
+	CABAC_ENTRY(103, -4, 69, -7, 74, -8, 71, -7, 76),
+	CABAC_ENTRY(104, -8, 88, -9, 88, -13, 98, -22, 125),
+
+	/* Table 9-19 – Values of variables m and n for ctxIdx from 105 to 165 */
+	CABAC_ENTRY(105, -2, 85, -13, 103, -4, 86, -7, 93),
+	CABAC_ENTRY(106, -6, 78, -13, 91, -12, 88, -11, 87),
+	CABAC_ENTRY(107, -1, 75, -9, 89, -5, 82, -3, 77),
+	CABAC_ENTRY(108, -7, 77, -14, 92, -3, 72, -5, 71),
+	CABAC_ENTRY(109, 2, 54, -8, 76, -4, 67, -4, 63),
+	CABAC_ENTRY(110, 5, 50, -12, 87, -8, 72, -4, 68),
+	CABAC_ENTRY(111, -3, 68, -23, 110, -16, 89, -12, 84),
+	CABAC_ENTRY(112, 1, 50, -24, 105, -9, 69, -7, 62),
+	CABAC_ENTRY(113, 6, 42, -10, 78, -1, 59, -7, 65),
+	CABAC_ENTRY(114, -4, 81, -20, 112, 5, 66, 8, 61),
+	CABAC_ENTRY(115, 1, 63, -17, 99, 4, 57, 5, 56),
+	CABAC_ENTRY(116, -4, 70, -78, 127, -4, 71, -2, 66),
+	CABAC_ENTRY(117, 0, 67, -70, 127, -2, 71, 1, 64),
+	CABAC_ENTRY(118, 2, 57, -50, 127, 2, 58, 0, 61),
+	CABAC_ENTRY(119, -2, 76, -46, 127, -1, 74, -2, 78),
+	CABAC_ENTRY(120, 11, 35, -4, 66, -4, 44, 1, 50),
+	CABAC_ENTRY(121, 4, 64, -5, 78, -1, 69, 7, 52),
+	CABAC_ENTRY(122, 1, 61, -4, 71, 0, 62, 10, 35),
+	CABAC_ENTRY(123, 11, 35, -8, 72, -7, 51, 0, 44),
+	CABAC_ENTRY(124, 18, 25, 2, 59, -4, 47, 11, 38),
+	CABAC_ENTRY(125, 12, 24, -1, 55, -6, 42, 1, 45),
+	CABAC_ENTRY(126, 13, 29, -7, 70, -3, 41, 0, 46),
+	CABAC_ENTRY(127, 13, 36, -6, 75, -6, 53, 5, 44),
+	CABAC_ENTRY(128, -10, 93, -8, 89, 8, 76, 31, 17),
+	CABAC_ENTRY(129, -7, 73, -34, 119, -9, 78, 1, 51),
+	CABAC_ENTRY(130, -2, 73, -3, 75, -11, 83, 7, 50),
+	CABAC_ENTRY(131, 13, 46, 32, 20, 9, 52, 28, 19),
+	CABAC_ENTRY(132, 9, 49, 30, 22, 0, 67, 16, 33),
+	CABAC_ENTRY(133, -7, 100, -44, 127, -5, 90, 14, 62),
+	CABAC_ENTRY(134, 9, 53, 0, 54, 1, 67, -13, 108),
+	CABAC_ENTRY(135, 2, 53, -5, 61, -15, 72, -15, 100),
+	CABAC_ENTRY(136, 5, 53, 0, 58, -5, 75, -13, 101),
+	CABAC_ENTRY(137, -2, 61, -1, 60, -8, 80, -13, 91),
+	CABAC_ENTRY(138, 0, 56, -3, 61, -21, 83, -12, 94),
+	CABAC_ENTRY(139, 0, 56, -8, 67, -21, 64, -10, 88),
+	CABAC_ENTRY(140, -13, 63, -25, 84, -13, 31, -16, 84),
+	CABAC_ENTRY(141, -5, 60, -14, 74, -25, 64, -10, 86),
+	CABAC_ENTRY(142, -1, 62, -5, 65, -29, 94, -7, 83),
+	CABAC_ENTRY(143, 4, 57, 5, 52, 9, 75, -13, 87),
+	CABAC_ENTRY(144, -6, 69, 2, 57, 17, 63, -19, 94),
+	CABAC_ENTRY(145, 4, 57, 0, 61, -8, 74, 1, 70),
+	CABAC_ENTRY(146, 14, 39, -9, 69, -5, 35, 0, 72),
+	CABAC_ENTRY(147, 4, 51, -11, 70, -2, 27, -5, 74),
+	CABAC_ENTRY(148, 13, 68, 18, 55, 13, 91, 18, 59),
+	CABAC_ENTRY(149, 3, 64, -4, 71, 3, 65, -8, 102),
+	CABAC_ENTRY(150, 1, 61, 0, 58, -7, 69, -15, 100),
+	CABAC_ENTRY(151, 9, 63, 7, 61, 8, 77, 0, 95),
+	CABAC_ENTRY(152, 7, 50, 9, 41, -10, 66, -4, 75),
+	CABAC_ENTRY(153, 16, 39, 18, 25, 3, 62, 2, 72),
+	CABAC_ENTRY(154, 5, 44, 9, 32, -3, 68, -11, 75),
+	CABAC_ENTRY(155, 4, 52, 5, 43, -20, 81, -3, 71),
+	CABAC_ENTRY(156, 11, 48, 9, 47, 0, 30, 15, 46),
+	CABAC_ENTRY(157, -5, 60, 0, 44, 1, 7, -13, 69),
+	CABAC_ENTRY(158, -1, 59, 0, 51, -3, 23, 0, 62),
+	CABAC_ENTRY(159, 0, 59, 2, 46, -21, 74, 0, 65),
+	CABAC_ENTRY(160, 22, 33, 19, 38, 16, 66, 21, 37),
+	CABAC_ENTRY(161, 5, 44, -4, 66, -23, 124, -15, 72),
+	CABAC_ENTRY(162, 14, 43, 15, 38, 17, 37, 9, 57),
+	CABAC_ENTRY(163, -1, 78, 12, 42, 44, -18, 16, 54),
+	CABAC_ENTRY(164, 0, 60, 9, 34, 50, -34, 0, 62),
+	CABAC_ENTRY(165, 9, 69, 0, 89, -22, 127, 12, 72),
+
+	/* Table 9-20 – Values of variables m and n for ctxIdx from 166 to 226 */
+	CABAC_ENTRY(166, 11, 28, 4, 45, 4, 39, 24, 0),
+	CABAC_ENTRY(167, 2, 40, 10, 28, 0, 42, 15, 9),
+	CABAC_ENTRY(168, 3, 44, 10, 31, 7, 34, 8, 25),
+	CABAC_ENTRY(169, 0, 49, 33, -11, 11, 29, 13, 18),
+	CABAC_ENTRY(170, 0, 46, 52, -43, 8, 31, 15, 9),
+	CABAC_ENTRY(171, 2, 44, 18, 15, 6, 37, 13, 19),
+	CABAC_ENTRY(172, 2, 51, 28, 0, 7, 42, 10, 37),
+	CABAC_ENTRY(173, 0, 47, 35, -22, 3, 40, 12, 18),
+	CABAC_ENTRY(174, 4, 39, 38, -25, 8, 33, 6, 29),
+	CABAC_ENTRY(175, 2, 62, 34, 0, 13, 43, 20, 33),
+	CABAC_ENTRY(176, 6, 46, 39, -18, 13, 36, 15, 30),
+	CABAC_ENTRY(177, 0, 54, 32, -12, 4, 47, 4, 45),
+	CABAC_ENTRY(178, 3, 54, 102, -94, 3, 55, 1, 58),
+	CABAC_ENTRY(179, 2, 58, 0, 0, 2, 58, 0, 62),
+	CABAC_ENTRY(180, 4, 63, 56, -15, 6, 60, 7, 61),
+	CABAC_ENTRY(181, 6, 51, 33, -4, 8, 44, 12, 38),
+	CABAC_ENTRY(182, 6, 57, 29, 10, 11, 44, 11, 45),
+	CABAC_ENTRY(183, 7, 53, 37, -5, 14, 42, 15, 39),
+	CABAC_ENTRY(184, 6, 52, 51, -29, 7, 48, 11, 42),
+	CABAC_ENTRY(185, 6, 55, 39, -9, 4, 56, 13, 44),
+	CABAC_ENTRY(186, 11, 45, 52, -34, 4, 52, 16, 45),
+	CABAC_ENTRY(187, 14, 36, 69, -58, 13, 37, 12, 41),
+	CABAC_ENTRY(188, 8, 53, 67, -63, 9, 49, 10, 49),
+	CABAC_ENTRY(189, -1, 82, 44, -5, 19, 58, 30, 34),
+	CABAC_ENTRY(190, 7, 55, 32, 7, 10, 48, 18, 42),
+	CABAC_ENTRY(191, -3, 78, 55, -29, 12, 45, 10, 55),
+	CABAC_ENTRY(192, 15, 46, 32, 1, 0, 69, 17, 51),
+	CABAC_ENTRY(193, 22, 31, 0, 0, 20, 33, 17, 46),
+	CABAC_ENTRY(194, -1, 84, 27, 36, 8, 63, 0, 89),
+	CABAC_ENTRY(195, 25, 7, 33, -25, 35, -18, 26, -19),
+	CABAC_ENTRY(196, 30, -7, 34, -30, 33, -25, 22, -17),
+	CABAC_ENTRY(197, 28, 3, 36, -28, 28, -3, 26, -17),
+	CABAC_ENTRY(198, 28, 4, 38, -28, 24, 10, 30, -25),
+	CABAC_ENTRY(199, 32, 0, 38, -27, 27, 0, 28, -20),
+	CABAC_ENTRY(200, 34, -1, 34, -18, 34, -14, 33, -23),
+	CABAC_ENTRY(201, 30, 6, 35, -16, 52, -44, 37, -27),
+	CABAC_ENTRY(202, 30, 6, 34, -14, 39, -24, 33, -23),
+	CABAC_ENTRY(203, 32, 9, 32, -8, 19, 17, 40, -28),
+	CABAC_ENTRY(204, 31, 19, 37, -6, 31, 25, 38, -17),
+	CABAC_ENTRY(205, 26, 27, 35, 0, 36, 29, 33, -11),
+	CABAC_ENTRY(206, 26, 30, 30, 10, 24, 33, 40, -15),
+	CABAC_ENTRY(207, 37, 20, 28, 18, 34, 15, 41, -6),
+	CABAC_ENTRY(208, 28, 34, 26, 25, 30, 20, 38, 1),
+	CABAC_ENTRY(209, 17, 70, 29, 41, 22, 73, 41, 17),
+	CABAC_ENTRY(210, 1, 67, 0, 75, 20, 34, 30, -6),
+	CABAC_ENTRY(211, 5, 59, 2, 72, 19, 31, 27, 3),
+	CABAC_ENTRY(212, 9, 67, 8, 77, 27, 44, 26, 22),
+	CABAC_ENTRY(213, 16, 30, 14, 35, 19, 16, 37, -16),
+	CABAC_ENTRY(214, 18, 32, 18, 31, 15, 36, 35, -4),
+	CABAC_ENTRY(215, 18, 35, 17, 35, 15, 36, 38, -8),
+	CABAC_ENTRY(216, 22, 29, 21, 30, 21, 28, 38, -3),
+	CABAC_ENTRY(217, 24, 31, 17, 45, 25, 21, 37, 3),
+	CABAC_ENTRY(218, 23, 38, 20, 42, 30, 20, 38, 5),
+	CABAC_ENTRY(219, 18, 43, 18, 45, 31, 12, 42, 0),
+	CABAC_ENTRY(220, 20, 41, 27, 26, 27, 16, 35, 16),
+	CABAC_ENTRY(221, 11, 63, 16, 54, 24, 42, 39, 22),
+	CABAC_ENTRY(222, 9, 59, 7, 66, 0, 93, 14, 48),
+	CABAC_ENTRY(223, 9, 64, 16, 56, 14, 56, 27, 37),
+	CABAC_ENTRY(224, -1, 94, 11, 73, 15, 57, 21, 60),
+	CABAC_ENTRY(225, -2, 89, 10, 67, 26, 38, 12, 68),
+	CABAC_ENTRY(226, -9, 108, -10, 116, -24, 127, 2, 97),
+
+	/* Table 9-21 – Values of variables m and n for ctxIdx from 227 to 275 */
+	CABAC_ENTRY(227, -6, 76, -23, 112, -24, 115, -3, 71),
+	CABAC_ENTRY(228, -2, 44, -15, 71, -22, 82, -6, 42),
+	CABAC_ENTRY(229, 0, 45, -7, 61, -9, 62, -5, 50),
+	CABAC_ENTRY(230, 0, 52, 0, 53, 0, 53, -3, 54),
+	CABAC_ENTRY(231, -3, 64, -5, 66, 0, 59, -2, 62),
+	CABAC_ENTRY(232, -2, 59, -11, 77, -14, 85, 0, 58),
+	CABAC_ENTRY(233, -4, 70, -9, 80, -13, 89, 1, 63),
+	CABAC_ENTRY(234, -4, 75, -9, 84, -13, 94, -2, 72),
+	CABAC_ENTRY(235, -8, 82, -10, 87, -11, 92, -1, 74),
+	CABAC_ENTRY(236, -17, 102, -34, 127, -29, 127, -9, 91),
+	CABAC_ENTRY(237, -9, 77, -21, 101, -21, 100, -5, 67),
+	CABAC_ENTRY(238, 3, 24, -3, 39, -14, 57, -5, 27),
+	CABAC_ENTRY(239, 0, 42, -5, 53, -12, 67, -3, 39),
+	CABAC_ENTRY(240, 0, 48, -7, 61, -11, 71, -2, 44),
+	CABAC_ENTRY(241, 0, 55, -11, 75, -10, 77, 0, 46),
+	CABAC_ENTRY(242, -6, 59, -15, 77, -21, 85, -16, 64),
+	CABAC_ENTRY(243, -7, 71, -17, 91, -16, 88, -8, 68),
+	CABAC_ENTRY(244, -12, 83, -25, 107, -23, 104, -10, 78),
+	CABAC_ENTRY(245, -11, 87, -25, 111, -15, 98, -6, 77),
+	CABAC_ENTRY(246, -30, 119, -28, 122, -37, 127, -10, 86),
+	CABAC_ENTRY(247, 1, 58, -11, 76, -10, 82, -12, 92),
+	CABAC_ENTRY(248, -3, 29, -10, 44, -8, 48, -15, 55),
+	CABAC_ENTRY(249, -1, 36, -10, 52, -8, 61, -10, 60),
+	CABAC_ENTRY(250, 1, 38, -10, 57, -8, 66, -6, 62),
+	CABAC_ENTRY(251, 2, 43, -9, 58, -7, 70, -4, 65),
+	CABAC_ENTRY(252, -6, 55, -16, 72, -14, 75, -12, 73),
+	CABAC_ENTRY(253, 0, 58, -7, 69, -10, 79, -8, 76),
+	CABAC_ENTRY(254, 0, 64, -4, 69, -9, 83, -7, 80),
+	CABAC_ENTRY(255, -3, 74, -5, 74, -12, 92, -9, 88),
+	CABAC_ENTRY(256, -10, 90, -9, 86, -18, 108, -17, 110),
+	CABAC_ENTRY(257, 0, 70, 2, 66, -4, 79, -11, 97),
+	CABAC_ENTRY(258, -4, 29, -9, 34, -22, 69, -20, 84),
+	CABAC_ENTRY(259, 5, 31, 1, 32, -16, 75, -11, 79),
+	CABAC_ENTRY(260, 7, 42, 11, 31, -2, 58, -6, 73),
+	CABAC_ENTRY(261, 1, 59, 5, 52, 1, 58, -4, 74),
+	CABAC_ENTRY(262, -2, 58, -2, 55, -13, 78, -13, 86),
+	CABAC_ENTRY(263, -3, 72, -2, 67, -9, 83, -13, 96),
+	CABAC_ENTRY(264, -3, 81, 0, 73, -4, 81, -11, 97),
+	CABAC_ENTRY(265, -11, 97, -8, 89, -13, 99, -19, 117),
+	CABAC_ENTRY(266, 0, 58, 3, 52, -13, 81, -8, 78),
+	CABAC_ENTRY(267, 8, 5, 7, 4, -6, 38, -5, 33),
+	CABAC_ENTRY(268, 10, 14, 10, 8, -13, 62, -4, 48),
+	CABAC_ENTRY(269, 14, 18, 17, 8, -6, 58, -2, 53),
+	CABAC_ENTRY(270, 13, 27, 16, 19, -2, 59, -3, 62),
+	CABAC_ENTRY(271, 2, 40, 3, 37, -16, 73, -13, 71),
+	CABAC_ENTRY(272, 0, 58, -1, 61, -10, 76, -10, 79),
+	CABAC_ENTRY(273, -3, 70, -5, 73, -13, 86, -12, 86),
+	CABAC_ENTRY(274, -6, 79, -1, 70, -9, 83, -13, 90),
+	CABAC_ENTRY(275, -8, 85, -4, 78, -10, 87, -14, 97),
+
+	/* Table 9-22 – Values of variables m and n for ctxIdx from 277 to 337 */
+	CABAC_ENTRY(277, -13, 106, -21, 126, -22, 127, -6, 93),
+	CABAC_ENTRY(278, -16, 106, -23, 124, -25, 127, -6, 84),
+	CABAC_ENTRY(279, -10, 87, -20, 110, -25, 120, -8, 79),
+	CABAC_ENTRY(280, -21, 114, -26, 126, -27, 127, 0, 66),
+	CABAC_ENTRY(281, -18, 110, -25, 124, -19, 114, -1, 71),
+	CABAC_ENTRY(282, -14, 98, -17, 105, -23, 117, 0, 62),
+	CABAC_ENTRY(283, -22, 110, -27, 121, -25, 118, -2, 60),
+	CABAC_ENTRY(284, -21, 106, -27, 117, -26, 117, -2, 59),
+	CABAC_ENTRY(285, -18, 103, -17, 102, -24, 113, -5, 75),
+	CABAC_ENTRY(286, -21, 107, -26, 117, -28, 118, -3, 62),
+	CABAC_ENTRY(287, -23, 108, -27, 116, -31, 120, -4, 58),
+	CABAC_ENTRY(288, -26, 112, -33, 122, -37, 124, -9, 66),
+	CABAC_ENTRY(289, -10, 96, -10, 95, -10, 94, -1, 79),
+	CABAC_ENTRY(290, -12, 95, -14, 100, -15, 102, 0, 71),
+	CABAC_ENTRY(291, -5, 91, -8, 95, -10, 99, 3, 68),
+	CABAC_ENTRY(292, -9, 93, -17, 111, -13, 106, 10, 44),
+	CABAC_ENTRY(293, -22, 94, -28, 114, -50, 127, -7, 62),
+	CABAC_ENTRY(294, -5, 86, -6, 89, -5, 92, 15, 36),
+	CABAC_ENTRY(295, 9, 67, -2, 80, 17, 57, 14, 40),
+	CABAC_ENTRY(296, -4, 80, -4, 82, -5, 86, 16, 27),
+	CABAC_ENTRY(297, -10, 85, -9, 85, -13, 94, 12, 29),
+	CABAC_ENTRY(298, -1, 70, -8, 81, -12, 91, 1, 44),
+	CABAC_ENTRY(299, 7, 60, -1, 72, -2, 77, 20, 36),
+	CABAC_ENTRY(300, 9, 58, 5, 64, 0, 71, 18, 32),
+	CABAC_ENTRY(301, 5, 61, 1, 67, -1, 73, 5, 42),
+	CABAC_ENTRY(302, 12, 50, 9, 56, 4, 64, 1, 48),
+	CABAC_ENTRY(303, 15, 50, 0, 69, -7, 81, 10, 62),
+	CABAC_ENTRY(304, 18, 49, 1, 69, 5, 64, 17, 46),
+	CABAC_ENTRY(305, 17, 54, 7, 69, 15, 57, 9, 64),
+	CABAC_ENTRY(306, 10, 41, -7, 69, 1, 67, -12, 104),
+	CABAC_ENTRY(307, 7, 46, -6, 67, 0, 68, -11, 97),
+	CABAC_ENTRY(308, -1, 51, -16, 77, -10, 67, -16, 96),
+	CABAC_ENTRY(309, 7, 49, -2, 64, 1, 68, -7, 88),
+	CABAC_ENTRY(310, 8, 52, 2, 61, 0, 77, -8, 85),
+	CABAC_ENTRY(311, 9, 41, -6, 67, 2, 64, -7, 85),
+	CABAC_ENTRY(312, 6, 47, -3, 64, 0, 68, -9, 85),
+	CABAC_ENTRY(313, 2, 55, 2, 57, -5, 78, -13, 88),
+	CABAC_ENTRY(314, 13, 41, -3, 65, 7, 55, 4, 66),
+	CABAC_ENTRY(315, 10, 44, -3, 66, 5, 59, -3, 77),
+	CABAC_ENTRY(316, 6, 50, 0, 62, 2, 65, -3, 76),
+	CABAC_ENTRY(317, 5, 53, 9, 51, 14, 54, -6, 76),
+	CABAC_ENTRY(318, 13, 49, -1, 66, 15, 44, 10, 58),
+	CABAC_ENTRY(319, 4, 63, -2, 71, 5, 60, -1, 76),
+	CABAC_ENTRY(320, 6, 64, -2, 75, 2, 70, -1, 83),
+	CABAC_ENTRY(321, -2, 69, -1, 70, -2, 76, -7, 99),
+	CABAC_ENTRY(322, -2, 59, -9, 72, -18, 86, -14, 95),
+	CABAC_ENTRY(323, 6, 70, 14, 60, 12, 70, 2, 95),
+	CABAC_ENTRY(324, 10, 44, 16, 37, 5, 64, 0, 76),
+	CABAC_ENTRY(325, 9, 31, 0, 47, -12, 70, -5, 74),
+	CABAC_ENTRY(326, 12, 43, 18, 35, 11, 55, 0, 70),
+	CABAC_ENTRY(327, 3, 53, 11, 37, 5, 56, -11, 75),
+	CABAC_ENTRY(328, 14, 34, 12, 41, 0, 69, 1, 68),
+	CABAC_ENTRY(329, 10, 38, 10, 41, 2, 65, 0, 65),
+	CABAC_ENTRY(330, -3, 52, 2, 48, -6, 74, -14, 73),
+	CABAC_ENTRY(331, 13, 40, 12, 41, 5, 54, 3, 62),
+	CABAC_ENTRY(332, 17, 32, 13, 41, 7, 54, 4, 62),
+	CABAC_ENTRY(333, 7, 44, 0, 59, -6, 76, -1, 68),
+	CABAC_ENTRY(334, 7, 38, 3, 50, -11, 82, -13, 75),
+	CABAC_ENTRY(335, 13, 50, 19, 40, -2, 77, 11, 55),
+	CABAC_ENTRY(336, 10, 57, 3, 66, -2, 77, 5, 64),
+	CABAC_ENTRY(337, 26, 43, 18, 50, 25, 42, 12, 70),
+
+	/* Table 9-23 – Values of variables m and n for ctxIdx from 338 to 398 */
+	CABAC_ENTRY(338, 14, 11, 19, -6, 17, -13, 15, 6),
+	CABAC_ENTRY(339, 11, 14, 18, -6, 16, -9, 6, 19),
+	CABAC_ENTRY(340, 9, 11, 14, 0, 17, -12, 7, 16),
+	CABAC_ENTRY(341, 18, 11, 26, -12, 27, -21, 12, 14),
+	CABAC_ENTRY(342, 21, 9, 31, -16, 37, -30, 18, 13),
+	CABAC_ENTRY(343, 23, -2, 33, -25, 41, -40, 13, 11),
+	CABAC_ENTRY(344, 32, -15, 33, -22, 42, -41, 13, 15),
+	CABAC_ENTRY(345, 32, -15, 37, -28, 48, -47, 15, 16),
+	CABAC_ENTRY(346, 34, -21, 39, -30, 39, -32, 12, 23),
+	CABAC_ENTRY(347, 39, -23, 42, -30, 46, -40, 13, 23),
+	CABAC_ENTRY(348, 42, -33, 47, -42, 52, -51, 15, 20),
+	CABAC_ENTRY(349, 41, -31, 45, -36, 46, -41, 14, 26),
+	CABAC_ENTRY(350, 46, -28, 49, -34, 52, -39, 14, 44),
+	CABAC_ENTRY(351, 38, -12, 41, -17, 43, -19, 17, 40),
+	CABAC_ENTRY(352, 21, 29, 32, 9, 32, 11, 17, 47),
+	CABAC_ENTRY(353, 45, -24, 69, -71, 61, -55, 24, 17),
+	CABAC_ENTRY(354, 53, -45, 63, -63, 56, -46, 21, 21),
+	CABAC_ENTRY(355, 48, -26, 66, -64, 62, -50, 25, 22),
+	CABAC_ENTRY(356, 65, -43, 77, -74, 81, -67, 31, 27),
+	CABAC_ENTRY(357, 43, -19, 54, -39, 45, -20, 22, 29),
+	CABAC_ENTRY(358, 39, -10, 52, -35, 35, -2, 19, 35),
+	CABAC_ENTRY(359, 30, 9, 41, -10, 28, 15, 14, 50),
+	CABAC_ENTRY(360, 18, 26, 36, 0, 34, 1, 10, 57),
+	CABAC_ENTRY(361, 20, 27, 40, -1, 39, 1, 7, 63),
+	CABAC_ENTRY(362, 0, 57, 30, 14, 30, 17, -2, 77),
+	CABAC_ENTRY(363, -14, 82, 28, 26, 20, 38, -4, 82),
+	CABAC_ENTRY(364, -5, 75, 23, 37, 18, 45, -3, 94),
+	CABAC_ENTRY(365, -19, 97, 12, 55, 15, 54, 9, 69),
+	CABAC_ENTRY(366, -35, 125, 11, 65, 0, 79, -12, 109),
+	CABAC_ENTRY(367, 27, 0, 37, -33, 36, -16, 36, -35),
+	CABAC_ENTRY(368, 28, 0, 39, -36, 37, -14, 36, -34),
+	CABAC_ENTRY(369, 31, -4, 40, -37, 37, -17, 32, -26),
+	CABAC_ENTRY(370, 27, 6, 38, -30, 32, 1, 37, -30),
+	CABAC_ENTRY(371, 34, 8, 46, -33, 34, 15, 44, -32),
+	CABAC_ENTRY(372, 30, 10, 42, -30, 29, 15, 34, -18),
+	CABAC_ENTRY(373, 24, 22, 40, -24, 24, 25, 34, -15),
+	CABAC_ENTRY(374, 33, 19, 49, -29, 34, 22, 40, -15),
+	CABAC_ENTRY(375, 22, 32, 38, -12, 31, 16, 33, -7),
+	CABAC_ENTRY(376, 26, 31, 40, -10, 35, 18, 35, -5),
+	CABAC_ENTRY(377, 21, 41, 38, -3, 31, 28, 33, 0),
+	CABAC_ENTRY(378, 26, 44, 46, -5, 33, 41, 38, 2),
+	CABAC_ENTRY(379, 23, 47, 31, 20, 36, 28, 33, 13),
+	CABAC_ENTRY(380, 16, 65, 29, 30, 27, 47, 23, 35),
+	CABAC_ENTRY(381, 14, 71, 25, 44, 21, 62, 13, 58),
+	CABAC_ENTRY(382, 8, 60, 12, 48, 18, 31, 29, -3),
+	CABAC_ENTRY(383, 6, 63, 11, 49, 19, 26, 26, 0),
+	CABAC_ENTRY(384, 17, 65, 26, 45, 36, 24, 22, 30),
+	CABAC_ENTRY(385, 21, 24, 22, 22, 24, 23, 31, -7),
+	CABAC_ENTRY(386, 23, 20, 23, 22, 27, 16, 35, -15),
+	CABAC_ENTRY(387, 26, 23, 27, 21, 24, 30, 34, -3),
+	CABAC_ENTRY(388, 27, 32, 33, 20, 31, 29, 34, 3),
+	CABAC_ENTRY(389, 28, 23, 26, 28, 22, 41, 36, -1),
+	CABAC_ENTRY(390, 28, 24, 30, 24, 22, 42, 34, 5),
+	CABAC_ENTRY(391, 23, 40, 27, 34, 16, 60, 32, 11),
+	CABAC_ENTRY(392, 24, 32, 18, 42, 15, 52, 35, 5),
+	CABAC_ENTRY(393, 28, 29, 25, 39, 14, 60, 34, 12),
+	CABAC_ENTRY(394, 23, 42, 18, 50, 3, 78, 39, 11),
+	CABAC_ENTRY(395, 19, 57, 12, 70, -16, 123, 30, 29),
+	CABAC_ENTRY(396, 22, 53, 21, 54, 21, 53, 34, 26),
+	CABAC_ENTRY(397, 22, 61, 14, 71, 22, 56, 29, 39),
+	CABAC_ENTRY(398, 11, 86, 11, 83, 25, 61, 19, 66),
+
+	/* Values of variables m and n for ctxIdx from 399 to 463 (not documented) */
+	CABAC_ENTRY(399, 12, 40, 25, 32, 21, 33, 31, 21),
+	CABAC_ENTRY(400, 11, 51, 21, 49, 19, 50, 31, 31),
+	CABAC_ENTRY(401, 14, 59, 21, 54, 17, 61, 25, 50),
+	CABAC_ENTRY(402, -4, 79, -5, 85, -3, 78, -17, 120),
+	CABAC_ENTRY(403, -7, 71, -6, 81, -8, 74, -20, 112),
+	CABAC_ENTRY(404, -5, 69, -10, 77, -9, 72, -18, 114),
+	CABAC_ENTRY(405, -9, 70, -7, 81, -10, 72, -11, 85),
+	CABAC_ENTRY(406, -8, 66, -17, 80, -18, 75, -15, 92),
+	CABAC_ENTRY(407, -10, 68, -18, 73, -12, 71, -14, 89),
+	CABAC_ENTRY(408, -19, 73, -4, 74, -11, 63, -26, 71),
+	CABAC_ENTRY(409, -12, 69, -10, 83, -5, 70, -15, 81),
+	CABAC_ENTRY(410, -16, 70, -9, 71, -17, 75, -14, 80),
+	CABAC_ENTRY(411, -15, 67, -9, 67, -14, 72, 0, 68),
+	CABAC_ENTRY(412, -20, 62, -1, 61, -16, 67, -14, 70),
+	CABAC_ENTRY(413, -19, 70, -8, 66, -8, 53, -24, 56),
+	CABAC_ENTRY(414, -16, 66, -14, 66, -14, 59, -23, 68),
+	CABAC_ENTRY(415, -22, 65, 0, 59, -9, 52, -24, 50),
+	CABAC_ENTRY(416, -20, 63, 2, 59, -11, 68, -11, 74),
+	CABAC_ENTRY(417, 9, -2, 17, -10, 9, -2, 23, -13),
+	CABAC_ENTRY(418, 26, -9, 32, -13, 30, -10, 26, -13),
+	CABAC_ENTRY(419, 33, -9, 42, -9, 31, -4, 40, -15),
+	CABAC_ENTRY(420, 39, -7, 49, -5, 33, -1, 49, -14),
+	CABAC_ENTRY(421, 41, -2, 53, 0, 33, 7, 44, 3),
+	CABAC_ENTRY(422, 45, 3, 64, 3, 31, 12, 45, 6),
+	CABAC_ENTRY(423, 49, 9, 68, 10, 37, 23, 44, 34),
+	CABAC_ENTRY(424, 45, 27, 66, 27, 31, 38, 33, 54),
+	CABAC_ENTRY(425, 36, 59, 47, 57, 20, 64, 19, 82),
+	CABAC_ENTRY(426, -6, 66, -5, 71, -9, 71, -3, 75),
+	CABAC_ENTRY(427, -7, 35, 0, 24, -7, 37, -1, 23),
+	CABAC_ENTRY(428, -7, 42, -1, 36, -8, 44, 1, 34),
+	CABAC_ENTRY(429, -8, 45, -2, 42, -11, 49, 1, 43),
+	CABAC_ENTRY(430, -5, 48, -2, 52, -10, 56, 0, 54),
+	CABAC_ENTRY(431, -12, 56, -9, 57, -12, 59, -2, 55),
+	CABAC_ENTRY(432, -6, 60, -6, 63, -8, 63, 0, 61),
+	CABAC_ENTRY(433, -5, 62, -4, 65, -9, 67, 1, 64),
+	CABAC_ENTRY(434, -8, 66, -4, 67, -6, 68, 0, 68),
+	CABAC_ENTRY(435, -8, 76, -7, 82, -10, 79, -9, 92),
+	CABAC_ENTRY(436, -5, 85, -3, 81, -3, 78, -14, 106),
+	CABAC_ENTRY(437, -6, 81, -3, 76, -8, 74, -13, 97),
+	CABAC_ENTRY(438, -10, 77, -7, 72, -9, 72, -15, 90),
+	CABAC_ENTRY(439, -7, 81, -6, 78, -10, 72, -12, 90),
+	CABAC_ENTRY(440, -17, 80, -12, 72, -18, 75, -18, 88),
+	CABAC_ENTRY(441, -18, 73, -14, 68, -12, 71, -10, 73),
+	CABAC_ENTRY(442, -4, 74, -3, 70, -11, 63, -9, 79),
+	CABAC_ENTRY(443, -10, 83, -6, 76, -5, 70, -14, 86),
+	CABAC_ENTRY(444, -9, 71, -5, 66, -17, 75, -10, 73),
+	CABAC_ENTRY(445, -9, 67, -5, 62, -14, 72, -10, 70),
+	CABAC_ENTRY(446, -1, 61, 0, 57, -16, 67, -10, 69),
+	CABAC_ENTRY(447, -8, 66, -4, 61, -8, 53, -5, 66),
+	CABAC_ENTRY(448, -14, 66, -9, 60, -14, 59, -9, 64),
+	CABAC_ENTRY(449, 0, 59, 1, 54, -9, 52, -5, 58),
+	CABAC_ENTRY(450, 2, 59, 2, 58, -11, 68, 2, 59),
+	CABAC_ENTRY(451, 21, -13, 17, -10, 9, -2, 21, -10),
+	CABAC_ENTRY(452, 33, -14, 32, -13, 30, -10, 24, -11),
+	CABAC_ENTRY(453, 39, -7, 42, -9, 31, -4, 28, -8),
+	CABAC_ENTRY(454, 46, -2, 49, -5, 33, -1, 28, -1),
+	CABAC_ENTRY(455, 51, 2, 53, 0, 33, 7, 29, 3),
+	CABAC_ENTRY(456, 60, 6, 64, 3, 31, 12, 29, 9),
+	CABAC_ENTRY(457, 61, 17, 68, 10, 37, 23, 35, 20),
+	CABAC_ENTRY(458, 55, 34, 66, 27, 31, 38, 29, 36),
+	CABAC_ENTRY(459, 42, 62, 47, 57, 20, 64, 14, 67),
+};
+
+#endif /* RKVDEC_H264_CABAC_H_ */
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 4fc167b42cf0..2fb19c5b1735 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -14,6 +14,7 @@
 
 #include "rkvdec.h"
 #include "rkvdec-regs.h"
+#include "rkvdec-h264-cabac.h"
 
 /* Size with u32 units. */
 #define RKV_CABAC_INIT_BUFFER_SIZE	(3680 + 128)
@@ -117,505 +118,6 @@ struct rkvdec_h264_ctx {
 	struct rkvdec_h264_reflists reflists;
 };
 
-#define CABAC_ENTRY(ctxidx, idc0_m, idc0_n, idc1_m, idc1_n,		\
-	    idc2_m, idc2_n, intra_m, intra_n)			\
-	[0][(ctxidx)] = {idc0_m, idc0_n},			\
-	[1][(ctxidx)] = {idc1_m, idc1_n},			\
-	[2][(ctxidx)] = {idc2_m, idc2_n},			\
-	[3][(ctxidx)] = {intra_m, intra_n}
-
-/*
- * Constant CABAC table.
- * Built from the tables described in section '9.3.1.1 Initialisation process
- * for context variables' of the H264 spec.
- */
-static const s8 rkvdec_h264_cabac_table[4][464][2] = {
-	/* Table 9-12 – Values of variables m and n for ctxIdx from 0 to 10 */
-	CABAC_ENTRY(0, 20, -15, 20, -15, 20, -15, 20, -15),
-	CABAC_ENTRY(1, 2, 54, 2, 54, 2, 54, 2, 54),
-	CABAC_ENTRY(2, 3, 74, 3, 74, 3, 74, 3, 74),
-	CABAC_ENTRY(3, 20, -15, 20, -15, 20, -15, 20, -15),
-	CABAC_ENTRY(4, 2, 54, 2, 54, 2, 54, 2, 54),
-	CABAC_ENTRY(5, 3, 74, 3, 74, 3, 74, 3, 74),
-	CABAC_ENTRY(6, -28, 127, -28, 127, -28, 127, -28, 127),
-	CABAC_ENTRY(7, -23, 104, -23, 104, -23, 104, -23, 104),
-	CABAC_ENTRY(8, -6, 53, -6, 53, -6, 53, -6, 53),
-	CABAC_ENTRY(9, -1, 54, -1, 54, -1, 54, -1, 54),
-	CABAC_ENTRY(10, 7, 51, 7, 51, 7, 51, 7, 51),
-
-	/* Table 9-13 – Values of variables m and n for ctxIdx from 11 to 23 */
-	CABAC_ENTRY(11, 23, 33, 22, 25, 29, 16, 0, 0),
-	CABAC_ENTRY(12, 23, 2, 34, 0, 25, 0, 0, 0),
-	CABAC_ENTRY(13, 21, 0, 16, 0, 14, 0, 0, 0),
-	CABAC_ENTRY(14, 1, 9, -2, 9, -10, 51, 0, 0),
-	CABAC_ENTRY(15, 0, 49, 4, 41, -3, 62, 0, 0),
-	CABAC_ENTRY(16, -37, 118, -29, 118, -27, 99, 0, 0),
-	CABAC_ENTRY(17, 5, 57, 2, 65, 26, 16, 0, 0),
-	CABAC_ENTRY(18, -13, 78, -6, 71, -4, 85, 0, 0),
-	CABAC_ENTRY(19, -11, 65, -13, 79, -24, 102, 0, 0),
-	CABAC_ENTRY(20, 1, 62, 5, 52, 5, 57, 0, 0),
-	CABAC_ENTRY(21, 12, 49, 9, 50, 6, 57, 0, 0),
-	CABAC_ENTRY(22, -4, 73, -3, 70, -17, 73, 0, 0),
-	CABAC_ENTRY(23, 17, 50, 10, 54, 14, 57, 0, 0),
-
-	/* Table 9-14 – Values of variables m and n for ctxIdx from 24 to 39 */
-	CABAC_ENTRY(24, 18, 64, 26, 34, 20, 40, 0, 0),
-	CABAC_ENTRY(25, 9, 43, 19, 22, 20, 10, 0, 0),
-	CABAC_ENTRY(26, 29, 0, 40, 0, 29, 0, 0, 0),
-	CABAC_ENTRY(27, 26, 67, 57, 2, 54, 0, 0, 0),
-	CABAC_ENTRY(28, 16, 90, 41, 36, 37, 42, 0, 0),
-	CABAC_ENTRY(29, 9, 104, 26, 69, 12, 97, 0, 0),
-	CABAC_ENTRY(30, -46, 127, -45, 127, -32, 127, 0, 0),
-	CABAC_ENTRY(31, -20, 104, -15, 101, -22, 117, 0, 0),
-	CABAC_ENTRY(32, 1, 67, -4, 76, -2, 74, 0, 0),
-	CABAC_ENTRY(33, -13, 78, -6, 71, -4, 85, 0, 0),
-	CABAC_ENTRY(34, -11, 65, -13, 79, -24, 102, 0, 0),
-	CABAC_ENTRY(35, 1, 62, 5, 52, 5, 57, 0, 0),
-	CABAC_ENTRY(36, -6, 86, 6, 69, -6, 93, 0, 0),
-	CABAC_ENTRY(37, -17, 95, -13, 90, -14, 88, 0, 0),
-	CABAC_ENTRY(38, -6, 61, 0, 52, -6, 44, 0, 0),
-	CABAC_ENTRY(39, 9, 45, 8, 43, 4, 55, 0, 0),
-
-	/* Table 9-15 – Values of variables m and n for ctxIdx from 40 to 53 */
-	CABAC_ENTRY(40, -3, 69, -2, 69, -11, 89, 0, 0),
-	CABAC_ENTRY(41, -6, 81, -5, 82, -15, 103, 0, 0),
-	CABAC_ENTRY(42, -11, 96, -10, 96, -21, 116, 0, 0),
-	CABAC_ENTRY(43, 6, 55, 2, 59, 19, 57, 0, 0),
-	CABAC_ENTRY(44, 7, 67, 2, 75, 20, 58, 0, 0),
-	CABAC_ENTRY(45, -5, 86, -3, 87, 4, 84, 0, 0),
-	CABAC_ENTRY(46, 2, 88, -3, 100, 6, 96, 0, 0),
-	CABAC_ENTRY(47, 0, 58, 1, 56, 1, 63, 0, 0),
-	CABAC_ENTRY(48, -3, 76, -3, 74, -5, 85, 0, 0),
-	CABAC_ENTRY(49, -10, 94, -6, 85, -13, 106, 0, 0),
-	CABAC_ENTRY(50, 5, 54, 0, 59, 5, 63, 0, 0),
-	CABAC_ENTRY(51, 4, 69, -3, 81, 6, 75, 0, 0),
-	CABAC_ENTRY(52, -3, 81, -7, 86, -3, 90, 0, 0),
-	CABAC_ENTRY(53, 0, 88, -5, 95, -1, 101, 0, 0),
-
-	/* Table 9-16 – Values of variables m and n for ctxIdx from 54 to 59 */
-	CABAC_ENTRY(54, -7, 67, -1, 66, 3, 55, 0, 0),
-	CABAC_ENTRY(55, -5, 74, -1, 77, -4, 79, 0, 0),
-	CABAC_ENTRY(56, -4, 74, 1, 70, -2, 75, 0, 0),
-	CABAC_ENTRY(57, -5, 80, -2, 86, -12, 97, 0, 0),
-	CABAC_ENTRY(58, -7, 72, -5, 72, -7, 50, 0, 0),
-	CABAC_ENTRY(59, 1, 58, 0, 61, 1, 60, 0, 0),
-
-	/* Table 9-17 – Values of variables m and n for ctxIdx from 60 to 69 */
-	CABAC_ENTRY(60, 0, 41, 0, 41, 0, 41, 0, 41),
-	CABAC_ENTRY(61, 0, 63, 0, 63, 0, 63, 0, 63),
-	CABAC_ENTRY(62, 0, 63, 0, 63, 0, 63, 0, 63),
-	CABAC_ENTRY(63, 0, 63, 0, 63, 0, 63, 0, 63),
-	CABAC_ENTRY(64, -9, 83, -9, 83, -9, 83, -9, 83),
-	CABAC_ENTRY(65, 4, 86, 4, 86, 4, 86, 4, 86),
-	CABAC_ENTRY(66, 0, 97, 0, 97, 0, 97, 0, 97),
-	CABAC_ENTRY(67, -7, 72, -7, 72, -7, 72, -7, 72),
-	CABAC_ENTRY(68, 13, 41, 13, 41, 13, 41, 13, 41),
-	CABAC_ENTRY(69, 3, 62, 3, 62, 3, 62, 3, 62),
-
-	/* Table 9-18 – Values of variables m and n for ctxIdx from 70 to 104 */
-	CABAC_ENTRY(70, 0, 45, 13, 15, 7, 34, 0, 11),
-	CABAC_ENTRY(71, -4, 78, 7, 51, -9, 88, 1, 55),
-	CABAC_ENTRY(72, -3, 96, 2, 80, -20, 127, 0, 69),
-	CABAC_ENTRY(73, -27, 126, -39, 127, -36, 127, -17, 127),
-	CABAC_ENTRY(74, -28, 98, -18, 91, -17, 91, -13, 102),
-	CABAC_ENTRY(75, -25, 101, -17, 96, -14, 95, 0, 82),
-	CABAC_ENTRY(76, -23, 67, -26, 81, -25, 84, -7, 74),
-	CABAC_ENTRY(77, -28, 82, -35, 98, -25, 86, -21, 107),
-	CABAC_ENTRY(78, -20, 94, -24, 102, -12, 89, -27, 127),
-	CABAC_ENTRY(79, -16, 83, -23, 97, -17, 91, -31, 127),
-	CABAC_ENTRY(80, -22, 110, -27, 119, -31, 127, -24, 127),
-	CABAC_ENTRY(81, -21, 91, -24, 99, -14, 76, -18, 95),
-	CABAC_ENTRY(82, -18, 102, -21, 110, -18, 103, -27, 127),
-	CABAC_ENTRY(83, -13, 93, -18, 102, -13, 90, -21, 114),
-	CABAC_ENTRY(84, -29, 127, -36, 127, -37, 127, -30, 127),
-	CABAC_ENTRY(85, -7, 92, 0, 80, 11, 80, -17, 123),
-	CABAC_ENTRY(86, -5, 89, -5, 89, 5, 76, -12, 115),
-	CABAC_ENTRY(87, -7, 96, -7, 94, 2, 84, -16, 122),
-	CABAC_ENTRY(88, -13, 108, -4, 92, 5, 78, -11, 115),
-	CABAC_ENTRY(89, -3, 46, 0, 39, -6, 55, -12, 63),
-	CABAC_ENTRY(90, -1, 65, 0, 65, 4, 61, -2, 68),
-	CABAC_ENTRY(91, -1, 57, -15, 84, -14, 83, -15, 84),
-	CABAC_ENTRY(92, -9, 93, -35, 127, -37, 127, -13, 104),
-	CABAC_ENTRY(93, -3, 74, -2, 73, -5, 79, -3, 70),
-	CABAC_ENTRY(94, -9, 92, -12, 104, -11, 104, -8, 93),
-	CABAC_ENTRY(95, -8, 87, -9, 91, -11, 91, -10, 90),
-	CABAC_ENTRY(96, -23, 126, -31, 127, -30, 127, -30, 127),
-	CABAC_ENTRY(97, 5, 54, 3, 55, 0, 65, -1, 74),
-	CABAC_ENTRY(98, 6, 60, 7, 56, -2, 79, -6, 97),
-	CABAC_ENTRY(99, 6, 59, 7, 55, 0, 72, -7, 91),
-	CABAC_ENTRY(100, 6, 69, 8, 61, -4, 92, -20, 127),
-	CABAC_ENTRY(101, -1, 48, -3, 53, -6, 56, -4, 56),
-	CABAC_ENTRY(102, 0, 68, 0, 68, 3, 68, -5, 82),
-	CABAC_ENTRY(103, -4, 69, -7, 74, -8, 71, -7, 76),
-	CABAC_ENTRY(104, -8, 88, -9, 88, -13, 98, -22, 125),
-
-	/* Table 9-19 – Values of variables m and n for ctxIdx from 105 to 165 */
-	CABAC_ENTRY(105, -2, 85, -13, 103, -4, 86, -7, 93),
-	CABAC_ENTRY(106, -6, 78, -13, 91, -12, 88, -11, 87),
-	CABAC_ENTRY(107, -1, 75, -9, 89, -5, 82, -3, 77),
-	CABAC_ENTRY(108, -7, 77, -14, 92, -3, 72, -5, 71),
-	CABAC_ENTRY(109, 2, 54, -8, 76, -4, 67, -4, 63),
-	CABAC_ENTRY(110, 5, 50, -12, 87, -8, 72, -4, 68),
-	CABAC_ENTRY(111, -3, 68, -23, 110, -16, 89, -12, 84),
-	CABAC_ENTRY(112, 1, 50, -24, 105, -9, 69, -7, 62),
-	CABAC_ENTRY(113, 6, 42, -10, 78, -1, 59, -7, 65),
-	CABAC_ENTRY(114, -4, 81, -20, 112, 5, 66, 8, 61),
-	CABAC_ENTRY(115, 1, 63, -17, 99, 4, 57, 5, 56),
-	CABAC_ENTRY(116, -4, 70, -78, 127, -4, 71, -2, 66),
-	CABAC_ENTRY(117, 0, 67, -70, 127, -2, 71, 1, 64),
-	CABAC_ENTRY(118, 2, 57, -50, 127, 2, 58, 0, 61),
-	CABAC_ENTRY(119, -2, 76, -46, 127, -1, 74, -2, 78),
-	CABAC_ENTRY(120, 11, 35, -4, 66, -4, 44, 1, 50),
-	CABAC_ENTRY(121, 4, 64, -5, 78, -1, 69, 7, 52),
-	CABAC_ENTRY(122, 1, 61, -4, 71, 0, 62, 10, 35),
-	CABAC_ENTRY(123, 11, 35, -8, 72, -7, 51, 0, 44),
-	CABAC_ENTRY(124, 18, 25, 2, 59, -4, 47, 11, 38),
-	CABAC_ENTRY(125, 12, 24, -1, 55, -6, 42, 1, 45),
-	CABAC_ENTRY(126, 13, 29, -7, 70, -3, 41, 0, 46),
-	CABAC_ENTRY(127, 13, 36, -6, 75, -6, 53, 5, 44),
-	CABAC_ENTRY(128, -10, 93, -8, 89, 8, 76, 31, 17),
-	CABAC_ENTRY(129, -7, 73, -34, 119, -9, 78, 1, 51),
-	CABAC_ENTRY(130, -2, 73, -3, 75, -11, 83, 7, 50),
-	CABAC_ENTRY(131, 13, 46, 32, 20, 9, 52, 28, 19),
-	CABAC_ENTRY(132, 9, 49, 30, 22, 0, 67, 16, 33),
-	CABAC_ENTRY(133, -7, 100, -44, 127, -5, 90, 14, 62),
-	CABAC_ENTRY(134, 9, 53, 0, 54, 1, 67, -13, 108),
-	CABAC_ENTRY(135, 2, 53, -5, 61, -15, 72, -15, 100),
-	CABAC_ENTRY(136, 5, 53, 0, 58, -5, 75, -13, 101),
-	CABAC_ENTRY(137, -2, 61, -1, 60, -8, 80, -13, 91),
-	CABAC_ENTRY(138, 0, 56, -3, 61, -21, 83, -12, 94),
-	CABAC_ENTRY(139, 0, 56, -8, 67, -21, 64, -10, 88),
-	CABAC_ENTRY(140, -13, 63, -25, 84, -13, 31, -16, 84),
-	CABAC_ENTRY(141, -5, 60, -14, 74, -25, 64, -10, 86),
-	CABAC_ENTRY(142, -1, 62, -5, 65, -29, 94, -7, 83),
-	CABAC_ENTRY(143, 4, 57, 5, 52, 9, 75, -13, 87),
-	CABAC_ENTRY(144, -6, 69, 2, 57, 17, 63, -19, 94),
-	CABAC_ENTRY(145, 4, 57, 0, 61, -8, 74, 1, 70),
-	CABAC_ENTRY(146, 14, 39, -9, 69, -5, 35, 0, 72),
-	CABAC_ENTRY(147, 4, 51, -11, 70, -2, 27, -5, 74),
-	CABAC_ENTRY(148, 13, 68, 18, 55, 13, 91, 18, 59),
-	CABAC_ENTRY(149, 3, 64, -4, 71, 3, 65, -8, 102),
-	CABAC_ENTRY(150, 1, 61, 0, 58, -7, 69, -15, 100),
-	CABAC_ENTRY(151, 9, 63, 7, 61, 8, 77, 0, 95),
-	CABAC_ENTRY(152, 7, 50, 9, 41, -10, 66, -4, 75),
-	CABAC_ENTRY(153, 16, 39, 18, 25, 3, 62, 2, 72),
-	CABAC_ENTRY(154, 5, 44, 9, 32, -3, 68, -11, 75),
-	CABAC_ENTRY(155, 4, 52, 5, 43, -20, 81, -3, 71),
-	CABAC_ENTRY(156, 11, 48, 9, 47, 0, 30, 15, 46),
-	CABAC_ENTRY(157, -5, 60, 0, 44, 1, 7, -13, 69),
-	CABAC_ENTRY(158, -1, 59, 0, 51, -3, 23, 0, 62),
-	CABAC_ENTRY(159, 0, 59, 2, 46, -21, 74, 0, 65),
-	CABAC_ENTRY(160, 22, 33, 19, 38, 16, 66, 21, 37),
-	CABAC_ENTRY(161, 5, 44, -4, 66, -23, 124, -15, 72),
-	CABAC_ENTRY(162, 14, 43, 15, 38, 17, 37, 9, 57),
-	CABAC_ENTRY(163, -1, 78, 12, 42, 44, -18, 16, 54),
-	CABAC_ENTRY(164, 0, 60, 9, 34, 50, -34, 0, 62),
-	CABAC_ENTRY(165, 9, 69, 0, 89, -22, 127, 12, 72),
-
-	/* Table 9-20 – Values of variables m and n for ctxIdx from 166 to 226 */
-	CABAC_ENTRY(166, 11, 28, 4, 45, 4, 39, 24, 0),
-	CABAC_ENTRY(167, 2, 40, 10, 28, 0, 42, 15, 9),
-	CABAC_ENTRY(168, 3, 44, 10, 31, 7, 34, 8, 25),
-	CABAC_ENTRY(169, 0, 49, 33, -11, 11, 29, 13, 18),
-	CABAC_ENTRY(170, 0, 46, 52, -43, 8, 31, 15, 9),
-	CABAC_ENTRY(171, 2, 44, 18, 15, 6, 37, 13, 19),
-	CABAC_ENTRY(172, 2, 51, 28, 0, 7, 42, 10, 37),
-	CABAC_ENTRY(173, 0, 47, 35, -22, 3, 40, 12, 18),
-	CABAC_ENTRY(174, 4, 39, 38, -25, 8, 33, 6, 29),
-	CABAC_ENTRY(175, 2, 62, 34, 0, 13, 43, 20, 33),
-	CABAC_ENTRY(176, 6, 46, 39, -18, 13, 36, 15, 30),
-	CABAC_ENTRY(177, 0, 54, 32, -12, 4, 47, 4, 45),
-	CABAC_ENTRY(178, 3, 54, 102, -94, 3, 55, 1, 58),
-	CABAC_ENTRY(179, 2, 58, 0, 0, 2, 58, 0, 62),
-	CABAC_ENTRY(180, 4, 63, 56, -15, 6, 60, 7, 61),
-	CABAC_ENTRY(181, 6, 51, 33, -4, 8, 44, 12, 38),
-	CABAC_ENTRY(182, 6, 57, 29, 10, 11, 44, 11, 45),
-	CABAC_ENTRY(183, 7, 53, 37, -5, 14, 42, 15, 39),
-	CABAC_ENTRY(184, 6, 52, 51, -29, 7, 48, 11, 42),
-	CABAC_ENTRY(185, 6, 55, 39, -9, 4, 56, 13, 44),
-	CABAC_ENTRY(186, 11, 45, 52, -34, 4, 52, 16, 45),
-	CABAC_ENTRY(187, 14, 36, 69, -58, 13, 37, 12, 41),
-	CABAC_ENTRY(188, 8, 53, 67, -63, 9, 49, 10, 49),
-	CABAC_ENTRY(189, -1, 82, 44, -5, 19, 58, 30, 34),
-	CABAC_ENTRY(190, 7, 55, 32, 7, 10, 48, 18, 42),
-	CABAC_ENTRY(191, -3, 78, 55, -29, 12, 45, 10, 55),
-	CABAC_ENTRY(192, 15, 46, 32, 1, 0, 69, 17, 51),
-	CABAC_ENTRY(193, 22, 31, 0, 0, 20, 33, 17, 46),
-	CABAC_ENTRY(194, -1, 84, 27, 36, 8, 63, 0, 89),
-	CABAC_ENTRY(195, 25, 7, 33, -25, 35, -18, 26, -19),
-	CABAC_ENTRY(196, 30, -7, 34, -30, 33, -25, 22, -17),
-	CABAC_ENTRY(197, 28, 3, 36, -28, 28, -3, 26, -17),
-	CABAC_ENTRY(198, 28, 4, 38, -28, 24, 10, 30, -25),
-	CABAC_ENTRY(199, 32, 0, 38, -27, 27, 0, 28, -20),
-	CABAC_ENTRY(200, 34, -1, 34, -18, 34, -14, 33, -23),
-	CABAC_ENTRY(201, 30, 6, 35, -16, 52, -44, 37, -27),
-	CABAC_ENTRY(202, 30, 6, 34, -14, 39, -24, 33, -23),
-	CABAC_ENTRY(203, 32, 9, 32, -8, 19, 17, 40, -28),
-	CABAC_ENTRY(204, 31, 19, 37, -6, 31, 25, 38, -17),
-	CABAC_ENTRY(205, 26, 27, 35, 0, 36, 29, 33, -11),
-	CABAC_ENTRY(206, 26, 30, 30, 10, 24, 33, 40, -15),
-	CABAC_ENTRY(207, 37, 20, 28, 18, 34, 15, 41, -6),
-	CABAC_ENTRY(208, 28, 34, 26, 25, 30, 20, 38, 1),
-	CABAC_ENTRY(209, 17, 70, 29, 41, 22, 73, 41, 17),
-	CABAC_ENTRY(210, 1, 67, 0, 75, 20, 34, 30, -6),
-	CABAC_ENTRY(211, 5, 59, 2, 72, 19, 31, 27, 3),
-	CABAC_ENTRY(212, 9, 67, 8, 77, 27, 44, 26, 22),
-	CABAC_ENTRY(213, 16, 30, 14, 35, 19, 16, 37, -16),
-	CABAC_ENTRY(214, 18, 32, 18, 31, 15, 36, 35, -4),
-	CABAC_ENTRY(215, 18, 35, 17, 35, 15, 36, 38, -8),
-	CABAC_ENTRY(216, 22, 29, 21, 30, 21, 28, 38, -3),
-	CABAC_ENTRY(217, 24, 31, 17, 45, 25, 21, 37, 3),
-	CABAC_ENTRY(218, 23, 38, 20, 42, 30, 20, 38, 5),
-	CABAC_ENTRY(219, 18, 43, 18, 45, 31, 12, 42, 0),
-	CABAC_ENTRY(220, 20, 41, 27, 26, 27, 16, 35, 16),
-	CABAC_ENTRY(221, 11, 63, 16, 54, 24, 42, 39, 22),
-	CABAC_ENTRY(222, 9, 59, 7, 66, 0, 93, 14, 48),
-	CABAC_ENTRY(223, 9, 64, 16, 56, 14, 56, 27, 37),
-	CABAC_ENTRY(224, -1, 94, 11, 73, 15, 57, 21, 60),
-	CABAC_ENTRY(225, -2, 89, 10, 67, 26, 38, 12, 68),
-	CABAC_ENTRY(226, -9, 108, -10, 116, -24, 127, 2, 97),
-
-	/* Table 9-21 – Values of variables m and n for ctxIdx from 227 to 275 */
-	CABAC_ENTRY(227, -6, 76, -23, 112, -24, 115, -3, 71),
-	CABAC_ENTRY(228, -2, 44, -15, 71, -22, 82, -6, 42),
-	CABAC_ENTRY(229, 0, 45, -7, 61, -9, 62, -5, 50),
-	CABAC_ENTRY(230, 0, 52, 0, 53, 0, 53, -3, 54),
-	CABAC_ENTRY(231, -3, 64, -5, 66, 0, 59, -2, 62),
-	CABAC_ENTRY(232, -2, 59, -11, 77, -14, 85, 0, 58),
-	CABAC_ENTRY(233, -4, 70, -9, 80, -13, 89, 1, 63),
-	CABAC_ENTRY(234, -4, 75, -9, 84, -13, 94, -2, 72),
-	CABAC_ENTRY(235, -8, 82, -10, 87, -11, 92, -1, 74),
-	CABAC_ENTRY(236, -17, 102, -34, 127, -29, 127, -9, 91),
-	CABAC_ENTRY(237, -9, 77, -21, 101, -21, 100, -5, 67),
-	CABAC_ENTRY(238, 3, 24, -3, 39, -14, 57, -5, 27),
-	CABAC_ENTRY(239, 0, 42, -5, 53, -12, 67, -3, 39),
-	CABAC_ENTRY(240, 0, 48, -7, 61, -11, 71, -2, 44),
-	CABAC_ENTRY(241, 0, 55, -11, 75, -10, 77, 0, 46),
-	CABAC_ENTRY(242, -6, 59, -15, 77, -21, 85, -16, 64),
-	CABAC_ENTRY(243, -7, 71, -17, 91, -16, 88, -8, 68),
-	CABAC_ENTRY(244, -12, 83, -25, 107, -23, 104, -10, 78),
-	CABAC_ENTRY(245, -11, 87, -25, 111, -15, 98, -6, 77),
-	CABAC_ENTRY(246, -30, 119, -28, 122, -37, 127, -10, 86),
-	CABAC_ENTRY(247, 1, 58, -11, 76, -10, 82, -12, 92),
-	CABAC_ENTRY(248, -3, 29, -10, 44, -8, 48, -15, 55),
-	CABAC_ENTRY(249, -1, 36, -10, 52, -8, 61, -10, 60),
-	CABAC_ENTRY(250, 1, 38, -10, 57, -8, 66, -6, 62),
-	CABAC_ENTRY(251, 2, 43, -9, 58, -7, 70, -4, 65),
-	CABAC_ENTRY(252, -6, 55, -16, 72, -14, 75, -12, 73),
-	CABAC_ENTRY(253, 0, 58, -7, 69, -10, 79, -8, 76),
-	CABAC_ENTRY(254, 0, 64, -4, 69, -9, 83, -7, 80),
-	CABAC_ENTRY(255, -3, 74, -5, 74, -12, 92, -9, 88),
-	CABAC_ENTRY(256, -10, 90, -9, 86, -18, 108, -17, 110),
-	CABAC_ENTRY(257, 0, 70, 2, 66, -4, 79, -11, 97),
-	CABAC_ENTRY(258, -4, 29, -9, 34, -22, 69, -20, 84),
-	CABAC_ENTRY(259, 5, 31, 1, 32, -16, 75, -11, 79),
-	CABAC_ENTRY(260, 7, 42, 11, 31, -2, 58, -6, 73),
-	CABAC_ENTRY(261, 1, 59, 5, 52, 1, 58, -4, 74),
-	CABAC_ENTRY(262, -2, 58, -2, 55, -13, 78, -13, 86),
-	CABAC_ENTRY(263, -3, 72, -2, 67, -9, 83, -13, 96),
-	CABAC_ENTRY(264, -3, 81, 0, 73, -4, 81, -11, 97),
-	CABAC_ENTRY(265, -11, 97, -8, 89, -13, 99, -19, 117),
-	CABAC_ENTRY(266, 0, 58, 3, 52, -13, 81, -8, 78),
-	CABAC_ENTRY(267, 8, 5, 7, 4, -6, 38, -5, 33),
-	CABAC_ENTRY(268, 10, 14, 10, 8, -13, 62, -4, 48),
-	CABAC_ENTRY(269, 14, 18, 17, 8, -6, 58, -2, 53),
-	CABAC_ENTRY(270, 13, 27, 16, 19, -2, 59, -3, 62),
-	CABAC_ENTRY(271, 2, 40, 3, 37, -16, 73, -13, 71),
-	CABAC_ENTRY(272, 0, 58, -1, 61, -10, 76, -10, 79),
-	CABAC_ENTRY(273, -3, 70, -5, 73, -13, 86, -12, 86),
-	CABAC_ENTRY(274, -6, 79, -1, 70, -9, 83, -13, 90),
-	CABAC_ENTRY(275, -8, 85, -4, 78, -10, 87, -14, 97),
-
-	/* Table 9-22 – Values of variables m and n for ctxIdx from 277 to 337 */
-	CABAC_ENTRY(277, -13, 106, -21, 126, -22, 127, -6, 93),
-	CABAC_ENTRY(278, -16, 106, -23, 124, -25, 127, -6, 84),
-	CABAC_ENTRY(279, -10, 87, -20, 110, -25, 120, -8, 79),
-	CABAC_ENTRY(280, -21, 114, -26, 126, -27, 127, 0, 66),
-	CABAC_ENTRY(281, -18, 110, -25, 124, -19, 114, -1, 71),
-	CABAC_ENTRY(282, -14, 98, -17, 105, -23, 117, 0, 62),
-	CABAC_ENTRY(283, -22, 110, -27, 121, -25, 118, -2, 60),
-	CABAC_ENTRY(284, -21, 106, -27, 117, -26, 117, -2, 59),
-	CABAC_ENTRY(285, -18, 103, -17, 102, -24, 113, -5, 75),
-	CABAC_ENTRY(286, -21, 107, -26, 117, -28, 118, -3, 62),
-	CABAC_ENTRY(287, -23, 108, -27, 116, -31, 120, -4, 58),
-	CABAC_ENTRY(288, -26, 112, -33, 122, -37, 124, -9, 66),
-	CABAC_ENTRY(289, -10, 96, -10, 95, -10, 94, -1, 79),
-	CABAC_ENTRY(290, -12, 95, -14, 100, -15, 102, 0, 71),
-	CABAC_ENTRY(291, -5, 91, -8, 95, -10, 99, 3, 68),
-	CABAC_ENTRY(292, -9, 93, -17, 111, -13, 106, 10, 44),
-	CABAC_ENTRY(293, -22, 94, -28, 114, -50, 127, -7, 62),
-	CABAC_ENTRY(294, -5, 86, -6, 89, -5, 92, 15, 36),
-	CABAC_ENTRY(295, 9, 67, -2, 80, 17, 57, 14, 40),
-	CABAC_ENTRY(296, -4, 80, -4, 82, -5, 86, 16, 27),
-	CABAC_ENTRY(297, -10, 85, -9, 85, -13, 94, 12, 29),
-	CABAC_ENTRY(298, -1, 70, -8, 81, -12, 91, 1, 44),
-	CABAC_ENTRY(299, 7, 60, -1, 72, -2, 77, 20, 36),
-	CABAC_ENTRY(300, 9, 58, 5, 64, 0, 71, 18, 32),
-	CABAC_ENTRY(301, 5, 61, 1, 67, -1, 73, 5, 42),
-	CABAC_ENTRY(302, 12, 50, 9, 56, 4, 64, 1, 48),
-	CABAC_ENTRY(303, 15, 50, 0, 69, -7, 81, 10, 62),
-	CABAC_ENTRY(304, 18, 49, 1, 69, 5, 64, 17, 46),
-	CABAC_ENTRY(305, 17, 54, 7, 69, 15, 57, 9, 64),
-	CABAC_ENTRY(306, 10, 41, -7, 69, 1, 67, -12, 104),
-	CABAC_ENTRY(307, 7, 46, -6, 67, 0, 68, -11, 97),
-	CABAC_ENTRY(308, -1, 51, -16, 77, -10, 67, -16, 96),
-	CABAC_ENTRY(309, 7, 49, -2, 64, 1, 68, -7, 88),
-	CABAC_ENTRY(310, 8, 52, 2, 61, 0, 77, -8, 85),
-	CABAC_ENTRY(311, 9, 41, -6, 67, 2, 64, -7, 85),
-	CABAC_ENTRY(312, 6, 47, -3, 64, 0, 68, -9, 85),
-	CABAC_ENTRY(313, 2, 55, 2, 57, -5, 78, -13, 88),
-	CABAC_ENTRY(314, 13, 41, -3, 65, 7, 55, 4, 66),
-	CABAC_ENTRY(315, 10, 44, -3, 66, 5, 59, -3, 77),
-	CABAC_ENTRY(316, 6, 50, 0, 62, 2, 65, -3, 76),
-	CABAC_ENTRY(317, 5, 53, 9, 51, 14, 54, -6, 76),
-	CABAC_ENTRY(318, 13, 49, -1, 66, 15, 44, 10, 58),
-	CABAC_ENTRY(319, 4, 63, -2, 71, 5, 60, -1, 76),
-	CABAC_ENTRY(320, 6, 64, -2, 75, 2, 70, -1, 83),
-	CABAC_ENTRY(321, -2, 69, -1, 70, -2, 76, -7, 99),
-	CABAC_ENTRY(322, -2, 59, -9, 72, -18, 86, -14, 95),
-	CABAC_ENTRY(323, 6, 70, 14, 60, 12, 70, 2, 95),
-	CABAC_ENTRY(324, 10, 44, 16, 37, 5, 64, 0, 76),
-	CABAC_ENTRY(325, 9, 31, 0, 47, -12, 70, -5, 74),
-	CABAC_ENTRY(326, 12, 43, 18, 35, 11, 55, 0, 70),
-	CABAC_ENTRY(327, 3, 53, 11, 37, 5, 56, -11, 75),
-	CABAC_ENTRY(328, 14, 34, 12, 41, 0, 69, 1, 68),
-	CABAC_ENTRY(329, 10, 38, 10, 41, 2, 65, 0, 65),
-	CABAC_ENTRY(330, -3, 52, 2, 48, -6, 74, -14, 73),
-	CABAC_ENTRY(331, 13, 40, 12, 41, 5, 54, 3, 62),
-	CABAC_ENTRY(332, 17, 32, 13, 41, 7, 54, 4, 62),
-	CABAC_ENTRY(333, 7, 44, 0, 59, -6, 76, -1, 68),
-	CABAC_ENTRY(334, 7, 38, 3, 50, -11, 82, -13, 75),
-	CABAC_ENTRY(335, 13, 50, 19, 40, -2, 77, 11, 55),
-	CABAC_ENTRY(336, 10, 57, 3, 66, -2, 77, 5, 64),
-	CABAC_ENTRY(337, 26, 43, 18, 50, 25, 42, 12, 70),
-
-	/* Table 9-23 – Values of variables m and n for ctxIdx from 338 to 398 */
-	CABAC_ENTRY(338, 14, 11, 19, -6, 17, -13, 15, 6),
-	CABAC_ENTRY(339, 11, 14, 18, -6, 16, -9, 6, 19),
-	CABAC_ENTRY(340, 9, 11, 14, 0, 17, -12, 7, 16),
-	CABAC_ENTRY(341, 18, 11, 26, -12, 27, -21, 12, 14),
-	CABAC_ENTRY(342, 21, 9, 31, -16, 37, -30, 18, 13),
-	CABAC_ENTRY(343, 23, -2, 33, -25, 41, -40, 13, 11),
-	CABAC_ENTRY(344, 32, -15, 33, -22, 42, -41, 13, 15),
-	CABAC_ENTRY(345, 32, -15, 37, -28, 48, -47, 15, 16),
-	CABAC_ENTRY(346, 34, -21, 39, -30, 39, -32, 12, 23),
-	CABAC_ENTRY(347, 39, -23, 42, -30, 46, -40, 13, 23),
-	CABAC_ENTRY(348, 42, -33, 47, -42, 52, -51, 15, 20),
-	CABAC_ENTRY(349, 41, -31, 45, -36, 46, -41, 14, 26),
-	CABAC_ENTRY(350, 46, -28, 49, -34, 52, -39, 14, 44),
-	CABAC_ENTRY(351, 38, -12, 41, -17, 43, -19, 17, 40),
-	CABAC_ENTRY(352, 21, 29, 32, 9, 32, 11, 17, 47),
-	CABAC_ENTRY(353, 45, -24, 69, -71, 61, -55, 24, 17),
-	CABAC_ENTRY(354, 53, -45, 63, -63, 56, -46, 21, 21),
-	CABAC_ENTRY(355, 48, -26, 66, -64, 62, -50, 25, 22),
-	CABAC_ENTRY(356, 65, -43, 77, -74, 81, -67, 31, 27),
-	CABAC_ENTRY(357, 43, -19, 54, -39, 45, -20, 22, 29),
-	CABAC_ENTRY(358, 39, -10, 52, -35, 35, -2, 19, 35),
-	CABAC_ENTRY(359, 30, 9, 41, -10, 28, 15, 14, 50),
-	CABAC_ENTRY(360, 18, 26, 36, 0, 34, 1, 10, 57),
-	CABAC_ENTRY(361, 20, 27, 40, -1, 39, 1, 7, 63),
-	CABAC_ENTRY(362, 0, 57, 30, 14, 30, 17, -2, 77),
-	CABAC_ENTRY(363, -14, 82, 28, 26, 20, 38, -4, 82),
-	CABAC_ENTRY(364, -5, 75, 23, 37, 18, 45, -3, 94),
-	CABAC_ENTRY(365, -19, 97, 12, 55, 15, 54, 9, 69),
-	CABAC_ENTRY(366, -35, 125, 11, 65, 0, 79, -12, 109),
-	CABAC_ENTRY(367, 27, 0, 37, -33, 36, -16, 36, -35),
-	CABAC_ENTRY(368, 28, 0, 39, -36, 37, -14, 36, -34),
-	CABAC_ENTRY(369, 31, -4, 40, -37, 37, -17, 32, -26),
-	CABAC_ENTRY(370, 27, 6, 38, -30, 32, 1, 37, -30),
-	CABAC_ENTRY(371, 34, 8, 46, -33, 34, 15, 44, -32),
-	CABAC_ENTRY(372, 30, 10, 42, -30, 29, 15, 34, -18),
-	CABAC_ENTRY(373, 24, 22, 40, -24, 24, 25, 34, -15),
-	CABAC_ENTRY(374, 33, 19, 49, -29, 34, 22, 40, -15),
-	CABAC_ENTRY(375, 22, 32, 38, -12, 31, 16, 33, -7),
-	CABAC_ENTRY(376, 26, 31, 40, -10, 35, 18, 35, -5),
-	CABAC_ENTRY(377, 21, 41, 38, -3, 31, 28, 33, 0),
-	CABAC_ENTRY(378, 26, 44, 46, -5, 33, 41, 38, 2),
-	CABAC_ENTRY(379, 23, 47, 31, 20, 36, 28, 33, 13),
-	CABAC_ENTRY(380, 16, 65, 29, 30, 27, 47, 23, 35),
-	CABAC_ENTRY(381, 14, 71, 25, 44, 21, 62, 13, 58),
-	CABAC_ENTRY(382, 8, 60, 12, 48, 18, 31, 29, -3),
-	CABAC_ENTRY(383, 6, 63, 11, 49, 19, 26, 26, 0),
-	CABAC_ENTRY(384, 17, 65, 26, 45, 36, 24, 22, 30),
-	CABAC_ENTRY(385, 21, 24, 22, 22, 24, 23, 31, -7),
-	CABAC_ENTRY(386, 23, 20, 23, 22, 27, 16, 35, -15),
-	CABAC_ENTRY(387, 26, 23, 27, 21, 24, 30, 34, -3),
-	CABAC_ENTRY(388, 27, 32, 33, 20, 31, 29, 34, 3),
-	CABAC_ENTRY(389, 28, 23, 26, 28, 22, 41, 36, -1),
-	CABAC_ENTRY(390, 28, 24, 30, 24, 22, 42, 34, 5),
-	CABAC_ENTRY(391, 23, 40, 27, 34, 16, 60, 32, 11),
-	CABAC_ENTRY(392, 24, 32, 18, 42, 15, 52, 35, 5),
-	CABAC_ENTRY(393, 28, 29, 25, 39, 14, 60, 34, 12),
-	CABAC_ENTRY(394, 23, 42, 18, 50, 3, 78, 39, 11),
-	CABAC_ENTRY(395, 19, 57, 12, 70, -16, 123, 30, 29),
-	CABAC_ENTRY(396, 22, 53, 21, 54, 21, 53, 34, 26),
-	CABAC_ENTRY(397, 22, 61, 14, 71, 22, 56, 29, 39),
-	CABAC_ENTRY(398, 11, 86, 11, 83, 25, 61, 19, 66),
-
-	/* Values of variables m and n for ctxIdx from 399 to 463 (not documented) */
-	CABAC_ENTRY(399, 12, 40, 25, 32, 21, 33, 31, 21),
-	CABAC_ENTRY(400, 11, 51, 21, 49, 19, 50, 31, 31),
-	CABAC_ENTRY(401, 14, 59, 21, 54, 17, 61, 25, 50),
-	CABAC_ENTRY(402, -4, 79, -5, 85, -3, 78, -17, 120),
-	CABAC_ENTRY(403, -7, 71, -6, 81, -8, 74, -20, 112),
-	CABAC_ENTRY(404, -5, 69, -10, 77, -9, 72, -18, 114),
-	CABAC_ENTRY(405, -9, 70, -7, 81, -10, 72, -11, 85),
-	CABAC_ENTRY(406, -8, 66, -17, 80, -18, 75, -15, 92),
-	CABAC_ENTRY(407, -10, 68, -18, 73, -12, 71, -14, 89),
-	CABAC_ENTRY(408, -19, 73, -4, 74, -11, 63, -26, 71),
-	CABAC_ENTRY(409, -12, 69, -10, 83, -5, 70, -15, 81),
-	CABAC_ENTRY(410, -16, 70, -9, 71, -17, 75, -14, 80),
-	CABAC_ENTRY(411, -15, 67, -9, 67, -14, 72, 0, 68),
-	CABAC_ENTRY(412, -20, 62, -1, 61, -16, 67, -14, 70),
-	CABAC_ENTRY(413, -19, 70, -8, 66, -8, 53, -24, 56),
-	CABAC_ENTRY(414, -16, 66, -14, 66, -14, 59, -23, 68),
-	CABAC_ENTRY(415, -22, 65, 0, 59, -9, 52, -24, 50),
-	CABAC_ENTRY(416, -20, 63, 2, 59, -11, 68, -11, 74),
-	CABAC_ENTRY(417, 9, -2, 17, -10, 9, -2, 23, -13),
-	CABAC_ENTRY(418, 26, -9, 32, -13, 30, -10, 26, -13),
-	CABAC_ENTRY(419, 33, -9, 42, -9, 31, -4, 40, -15),
-	CABAC_ENTRY(420, 39, -7, 49, -5, 33, -1, 49, -14),
-	CABAC_ENTRY(421, 41, -2, 53, 0, 33, 7, 44, 3),
-	CABAC_ENTRY(422, 45, 3, 64, 3, 31, 12, 45, 6),
-	CABAC_ENTRY(423, 49, 9, 68, 10, 37, 23, 44, 34),
-	CABAC_ENTRY(424, 45, 27, 66, 27, 31, 38, 33, 54),
-	CABAC_ENTRY(425, 36, 59, 47, 57, 20, 64, 19, 82),
-	CABAC_ENTRY(426, -6, 66, -5, 71, -9, 71, -3, 75),
-	CABAC_ENTRY(427, -7, 35, 0, 24, -7, 37, -1, 23),
-	CABAC_ENTRY(428, -7, 42, -1, 36, -8, 44, 1, 34),
-	CABAC_ENTRY(429, -8, 45, -2, 42, -11, 49, 1, 43),
-	CABAC_ENTRY(430, -5, 48, -2, 52, -10, 56, 0, 54),
-	CABAC_ENTRY(431, -12, 56, -9, 57, -12, 59, -2, 55),
-	CABAC_ENTRY(432, -6, 60, -6, 63, -8, 63, 0, 61),
-	CABAC_ENTRY(433, -5, 62, -4, 65, -9, 67, 1, 64),
-	CABAC_ENTRY(434, -8, 66, -4, 67, -6, 68, 0, 68),
-	CABAC_ENTRY(435, -8, 76, -7, 82, -10, 79, -9, 92),
-	CABAC_ENTRY(436, -5, 85, -3, 81, -3, 78, -14, 106),
-	CABAC_ENTRY(437, -6, 81, -3, 76, -8, 74, -13, 97),
-	CABAC_ENTRY(438, -10, 77, -7, 72, -9, 72, -15, 90),
-	CABAC_ENTRY(439, -7, 81, -6, 78, -10, 72, -12, 90),
-	CABAC_ENTRY(440, -17, 80, -12, 72, -18, 75, -18, 88),
-	CABAC_ENTRY(441, -18, 73, -14, 68, -12, 71, -10, 73),
-	CABAC_ENTRY(442, -4, 74, -3, 70, -11, 63, -9, 79),
-	CABAC_ENTRY(443, -10, 83, -6, 76, -5, 70, -14, 86),
-	CABAC_ENTRY(444, -9, 71, -5, 66, -17, 75, -10, 73),
-	CABAC_ENTRY(445, -9, 67, -5, 62, -14, 72, -10, 70),
-	CABAC_ENTRY(446, -1, 61, 0, 57, -16, 67, -10, 69),
-	CABAC_ENTRY(447, -8, 66, -4, 61, -8, 53, -5, 66),
-	CABAC_ENTRY(448, -14, 66, -9, 60, -14, 59, -9, 64),
-	CABAC_ENTRY(449, 0, 59, 1, 54, -9, 52, -5, 58),
-	CABAC_ENTRY(450, 2, 59, 2, 58, -11, 68, 2, 59),
-	CABAC_ENTRY(451, 21, -13, 17, -10, 9, -2, 21, -10),
-	CABAC_ENTRY(452, 33, -14, 32, -13, 30, -10, 24, -11),
-	CABAC_ENTRY(453, 39, -7, 42, -9, 31, -4, 28, -8),
-	CABAC_ENTRY(454, 46, -2, 49, -5, 33, -1, 28, -1),
-	CABAC_ENTRY(455, 51, 2, 53, 0, 33, 7, 29, 3),
-	CABAC_ENTRY(456, 60, 6, 64, 3, 31, 12, 29, 9),
-	CABAC_ENTRY(457, 61, 17, 68, 10, 37, 23, 35, 20),
-	CABAC_ENTRY(458, 55, 34, 66, 27, 31, 38, 29, 36),
-	CABAC_ENTRY(459, 42, 62, 47, 57, 20, 64, 14, 67),
-};
-
 static void set_ps_field(u32 *buf, struct rkvdec_ps_field field, u32 value)
 {
 	u8 bit = field.offset % 32, word = field.offset / 32;
-- 
2.44.2

[PATCH v2 2/4] media: rockchip: Introduce the rkvdec2 driver

[Detlev Casanova]

This driver supports the second generation of the Rockchip Video
decoder, also known as vdpu34x.
It is currently only used on the RK3588(s) SoC.

There are 2 decoders on the RK3588 SoC that can work in pair to decode
8K video at 30 FPS but currently, only using one core at a time is
supported.

Scheduling requests between the two cores will be implemented later.

The core supports H264, HEVC, VP9 and AVS2 decoding but this driver
currently only supports H264.

The driver is based on rkvdec and they may share some code in the
future.
The decision to make a different driver is mainly because rkvdec2 has
more features and can work with multiple cores.

The registers are mapped in a struct in RAM using bitfields. It is IO
copied to the HW when all values are configured.
The decision to use such a struct instead of writing buffers one by one
is based on the following reasons:
 - Rockchip cores are known to misbehave when registers are not written
   in address order,
 - Those cores also need the software to write all registers, even if
   they are written their default values or are not related to the task
   (this core will not start decoding some H264 frames if some VP9
   registers are not written to 0)
 - In the future, to support multiple cores, the scheduler could be
   optimized by storing the precomputed registers values and copy them
   to the HW as soos as a core becomes available.

This makes the code more readable and may bring performance improvements
in future features.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
---
 drivers/staging/media/Kconfig                |    1 +
 drivers/staging/media/Makefile               |    1 +
 drivers/staging/media/rkvdec2/Kconfig        |   15 +
 drivers/staging/media/rkvdec2/Makefile       |    3 +
 drivers/staging/media/rkvdec2/TODO           |   12 +
 drivers/staging/media/rkvdec2/rkvdec2-h264.c |  739 +++++++++++
 drivers/staging/media/rkvdec2/rkvdec2-regs.h |  345 +++++
 drivers/staging/media/rkvdec2/rkvdec2.c      | 1253 ++++++++++++++++++
 drivers/staging/media/rkvdec2/rkvdec2.h      |  130 ++
 9 files changed, 2499 insertions(+)
 create mode 100644 drivers/staging/media/rkvdec2/Kconfig
 create mode 100644 drivers/staging/media/rkvdec2/Makefile
 create mode 100644 drivers/staging/media/rkvdec2/TODO
 create mode 100644 drivers/staging/media/rkvdec2/rkvdec2-h264.c
 create mode 100644 drivers/staging/media/rkvdec2/rkvdec2-regs.h
 create mode 100644 drivers/staging/media/rkvdec2/rkvdec2.c
 create mode 100644 drivers/staging/media/rkvdec2/rkvdec2.h

diff --git a/drivers/staging/media/Kconfig b/drivers/staging/media/Kconfig
index 554c2e475ce3..7f377d37e670 100644
--- a/drivers/staging/media/Kconfig
+++ b/drivers/staging/media/Kconfig
@@ -35,6 +35,7 @@ source "drivers/staging/media/meson/vdec/Kconfig"
 source "drivers/staging/media/omap4iss/Kconfig"
 
 source "drivers/staging/media/rkvdec/Kconfig"
+source "drivers/staging/media/rkvdec2/Kconfig"
 
 source "drivers/staging/media/starfive/Kconfig"
 
diff --git a/drivers/staging/media/Makefile b/drivers/staging/media/Makefile
index dcaeeca0ee6d..0a2d89db32b2 100644
--- a/drivers/staging/media/Makefile
+++ b/drivers/staging/media/Makefile
@@ -6,6 +6,7 @@ obj-$(CONFIG_VIDEO_MAX96712)	+= max96712/
 obj-$(CONFIG_VIDEO_MESON_VDEC)	+= meson/vdec/
 obj-$(CONFIG_VIDEO_OMAP4)	+= omap4iss/
 obj-$(CONFIG_VIDEO_ROCKCHIP_VDEC)	+= rkvdec/
+obj-$(CONFIG_VIDEO_ROCKCHIP_VDEC2)	+= rkvdec2/
 obj-$(CONFIG_VIDEO_STARFIVE_CAMSS)	+= starfive/
 obj-$(CONFIG_VIDEO_SUNXI)	+= sunxi/
 obj-$(CONFIG_VIDEO_TEGRA)	+= tegra-video/
diff --git a/drivers/staging/media/rkvdec2/Kconfig b/drivers/staging/media/rkvdec2/Kconfig
new file mode 100644
index 000000000000..fd505cb7aff9
--- /dev/null
+++ b/drivers/staging/media/rkvdec2/Kconfig
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0
+config VIDEO_ROCKCHIP_VDEC2
+	tristate "Rockchip Video Decoder driver 2"
+	depends on ARCH_ROCKCHIP || COMPILE_TEST
+	depends on VIDEO_DEV
+	select MEDIA_CONTROLLER
+	select VIDEOBUF2_DMA_CONTIG
+	select VIDEOBUF2_VMALLOC
+	select V4L2_MEM2MEM_DEV
+	select V4L2_H264
+	help
+	  Support for the Rockchip Video Decoder 2 IP present on Rockchip SoCs,
+	  which accelerates video decoding.
+	  To compile this driver as a module, choose M here: the module
+	  will be called rockchip-vdec2.
diff --git a/drivers/staging/media/rkvdec2/Makefile b/drivers/staging/media/rkvdec2/Makefile
new file mode 100644
index 000000000000..b5a6ac701970
--- /dev/null
+++ b/drivers/staging/media/rkvdec2/Makefile
@@ -0,0 +1,3 @@
+obj-$(CONFIG_VIDEO_ROCKCHIP_VDEC2) += rockchip-vdec2.o
+
+rockchip-vdec2-y += rkvdec2.o rkvdec2-h264.o
diff --git a/drivers/staging/media/rkvdec2/TODO b/drivers/staging/media/rkvdec2/TODO
new file mode 100644
index 000000000000..5c78d9445e4c
--- /dev/null
+++ b/drivers/staging/media/rkvdec2/TODO
@@ -0,0 +1,12 @@
+* H264 features
+
+  - Fluster score currently is 129/135 for JVT-AVC_V1.
+    This probably won't improve.
+
+* Support for HEVC and VP9 are planned for this driver.
+
+  First, the h264 backend needs to be stabilized.
+
+* Evaluate sharing code with rkvdec
+
+  As rkvdec is still in staging, this driver stays there as well.
diff --git a/drivers/staging/media/rkvdec2/rkvdec2-h264.c b/drivers/staging/media/rkvdec2/rkvdec2-h264.c
new file mode 100644
index 000000000000..39965d0a720b
--- /dev/null
+++ b/drivers/staging/media/rkvdec2/rkvdec2-h264.c
@@ -0,0 +1,739 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Rockchip Video Decoder 2 H264 backend
+ *
+ * Copyright (C) 2024 Collabora, Ltd.
+ *  Detlev Casanova <detlev.casanova@collabora.com>
+ *
+ * Based on rkvdec driver by Boris Brezillon <boris.brezillon@collabora.com>
+ */
+
+#include <media/v4l2-h264.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "rkvdec2.h"
+#include "rkvdec2-regs.h"
+/* This header will move to a rockchip/common folder when de-staging */
+#include "../rkvdec/rkvdec-h264-cabac.h"
+
+#define RKVDEC_NUM_REFLIST		3
+
+struct rkvdec2_h264_scaling_list {
+	u8 scaling_list_4x4[6][16];
+	u8 scaling_list_8x8[6][64];
+	u8 padding[128];
+};
+
+struct rkvdec2_sps {
+	u16 seq_parameter_set_id:			4;
+	u16 profile_idc:				8;
+	u16 constraint_set3_flag:			1;
+	u16 chroma_format_idc:				2;
+	u16 bit_depth_luma:				3;
+	u16 bit_depth_chroma:				3;
+	u16 qpprime_y_zero_transform_bypass_flag:	1;
+	u16 log2_max_frame_num_minus4:			4;
+	u16 max_num_ref_frames:				5;
+	u16 pic_order_cnt_type:				2;
+	u16 log2_max_pic_order_cnt_lsb_minus4:		4;
+	u16 delta_pic_order_always_zero_flag:		1;
+	u16 pic_width_in_mbs:				12;
+	u16 pic_height_in_mbs:				12;
+	u16 frame_mbs_only_flag:			1;
+	u16 mb_adaptive_frame_field_flag:		1;
+	u16 direct_8x8_inference_flag:			1;
+	u16 mvc_extension_enable:			1;
+	u16 num_views:					2;
+
+	u16 reserved_bits:				12;
+	u16 reserved[11];
+} __packed;
+
+struct rkvdec2_pps {
+	u16 pic_parameter_set_id:				8;
+	u16 pps_seq_parameter_set_id:				5;
+	u16 entropy_coding_mode_flag:				1;
+	u16 bottom_field_pic_order_in_frame_present_flag:	1;
+	u16 num_ref_idx_l0_default_active_minus1:		5;
+	u16 num_ref_idx_l1_default_active_minus1:		5;
+	u16 weighted_pred_flag:					1;
+	u16 weighted_bipred_idc:				2;
+	u16 pic_init_qp_minus26:				7;
+	u16 pic_init_qs_minus26:				6;
+	u16 chroma_qp_index_offset:				5;
+	u16 deblocking_filter_control_present_flag:		1;
+	u16 constrained_intra_pred_flag:			1;
+	u16 redundant_pic_cnt_present:				1;
+	u16 transform_8x8_mode_flag:				1;
+	u16 second_chroma_qp_index_offset:			5;
+	u16 scaling_list_enable_flag:				1;
+	u32 scaling_list_address;
+	u16 is_longterm;
+
+	u8 reserved[3];
+} __packed;
+
+struct rkvdec2_rps_entry {
+	u32 dpb_info0:		5;
+	u32 bottom_flag0:	1;
+	u32 view_index_off0:	1;
+	u32 dpb_info1:		5;
+	u32 bottom_flag1:	1;
+	u32 view_index_off1:	1;
+	u32 dpb_info2:		5;
+	u32 bottom_flag2:	1;
+	u32 view_index_off2:	1;
+	u32 dpb_info3:		5;
+	u32 bottom_flag3:	1;
+	u32 view_index_off3:	1;
+	u32 dpb_info4:		5;
+	u32 bottom_flag4:	1;
+	u32 view_index_off4:	1;
+	u32 dpb_info5:		5;
+	u32 bottom_flag5:	1;
+	u32 view_index_off5:	1;
+	u32 dpb_info6:		5;
+	u32 bottom_flag6:	1;
+	u32 view_index_off6:	1;
+	u32 dpb_info7:		5;
+	u32 bottom_flag7:	1;
+	u32 view_index_off7:	1;
+} __packed;
+
+struct rkvdec2_rps {
+	u16 frame_num[16];
+	u32 reserved0;
+	struct rkvdec2_rps_entry entries[12];
+	u32 reserved1[66];
+}  __packed;
+
+struct rkvdec2_sps_pps {
+	struct rkvdec2_sps sps;
+	struct rkvdec2_pps pps;
+}  __packed;
+
+/* Data structure describing auxiliary buffer format. */
+struct rkvdec2_h264_priv_tbl {
+	u32 cabac_table[928];
+	struct rkvdec2_h264_scaling_list scaling_list;
+	struct rkvdec2_sps_pps param_set[256];
+	struct rkvdec2_rps rps;
+};
+
+struct rkvdec2_h264_reflists {
+	struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN];
+	struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN];
+	struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN];
+};
+
+struct rkvdec2_h264_run {
+	struct rkvdec2_run					base;
+	const struct v4l2_ctrl_h264_decode_params		*decode_params;
+	const struct v4l2_ctrl_h264_sps				*sps;
+	const struct v4l2_ctrl_h264_pps				*pps;
+	const struct v4l2_ctrl_h264_scaling_matrix		*scaling_matrix;
+	struct vb2_buffer *ref_buf[V4L2_H264_NUM_DPB_ENTRIES];
+};
+
+struct rkvdec2_h264_ctx {
+	struct rkvdec2_aux_buf		priv_tbl;
+	struct rkvdec2_h264_reflists	reflists;
+	struct rkvdec2_regs_h264	regs;
+};
+
+static void assemble_hw_pps(struct rkvdec2_ctx *ctx,
+			    struct rkvdec2_h264_run *run)
+{
+	struct rkvdec2_h264_ctx *h264_ctx = ctx->priv;
+	const struct v4l2_ctrl_h264_sps *sps = run->sps;
+	const struct v4l2_ctrl_h264_pps *pps = run->pps;
+	const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params;
+	const struct v4l2_h264_dpb_entry *dpb = dec_params->dpb;
+	struct rkvdec2_h264_priv_tbl *priv_tbl = h264_ctx->priv_tbl.cpu;
+	struct rkvdec2_sps_pps *hw_ps;
+	dma_addr_t scaling_list_address;
+	u32 scaling_distance;
+	u32 i;
+
+	/*
+	 * HW read the SPS/PPS information from PPS packet index by PPS id.
+	 * offset from the base can be calculated by PPS_id * 32 (size per PPS
+	 * packet unit). so the driver copy SPS/PPS information to the exact PPS
+	 * packet unit for HW accessing.
+	 */
+	hw_ps = &priv_tbl->param_set[pps->pic_parameter_set_id];
+	memset(hw_ps, 0, sizeof(*hw_ps));
+
+	/* write sps */
+	hw_ps->sps.seq_parameter_set_id = 0xf;
+	hw_ps->sps.profile_idc = 0xff;
+	hw_ps->sps.constraint_set3_flag = 1;
+	hw_ps->sps.chroma_format_idc = sps->chroma_format_idc;
+	hw_ps->sps.bit_depth_luma = sps->bit_depth_luma_minus8;
+	hw_ps->sps.bit_depth_chroma = sps->bit_depth_chroma_minus8;
+	hw_ps->sps.qpprime_y_zero_transform_bypass_flag = 0;
+	hw_ps->sps.log2_max_frame_num_minus4 = sps->log2_max_frame_num_minus4;
+	hw_ps->sps.max_num_ref_frames = sps->max_num_ref_frames;
+	hw_ps->sps.pic_order_cnt_type = sps->pic_order_cnt_type;
+	hw_ps->sps.log2_max_pic_order_cnt_lsb_minus4 =
+		sps->log2_max_pic_order_cnt_lsb_minus4;
+	hw_ps->sps.delta_pic_order_always_zero_flag =
+		!!(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO);
+	hw_ps->sps.mvc_extension_enable = 1;
+	hw_ps->sps.num_views = 1;
+
+	/*
+	 * Use the SPS values since they are already in macroblocks
+	 * dimensions, height can be field height (halved) if
+	 * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set and also it allows
+	 * decoding smaller images into larger allocation which can be used
+	 * to implementing SVC spatial layer support.
+	 */
+	hw_ps->sps.pic_width_in_mbs = sps->pic_width_in_mbs_minus1 + 1;
+	hw_ps->sps.pic_height_in_mbs = sps->pic_height_in_map_units_minus1 + 1;
+	hw_ps->sps.frame_mbs_only_flag =
+		!!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY);
+	hw_ps->sps.mb_adaptive_frame_field_flag =
+		!!(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD);
+	hw_ps->sps.direct_8x8_inference_flag =
+		!!(sps->flags & V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE);
+
+	/* write pps */
+	hw_ps->pps.pic_parameter_set_id = 0xff;
+	hw_ps->pps.pps_seq_parameter_set_id = 0x1f;
+	hw_ps->pps.entropy_coding_mode_flag =
+		!!(pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE);
+	hw_ps->pps.bottom_field_pic_order_in_frame_present_flag =
+		!!(pps->flags & V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT);
+	hw_ps->pps.num_ref_idx_l0_default_active_minus1 =
+		pps->num_ref_idx_l0_default_active_minus1;
+	hw_ps->pps.num_ref_idx_l1_default_active_minus1 =
+		pps->num_ref_idx_l1_default_active_minus1;
+	hw_ps->pps.weighted_pred_flag =
+		!!(pps->flags & V4L2_H264_PPS_FLAG_WEIGHTED_PRED);
+	hw_ps->pps.weighted_bipred_idc = pps->weighted_bipred_idc;
+	hw_ps->pps.pic_init_qp_minus26 = pps->pic_init_qp_minus26;
+	hw_ps->pps.pic_init_qs_minus26 = pps->pic_init_qs_minus26;
+	hw_ps->pps.chroma_qp_index_offset = pps->chroma_qp_index_offset;
+	hw_ps->pps.deblocking_filter_control_present_flag =
+		!!(pps->flags & V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT);
+	hw_ps->pps.constrained_intra_pred_flag =
+		!!(pps->flags & V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED);
+	hw_ps->pps.redundant_pic_cnt_present =
+		!!(pps->flags & V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT);
+	hw_ps->pps.transform_8x8_mode_flag =
+		!!(pps->flags & V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE);
+	hw_ps->pps.second_chroma_qp_index_offset = pps->second_chroma_qp_index_offset;
+	hw_ps->pps.scaling_list_enable_flag =
+		!!(pps->flags & V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT);
+
+	/*
+	 * To be on the safe side, program the scaling matrix address
+	 *
+	 * With this set here,
+	 *  RKVDEC_SWREG12_SENCODARY_EN:sw_scanlist_addr_valid_en
+	 * can stay at 0
+	 */
+	scaling_distance = offsetof(struct rkvdec2_h264_priv_tbl, scaling_list);
+	scaling_list_address = h264_ctx->priv_tbl.dma + scaling_distance;
+	hw_ps->pps.scaling_list_address = scaling_list_address;
+
+	for (i = 0; i < ARRAY_SIZE(dec_params->dpb); i++) {
+		if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)
+			hw_ps->pps.is_longterm |= (1 << i);
+	}
+}
+
+static void lookup_ref_buf_idx(struct rkvdec2_ctx *ctx,
+			       struct rkvdec2_h264_run *run)
+{
+	const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params;
+	u32 i;
+
+	for (i = 0; i < ARRAY_SIZE(dec_params->dpb); i++) {
+		struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
+		const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb;
+		struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q;
+		struct vb2_buffer *buf = NULL;
+
+		if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) {
+			buf = vb2_find_buffer(cap_q, dpb[i].reference_ts);
+			if (!buf) {
+				dev_dbg(ctx->dev->dev, "No buffer for reference_ts %llu",
+					dpb[i].reference_ts);
+			}
+		}
+
+		run->ref_buf[i] = buf;
+	}
+}
+
+static void set_dpb_info(struct rkvdec2_rps_entry *entries,
+			 u8 reflist,
+			 u8 refnum,
+			 u8 info,
+			 bool bottom)
+{
+	struct rkvdec2_rps_entry *entry = &entries[(reflist * 4) + refnum / 8];
+	u8 idx = refnum % 8;
+
+	switch (idx) {
+	case 0:
+		entry->dpb_info0 = info;
+		entry->bottom_flag0 = bottom;
+		break;
+	case 1:
+		entry->dpb_info1 = info;
+		entry->bottom_flag1 = bottom;
+		break;
+	case 2:
+		entry->dpb_info2 = info;
+		entry->bottom_flag2 = bottom;
+		break;
+	case 3:
+		entry->dpb_info3 = info;
+		entry->bottom_flag3 = bottom;
+		break;
+	case 4:
+		entry->dpb_info4 = info;
+		entry->bottom_flag4 = bottom;
+		break;
+	case 5:
+		entry->dpb_info5 = info;
+		entry->bottom_flag5 = bottom;
+		break;
+	case 6:
+		entry->dpb_info6 = info;
+		entry->bottom_flag6 = bottom;
+		break;
+	case 7:
+		entry->dpb_info7 = info;
+		entry->bottom_flag7 = bottom;
+		break;
+	}
+}
+
+static void assemble_hw_rps(struct rkvdec2_ctx *ctx,
+			    struct v4l2_h264_reflist_builder *builder,
+			    struct rkvdec2_h264_run *run)
+{
+	const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params;
+	const struct v4l2_h264_dpb_entry *dpb = dec_params->dpb;
+	struct rkvdec2_h264_ctx *h264_ctx = ctx->priv;
+	struct rkvdec2_h264_priv_tbl *priv_tbl = h264_ctx->priv_tbl.cpu;
+
+	struct rkvdec2_rps *hw_rps = &priv_tbl->rps;
+	u32 i, j;
+
+	memset(hw_rps, 0, sizeof(priv_tbl->rps));
+
+	/*
+	 * Assign an invalid pic_num if DPB entry at that position is inactive.
+	 * If we assign 0 in that position hardware will treat that as a real
+	 * reference picture with pic_num 0, triggering output picture
+	 * corruption.
+	 */
+	for (i = 0; i < ARRAY_SIZE(dec_params->dpb); i++) {
+		if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
+			continue;
+
+		hw_rps->frame_num[i] = builder->refs[i].frame_num;
+	}
+
+	for (j = 0; j < RKVDEC_NUM_REFLIST; j++) {
+		for (i = 0; i < builder->num_valid; i++) {
+			struct v4l2_h264_reference *ref;
+			bool dpb_valid;
+			bool bottom;
+
+			switch (j) {
+			case 0:
+				ref = &h264_ctx->reflists.p[i];
+				break;
+			case 1:
+				ref = &h264_ctx->reflists.b0[i];
+				break;
+			case 2:
+				ref = &h264_ctx->reflists.b1[i];
+				break;
+			}
+
+			if (WARN_ON(ref->index >= ARRAY_SIZE(dec_params->dpb)))
+				continue;
+
+			dpb_valid = !!(run->ref_buf[ref->index]);
+			bottom = ref->fields == V4L2_H264_BOTTOM_FIELD_REF;
+
+			set_dpb_info(hw_rps->entries, j, i, ref->index | (dpb_valid << 4), bottom);
+		}
+	}
+}
+
+static void assemble_hw_scaling_list(struct rkvdec2_ctx *ctx,
+				     struct rkvdec2_h264_run *run)
+{
+	const struct v4l2_ctrl_h264_scaling_matrix *scaling = run->scaling_matrix;
+	const struct v4l2_ctrl_h264_pps *pps = run->pps;
+	struct rkvdec2_h264_ctx *h264_ctx = ctx->priv;
+	struct rkvdec2_h264_priv_tbl *tbl = h264_ctx->priv_tbl.cpu;
+
+	if (!(pps->flags & V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT))
+		return;
+
+	BUILD_BUG_ON(sizeof(tbl->scaling_list.scaling_list_4x4) !=
+		     sizeof(scaling->scaling_list_4x4));
+	BUILD_BUG_ON(sizeof(tbl->scaling_list.scaling_list_8x8) !=
+		     sizeof(scaling->scaling_list_8x8));
+
+	memcpy(tbl->scaling_list.scaling_list_4x4,
+	       scaling->scaling_list_4x4,
+	       sizeof(scaling->scaling_list_4x4));
+
+	memcpy(tbl->scaling_list.scaling_list_8x8,
+	       scaling->scaling_list_8x8,
+	       sizeof(scaling->scaling_list_8x8));
+}
+
+static inline void rkvdec2_memcpy_toio(void __iomem *dst, void *src, size_t len)
+{
+#ifdef CONFIG_ARM64
+	__iowrite32_copy(dst, src, len);
+#elif defined(CONFIG_ARM)
+	memcpy_toio(dst, src, len);
+#endif
+}
+
+static void rkvdec2_write_regs(struct rkvdec2_ctx *ctx)
+{
+	struct rkvdec2_dev *rkvdec = ctx->dev;
+	struct rkvdec2_h264_ctx *h264_ctx = ctx->priv;
+
+	rkvdec2_memcpy_toio(rkvdec->regs + OFFSET_COMMON_REGS,
+			    &h264_ctx->regs.common,
+			    sizeof(h264_ctx->regs.common));
+	rkvdec2_memcpy_toio(rkvdec->regs + OFFSET_CODEC_PARAMS_REGS,
+			    &h264_ctx->regs.h264_param,
+			    sizeof(h264_ctx->regs.h264_param));
+	rkvdec2_memcpy_toio(rkvdec->regs + OFFSET_COMMON_ADDR_REGS,
+			    &h264_ctx->regs.common_addr,
+			    sizeof(h264_ctx->regs.common_addr));
+	rkvdec2_memcpy_toio(rkvdec->regs + OFFSET_CODEC_ADDR_REGS,
+			    &h264_ctx->regs.h264_addr,
+			    sizeof(h264_ctx->regs.h264_addr));
+	rkvdec2_memcpy_toio(rkvdec->regs + OFFSET_POC_HIGHBIT_REGS,
+			    &h264_ctx->regs.h264_highpoc,
+			    sizeof(h264_ctx->regs.h264_highpoc));
+}
+
+static void config_registers(struct rkvdec2_ctx *ctx,
+			     struct rkvdec2_h264_run *run)
+{
+	const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params;
+	const struct v4l2_ctrl_h264_sps *sps = run->sps;
+	const struct v4l2_h264_dpb_entry *dpb = dec_params->dpb;
+	struct rkvdec2_h264_ctx *h264_ctx = ctx->priv;
+	dma_addr_t priv_start_addr = h264_ctx->priv_tbl.dma;
+	const struct v4l2_pix_format_mplane *dst_fmt;
+	struct vb2_v4l2_buffer *src_buf = run->base.bufs.src;
+	struct vb2_v4l2_buffer *dst_buf = run->base.bufs.dst;
+	struct rkvdec2_regs_h264 *regs = &h264_ctx->regs;
+	const struct v4l2_format *f;
+	dma_addr_t rlc_addr;
+	dma_addr_t dst_addr;
+	u32 hor_virstride = 0;
+	u32 ver_virstride = 0;
+	u32 y_virstride = 0;
+	u32 offset;
+	u32 pixels;
+	u32 i;
+
+	memset(regs, 0, sizeof(*regs));
+
+	/* Set H264 mode */
+	regs->common.reg009.dec_mode = RKVDEC2_MODE_H264;
+
+	/* Set config */
+	regs->common.reg011.buf_empty_en = 1;
+	regs->common.reg011.dec_clkgate_e = 1;
+	regs->common.reg011.dec_timeout_e = 1;
+	regs->common.reg011.pix_range_detection_e = 1;
+
+	/* Set IDR flag */
+	regs->common.reg013.cur_pic_is_idr =
+		!!(dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_IDR_PIC);
+
+	/* Set input stream length */
+	regs->common.stream_len = vb2_get_plane_payload(&src_buf->vb2_buf, 0);
+
+	/* Set max slice number */
+	regs->common.reg017.slice_num = MAX_SLICE_NUMBER;
+
+	/* Set strides */
+	f = &ctx->decoded_fmt;
+	dst_fmt = &f->fmt.pix_mp;
+	hor_virstride = (sps->bit_depth_luma_minus8 + 8) * dst_fmt->width / 8;
+	ver_virstride = round_up(dst_fmt->height, 16);
+	y_virstride = hor_virstride * ver_virstride;
+	pixels = dst_fmt->height * dst_fmt->width;
+
+	regs->common.reg018.y_hor_virstride = hor_virstride / 16;
+	regs->common.reg019.uv_hor_virstride = hor_virstride / 16;
+	regs->common.reg020.y_virstride = y_virstride / 16;
+
+	/* Activate block gating */
+	regs->common.reg026.swreg_block_gating_e = 0xfffef;
+	regs->common.reg026.reg_cfg_gating_en = 1;
+
+	/* Set timeout threshold */
+	if (pixels < RKVDEC2_1080P_PIXELS)
+		regs->common.timeout_threshold = RKVDEC2_TIMEOUT_1080p;
+	else if (pixels < RKVDEC2_4K_PIXELS)
+		regs->common.timeout_threshold = RKVDEC2_TIMEOUT_4K;
+	else if (pixels < RKVDEC2_8K_PIXELS)
+		regs->common.timeout_threshold = RKVDEC2_TIMEOUT_8K;
+
+	/* Set TOP and BOTTOM POCs */
+	regs->h264_param.cur_top_poc = dec_params->top_field_order_cnt;
+	regs->h264_param.cur_bot_poc = dec_params->bottom_field_order_cnt;
+
+	/* Set ref pic address & poc */
+	for (i = 0; i < ARRAY_SIZE(dec_params->dpb); i++) {
+		struct vb2_buffer *vb_buf = run->ref_buf[i];
+		dma_addr_t buf_dma;
+
+		/*
+		 * If a DPB entry is unused or invalid, address of current destination
+		 * buffer is returned.
+		 */
+		if (!vb_buf)
+			vb_buf = &dst_buf->vb2_buf;
+
+		buf_dma = vb2_dma_contig_plane_dma_addr(vb_buf, 0);
+
+		/* Set reference addresses */
+		regs->h264_addr.ref_base[i] = buf_dma;
+
+		/* Set COLMV addresses */
+		regs->h264_addr.colmv_base[i] = buf_dma + ctx->colmv_offset;
+
+		struct rkvdec2_h264_ref_info *ref_info =
+			&regs->h264_param.ref_info_regs[i / 4].ref_info[i % 4];
+
+		ref_info->ref_field =
+			!!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_FIELD);
+		ref_info->ref_colmv_use_flag =
+			!!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE);
+		ref_info->ref_topfield_used =
+			!!(dpb[i].fields & V4L2_H264_TOP_FIELD_REF);
+		ref_info->ref_botfield_used =
+			!!(dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF);
+
+		regs->h264_param.ref_pocs[i * 2] =
+			dpb[i].top_field_order_cnt;
+		regs->h264_param.ref_pocs[i * 2 + 1] =
+			dpb[i].bottom_field_order_cnt;
+	}
+
+	/* Set rlc base address (input stream) */
+	rlc_addr = vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf, 0);
+	regs->common_addr.rlc_base = rlc_addr;
+	regs->common_addr.rlcwrite_base = rlc_addr;
+
+	/* Set output base address */
+	dst_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
+	regs->common_addr.decout_base = dst_addr;
+
+	/* Set colmv address */
+	regs->common_addr.colmv_cur_base = dst_addr + ctx->colmv_offset;
+
+	/* Set RCB addresses */
+	for (i = 0; i < RKVDEC2_RCB_COUNT; i++)
+		regs->common_addr.rcb_base[i] = ctx->rcb_bufs[i].dma;
+
+	/* Set hw pps address */
+	offset = offsetof(struct rkvdec2_h264_priv_tbl, param_set);
+	regs->h264_addr.pps_base = priv_start_addr + offset;
+
+	/* Set hw rps address */
+	offset = offsetof(struct rkvdec2_h264_priv_tbl, rps);
+	regs->h264_addr.rps_base = priv_start_addr + offset;
+
+	/* Set cabac table */
+	offset = offsetof(struct rkvdec2_h264_priv_tbl, cabac_table);
+	regs->h264_addr.cabactbl_base = priv_start_addr + offset;
+
+	rkvdec2_write_regs(ctx);
+}
+
+#define RKVDEC_H264_MAX_DEPTH_IN_BYTES		2
+
+static int rkvdec2_h264_adjust_fmt(struct rkvdec2_ctx *ctx,
+				   struct v4l2_format *f)
+{
+	struct v4l2_pix_format_mplane *fmt = &f->fmt.pix_mp;
+
+	fmt->num_planes = 1;
+	if (!fmt->plane_fmt[0].sizeimage)
+		fmt->plane_fmt[0].sizeimage = fmt->width * fmt->height *
+					      RKVDEC_H264_MAX_DEPTH_IN_BYTES;
+	return 0;
+}
+
+static int rkvdec2_h264_validate_sps(struct rkvdec2_ctx *ctx,
+				     const struct v4l2_ctrl_h264_sps *sps)
+{
+	unsigned int width, height;
+
+	/*
+	 * TODO: The hardware supports 10-bit and 4:2:2 profiles,
+	 * but it's currently broken in the driver.
+	 * Reject them for now, until it's fixed.
+	 */
+	if (sps->chroma_format_idc > 1)
+		/* Only 4:0:0 and 4:2:0 are supported */
+		return -EINVAL;
+	if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
+		/* Luma and chroma bit depth mismatch */
+		return -EINVAL;
+	if (sps->bit_depth_luma_minus8 != 0)
+		/* Only 8-bit is supported */
+		return -EINVAL;
+
+	width = (sps->pic_width_in_mbs_minus1 + 1) * 16;
+	height = (sps->pic_height_in_map_units_minus1 + 1) * 16;
+
+	/*
+	 * When frame_mbs_only_flag is not set, this is field height,
+	 * which is half the final height (see (7-18) in the
+	 * specification)
+	 */
+	if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
+		height *= 2;
+
+	if (width > ctx->coded_fmt.fmt.pix_mp.width ||
+	    height > ctx->coded_fmt.fmt.pix_mp.height)
+		return -EINVAL;
+
+	return 0;
+}
+
+static int rkvdec2_h264_start(struct rkvdec2_ctx *ctx)
+{
+	struct rkvdec2_dev *rkvdec = ctx->dev;
+	struct rkvdec2_h264_priv_tbl *priv_tbl;
+	struct rkvdec2_h264_ctx *h264_ctx;
+	struct v4l2_ctrl *ctrl;
+	int ret;
+
+	ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl,
+			      V4L2_CID_STATELESS_H264_SPS);
+	if (!ctrl)
+		return -EINVAL;
+
+	ret = rkvdec2_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
+	if (ret)
+		return ret;
+
+	h264_ctx = kzalloc(sizeof(*h264_ctx), GFP_KERNEL);
+	if (!h264_ctx)
+		return -ENOMEM;
+
+	priv_tbl = dma_alloc_coherent(rkvdec->dev, sizeof(*priv_tbl),
+				      &h264_ctx->priv_tbl.dma, GFP_KERNEL);
+	if (!priv_tbl) {
+		ret = -ENOMEM;
+		goto err_free_ctx;
+	}
+
+	h264_ctx->priv_tbl.size = sizeof(*priv_tbl);
+	h264_ctx->priv_tbl.cpu = priv_tbl;
+	memcpy(priv_tbl->cabac_table, rkvdec_h264_cabac_table,
+	       sizeof(rkvdec_h264_cabac_table));
+
+	ctx->priv = h264_ctx;
+	return 0;
+
+err_free_ctx:
+	kfree(h264_ctx);
+	return ret;
+}
+
+static void rkvdec2_h264_stop(struct rkvdec2_ctx *ctx)
+{
+	struct rkvdec2_h264_ctx *h264_ctx = ctx->priv;
+	struct rkvdec2_dev *rkvdec = ctx->dev;
+
+	dma_free_coherent(rkvdec->dev, h264_ctx->priv_tbl.size,
+			  h264_ctx->priv_tbl.cpu, h264_ctx->priv_tbl.dma);
+	kfree(h264_ctx);
+}
+
+static void rkvdec2_h264_run_preamble(struct rkvdec2_ctx *ctx,
+				      struct rkvdec2_h264_run *run)
+{
+	struct v4l2_ctrl *ctrl;
+
+	ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl,
+			      V4L2_CID_STATELESS_H264_DECODE_PARAMS);
+	run->decode_params = ctrl ? ctrl->p_cur.p : NULL;
+	ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl,
+			      V4L2_CID_STATELESS_H264_SPS);
+	run->sps = ctrl ? ctrl->p_cur.p : NULL;
+	ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl,
+			      V4L2_CID_STATELESS_H264_PPS);
+	run->pps = ctrl ? ctrl->p_cur.p : NULL;
+	ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl,
+			      V4L2_CID_STATELESS_H264_SCALING_MATRIX);
+	run->scaling_matrix = ctrl ? ctrl->p_cur.p : NULL;
+
+	rkvdec2_run_preamble(ctx, &run->base);
+}
+
+static int rkvdec2_h264_run(struct rkvdec2_ctx *ctx)
+{
+	struct v4l2_h264_reflist_builder reflist_builder;
+	struct rkvdec2_dev *rkvdec = ctx->dev;
+	struct rkvdec2_h264_ctx *h264_ctx = ctx->priv;
+	struct rkvdec2_h264_run run;
+
+	rkvdec2_h264_run_preamble(ctx, &run);
+
+	/* Build the P/B{0,1} ref lists. */
+	v4l2_h264_init_reflist_builder(&reflist_builder, run.decode_params,
+				       run.sps, run.decode_params->dpb);
+	v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p);
+	v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0,
+				    h264_ctx->reflists.b1);
+
+	assemble_hw_scaling_list(ctx, &run);
+	assemble_hw_pps(ctx, &run);
+	lookup_ref_buf_idx(ctx, &run);
+	assemble_hw_rps(ctx, &reflist_builder, &run);
+
+	config_registers(ctx, &run);
+
+	rkvdec2_run_postamble(ctx, &run.base);
+
+	schedule_delayed_work(&rkvdec->watchdog_work, msecs_to_jiffies(2000));
+
+	/* Start decoding! */
+	writel(RKVDEC2_REG_DEC_E_BIT, rkvdec->regs + RKVDEC2_REG_DEC_E);
+
+	return 0;
+}
+
+static int rkvdec2_h264_try_ctrl(struct rkvdec2_ctx *ctx, struct v4l2_ctrl *ctrl)
+{
+	if (ctrl->id == V4L2_CID_STATELESS_H264_SPS)
+		return rkvdec2_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
+
+	return 0;
+}
+
+const struct rkvdec2_coded_fmt_ops rkvdec2_h264_fmt_ops = {
+	.adjust_fmt = rkvdec2_h264_adjust_fmt,
+	.start = rkvdec2_h264_start,
+	.stop = rkvdec2_h264_stop,
+	.run = rkvdec2_h264_run,
+	.try_ctrl = rkvdec2_h264_try_ctrl,
+};
diff --git a/drivers/staging/media/rkvdec2/rkvdec2-regs.h b/drivers/staging/media/rkvdec2/rkvdec2-regs.h
new file mode 100644
index 000000000000..971df554fc10
--- /dev/null
+++ b/drivers/staging/media/rkvdec2/rkvdec2-regs.h
@@ -0,0 +1,345 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Rockchip Video Decoder 2 driver registers description
+ *
+ * Copyright (C) 2024 Collabora, Ltd.
+ *  Detlev Casanova <detlev.casanova@collabora.com>
+ */
+
+#ifndef _RKVDEC_REGS_H_
+#define _RKVDEC_REGS_H_
+
+#define OFFSET_COMMON_REGS		(8 * sizeof(u32))
+#define OFFSET_CODEC_PARAMS_REGS	(64 * sizeof(u32))
+#define OFFSET_COMMON_ADDR_REGS		(128 * sizeof(u32))
+#define OFFSET_CODEC_ADDR_REGS		(160 * sizeof(u32))
+#define OFFSET_POC_HIGHBIT_REGS		(200 * sizeof(u32))
+
+#define RKVDEC2_MODE_HEVC	0
+#define RKVDEC2_MODE_H264	1
+#define RKVDEC2_MODE_VP9	2
+#define RKVDEC2_MODE_AVS2	3
+
+#define MAX_SLICE_NUMBER	0x3fff
+
+#define RKVDEC2_1080P_PIXELS		(1920 * 1080)
+#define RKVDEC2_4K_PIXELS		(4096 * 2304)
+#define RKVDEC2_8K_PIXELS		(7680 * 4320)
+#define RKVDEC2_TIMEOUT_1080p		(0xefffff)
+#define RKVDEC2_TIMEOUT_4K		(0x2cfffff)
+#define RKVDEC2_TIMEOUT_8K		(0x4ffffff)
+
+#define RKVDEC2_REG_DEC_E		0x028
+#define RKVDEC2_REG_DEC_E_BIT		1
+
+#define RKVDEC2_REG_IMPORTANT_EN	0x02c
+#define RKVDEC2_REG_DEC_IRQ_DISABLE	BIT(4)
+
+#define RKVDEC2_REG_STA_INT		0x380
+#define STA_INT_DEC_RDY_STA		BIT(2)
+
+/* base: OFFSET_COMMON_REGS */
+struct rkvdec2_regs_common {
+	struct rkvdec2_in_out {
+		u32 in_endian		: 1;
+		u32 in_swap32_e		: 1;
+		u32 in_swap64_e		: 1;
+		u32 str_endian		: 1;
+		u32 str_swap32_e	: 1;
+		u32 str_swap64_e	: 1;
+		u32 out_endian		: 1;
+		u32 out_swap32_e	: 1;
+		u32 out_cbcr_swap	: 1;
+		u32 out_swap64_e	: 1;
+		u32 reserved		: 22;
+	} reg008;
+
+	struct rkvdec2_dec_mode {
+		u32 dec_mode	: 10;
+		u32 reserved	: 22;
+	} reg009;
+
+	struct rkvdec2_dec_e {
+		u32 dec_e	: 1;
+		u32 reserved	: 31;
+	} reg010;
+
+	struct rkvdec2_important_en {
+		u32 reserved			: 1;
+		u32 dec_clkgate_e		: 1;
+		u32 dec_e_strmd_clkgate_dis	: 1;
+		u32 reserved0			: 1;
+
+		u32 dec_irq_dis			: 1;
+		u32 dec_timeout_e		: 1;
+		u32 buf_empty_en		: 1;
+		u32 reserved1			: 3;
+
+		u32 dec_e_rewrite_valid		: 1;
+		u32 reserved2			: 9;
+		u32 softrst_en_p		: 1;
+		u32 force_softreset_valid	: 1;
+		u32 reserved3			: 2;
+		u32 pix_range_detection_e	: 1;
+		u32 reserved4			: 7;
+	} reg011;
+
+	struct rkvdec2_sencodary_en {
+		u32 wr_ddr_align_en		: 1;
+		u32 colmv_compress_en		: 1;
+		u32 fbc_e			: 1;
+		u32 reserved0			: 1;
+
+		u32 buspr_slot_disable		: 1;
+		u32 error_info_en		: 1;
+		u32 info_collect_en		: 1;
+		u32 wait_reset_en		: 1;
+
+		u32 scanlist_addr_valid_en	: 1;
+		u32 scale_down_en		: 1;
+		u32 error_cfg_wr_disable	: 1;
+		u32 reserved1			: 21;
+	} reg012;
+
+	struct rkvdec2_en_mode_set {
+		u32 timeout_mode		: 1;
+		u32 req_timeout_rst_sel		: 1;
+		u32 reserved0			: 1;
+		u32 dec_commonirq_mode		: 1;
+		u32 reserved1			: 2;
+		u32 stmerror_waitdecfifo_empty	: 1;
+		u32 reserved2			: 2;
+		u32 h26x_streamd_error_mode	: 1;
+		u32 reserved3			: 2;
+		u32 allow_not_wr_unref_bframe	: 1;
+		u32 fbc_output_wr_disable	: 1;
+		u32 reserved4			: 1;
+		u32 colmv_error_mode		: 1;
+
+		u32 reserved5			: 2;
+		u32 h26x_error_mode		: 1;
+		u32 reserved6			: 2;
+		u32 ycacherd_prior		: 1;
+		u32 reserved7			: 2;
+		u32 cur_pic_is_idr		: 1;
+		u32 reserved8			: 1;
+		u32 right_auto_rst_disable	: 1;
+		u32 frame_end_err_rst_flag	: 1;
+		u32 rd_prior_mode		: 1;
+		u32 rd_ctrl_prior_mode		: 1;
+		u32 reserved9			: 1;
+		u32 filter_outbuf_mode		: 1;
+	} reg013;
+
+	struct rkvdec2_fbc_param_set {
+		u32 fbc_force_uncompress	: 1;
+
+		u32 reserved0			: 2;
+		u32 allow_16x8_cp_flag		: 1;
+		u32 reserved1			: 2;
+
+		u32 fbc_h264_exten_4or8_flag	: 1;
+		u32 reserved2			: 25;
+	} reg014;
+
+	struct rkvdec2_stream_param_set {
+		u32 rlc_mode_direct_write	: 1;
+		u32 rlc_mode			: 1;
+		u32 reserved0			: 3;
+
+		u32 strm_start_bit		: 7;
+		u32 reserved1			: 20;
+	} reg015;
+
+	u32 stream_len;
+
+	struct rkvdec2_slice_number {
+		u32 slice_num	: 25;
+		u32 reserved	: 7;
+	} reg017;
+
+	struct rkvdec2_y_hor_stride {
+		u32 y_hor_virstride	: 16;
+		u32 reserved		: 16;
+	} reg018;
+
+	struct rkvdec2_uv_hor_stride {
+		u32 uv_hor_virstride	: 16;
+		u32 reserved		: 16;
+	} reg019;
+
+	struct rkvdec2_y_stride {
+		u32 y_virstride		: 28;
+		u32 reserved		: 4;
+	} reg020;
+
+	struct rkvdec2_error_ctrl_set {
+		u32 inter_error_prc_mode		: 1;
+		u32 error_intra_mode			: 1;
+		u32 error_deb_en			: 1;
+		u32 picidx_replace			: 5;
+		u32 error_spread_e			: 1;
+		u32 reserved0				: 3;
+		u32 error_inter_pred_cross_slice	: 1;
+		u32 reserved1				: 11;
+		u32 roi_error_ctu_cal_en		: 1;
+		u32 reserved2				: 7;
+	} reg021;
+
+	struct rkvdec2_err_roi_ctu_offset_start {
+		u32 roi_x_ctu_offset_st	: 12;
+		u32 reserved0		: 4;
+		u32 roi_y_ctu_offset_st	: 12;
+		u32 reserved1		: 4;
+	} reg022;
+
+	struct rkvdec2_err_roi_ctu_offset_end {
+		u32 roi_x_ctu_offset_end	: 12;
+		u32 reserved0			: 4;
+		u32 roi_y_ctu_offset_end	: 12;
+		u32 reserved1			: 4;
+	} reg023;
+
+	struct rkvdec2_cabac_error_en_lowbits {
+		u32 cabac_err_en_lowbits	: 32;
+	} reg024;
+
+	struct rkvdec2_cabac_error_en_highbits {
+		u32 cabac_err_en_highbits	: 30;
+		u32 reserved			: 2;
+	} reg025;
+
+	struct rkvdec2_block_gating_en {
+		u32 swreg_block_gating_e	: 20;
+		u32 reserved			: 11;
+		u32 reg_cfg_gating_en		: 1;
+	} reg026;
+
+	struct SW027_CORE_SAFE_PIXELS {
+		u32 core_safe_x_pixels	: 16;
+		u32 core_safe_y_pixels	: 16;
+	} reg027;
+
+	struct rkvdec2_multiply_core_ctrl {
+		u32 swreg_vp9_wr_prob_idx	: 3;
+		u32 reserved0			: 1;
+		u32 swreg_vp9_rd_prob_idx	: 3;
+		u32 reserved1			: 1;
+
+		u32 swreg_ref_req_advance_flag	: 1;
+		u32 sw_colmv_req_advance_flag	: 1;
+		u32 sw_poc_only_highbit_flag	: 1;
+		u32 sw_poc_arb_flag		: 1;
+
+		u32 reserved2			: 4;
+		u32 sw_film_idx			: 10;
+		u32 reserved3			: 2;
+		u32 sw_pu_req_mismatch_dis	: 1;
+		u32 sw_colmv_req_mismatch_dis	: 1;
+		u32 reserved4			: 2;
+	} reg028;
+
+	struct SW029_SCALE_DOWN_CTRL {
+		u32 scale_down_hor_ratio	: 2;
+		u32 reserved0			: 6;
+		u32 scale_down_vrz_ratio	: 2;
+		u32 reserved1			: 22;
+	} reg029;
+
+	struct SW032_Y_SCALE_DOWN_TILE8x8_HOR_STRIDE {
+		u32 y_scale_down_hor_stride	: 20;
+		u32 reserved0			: 12;
+	} reg030;
+
+	struct SW031_UV_SCALE_DOWN_TILE8x8_HOR_STRIDE {
+		u32 uv_scale_down_hor_stride	: 20;
+		u32 reserved0			: 12;
+	} reg031;
+
+	u32 timeout_threshold;
+} __packed;
+
+/* base: OFFSET_COMMON_ADDR_REGS */
+struct rkvdec2_regs_common_addr {
+	u32 rlc_base;
+	u32 rlcwrite_base;
+	u32 decout_base;
+	u32 colmv_cur_base;
+	u32 error_ref_base;
+	u32 rcb_base[10];
+} __packed;
+
+/* base: OFFSET_CODEC_PARAMS_REGS */
+struct rkvdec2_regs_h264_params {
+	struct rkvdec2_h26x_set {
+		u32 h26x_frame_orslice		: 1;
+		u32 h26x_rps_mode		: 1;
+		u32 h26x_stream_mode		: 1;
+		u32 h26x_stream_lastpacket	: 1;
+		u32 h264_firstslice_flag	: 1;
+		u32 reserved			: 27;
+	} reg064;
+
+	u32 cur_top_poc;
+	u32 cur_bot_poc;
+	u32 ref_pocs[32];
+
+	struct rkvdec2_h264_info {
+		struct rkvdec2_h264_ref_info {
+			u32 ref_field		: 1;
+			u32 ref_topfield_used	: 1;
+			u32 ref_botfield_used	: 1;
+			u32 ref_colmv_use_flag	: 1;
+			u32 ref_reserved	: 4;
+		} __packed ref_info[4];
+	} __packed ref_info_regs[4];
+
+	u32 reserved_103_111[9];
+
+	struct rkvdec2_error_ref_info {
+		u32 avs2_ref_error_field	: 1;
+		u32 avs2_ref_error_topfield	: 1;
+		u32 ref_error_topfield_used	: 1;
+		u32 ref_error_botfield_used	: 1;
+		u32 reserved			: 28;
+	} reg112;
+} __packed;
+
+/* base: OFFSET_CODEC_ADDR_REGS */
+struct rkvdec2_regs_h264_addr {
+	u32 reserved_160;
+	u32 pps_base;
+	u32 reserved_162;
+	u32 rps_base;
+	u32 ref_base[16];
+	u32 scanlist_addr;
+	u32 colmv_base[16];
+	u32 cabactbl_base;
+} __packed;
+
+struct rkvdec2_regs_h264_highpoc {
+	struct rkvdec2_ref_poc_highbit {
+		u32 ref0_poc_highbit	: 4;
+		u32 ref1_poc_highbit	: 4;
+		u32 ref2_poc_highbit	: 4;
+		u32 ref3_poc_highbit	: 4;
+		u32 ref4_poc_highbit	: 4;
+		u32 ref5_poc_highbit	: 4;
+		u32 ref6_poc_highbit	: 4;
+		u32 ref7_poc_highbit	: 4;
+	} reg200[4];
+	struct rkvdec2_cur_poc_highbit {
+		u32 cur_poc_highbit	: 4;
+		u32 reserved		: 28;
+	} reg204;
+} __packed;
+
+struct rkvdec2_regs_h264 {
+	struct rkvdec2_regs_common		common;
+	struct rkvdec2_regs_h264_params		h264_param;
+	struct rkvdec2_regs_common_addr		common_addr;
+	struct rkvdec2_regs_h264_addr		h264_addr;
+	struct rkvdec2_regs_h264_highpoc	h264_highpoc;
+} __packed;
+
+#endif /* __RKVDEC_REGS_H__ */
diff --git a/drivers/staging/media/rkvdec2/rkvdec2.c b/drivers/staging/media/rkvdec2/rkvdec2.c
new file mode 100644
index 000000000000..464eec67521b
--- /dev/null
+++ b/drivers/staging/media/rkvdec2/rkvdec2.c
@@ -0,0 +1,1253 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Rockchip Video Decoder 2 driver
+ *
+ * Copyright (C) 2024 Collabora, Ltd.
+ *  Detlev Casanova <detlev.casanova@collabora.com>
+ *
+ * Based on rkvdec driver by Boris Brezillon <boris.brezillon@collabora.com>
+ */
+
+#include <linux/clk.h>
+#include <linux/genalloc.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/pm.h>
+#include <linux/pm_runtime.h>
+#include <linux/slab.h>
+#include <linux/videodev2.h>
+#include <linux/workqueue.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-vmalloc.h>
+
+#include "rkvdec2.h"
+
+static int rkvdec2_try_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct rkvdec2_ctx *ctx = container_of(ctrl->handler, struct rkvdec2_ctx, ctrl_hdl);
+	const struct rkvdec2_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+
+	if (desc->ops->try_ctrl)
+		return desc->ops->try_ctrl(ctx, ctrl);
+
+	return 0;
+}
+
+static const struct v4l2_ctrl_ops rkvdec2_ctrl_ops = {
+	.try_ctrl = rkvdec2_try_ctrl,
+};
+
+static const struct rkvdec2_ctrl_desc rkvdec2_h264_ctrl_descs[] = {
+	{
+		.cfg.id = V4L2_CID_STATELESS_H264_DECODE_PARAMS,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_H264_SPS,
+		.cfg.ops = &rkvdec2_ctrl_ops,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_H264_PPS,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_H264_SCALING_MATRIX,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_H264_DECODE_MODE,
+		.cfg.min = V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED,
+		.cfg.max = V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED,
+		.cfg.def = V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_H264_START_CODE,
+		.cfg.min = V4L2_STATELESS_H264_START_CODE_ANNEX_B,
+		.cfg.def = V4L2_STATELESS_H264_START_CODE_ANNEX_B,
+		.cfg.max = V4L2_STATELESS_H264_START_CODE_ANNEX_B,
+	},
+	{
+		.cfg.id = V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+		.cfg.min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
+		.cfg.max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
+		.cfg.menu_skip_mask =
+			BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED),
+		.cfg.def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
+	},
+	{
+		.cfg.id = V4L2_CID_MPEG_VIDEO_H264_LEVEL,
+		.cfg.min = V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
+		.cfg.max = V4L2_MPEG_VIDEO_H264_LEVEL_6_1,
+	},
+};
+
+static const struct rkvdec2_ctrls rkvdec2_h264_ctrls = {
+	.ctrls = rkvdec2_h264_ctrl_descs,
+	.num_ctrls = ARRAY_SIZE(rkvdec2_h264_ctrl_descs),
+};
+
+static const u32 rkvdec2_h264_decoded_fmts[] = {
+	V4L2_PIX_FMT_NV12
+};
+
+static const struct rkvdec2_coded_fmt_desc rkvdec2_coded_fmts[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_H264_SLICE,
+		.frmsize = {
+			.min_width = 16,
+			.max_width =  65520,
+			.step_width = 16,
+			.min_height = 16,
+			.max_height =  65520,
+			.step_height = 16,
+		},
+		.ctrls = &rkvdec2_h264_ctrls,
+		.ops = &rkvdec2_h264_fmt_ops,
+		.num_decoded_fmts = ARRAY_SIZE(rkvdec2_h264_decoded_fmts),
+		.decoded_fmts = rkvdec2_h264_decoded_fmts,
+		.subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
+	},
+};
+
+enum rcb_axis {
+	PIC_WIDTH = 0,
+	PIC_HEIGHT = 1
+};
+
+struct rcb_size_info {
+	u8 multiplier;
+	enum rcb_axis axis;
+};
+
+static struct rcb_size_info rcb_sizes[] = {
+	{6,	PIC_WIDTH},	// intrar
+	{1,	PIC_WIDTH},	// transdr (Is actually 0.4*pic_width)
+	{1,	PIC_HEIGHT},	// transdc (Is actually 0.1*pic_height)
+	{3,	PIC_WIDTH},	// streamdr
+	{6,	PIC_WIDTH},	// interr
+	{3,	PIC_HEIGHT},	// interc
+	{22,	PIC_WIDTH},	// dblkr
+	{6,	PIC_WIDTH},	// saor
+	{11,	PIC_WIDTH},	// fbcr
+	{67,	PIC_HEIGHT},	// filtc col
+};
+
+#define RCB_SIZE(n) (rcb_sizes[(n)].multiplier * (rcb_sizes[(n)].axis ? height : width))
+
+static const struct rkvdec2_coded_fmt_desc *
+rkvdec2_find_coded_fmt_desc(u32 fourcc)
+{
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(rkvdec2_coded_fmts); i++) {
+		if (rkvdec2_coded_fmts[i].fourcc == fourcc)
+			return &rkvdec2_coded_fmts[i];
+	}
+
+	return NULL;
+}
+
+static void rkvdec2_reset_fmt(struct rkvdec2_ctx *ctx, struct v4l2_format *f,
+			      u32 fourcc)
+{
+	memset(f, 0, sizeof(*f));
+	f->fmt.pix_mp.pixelformat = fourcc;
+	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
+	f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
+}
+
+static void rkvdec2_reset_coded_fmt(struct rkvdec2_ctx *ctx)
+{
+	struct v4l2_format *f = &ctx->coded_fmt;
+
+	ctx->coded_fmt_desc = &rkvdec2_coded_fmts[0];
+	rkvdec2_reset_fmt(ctx, f, ctx->coded_fmt_desc->fourcc);
+
+	f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	f->fmt.pix_mp.width = ctx->coded_fmt_desc->frmsize.min_width;
+	f->fmt.pix_mp.height = ctx->coded_fmt_desc->frmsize.min_height;
+
+	if (ctx->coded_fmt_desc->ops->adjust_fmt)
+		ctx->coded_fmt_desc->ops->adjust_fmt(ctx, f);
+}
+
+static void rkvdec2_reset_decoded_fmt(struct rkvdec2_ctx *ctx)
+{
+	struct v4l2_format *f = &ctx->decoded_fmt;
+
+	rkvdec2_reset_fmt(ctx, f, ctx->coded_fmt_desc->decoded_fmts[0]);
+	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	v4l2_fill_pixfmt_mp(&f->fmt.pix_mp,
+			    ctx->coded_fmt_desc->decoded_fmts[0],
+			    ctx->coded_fmt.fmt.pix_mp.width,
+			    ctx->coded_fmt.fmt.pix_mp.height);
+
+	ctx->colmv_offset = f->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+	f->fmt.pix_mp.plane_fmt[0].sizeimage += 128 *
+		DIV_ROUND_UP(f->fmt.pix_mp.width, 16) *
+		DIV_ROUND_UP(f->fmt.pix_mp.height, 16);
+}
+
+static int rkvdec2_enum_framesizes(struct file *file, void *priv,
+				   struct v4l2_frmsizeenum *fsize)
+{
+	const struct rkvdec2_coded_fmt_desc *fmt;
+
+	if (fsize->index != 0)
+		return -EINVAL;
+
+	fmt = rkvdec2_find_coded_fmt_desc(fsize->pixel_format);
+	if (!fmt)
+		return -EINVAL;
+
+	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+	fsize->stepwise = fmt->frmsize;
+	return 0;
+}
+
+static int rkvdec2_querycap(struct file *file, void *priv,
+			    struct v4l2_capability *cap)
+{
+	struct rkvdec2_dev *rkvdec = video_drvdata(file);
+	struct video_device *vdev = video_devdata(file);
+
+	strscpy(cap->driver, rkvdec->dev->driver->name,
+		sizeof(cap->driver));
+	strscpy(cap->card, vdev->name, sizeof(cap->card));
+	snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+		 rkvdec->dev->driver->name);
+	return 0;
+}
+
+static int rkvdec2_try_capture_fmt(struct file *file, void *priv,
+				   struct v4l2_format *f)
+{
+	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+	struct rkvdec2_ctx *ctx = fh_to_rkvdec2_ctx(priv);
+	const struct rkvdec2_coded_fmt_desc *coded_desc;
+	unsigned int i;
+
+	/*
+	 * The codec context should point to a coded format desc, if the format
+	 * on the coded end has not been set yet, it should point to the
+	 * default value.
+	 */
+	coded_desc = ctx->coded_fmt_desc;
+	if (WARN_ON(!coded_desc))
+		return -EINVAL;
+
+	for (i = 0; i < coded_desc->num_decoded_fmts; i++) {
+		if (coded_desc->decoded_fmts[i] == pix_mp->pixelformat)
+			break;
+	}
+
+	if (i == coded_desc->num_decoded_fmts)
+		pix_mp->pixelformat = coded_desc->decoded_fmts[0];
+
+	/* Always apply the frmsize constraint of the coded end. */
+	pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width);
+	pix_mp->height = max(pix_mp->height, ctx->coded_fmt.fmt.pix_mp.height);
+	v4l2_apply_frmsize_constraints(&pix_mp->width,
+				       &pix_mp->height,
+				       &coded_desc->frmsize);
+
+	v4l2_fill_pixfmt_mp(pix_mp, pix_mp->pixelformat,
+			    pix_mp->width, pix_mp->height);
+
+	pix_mp->plane_fmt[0].sizeimage +=
+		128 *
+		DIV_ROUND_UP(pix_mp->width, 16) *
+		DIV_ROUND_UP(pix_mp->height, 16);
+
+	pix_mp->field = V4L2_FIELD_NONE;
+
+	return 0;
+}
+
+static int rkvdec2_try_output_fmt(struct file *file, void *priv,
+				  struct v4l2_format *f)
+{
+	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+	struct rkvdec2_ctx *ctx = fh_to_rkvdec2_ctx(priv);
+	const struct rkvdec2_coded_fmt_desc *desc;
+
+	desc = rkvdec2_find_coded_fmt_desc(pix_mp->pixelformat);
+	if (!desc) {
+		pix_mp->pixelformat = rkvdec2_coded_fmts[0].fourcc;
+		desc = &rkvdec2_coded_fmts[0];
+	}
+
+	v4l2_apply_frmsize_constraints(&pix_mp->width,
+				       &pix_mp->height,
+				       &desc->frmsize);
+
+	pix_mp->field = V4L2_FIELD_NONE;
+	/* All coded formats are considered single planar for now. */
+	pix_mp->num_planes = 1;
+
+	if (desc->ops->adjust_fmt) {
+		int ret;
+
+		ret = desc->ops->adjust_fmt(ctx, f);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+static int rkvdec2_s_capture_fmt(struct file *file, void *priv,
+				 struct v4l2_format *f)
+{
+	struct rkvdec2_ctx *ctx = fh_to_rkvdec2_ctx(priv);
+	struct vb2_queue *vq;
+	int ret;
+
+	/* Change not allowed if queue is busy */
+	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
+			     V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+	if (vb2_is_busy(vq))
+		return -EBUSY;
+
+	ret = rkvdec2_try_capture_fmt(file, priv, f);
+	if (ret)
+		return ret;
+
+	ctx->decoded_fmt = *f;
+	return 0;
+}
+
+static int rkvdec2_s_output_fmt(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	struct rkvdec2_ctx *ctx = fh_to_rkvdec2_ctx(priv);
+	struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
+	const struct rkvdec2_coded_fmt_desc *desc;
+	struct v4l2_format *cap_fmt;
+	struct vb2_queue *peer_vq, *vq;
+	int ret;
+
+	/*
+	 * In order to support dynamic resolution change, the decoder admits
+	 * a resolution change, as long as the pixelformat remains. Can't be
+	 * done if streaming.
+	 */
+	vq = v4l2_m2m_get_vq(m2m_ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+	if (vb2_is_streaming(vq) ||
+	    (vb2_is_busy(vq) &&
+	     f->fmt.pix_mp.pixelformat != ctx->coded_fmt.fmt.pix_mp.pixelformat))
+		return -EBUSY;
+
+	/*
+	 * Since format change on the OUTPUT queue will reset the CAPTURE
+	 * queue, we can't allow doing so when the CAPTURE queue has buffers
+	 * allocated.
+	 */
+	peer_vq = v4l2_m2m_get_vq(m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+	if (vb2_is_busy(peer_vq))
+		return -EBUSY;
+
+	ret = rkvdec2_try_output_fmt(file, priv, f);
+	if (ret)
+		return ret;
+
+	desc = rkvdec2_find_coded_fmt_desc(f->fmt.pix_mp.pixelformat);
+	if (!desc)
+		return -EINVAL;
+	ctx->coded_fmt_desc = desc;
+	ctx->coded_fmt = *f;
+
+	/*
+	 * Current decoded format might have become invalid with newly
+	 * selected codec, so reset it to default just to be safe and
+	 * keep internal driver state sane. User is mandated to set
+	 * the decoded format again after we return, so we don't need
+	 * anything smarter.
+	 *
+	 * Note that this will propagate any size changes to the decoded format.
+	 */
+	rkvdec2_reset_decoded_fmt(ctx);
+
+	/* Propagate colorspace information to capture. */
+	cap_fmt = &ctx->decoded_fmt;
+	cap_fmt->fmt.pix_mp.colorspace = f->fmt.pix_mp.colorspace;
+	cap_fmt->fmt.pix_mp.xfer_func = f->fmt.pix_mp.xfer_func;
+	cap_fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc;
+	cap_fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization;
+
+	/* Enable format specific queue features */
+	vq->subsystem_flags |= desc->subsystem_flags;
+
+	return 0;
+}
+
+static int rkvdec2_g_output_fmt(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	struct rkvdec2_ctx *ctx = fh_to_rkvdec2_ctx(priv);
+
+	*f = ctx->coded_fmt;
+	return 0;
+}
+
+static int rkvdec2_g_capture_fmt(struct file *file, void *priv,
+				 struct v4l2_format *f)
+{
+	struct rkvdec2_ctx *ctx = fh_to_rkvdec2_ctx(priv);
+
+	*f = ctx->decoded_fmt;
+	return 0;
+}
+
+static int rkvdec2_enum_output_fmt(struct file *file, void *priv,
+				   struct v4l2_fmtdesc *f)
+{
+	if (f->index >= ARRAY_SIZE(rkvdec2_coded_fmts))
+		return -EINVAL;
+
+	f->pixelformat = rkvdec2_coded_fmts[f->index].fourcc;
+	return 0;
+}
+
+static int rkvdec2_enum_capture_fmt(struct file *file, void *priv,
+				    struct v4l2_fmtdesc *f)
+{
+	struct rkvdec2_ctx *ctx = fh_to_rkvdec2_ctx(priv);
+
+	if (WARN_ON(!ctx->coded_fmt_desc))
+		return -EINVAL;
+
+	if (f->index >= ctx->coded_fmt_desc->num_decoded_fmts)
+		return -EINVAL;
+
+	f->pixelformat = ctx->coded_fmt_desc->decoded_fmts[f->index];
+	return 0;
+}
+
+static const struct v4l2_ioctl_ops rkvdec2_ioctl_ops = {
+	.vidioc_querycap = rkvdec2_querycap,
+	.vidioc_enum_framesizes = rkvdec2_enum_framesizes,
+
+	.vidioc_try_fmt_vid_cap_mplane = rkvdec2_try_capture_fmt,
+	.vidioc_try_fmt_vid_out_mplane = rkvdec2_try_output_fmt,
+	.vidioc_s_fmt_vid_out_mplane = rkvdec2_s_output_fmt,
+	.vidioc_s_fmt_vid_cap_mplane = rkvdec2_s_capture_fmt,
+	.vidioc_g_fmt_vid_out_mplane = rkvdec2_g_output_fmt,
+	.vidioc_g_fmt_vid_cap_mplane = rkvdec2_g_capture_fmt,
+	.vidioc_enum_fmt_vid_out = rkvdec2_enum_output_fmt,
+	.vidioc_enum_fmt_vid_cap = rkvdec2_enum_capture_fmt,
+
+	.vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs,
+	.vidioc_querybuf = v4l2_m2m_ioctl_querybuf,
+	.vidioc_qbuf = v4l2_m2m_ioctl_qbuf,
+	.vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
+	.vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
+	.vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
+	.vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
+
+	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+
+	.vidioc_streamon = v4l2_m2m_ioctl_streamon,
+	.vidioc_streamoff = v4l2_m2m_ioctl_streamoff,
+
+	.vidioc_decoder_cmd = v4l2_m2m_ioctl_stateless_decoder_cmd,
+	.vidioc_try_decoder_cmd = v4l2_m2m_ioctl_stateless_try_decoder_cmd,
+};
+
+static int rkvdec2_queue_setup(struct vb2_queue *vq, unsigned int *num_buffers,
+			       unsigned int *num_planes, unsigned int sizes[],
+			      struct device *alloc_devs[])
+{
+	struct rkvdec2_ctx *ctx = vb2_get_drv_priv(vq);
+	struct v4l2_format *f;
+	unsigned int i;
+
+	if (V4L2_TYPE_IS_OUTPUT(vq->type))
+		f = &ctx->coded_fmt;
+	else
+		f = &ctx->decoded_fmt;
+
+	if (*num_planes) {
+		if (*num_planes != f->fmt.pix_mp.num_planes)
+			return -EINVAL;
+
+		for (i = 0; i < f->fmt.pix_mp.num_planes; i++) {
+			if (sizes[i] < f->fmt.pix_mp.plane_fmt[i].sizeimage)
+				return -EINVAL;
+		}
+	} else {
+		*num_planes = f->fmt.pix_mp.num_planes;
+		for (i = 0; i < f->fmt.pix_mp.num_planes; i++)
+			sizes[i] = f->fmt.pix_mp.plane_fmt[i].sizeimage;
+	}
+
+	return 0;
+}
+
+static int rkvdec2_buf_prepare(struct vb2_buffer *vb)
+{
+	struct vb2_queue *vq = vb->vb2_queue;
+	struct rkvdec2_ctx *ctx = vb2_get_drv_priv(vq);
+	struct v4l2_format *f;
+	unsigned int i;
+
+	if (V4L2_TYPE_IS_OUTPUT(vq->type))
+		f = &ctx->coded_fmt;
+	else
+		f = &ctx->decoded_fmt;
+
+	for (i = 0; i < f->fmt.pix_mp.num_planes; ++i) {
+		u32 sizeimage = f->fmt.pix_mp.plane_fmt[i].sizeimage;
+
+		if (vb2_plane_size(vb, i) < sizeimage)
+			return -EINVAL;
+	}
+
+	/*
+	 * Buffer's bytesused must be written by driver for CAPTURE buffers.
+	 * (for OUTPUT buffers, if userspace passes 0 bytesused, v4l2-core sets
+	 * it to buffer length).
+	 */
+	if (V4L2_TYPE_IS_CAPTURE(vq->type))
+		vb2_set_plane_payload(vb, 0, f->fmt.pix_mp.plane_fmt[0].sizeimage);
+
+	return 0;
+}
+
+static void rkvdec2_buf_queue(struct vb2_buffer *vb)
+{
+	struct rkvdec2_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+
+	v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, vbuf);
+}
+
+static int rkvdec2_buf_out_validate(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+
+	vbuf->field = V4L2_FIELD_NONE;
+	return 0;
+}
+
+static void rkvdec2_buf_request_complete(struct vb2_buffer *vb)
+{
+	struct rkvdec2_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_ctrl_request_complete(vb->req_obj.req, &ctx->ctrl_hdl);
+}
+
+static void rkvdec2_free_rcb(struct rkvdec2_ctx *ctx)
+{
+	u32 width, height;
+	int i;
+
+	width = ctx->decoded_fmt.fmt.pix_mp.width;
+	height = ctx->decoded_fmt.fmt.pix_mp.height;
+
+	for (i = 0; i < RKVDEC2_RCB_COUNT; i++) {
+		if (!ctx->rcb_bufs[i].cpu)
+			continue;
+
+		switch (ctx->rcb_bufs[i].type) {
+		case RKVDEC2_ALLOC_SRAM:
+			gen_pool_free(ctx->dev->sram_pool,
+				      (unsigned long)ctx->rcb_bufs[i].cpu,
+				      RCB_SIZE(i));
+			break;
+		case RKVDEC2_ALLOC_DMA:
+			dma_free_coherent(ctx->dev->dev,
+					  RCB_SIZE(i),
+					  ctx->rcb_bufs[i].cpu,
+					  ctx->rcb_bufs[i].dma);
+			break;
+		}
+	}
+}
+
+static int rkvdec2_allocate_rcb(struct rkvdec2_ctx *ctx)
+{
+	int ret, i;
+	u32 width, height;
+
+	memset(ctx->rcb_bufs, 0, sizeof(*ctx->rcb_bufs));
+
+	width = ctx->decoded_fmt.fmt.pix_mp.width;
+	height = ctx->decoded_fmt.fmt.pix_mp.height;
+
+	for (i = 0; i < RKVDEC2_RCB_COUNT; i++) {
+		void *cpu = NULL;
+		dma_addr_t dma;
+		size_t rcb_size = RCB_SIZE(i);
+		enum rkvdec2_alloc_type alloc_type = RKVDEC2_ALLOC_SRAM;
+
+		if (ctx->dev->sram_pool) {
+			cpu = gen_pool_dma_zalloc_align(ctx->dev->sram_pool,
+							rcb_size,
+							&dma,
+							64);
+		}
+
+		/* Fallback to RAM */
+		if (!cpu) {
+			cpu = dma_alloc_coherent(ctx->dev->dev,
+						 rcb_size,
+						 &dma,
+						 GFP_KERNEL);
+			alloc_type = RKVDEC2_ALLOC_DMA;
+		}
+
+		if (!cpu) {
+			ret = -ENOMEM;
+			goto err_alloc;
+		}
+
+		ctx->rcb_bufs[i].cpu = cpu;
+		ctx->rcb_bufs[i].dma = dma;
+		ctx->rcb_bufs[i].size = rcb_size;
+		ctx->rcb_bufs[i].type = alloc_type;
+	}
+
+	return 0;
+
+err_alloc:
+	rkvdec2_free_rcb(ctx);
+
+	return ret;
+}
+
+static int rkvdec2_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+	struct rkvdec2_ctx *ctx = vb2_get_drv_priv(q);
+	const struct rkvdec2_coded_fmt_desc *desc;
+	int ret;
+
+	if (V4L2_TYPE_IS_CAPTURE(q->type))
+		return 0;
+
+	desc = ctx->coded_fmt_desc;
+	if (WARN_ON(!desc))
+		return -EINVAL;
+
+	ret = rkvdec2_allocate_rcb(ctx);
+	if (ret)
+		return ret;
+
+	if (desc->ops->start) {
+		ret = desc->ops->start(ctx);
+		if (ret)
+			goto err_ops_start;
+	}
+
+	return 0;
+
+err_ops_start:
+	rkvdec2_free_rcb(ctx);
+
+	return ret;
+}
+
+static void rkvdec2_queue_cleanup(struct vb2_queue *vq, u32 state)
+{
+	struct rkvdec2_ctx *ctx = vb2_get_drv_priv(vq);
+
+	while (true) {
+		struct vb2_v4l2_buffer *vbuf;
+
+		if (V4L2_TYPE_IS_OUTPUT(vq->type))
+			vbuf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+		else
+			vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+
+		if (!vbuf)
+			break;
+
+		v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req,
+					   &ctx->ctrl_hdl);
+		v4l2_m2m_buf_done(vbuf, state);
+	}
+}
+
+static void rkvdec2_stop_streaming(struct vb2_queue *q)
+{
+	struct rkvdec2_ctx *ctx = vb2_get_drv_priv(q);
+
+	if (V4L2_TYPE_IS_OUTPUT(q->type)) {
+		const struct rkvdec2_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+
+		if (WARN_ON(!desc))
+			return;
+
+		if (desc->ops->stop)
+			desc->ops->stop(ctx);
+
+		rkvdec2_free_rcb(ctx);
+	}
+
+	rkvdec2_queue_cleanup(q, VB2_BUF_STATE_ERROR);
+}
+
+static const struct vb2_ops rkvdec2_queue_ops = {
+	.queue_setup = rkvdec2_queue_setup,
+	.buf_prepare = rkvdec2_buf_prepare,
+	.buf_queue = rkvdec2_buf_queue,
+	.buf_out_validate = rkvdec2_buf_out_validate,
+	.buf_request_complete = rkvdec2_buf_request_complete,
+	.start_streaming = rkvdec2_start_streaming,
+	.stop_streaming = rkvdec2_stop_streaming,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+};
+
+static int rkvdec2_request_validate(struct media_request *req)
+{
+	unsigned int count;
+
+	count = vb2_request_buffer_cnt(req);
+	if (!count)
+		return -ENOENT;
+	else if (count > 1)
+		return -EINVAL;
+
+	return vb2_request_validate(req);
+}
+
+static const struct media_device_ops rkvdec2_media_ops = {
+	.req_validate = rkvdec2_request_validate,
+	.req_queue = v4l2_m2m_request_queue,
+};
+
+static void rkvdec2_job_finish_no_pm(struct rkvdec2_ctx *ctx,
+				     enum vb2_buffer_state result)
+{
+	if (ctx->coded_fmt_desc->ops->done) {
+		struct vb2_v4l2_buffer *src_buf, *dst_buf;
+
+		src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+		dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+		ctx->coded_fmt_desc->ops->done(ctx, src_buf, dst_buf, result);
+	}
+
+	v4l2_m2m_buf_done_and_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx,
+					 result);
+}
+
+static void rkvdec2_job_finish(struct rkvdec2_ctx *ctx,
+			       enum vb2_buffer_state result)
+{
+	struct rkvdec2_dev *rkvdec = ctx->dev;
+
+	pm_runtime_mark_last_busy(rkvdec->dev);
+	pm_runtime_put_autosuspend(rkvdec->dev);
+	rkvdec2_job_finish_no_pm(ctx, result);
+}
+
+void rkvdec2_run_preamble(struct rkvdec2_ctx *ctx, struct rkvdec2_run *run)
+{
+	struct media_request *src_req;
+
+	memset(run, 0, sizeof(*run));
+
+	run->bufs.src = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+	run->bufs.dst = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+
+	/* Apply request(s) controls if needed. */
+	src_req = run->bufs.src->vb2_buf.req_obj.req;
+	if (src_req)
+		v4l2_ctrl_request_setup(src_req, &ctx->ctrl_hdl);
+
+	v4l2_m2m_buf_copy_metadata(run->bufs.src, run->bufs.dst, true);
+}
+
+void rkvdec2_run_postamble(struct rkvdec2_ctx *ctx, struct rkvdec2_run *run)
+{
+	struct media_request *src_req = run->bufs.src->vb2_buf.req_obj.req;
+
+	if (src_req)
+		v4l2_ctrl_request_complete(src_req, &ctx->ctrl_hdl);
+}
+
+static void rkvdec2_device_run(void *priv)
+{
+	struct rkvdec2_ctx *ctx = priv;
+	struct rkvdec2_dev *rkvdec = ctx->dev;
+	const struct rkvdec2_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+	int ret;
+
+	if (WARN_ON(!desc))
+		return;
+
+	ret = pm_runtime_resume_and_get(rkvdec->dev);
+	if (ret < 0) {
+		rkvdec2_job_finish_no_pm(ctx, VB2_BUF_STATE_ERROR);
+		return;
+	}
+
+	ret = desc->ops->run(ctx);
+	if (ret)
+		rkvdec2_job_finish(ctx, VB2_BUF_STATE_ERROR);
+}
+
+static const struct v4l2_m2m_ops rkvdec2_m2m_ops = {
+	.device_run = rkvdec2_device_run,
+};
+
+static int rkvdec2_queue_init(void *priv,
+			      struct vb2_queue *src_vq,
+			      struct vb2_queue *dst_vq)
+{
+	struct rkvdec2_ctx *ctx = priv;
+	struct rkvdec2_dev *rkvdec = ctx->dev;
+	int ret;
+
+	src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	src_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+	src_vq->drv_priv = ctx;
+	src_vq->ops = &rkvdec2_queue_ops;
+	src_vq->mem_ops = &vb2_dma_contig_memops;
+
+	/*
+	 * No CPU access on the queues, so no kernel mapping needed.
+	 */
+	src_vq->dma_attrs = DMA_ATTR_NO_KERNEL_MAPPING;
+	src_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	src_vq->lock = &rkvdec->vdev_lock;
+	src_vq->dev = rkvdec->v4l2_dev.dev;
+	src_vq->supports_requests = true;
+	src_vq->requires_requests = true;
+
+	ret = vb2_queue_init(src_vq);
+	if (ret)
+		return ret;
+
+	dst_vq->bidirectional = true;
+	dst_vq->mem_ops = &vb2_dma_contig_memops;
+	dst_vq->dma_attrs = DMA_ATTR_NO_KERNEL_MAPPING;
+	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	dst_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+	dst_vq->drv_priv = ctx;
+	dst_vq->ops = &rkvdec2_queue_ops;
+	dst_vq->buf_struct_size = sizeof(struct rkvdec2_decoded_buffer);
+	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	dst_vq->lock = &rkvdec->vdev_lock;
+	dst_vq->dev = rkvdec->v4l2_dev.dev;
+
+	return vb2_queue_init(dst_vq);
+}
+
+static int rkvdec2_add_ctrls(struct rkvdec2_ctx *ctx,
+			     const struct rkvdec2_ctrls *ctrls)
+{
+	unsigned int i;
+
+	for (i = 0; i < ctrls->num_ctrls; i++) {
+		const struct v4l2_ctrl_config *cfg = &ctrls->ctrls[i].cfg;
+
+		v4l2_ctrl_new_custom(&ctx->ctrl_hdl, cfg, ctx);
+		if (ctx->ctrl_hdl.error)
+			return ctx->ctrl_hdl.error;
+	}
+
+	return 0;
+}
+
+static int rkvdec2_init_ctrls(struct rkvdec2_ctx *ctx)
+{
+	unsigned int i, nctrls = 0;
+	int ret;
+
+	for (i = 0; i < ARRAY_SIZE(rkvdec2_coded_fmts); i++)
+		nctrls += rkvdec2_coded_fmts[i].ctrls->num_ctrls;
+
+	v4l2_ctrl_handler_init(&ctx->ctrl_hdl, nctrls);
+
+	for (i = 0; i < ARRAY_SIZE(rkvdec2_coded_fmts); i++) {
+		ret = rkvdec2_add_ctrls(ctx, rkvdec2_coded_fmts[i].ctrls);
+		if (ret)
+			goto err_free_handler;
+	}
+
+	ret = v4l2_ctrl_handler_setup(&ctx->ctrl_hdl);
+	if (ret)
+		goto err_free_handler;
+
+	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
+	return 0;
+
+err_free_handler:
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+	return ret;
+}
+
+static int rkvdec2_open(struct file *filp)
+{
+	struct rkvdec2_dev *rkvdec = video_drvdata(filp);
+	struct rkvdec2_ctx *ctx;
+	int ret;
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx)
+		return -ENOMEM;
+
+	ctx->dev = rkvdec;
+	rkvdec2_reset_coded_fmt(ctx);
+	rkvdec2_reset_decoded_fmt(ctx);
+	v4l2_fh_init(&ctx->fh, video_devdata(filp));
+
+	ret = rkvdec2_init_ctrls(ctx);
+	if (ret)
+		goto err_free_ctx;
+
+	ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(rkvdec->m2m_dev, ctx,
+					    rkvdec2_queue_init);
+	if (IS_ERR(ctx->fh.m2m_ctx)) {
+		ret = PTR_ERR(ctx->fh.m2m_ctx);
+		goto err_cleanup_ctrls;
+	}
+
+	filp->private_data = &ctx->fh;
+	v4l2_fh_add(&ctx->fh);
+
+	return 0;
+
+err_cleanup_ctrls:
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+
+err_free_ctx:
+	kfree(ctx);
+	return ret;
+}
+
+static int rkvdec2_release(struct file *filp)
+{
+	struct rkvdec2_ctx *ctx = fh_to_rkvdec2_ctx(filp->private_data);
+
+	v4l2_fh_del(&ctx->fh);
+	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+	v4l2_fh_exit(&ctx->fh);
+	kfree(ctx);
+
+	return 0;
+}
+
+static const struct v4l2_file_operations rkvdec2_fops = {
+	.owner = THIS_MODULE,
+	.open = rkvdec2_open,
+	.release = rkvdec2_release,
+	.poll = v4l2_m2m_fop_poll,
+	.unlocked_ioctl = video_ioctl2,
+	.mmap = v4l2_m2m_fop_mmap,
+};
+
+static int rkvdec2_v4l2_init(struct rkvdec2_dev *rkvdec)
+{
+	int ret;
+
+	ret = v4l2_device_register(rkvdec->dev, &rkvdec->v4l2_dev);
+	if (ret) {
+		dev_err(rkvdec->dev, "Failed to register V4L2 device\n");
+		return ret;
+	}
+
+	rkvdec->m2m_dev = v4l2_m2m_init(&rkvdec2_m2m_ops);
+	if (IS_ERR(rkvdec->m2m_dev)) {
+		v4l2_err(&rkvdec->v4l2_dev, "Failed to init mem2mem device\n");
+		ret = PTR_ERR(rkvdec->m2m_dev);
+		goto err_unregister_v4l2;
+	}
+
+	rkvdec->mdev.dev = rkvdec->dev;
+	strscpy(rkvdec->mdev.model, "rkvdec2", sizeof(rkvdec->mdev.model));
+	strscpy(rkvdec->mdev.bus_info, "platform:rkvdec2",
+		sizeof(rkvdec->mdev.bus_info));
+	media_device_init(&rkvdec->mdev);
+	rkvdec->mdev.ops = &rkvdec2_media_ops;
+	rkvdec->v4l2_dev.mdev = &rkvdec->mdev;
+
+	rkvdec->vdev.lock = &rkvdec->vdev_lock;
+	rkvdec->vdev.v4l2_dev = &rkvdec->v4l2_dev;
+	rkvdec->vdev.fops = &rkvdec2_fops;
+	rkvdec->vdev.release = video_device_release_empty;
+	rkvdec->vdev.vfl_dir = VFL_DIR_M2M;
+	rkvdec->vdev.device_caps = V4L2_CAP_STREAMING |
+				   V4L2_CAP_VIDEO_M2M_MPLANE;
+	rkvdec->vdev.ioctl_ops = &rkvdec2_ioctl_ops;
+	video_set_drvdata(&rkvdec->vdev, rkvdec);
+	strscpy(rkvdec->vdev.name, "rkvdec2", sizeof(rkvdec->vdev.name));
+
+	ret = video_register_device(&rkvdec->vdev, VFL_TYPE_VIDEO, -1);
+	if (ret) {
+		v4l2_err(&rkvdec->v4l2_dev, "Failed to register video device\n");
+		goto err_cleanup_mc;
+	}
+
+	ret = v4l2_m2m_register_media_controller(rkvdec->m2m_dev, &rkvdec->vdev,
+						 MEDIA_ENT_F_PROC_VIDEO_DECODER);
+	if (ret) {
+		v4l2_err(&rkvdec->v4l2_dev,
+			 "Failed to initialize V4L2 M2M media controller\n");
+		goto err_unregister_vdev;
+	}
+
+	ret = media_device_register(&rkvdec->mdev);
+	if (ret) {
+		v4l2_err(&rkvdec->v4l2_dev, "Failed to register media device\n");
+		goto err_unregister_mc;
+	}
+
+	return 0;
+
+err_unregister_mc:
+	v4l2_m2m_unregister_media_controller(rkvdec->m2m_dev);
+
+err_unregister_vdev:
+	video_unregister_device(&rkvdec->vdev);
+
+err_cleanup_mc:
+	media_device_cleanup(&rkvdec->mdev);
+	v4l2_m2m_release(rkvdec->m2m_dev);
+
+err_unregister_v4l2:
+	v4l2_device_unregister(&rkvdec->v4l2_dev);
+	return ret;
+}
+
+static void rkvdec2_v4l2_cleanup(struct rkvdec2_dev *rkvdec)
+{
+	media_device_unregister(&rkvdec->mdev);
+	v4l2_m2m_unregister_media_controller(rkvdec->m2m_dev);
+	video_unregister_device(&rkvdec->vdev);
+	media_device_cleanup(&rkvdec->mdev);
+	v4l2_m2m_release(rkvdec->m2m_dev);
+	v4l2_device_unregister(&rkvdec->v4l2_dev);
+}
+
+static irqreturn_t rkvdec2_irq_handler(int irq, void *priv)
+{
+	struct rkvdec2_dev *rkvdec = priv;
+	enum vb2_buffer_state state;
+	u32 status;
+
+	status = readl(rkvdec->regs + RKVDEC2_REG_STA_INT);
+	state = (status & STA_INT_DEC_RDY_STA) ?
+		VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
+
+	/* Clear interrupt status */
+	writel(0, rkvdec->regs + RKVDEC2_REG_STA_INT);
+	if (cancel_delayed_work(&rkvdec->watchdog_work)) {
+		struct rkvdec2_ctx *ctx;
+
+		ctx = v4l2_m2m_get_curr_priv(rkvdec->m2m_dev);
+		rkvdec2_job_finish(ctx, state);
+	}
+
+	return IRQ_HANDLED;
+}
+
+static void rkvdec2_watchdog_func(struct work_struct *work)
+{
+	struct rkvdec2_dev *rkvdec = container_of(to_delayed_work(work), struct rkvdec2_dev,
+			      watchdog_work);
+	struct rkvdec2_ctx *ctx;
+
+	ctx = v4l2_m2m_get_curr_priv(rkvdec->m2m_dev);
+	if (ctx) {
+		dev_err(rkvdec->dev, "Frame processing timed out!\n");
+		writel(RKVDEC2_REG_DEC_IRQ_DISABLE, rkvdec->regs + RKVDEC2_REG_IMPORTANT_EN);
+		writel(0, rkvdec->regs + RKVDEC2_REG_DEC_E);
+		rkvdec2_job_finish(ctx, VB2_BUF_STATE_ERROR);
+	}
+}
+
+static const struct of_device_id of_rkvdec2_match[] = {
+	{ .compatible = "rockchip,rk3588-vdec" },
+	{ /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, of_rkvdec2_match);
+
+static const char * const rkvdec2_clk_names[] = {
+	"axi",
+	"ahb",
+	"core",
+	"cabac",
+	"hevc_cabac",
+};
+
+/*
+ * Some SoCs, like RK3588 have multiple identical vdpu34x cores, but the
+ * kernel is currently missing support for multi-core handling. Exposing
+ * separate devices for each core to userspace is bad, since that does
+ * not allow scheduling tasks properly (and creates ABI). With this workaround
+ * the driver will only probe for the first core and early exit for the other
+ * cores. Once the driver gains multi-core support, the same technique
+ * for detecting the main core can be used to cluster all cores together.
+ */
+static int rkvdec2_disable_multicore(struct rkvdec2_dev *rkvdec)
+{
+	const char *compatible;
+	struct device_node *node;
+	int ret;
+
+	/* Intentionally ignores the fallback strings */
+	ret = of_property_read_string(rkvdec->dev->of_node, "compatible", &compatible);
+	if (ret)
+		return ret;
+
+	/* first compatible node found from the root node is considered the main core */
+	node = of_find_compatible_node(NULL, NULL, compatible);
+	if (!node)
+		return -EINVAL; /* broken DT? */
+
+	if (rkvdec->dev->of_node != node) {
+		dev_info(rkvdec->dev, "missing multi-core support, ignoring this instance\n");
+		return -ENODEV;
+	}
+
+	return 0;
+}
+
+static int rkvdec2_probe(struct platform_device *pdev)
+{
+	struct rkvdec2_dev *rkvdec;
+	unsigned int i;
+	int ret, irq;
+
+	rkvdec = devm_kzalloc(&pdev->dev, sizeof(*rkvdec), GFP_KERNEL);
+	if (!rkvdec)
+		return -ENOMEM;
+
+	platform_set_drvdata(pdev, rkvdec);
+	rkvdec->dev = &pdev->dev;
+
+	ret = rkvdec2_disable_multicore(rkvdec);
+	if (ret)
+		return ret;
+
+	mutex_init(&rkvdec->vdev_lock);
+	INIT_DELAYED_WORK(&rkvdec->watchdog_work, rkvdec2_watchdog_func);
+
+	rkvdec->clocks = devm_kcalloc(&pdev->dev, ARRAY_SIZE(rkvdec2_clk_names),
+				      sizeof(*rkvdec->clocks), GFP_KERNEL);
+	if (!rkvdec->clocks)
+		return -ENOMEM;
+
+	for (i = 0; i < ARRAY_SIZE(rkvdec2_clk_names); i++)
+		rkvdec->clocks[i].id = rkvdec2_clk_names[i];
+
+	ret = devm_clk_bulk_get(&pdev->dev, ARRAY_SIZE(rkvdec2_clk_names),
+				rkvdec->clocks);
+	if (ret)
+		return ret;
+
+	rkvdec->regs = devm_platform_ioremap_resource(pdev, 0);
+	if (IS_ERR(rkvdec->regs))
+		return PTR_ERR(rkvdec->regs);
+
+	/*
+	 * Without IOMMU support, keep DMA in the lower 32 bits.
+	 */
+	ret = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
+	if (ret) {
+		dev_err(&pdev->dev, "Could not set DMA coherent mask.\n");
+		return ret;
+	}
+
+	vb2_dma_contig_set_max_seg_size(&pdev->dev, DMA_BIT_MASK(32));
+
+	irq = platform_get_irq(pdev, 0);
+	if (irq <= 0)
+		return -ENXIO;
+
+	ret = devm_request_threaded_irq(&pdev->dev, irq, NULL,
+					rkvdec2_irq_handler, IRQF_ONESHOT,
+					dev_name(&pdev->dev), rkvdec);
+	if (ret) {
+		dev_err(&pdev->dev, "Could not request vdec2 IRQ\n");
+		return ret;
+	}
+
+	rkvdec->sram_pool = of_gen_pool_get(pdev->dev.of_node, "sram", 0);
+	if (!rkvdec->sram_pool)
+		dev_info(&pdev->dev, "No sram node, RCB will be stored in RAM\n");
+
+	pm_runtime_set_autosuspend_delay(&pdev->dev, 100);
+	pm_runtime_use_autosuspend(&pdev->dev);
+	pm_runtime_enable(&pdev->dev);
+
+	ret = rkvdec2_v4l2_init(rkvdec);
+	if (ret)
+		goto err_disable_runtime_pm;
+
+	return 0;
+
+err_disable_runtime_pm:
+	pm_runtime_dont_use_autosuspend(&pdev->dev);
+	pm_runtime_disable(&pdev->dev);
+
+	if (rkvdec->sram_pool)
+		gen_pool_destroy(rkvdec->sram_pool);
+
+	return ret;
+}
+
+static void rkvdec2_remove(struct platform_device *pdev)
+{
+	struct rkvdec2_dev *rkvdec = platform_get_drvdata(pdev);
+
+	cancel_delayed_work_sync(&rkvdec->watchdog_work);
+
+	rkvdec2_v4l2_cleanup(rkvdec);
+	pm_runtime_disable(&pdev->dev);
+	pm_runtime_dont_use_autosuspend(&pdev->dev);
+
+	if (rkvdec->sram_pool)
+		gen_pool_destroy(rkvdec->sram_pool);
+}
+
+#ifdef CONFIG_PM
+static int rkvdec2_runtime_resume(struct device *dev)
+{
+	struct rkvdec2_dev *rkvdec = dev_get_drvdata(dev);
+
+	return clk_bulk_prepare_enable(ARRAY_SIZE(rkvdec2_clk_names),
+				       rkvdec->clocks);
+}
+
+static int rkvdec2_runtime_suspend(struct device *dev)
+{
+	struct rkvdec2_dev *rkvdec = dev_get_drvdata(dev);
+
+	clk_bulk_disable_unprepare(ARRAY_SIZE(rkvdec2_clk_names),
+				   rkvdec->clocks);
+	return 0;
+}
+#endif
+
+static const struct dev_pm_ops rkvdec2_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+				pm_runtime_force_resume)
+	SET_RUNTIME_PM_OPS(rkvdec2_runtime_suspend, rkvdec2_runtime_resume, NULL)
+};
+
+static struct platform_driver rkvdec2_driver = {
+	.probe = rkvdec2_probe,
+	.remove_new = rkvdec2_remove,
+	.driver = {
+		   .name = "rkvdec2",
+		   .of_match_table = of_rkvdec2_match,
+		   .pm = &rkvdec2_pm_ops,
+	},
+};
+module_platform_driver(rkvdec2_driver);
+
+MODULE_AUTHOR("Detlev Casanova <detlev.casanova@collabora.com>");
+MODULE_DESCRIPTION("Rockchip Video Decoder 2 driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/staging/media/rkvdec2/rkvdec2.h b/drivers/staging/media/rkvdec2/rkvdec2.h
new file mode 100644
index 000000000000..8613051775e9
--- /dev/null
+++ b/drivers/staging/media/rkvdec2/rkvdec2.h
@@ -0,0 +1,130 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Rockchip Video Decoder 2 driver
+ *
+ * Copyright (C) 2024 Collabora, Ltd.
+ *  Detlev Casanova <detlev.casanova@collabora.com>
+ *
+ * Based on rkvdec driver by Boris Brezillon <boris.brezillon@collabora.com>
+ */
+#ifndef RKVDEC_H_
+#define RKVDEC_H_
+
+#include <linux/platform_device.h>
+#include <linux/videodev2.h>
+#include <linux/wait.h>
+#include <linux/clk.h>
+#include <linux/dma-mapping.h>
+
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "rkvdec2-regs.h"
+
+#define RKVDEC2_RCB_COUNT	10
+
+struct rkvdec2_ctx;
+
+enum rkvdec2_alloc_type {
+	RKVDEC2_ALLOC_SRAM,
+	RKVDEC2_ALLOC_DMA,
+};
+
+struct rkvdec2_aux_buf {
+	void *cpu;
+	dma_addr_t dma;
+	size_t size;
+	enum rkvdec2_alloc_type type;
+};
+
+struct rkvdec2_ctrl_desc {
+	struct v4l2_ctrl_config cfg;
+};
+
+struct rkvdec2_ctrls {
+	const struct rkvdec2_ctrl_desc *ctrls;
+	unsigned int num_ctrls;
+};
+
+struct rkvdec2_run {
+	struct {
+		struct vb2_v4l2_buffer *src;
+		struct vb2_v4l2_buffer *dst;
+	} bufs;
+};
+
+struct rkvdec2_decoded_buffer {
+	/* Must be the first field in this struct. */
+	struct v4l2_m2m_buffer base;
+};
+
+static inline struct rkvdec2_decoded_buffer *
+vb2_to_rkvdec2_decoded_buf(struct vb2_buffer *buf)
+{
+	return container_of(buf, struct rkvdec2_decoded_buffer,
+			    base.vb.vb2_buf);
+}
+
+struct rkvdec2_coded_fmt_ops {
+	int (*adjust_fmt)(struct rkvdec2_ctx *ctx,
+			  struct v4l2_format *f);
+	int (*start)(struct rkvdec2_ctx *ctx);
+	void (*stop)(struct rkvdec2_ctx *ctx);
+	int (*run)(struct rkvdec2_ctx *ctx);
+	void (*done)(struct rkvdec2_ctx *ctx, struct vb2_v4l2_buffer *src_buf,
+		     struct vb2_v4l2_buffer *dst_buf,
+		     enum vb2_buffer_state result);
+	int (*try_ctrl)(struct rkvdec2_ctx *ctx, struct v4l2_ctrl *ctrl);
+};
+
+struct rkvdec2_coded_fmt_desc {
+	u32 fourcc;
+	struct v4l2_frmsize_stepwise frmsize;
+	const struct rkvdec2_ctrls *ctrls;
+	const struct rkvdec2_coded_fmt_ops *ops;
+	unsigned int num_decoded_fmts;
+	const u32 *decoded_fmts;
+	u32 subsystem_flags;
+};
+
+struct rkvdec2_dev {
+	struct v4l2_device v4l2_dev;
+	struct media_device mdev;
+	struct video_device vdev;
+	struct v4l2_m2m_dev *m2m_dev;
+	struct device *dev;
+	struct clk_bulk_data *clocks;
+	void __iomem *regs;
+	struct gen_pool *sram_pool;
+	struct mutex vdev_lock; /* serializes ioctls */
+	struct delayed_work watchdog_work;
+};
+
+struct rkvdec2_ctx {
+	struct v4l2_fh fh;
+	struct v4l2_format coded_fmt;
+	struct v4l2_format decoded_fmt;
+	const struct rkvdec2_coded_fmt_desc *coded_fmt_desc;
+	struct v4l2_ctrl_handler ctrl_hdl;
+	struct rkvdec2_dev *dev;
+	struct rkvdec2_aux_buf rcb_bufs[RKVDEC2_RCB_COUNT];
+
+	u32 colmv_offset;
+
+	void *priv;
+};
+
+static inline struct rkvdec2_ctx *fh_to_rkvdec2_ctx(struct v4l2_fh *fh)
+{
+	return container_of(fh, struct rkvdec2_ctx, fh);
+}
+
+void rkvdec2_run_preamble(struct rkvdec2_ctx *ctx, struct rkvdec2_run *run);
+void rkvdec2_run_postamble(struct rkvdec2_ctx *ctx, struct rkvdec2_run *run);
+
+extern const struct rkvdec2_coded_fmt_ops rkvdec2_h264_fmt_ops;
+
+#endif /* RKVDEC_H_ */
-- 
2.44.2

[PATCH v2 3/4] media: dt-bindings: rockchip: Document RK3588 Video Decoder bindings

[Detlev Casanova]

Document the Rockchip RK3588 Video Decoder bindings.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
---
 .../bindings/media/rockchip,vdec.yaml         | 46 +++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/Documentation/devicetree/bindings/media/rockchip,vdec.yaml b/Documentation/devicetree/bindings/media/rockchip,vdec.yaml
index 08b02ec16755..22cb62faaa9b 100644
--- a/Documentation/devicetree/bindings/media/rockchip,vdec.yaml
+++ b/Documentation/devicetree/bindings/media/rockchip,vdec.yaml
@@ -17,6 +17,7 @@ properties:
   compatible:
     oneOf:
       - const: rockchip,rk3399-vdec
+      - const: rockchip,rk3588-vdec
       - items:
           - enum:
               - rockchip,rk3228-vdec
@@ -30,29 +31,56 @@ properties:
     maxItems: 1
 
   clocks:
+    minItems: 4
     items:
       - description: The Video Decoder AXI interface clock
       - description: The Video Decoder AHB interface clock
       - description: The Video Decoded CABAC clock
       - description: The Video Decoder core clock
+      - description: The Video decoder HEVC CABAC clock
 
   clock-names:
+    minItems: 4
     items:
       - const: axi
       - const: ahb
       - const: cabac
       - const: core
+      - const: hevc_cabac
 
   assigned-clocks: true
 
   assigned-clock-rates: true
 
+  resets:
+    items:
+      - description: The Video Decoder AXI interface reset
+      - description: The Video Decoder AHB interface reset
+      - description: The Video Decoded CABAC reset
+      - description: The Video Decoder core reset
+      - description: The Video decoder HEVC CABAC reset
+
+  reset-names:
+    items:
+      - const: rst_axi
+      - const: rst_ahb
+      - const: rst_cabac
+      - const: rst_core
+      - const: rst_hevc_cabac
+
   power-domains:
     maxItems: 1
 
   iommus:
     maxItems: 1
 
+  sram:
+    $ref: /schemas/types.yaml#/definitions/phandle
+    description: |
+      phandle to a reserved on-chip SRAM regions.
+      Some SoCs, like rk3588 provide on-chip SRAM to store temporary
+      buffers during decoding.
+
 required:
   - compatible
   - reg
@@ -61,6 +89,24 @@ required:
   - clock-names
   - power-domains
 
+allOf:
+  - if:
+      properties:
+        compatible:
+          contains:
+            const: rockchip,rk3588-vdec
+    then:
+      properties:
+        clocks:
+          minItems: 5
+        clock-names:
+          minItems: 5
+
+        resets:
+          minItems: 5
+        reset-names:
+          minItems: 5
+
 additionalProperties: false
 
 examples:
-- 
2.44.2

[PATCH v2 4/4] arm64: dts: rockchip: Add rkvdec2 Video Decoder on rk3588(s)

[Detlev Casanova]

Add the rkvdec2 Video Decoder to the RK3588s devicetree.

Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
---
 arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 50 +++++++++++++++++++++++
 1 file changed, 50 insertions(+)

diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
index 6ac5ac8b48ab..7690632f57f1 100644
--- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
@@ -2596,6 +2596,16 @@ system_sram2: sram@ff001000 {
 		ranges = <0x0 0x0 0xff001000 0xef000>;
 		#address-cells = <1>;
 		#size-cells = <1>;
+
+		vdec0_sram: rkvdec-sram@0 {
+			reg = <0x0 0x78000>;
+			pool;
+		};
+
+		vdec1_sram: rkvdec-sram@1 {
+			reg = <0x78000 0x77000>;
+			pool;
+		};
 	};
 
 	pinctrl: pinctrl {
@@ -2665,6 +2675,46 @@ gpio4: gpio@fec50000 {
 			#interrupt-cells = <2>;
 		};
 	};
+
+	vdec0: video-decoder@fdc38100 {
+		compatible = "rockchip,rk3588-vdec";
+		reg = <0x0 0xfdc38100 0x0 0x500>;
+		interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH 0>;
+		clocks = <&cru ACLK_RKVDEC0>, <&cru HCLK_RKVDEC0>, <&cru CLK_RKVDEC0_CA>,
+			 <&cru CLK_RKVDEC0_CORE>, <&cru CLK_RKVDEC0_HEVC_CA>;
+		clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac";
+		assigned-clocks = <&cru ACLK_RKVDEC0>, <&cru CLK_RKVDEC0_CORE>,
+				  <&cru CLK_RKVDEC0_CA>, <&cru CLK_RKVDEC0_HEVC_CA>;
+		assigned-clock-rates = <800000000>, <600000000>,
+				       <600000000>, <1000000000>;
+		resets = <&cru SRST_A_RKVDEC0>, <&cru SRST_H_RKVDEC0>, <&cru SRST_RKVDEC0_CA>,
+			 <&cru SRST_RKVDEC0_CORE>, <&cru SRST_RKVDEC0_HEVC_CA>;
+		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
+			      "rst_core", "rst_hevc_cabac";
+		power-domains = <&power RK3588_PD_RKVDEC0>;
+		sram = <&vdec0_sram>;
+		status = "okay";
+	};
+
+	vdec1: video-decoder@fdc40100 {
+		compatible = "rockchip,rk3588-vdec";
+		reg = <0x0 0xfdc40100 0x0 0x500>;
+		interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH 0>;
+		clocks = <&cru ACLK_RKVDEC1>, <&cru HCLK_RKVDEC1>, <&cru CLK_RKVDEC1_CA>,
+			 <&cru CLK_RKVDEC1_CORE>, <&cru CLK_RKVDEC1_HEVC_CA>;
+		clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac";
+		assigned-clocks = <&cru ACLK_RKVDEC1>, <&cru CLK_RKVDEC1_CORE>,
+				  <&cru CLK_RKVDEC1_CA>, <&cru CLK_RKVDEC1_HEVC_CA>;
+		assigned-clock-rates = <800000000>, <600000000>,
+				       <600000000>, <1000000000>;
+		resets = <&cru SRST_A_RKVDEC1>, <&cru SRST_H_RKVDEC1>, <&cru SRST_RKVDEC1_CA>,
+			 <&cru SRST_RKVDEC1_CORE>, <&cru SRST_RKVDEC1_HEVC_CA>;
+		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
+			      "rst_core", "rst_hevc_cabac";
+		power-domains = <&power RK3588_PD_RKVDEC1>;
+		sram = <&vdec1_sram>;
+		status = "okay";
+	};
 };
 
 #include "rk3588s-pinctrl.dtsi"
-- 
2.44.2

Re: [PATCH v2 4/4] arm64: dts: rockchip: Add rkvdec2 Video Decoder on rk3588(s)

[Jonas Karlman]

Hi Detlev,

On 2024-06-19 16:57, Detlev Casanova wrote:
> Add the rkvdec2 Video Decoder to the RK3588s devicetree.
> 
> Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
> ---
>  arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 50 +++++++++++++++++++++++
>  1 file changed, 50 insertions(+)
> 
> diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> index 6ac5ac8b48ab..7690632f57f1 100644
> --- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> +++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> @@ -2596,6 +2596,16 @@ system_sram2: sram@ff001000 {
>  		ranges = <0x0 0x0 0xff001000 0xef000>;
>  		#address-cells = <1>;
>  		#size-cells = <1>;
> +
> +		vdec0_sram: rkvdec-sram@0 {
> +			reg = <0x0 0x78000>;
> +			pool;
> +		};
> +
> +		vdec1_sram: rkvdec-sram@1 {
> +			reg = <0x78000 0x77000>;
> +			pool;
> +		};
>  	};
>  
>  	pinctrl: pinctrl {
> @@ -2665,6 +2675,46 @@ gpio4: gpio@fec50000 {
>  			#interrupt-cells = <2>;
>  		};
>  	};
> +
> +	vdec0: video-decoder@fdc38100 {
> +		compatible = "rockchip,rk3588-vdec";
> +		reg = <0x0 0xfdc38100 0x0 0x500>;
> +		interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH 0>;
> +		clocks = <&cru ACLK_RKVDEC0>, <&cru HCLK_RKVDEC0>, <&cru CLK_RKVDEC0_CA>,
> +			 <&cru CLK_RKVDEC0_CORE>, <&cru CLK_RKVDEC0_HEVC_CA>;
> +		clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac";
> +		assigned-clocks = <&cru ACLK_RKVDEC0>, <&cru CLK_RKVDEC0_CORE>,
> +				  <&cru CLK_RKVDEC0_CA>, <&cru CLK_RKVDEC0_HEVC_CA>;
> +		assigned-clock-rates = <800000000>, <600000000>,
> +				       <600000000>, <1000000000>;
> +		resets = <&cru SRST_A_RKVDEC0>, <&cru SRST_H_RKVDEC0>, <&cru SRST_RKVDEC0_CA>,
> +			 <&cru SRST_RKVDEC0_CORE>, <&cru SRST_RKVDEC0_HEVC_CA>;
> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
> +			      "rst_core", "rst_hevc_cabac";
> +		power-domains = <&power RK3588_PD_RKVDEC0>;
> +		sram = <&vdec0_sram>;
> +		status = "okay";
> +	};
> +
> +	vdec1: video-decoder@fdc40100 {
> +		compatible = "rockchip,rk3588-vdec";
> +		reg = <0x0 0xfdc40100 0x0 0x500>;
> +		interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH 0>;
> +		clocks = <&cru ACLK_RKVDEC1>, <&cru HCLK_RKVDEC1>, <&cru CLK_RKVDEC1_CA>,
> +			 <&cru CLK_RKVDEC1_CORE>, <&cru CLK_RKVDEC1_HEVC_CA>;
> +		clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac";
> +		assigned-clocks = <&cru ACLK_RKVDEC1>, <&cru CLK_RKVDEC1_CORE>,
> +				  <&cru CLK_RKVDEC1_CA>, <&cru CLK_RKVDEC1_HEVC_CA>;
> +		assigned-clock-rates = <800000000>, <600000000>,
> +				       <600000000>, <1000000000>;
> +		resets = <&cru SRST_A_RKVDEC1>, <&cru SRST_H_RKVDEC1>, <&cru SRST_RKVDEC1_CA>,
> +			 <&cru SRST_RKVDEC1_CORE>, <&cru SRST_RKVDEC1_HEVC_CA>;
> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
> +			      "rst_core", "rst_hevc_cabac";
> +		power-domains = <&power RK3588_PD_RKVDEC1>;
> +		sram = <&vdec1_sram>;
> +		status = "okay";
> +	};

This is still missing the iommus, please add the iommus, they should be
supported/same as the one used for e.g. VOP2:

  compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";

The VOP2 MMUs does have one extra mmu_cfg_mode flag in AUTO_GATING,
compared to the VDPU381 MMUs, however only the AV1D MMU should be
special on RK3588.

Please add the iommus :-)

Regards,
Jonas

>  };
>  
>  #include "rk3588s-pinctrl.dtsi"

Re: [PATCH v2 4/4] arm64: dts: rockchip: Add rkvdec2 Video Decoder on rk3588(s)

[Heiko Stübner]

Am Mittwoch, 19. Juni 2024, 16:57:21 CEST schrieb Detlev Casanova:
> Add the rkvdec2 Video Decoder to the RK3588s devicetree.
> 
> Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
> ---
>  arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 50 +++++++++++++++++++++++
>  1 file changed, 50 insertions(+)
> 
> diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> index 6ac5ac8b48ab..7690632f57f1 100644
> --- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> +++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> @@ -2596,6 +2596,16 @@ system_sram2: sram@ff001000 {
>  		ranges = <0x0 0x0 0xff001000 0xef000>;
>  		#address-cells = <1>;
>  		#size-cells = <1>;
> +
> +		vdec0_sram: rkvdec-sram@0 {
> +			reg = <0x0 0x78000>;
> +			pool;
> +		};
> +
> +		vdec1_sram: rkvdec-sram@1 {
> +			reg = <0x78000 0x77000>;
> +			pool;
> +		};
>  	};
>  
>  	pinctrl: pinctrl {
> @@ -2665,6 +2675,46 @@ gpio4: gpio@fec50000 {
>  			#interrupt-cells = <2>;
>  		};
>  	};
> +
> +	vdec0: video-decoder@fdc38100 {
> +		compatible = "rockchip,rk3588-vdec";
> +		reg = <0x0 0xfdc38100 0x0 0x500>;
> +		interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH 0>;
> +		clocks = <&cru ACLK_RKVDEC0>, <&cru HCLK_RKVDEC0>, <&cru CLK_RKVDEC0_CA>,
> +			 <&cru CLK_RKVDEC0_CORE>, <&cru CLK_RKVDEC0_HEVC_CA>;
> +		clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac";
> +		assigned-clocks = <&cru ACLK_RKVDEC0>, <&cru CLK_RKVDEC0_CORE>,
> +				  <&cru CLK_RKVDEC0_CA>, <&cru CLK_RKVDEC0_HEVC_CA>;
> +		assigned-clock-rates = <800000000>, <600000000>,
> +				       <600000000>, <1000000000>;
> +		resets = <&cru SRST_A_RKVDEC0>, <&cru SRST_H_RKVDEC0>, <&cru SRST_RKVDEC0_CA>,
> +			 <&cru SRST_RKVDEC0_CORE>, <&cru SRST_RKVDEC0_HEVC_CA>;
> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
> +			      "rst_core", "rst_hevc_cabac";
> +		power-domains = <&power RK3588_PD_RKVDEC0>;
> +		sram = <&vdec0_sram>;
> +		status = "okay";

okay is the default status, so is not needed when the node is always-on

> +	};
> +
> +	vdec1: video-decoder@fdc40100 {
> +		compatible = "rockchip,rk3588-vdec";
> +		reg = <0x0 0xfdc40100 0x0 0x500>;
> +		interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH 0>;
> +		clocks = <&cru ACLK_RKVDEC1>, <&cru HCLK_RKVDEC1>, <&cru CLK_RKVDEC1_CA>,
> +			 <&cru CLK_RKVDEC1_CORE>, <&cru CLK_RKVDEC1_HEVC_CA>;
> +		clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac";
> +		assigned-clocks = <&cru ACLK_RKVDEC1>, <&cru CLK_RKVDEC1_CORE>,
> +				  <&cru CLK_RKVDEC1_CA>, <&cru CLK_RKVDEC1_HEVC_CA>;
> +		assigned-clock-rates = <800000000>, <600000000>,
> +				       <600000000>, <1000000000>;
> +		resets = <&cru SRST_A_RKVDEC1>, <&cru SRST_H_RKVDEC1>, <&cru SRST_RKVDEC1_CA>,
> +			 <&cru SRST_RKVDEC1_CORE>, <&cru SRST_RKVDEC1_HEVC_CA>;
> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
> +			      "rst_core", "rst_hevc_cabac";
> +		power-domains = <&power RK3588_PD_RKVDEC1>;
> +		sram = <&vdec1_sram>;
> +		status = "okay";

same

> +	};
>  };
>  
>  #include "rk3588s-pinctrl.dtsi"
> 

Re: [PATCH v2 4/4] arm64: dts: rockchip: Add rkvdec2 Video Decoder on rk3588(s)

[Alex Bee]

Am 19.06.24 um 17:28 schrieb Jonas Karlman:
> Hi Detlev,
>
> On 2024-06-19 16:57, Detlev Casanova wrote:
>> Add the rkvdec2 Video Decoder to the RK3588s devicetree.
>>
>> Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
>> ---
>>   arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 50 +++++++++++++++++++++++
>>   1 file changed, 50 insertions(+)
>>
>> diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
>> index 6ac5ac8b48ab..7690632f57f1 100644
>> --- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
>> +++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
>> @@ -2596,6 +2596,16 @@ system_sram2: sram@ff001000 {
>>   		ranges = <0x0 0x0 0xff001000 0xef000>;
>>   		#address-cells = <1>;
>>   		#size-cells = <1>;
>> +
>> +		vdec0_sram: rkvdec-sram@0 {
>> +			reg = <0x0 0x78000>;
>> +			pool;
>> +		};
>> +
>> +		vdec1_sram: rkvdec-sram@1 {
>> +			reg = <0x78000 0x77000>;
>> +			pool;
>> +		};
>>   	};
>>   
>>   	pinctrl: pinctrl {
>> @@ -2665,6 +2675,46 @@ gpio4: gpio@fec50000 {
>>   			#interrupt-cells = <2>;
>>   		};
>>   	};
>> +
>> +	vdec0: video-decoder@fdc38100 {
>> +		compatible = "rockchip,rk3588-vdec";
>> +		reg = <0x0 0xfdc38100 0x0 0x500>;
>> +		interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH 0>;
>> +		clocks = <&cru ACLK_RKVDEC0>, <&cru HCLK_RKVDEC0>, <&cru CLK_RKVDEC0_CA>,
>> +			 <&cru CLK_RKVDEC0_CORE>, <&cru CLK_RKVDEC0_HEVC_CA>;
>> +		clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac";
>> +		assigned-clocks = <&cru ACLK_RKVDEC0>, <&cru CLK_RKVDEC0_CORE>,
>> +				  <&cru CLK_RKVDEC0_CA>, <&cru CLK_RKVDEC0_HEVC_CA>;
>> +		assigned-clock-rates = <800000000>, <600000000>,
>> +				       <600000000>, <1000000000>;
>> +		resets = <&cru SRST_A_RKVDEC0>, <&cru SRST_H_RKVDEC0>, <&cru SRST_RKVDEC0_CA>,
>> +			 <&cru SRST_RKVDEC0_CORE>, <&cru SRST_RKVDEC0_HEVC_CA>;
>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
>> +			      "rst_core", "rst_hevc_cabac";
>> +		power-domains = <&power RK3588_PD_RKVDEC0>;
>> +		sram = <&vdec0_sram>;
>> +		status = "okay";
>> +	};
>> +
>> +	vdec1: video-decoder@fdc40100 {
>> +		compatible = "rockchip,rk3588-vdec";
>> +		reg = <0x0 0xfdc40100 0x0 0x500>;
>> +		interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH 0>;
>> +		clocks = <&cru ACLK_RKVDEC1>, <&cru HCLK_RKVDEC1>, <&cru CLK_RKVDEC1_CA>,
>> +			 <&cru CLK_RKVDEC1_CORE>, <&cru CLK_RKVDEC1_HEVC_CA>;
>> +		clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac";
>> +		assigned-clocks = <&cru ACLK_RKVDEC1>, <&cru CLK_RKVDEC1_CORE>,
>> +				  <&cru CLK_RKVDEC1_CA>, <&cru CLK_RKVDEC1_HEVC_CA>;
>> +		assigned-clock-rates = <800000000>, <600000000>,
>> +				       <600000000>, <1000000000>;
>> +		resets = <&cru SRST_A_RKVDEC1>, <&cru SRST_H_RKVDEC1>, <&cru SRST_RKVDEC1_CA>,
>> +			 <&cru SRST_RKVDEC1_CORE>, <&cru SRST_RKVDEC1_HEVC_CA>;
>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
>> +			      "rst_core", "rst_hevc_cabac";
>> +		power-domains = <&power RK3588_PD_RKVDEC1>;
>> +		sram = <&vdec1_sram>;
>> +		status = "okay";
>> +	};
> This is still missing the iommus, please add the iommus, they should be
> supported/same as the one used for e.g. VOP2:
>
>    compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
>
> The VOP2 MMUs does have one extra mmu_cfg_mode flag in AUTO_GATING,
> compared to the VDPU381 MMUs, however only the AV1D MMU should be
> special on RK3588.
>
> Please add the iommus :-)
When looking add the vendor DT/iommu driver I'm seeing serval quirks
applied for vdec's iommus. Since it's rightly frowned upon adding such
boolean-quirk-properties to upstream devicetrees, we'd at least need
additional (fallback-) compatibles, even if it works with the iommu driver
as is (what I doubt, but haven't tested). We need to be able to apply those
quirks later without changing the devicetree (as usual) and I'm sure RK
devs haven't added these quirks for the personal amusement. If Detlev says
iommu is out of scope for this series (which is valid), I'd say it's fine
to leave them out for now (as no binding exists) and the HW works
(obviously) fine without them.

> Regards,
> Jonas
>
>>   };
>>   
>>   #include "rk3588s-pinctrl.dtsi"

Re: [PATCH v2 3/4] media: dt-bindings: rockchip: Document RK3588 Video Decoder bindings

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 3007 bytes --]

On Wed, Jun 19, 2024 at 10:57:20AM -0400, Detlev Casanova wrote:
> Document the Rockchip RK3588 Video Decoder bindings.
> 
> Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
> ---
>  .../bindings/media/rockchip,vdec.yaml         | 46 +++++++++++++++++++
>  1 file changed, 46 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/media/rockchip,vdec.yaml b/Documentation/devicetree/bindings/media/rockchip,vdec.yaml
> index 08b02ec16755..22cb62faaa9b 100644
> --- a/Documentation/devicetree/bindings/media/rockchip,vdec.yaml
> +++ b/Documentation/devicetree/bindings/media/rockchip,vdec.yaml
> @@ -17,6 +17,7 @@ properties:
>    compatible:
>      oneOf:
>        - const: rockchip,rk3399-vdec
> +      - const: rockchip,rk3588-vdec
>        - items:
>            - enum:
>                - rockchip,rk3228-vdec
> @@ -30,29 +31,56 @@ properties:
>      maxItems: 1
>  
>    clocks:
> +    minItems: 4
>      items:
>        - description: The Video Decoder AXI interface clock
>        - description: The Video Decoder AHB interface clock
>        - description: The Video Decoded CABAC clock
>        - description: The Video Decoder core clock
> +      - description: The Video decoder HEVC CABAC clock
>  
>    clock-names:
> +    minItems: 4
>      items:
>        - const: axi
>        - const: ahb
>        - const: cabac
>        - const: core
> +      - const: hevc_cabac
>  
>    assigned-clocks: true
>  
>    assigned-clock-rates: true
>  
> +  resets:
> +    items:
> +      - description: The Video Decoder AXI interface reset
> +      - description: The Video Decoder AHB interface reset
> +      - description: The Video Decoded CABAC reset
> +      - description: The Video Decoder core reset
> +      - description: The Video decoder HEVC CABAC reset
> +
> +  reset-names:
> +    items:
> +      - const: rst_axi
> +      - const: rst_ahb
> +      - const: rst_cabac
> +      - const: rst_core
> +      - const: rst_hevc_cabac
> +
>    power-domains:
>      maxItems: 1
>  
>    iommus:
>      maxItems: 1
>  
> +  sram:
> +    $ref: /schemas/types.yaml#/definitions/phandle
> +    description: |
> +      phandle to a reserved on-chip SRAM regions.
> +      Some SoCs, like rk3588 provide on-chip SRAM to store temporary
> +      buffers during decoding.
> +
>  required:
>    - compatible
>    - reg
> @@ -61,6 +89,24 @@ required:
>    - clock-names
>    - power-domains
>  
> +allOf:
> +  - if:
> +      properties:
> +        compatible:
> +          contains:
> +            const: rockchip,rk3588-vdec
> +    then:
> +      properties:
> +        clocks:
> +          minItems: 5
> +        clock-names:
> +          minItems: 5
> +
> +        resets:
> +          minItems: 5
> +        reset-names:
> +          minItems: 5

You need an else clause here to restrict sram, resets on the other
platforms and to cap clocks at maxItems: 4.

Thanks,
Conor.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v2 2/4] media: rockchip: Introduce the rkvdec2 driver

[Jianfeng Liu]

Hi Detlev,

On Wed, 19 Jun 2024 10:57:19 -0400, Detlev Casanova wrote:
>+	if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
>+		height *= 2;
>+
>+	if (width > ctx->coded_fmt.fmt.pix_mp.width ||
>+	    height > ctx->coded_fmt.fmt.pix_mp.height)
>+		return -EINVAL;

I did further invesatigation on chromium. I find that before real video
is decoded, chromium will call VIDIOC_STREAMON twice with value of
sps->flags 0:

At the first time width and height are 16; ctx->coded_fmt.fmt.pix_mp.width
and coded_fmt.fmt.pix_mp.height are 16, which are the min size of decoder;
At the second time width and height are still 16; while
coded_fmt.fmt.pix_mp.width is 1920 and coded_fmt.fmt.pix_mp.height is
1088, which are the real size of video.

So VIDIOC_STREAMON will fall at the first time call because sps->flags is
0 so V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set, and then height is
doubled to 32 which is larger than 16.

What do you think if we skip doubling height if sps->flags is 0 and at the
same time V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set? The following hack
did fix my chromium:

--- a/drivers/staging/media/rkvdec2/rkvdec2-h264.c
+++ b/drivers/staging/media/rkvdec2/rkvdec2-h264.c
@@ -767,7 +767,7 @@ static int rkvdec2_h264_validate_sps(struct rkvdec2_ctx *ctx,
         * which is half the final height (see (7-18) in the
         * specification)
         */
-       if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
+       if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY) && sps->flags)
                height *= 2;
 
        if (width > ctx->coded_fmt.fmt.pix_mp.width ||

Best regards
Jianfeng

Re: [PATCH v2 4/4] arm64: dts: rockchip: Add rkvdec2 Video Decoder on rk3588(s)

[Jonas Karlman]

Hi Alex,

On 2024-06-19 19:19, Alex Bee wrote:
> 
> Am 19.06.24 um 17:28 schrieb Jonas Karlman:
>> Hi Detlev,
>>
>> On 2024-06-19 16:57, Detlev Casanova wrote:
>>> Add the rkvdec2 Video Decoder to the RK3588s devicetree.
>>>
>>> Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
>>> ---
>>>   arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 50 +++++++++++++++++++++++
>>>   1 file changed, 50 insertions(+)
>>>
>>> diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
>>> index 6ac5ac8b48ab..7690632f57f1 100644
>>> --- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
>>> +++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
>>> @@ -2596,6 +2596,16 @@ system_sram2: sram@ff001000 {
>>>   		ranges = <0x0 0x0 0xff001000 0xef000>;
>>>   		#address-cells = <1>;
>>>   		#size-cells = <1>;
>>> +
>>> +		vdec0_sram: rkvdec-sram@0 {
>>> +			reg = <0x0 0x78000>;
>>> +			pool;
>>> +		};
>>> +
>>> +		vdec1_sram: rkvdec-sram@1 {
>>> +			reg = <0x78000 0x77000>;
>>> +			pool;
>>> +		};
>>>   	};
>>>   
>>>   	pinctrl: pinctrl {
>>> @@ -2665,6 +2675,46 @@ gpio4: gpio@fec50000 {
>>>   			#interrupt-cells = <2>;
>>>   		};
>>>   	};
>>> +
>>> +	vdec0: video-decoder@fdc38100 {
>>> +		compatible = "rockchip,rk3588-vdec";
>>> +		reg = <0x0 0xfdc38100 0x0 0x500>;
>>> +		interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH 0>;
>>> +		clocks = <&cru ACLK_RKVDEC0>, <&cru HCLK_RKVDEC0>, <&cru CLK_RKVDEC0_CA>,
>>> +			 <&cru CLK_RKVDEC0_CORE>, <&cru CLK_RKVDEC0_HEVC_CA>;
>>> +		clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac";
>>> +		assigned-clocks = <&cru ACLK_RKVDEC0>, <&cru CLK_RKVDEC0_CORE>,
>>> +				  <&cru CLK_RKVDEC0_CA>, <&cru CLK_RKVDEC0_HEVC_CA>;
>>> +		assigned-clock-rates = <800000000>, <600000000>,
>>> +				       <600000000>, <1000000000>;
>>> +		resets = <&cru SRST_A_RKVDEC0>, <&cru SRST_H_RKVDEC0>, <&cru SRST_RKVDEC0_CA>,
>>> +			 <&cru SRST_RKVDEC0_CORE>, <&cru SRST_RKVDEC0_HEVC_CA>;
>>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
>>> +			      "rst_core", "rst_hevc_cabac";
>>> +		power-domains = <&power RK3588_PD_RKVDEC0>;
>>> +		sram = <&vdec0_sram>;
>>> +		status = "okay";
>>> +	};
>>> +
>>> +	vdec1: video-decoder@fdc40100 {
>>> +		compatible = "rockchip,rk3588-vdec";
>>> +		reg = <0x0 0xfdc40100 0x0 0x500>;
>>> +		interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH 0>;
>>> +		clocks = <&cru ACLK_RKVDEC1>, <&cru HCLK_RKVDEC1>, <&cru CLK_RKVDEC1_CA>,
>>> +			 <&cru CLK_RKVDEC1_CORE>, <&cru CLK_RKVDEC1_HEVC_CA>;
>>> +		clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac";
>>> +		assigned-clocks = <&cru ACLK_RKVDEC1>, <&cru CLK_RKVDEC1_CORE>,
>>> +				  <&cru CLK_RKVDEC1_CA>, <&cru CLK_RKVDEC1_HEVC_CA>;
>>> +		assigned-clock-rates = <800000000>, <600000000>,
>>> +				       <600000000>, <1000000000>;
>>> +		resets = <&cru SRST_A_RKVDEC1>, <&cru SRST_H_RKVDEC1>, <&cru SRST_RKVDEC1_CA>,
>>> +			 <&cru SRST_RKVDEC1_CORE>, <&cru SRST_RKVDEC1_HEVC_CA>;
>>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
>>> +			      "rst_core", "rst_hevc_cabac";
>>> +		power-domains = <&power RK3588_PD_RKVDEC1>;
>>> +		sram = <&vdec1_sram>;
>>> +		status = "okay";
>>> +	};
>> This is still missing the iommus, please add the iommus, they should be
>> supported/same as the one used for e.g. VOP2:
>>
>>    compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
>>
>> The VOP2 MMUs does have one extra mmu_cfg_mode flag in AUTO_GATING,
>> compared to the VDPU381 MMUs, however only the AV1D MMU should be
>> special on RK3588.
>>
>> Please add the iommus :-)
> When looking add the vendor DT/iommu driver I'm seeing serval quirks
> applied for vdec's iommus. Since it's rightly frowned upon adding such
> boolean-quirk-properties to upstream devicetrees, we'd at least need
> additional (fallback-) compatibles, even if it works with the iommu driver
> as is (what I doubt, but haven't tested). We need to be able to apply those
> quirks later without changing the devicetree (as usual) and I'm sure RK
> devs haven't added these quirks for the personal amusement.

Based on what I investigated the hw should work similar, and the quirks
mostly seem related to optimizations and sw quirks, like do not zap each
line, keep it alive even when pm runtime say it is not in use and other
quirks that seem to be more of sw nature on how to best utilize the hw.

> If Detlev says
> iommu is out of scope for this series (which is valid), I'd say it's fine
> to leave them out for now (as no binding exists) and the HW works
> (obviously) fine without them.

Sure, use of MMU can be added later.

Regards,
Jonas

> 
>> Regards,
>> Jonas
>>
>>>   };
>>>   
>>>   #include "rk3588s-pinctrl.dtsi"

Re: [PATCH v2 4/4] arm64: dts: rockchip: Add rkvdec2 Video Decoder on rk3588(s)

[Krzysztof Kozlowski]

On 19/06/2024 16:57, Detlev Casanova wrote:
> Add the rkvdec2 Video Decoder to the RK3588s devicetree.
> 
> Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
> ---
>  arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 50 +++++++++++++++++++++++
>  1 file changed, 50 insertions(+)
> 
> diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> index 6ac5ac8b48ab..7690632f57f1 100644
> --- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> +++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> @@ -2596,6 +2596,16 @@ system_sram2: sram@ff001000 {
>  		ranges = <0x0 0x0 0xff001000 0xef000>;
>  		#address-cells = <1>;
>  		#size-cells = <1>;
> +
> +		vdec0_sram: rkvdec-sram@0 {
> +			reg = <0x0 0x78000>;
> +			pool;
> +		};
> +
> +		vdec1_sram: rkvdec-sram@1 {
> +			reg = <0x78000 0x77000>;
> +			pool;
> +		};
>  	};
>  
>  	pinctrl: pinctrl {
> @@ -2665,6 +2675,46 @@ gpio4: gpio@fec50000 {
>  			#interrupt-cells = <2>;
>  		};
>  	};
> +
> +	vdec0: video-decoder@fdc38100 {
> +		compatible = "rockchip,rk3588-vdec";
> +		reg = <0x0 0xfdc38100 0x0 0x500>;
> +		interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH 0>;
> +		clocks = <&cru ACLK_RKVDEC0>, <&cru HCLK_RKVDEC0>, <&cru CLK_RKVDEC0_CA>,
> +			 <&cru CLK_RKVDEC0_CORE>, <&cru CLK_RKVDEC0_HEVC_CA>;
> +		clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac";
> +		assigned-clocks = <&cru ACLK_RKVDEC0>, <&cru CLK_RKVDEC0_CORE>,
> +				  <&cru CLK_RKVDEC0_CA>, <&cru CLK_RKVDEC0_HEVC_CA>;
> +		assigned-clock-rates = <800000000>, <600000000>,
> +				       <600000000>, <1000000000>;
> +		resets = <&cru SRST_A_RKVDEC0>, <&cru SRST_H_RKVDEC0>, <&cru SRST_RKVDEC0_CA>,
> +			 <&cru SRST_RKVDEC0_CORE>, <&cru SRST_RKVDEC0_HEVC_CA>;
> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
> +			      "rst_core", "rst_hevc_cabac";
> +		power-domains = <&power RK3588_PD_RKVDEC0>;
> +		sram = <&vdec0_sram>;
> +		status = "okay";

Should not be needed. Where did you disable it?

> +	};
> +
> +	vdec1: video-decoder@fdc40100 {
> +		compatible = "rockchip,rk3588-vdec";
> +		reg = <0x0 0xfdc40100 0x0 0x500>;
> +		interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH 0>;
> +		clocks = <&cru ACLK_RKVDEC1>, <&cru HCLK_RKVDEC1>, <&cru CLK_RKVDEC1_CA>,
> +			 <&cru CLK_RKVDEC1_CORE>, <&cru CLK_RKVDEC1_HEVC_CA>;
> +		clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac";
> +		assigned-clocks = <&cru ACLK_RKVDEC1>, <&cru CLK_RKVDEC1_CORE>,
> +				  <&cru CLK_RKVDEC1_CA>, <&cru CLK_RKVDEC1_HEVC_CA>;
> +		assigned-clock-rates = <800000000>, <600000000>,
> +				       <600000000>, <1000000000>;
> +		resets = <&cru SRST_A_RKVDEC1>, <&cru SRST_H_RKVDEC1>, <&cru SRST_RKVDEC1_CA>,
> +			 <&cru SRST_RKVDEC1_CORE>, <&cru SRST_RKVDEC1_HEVC_CA>;
> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
> +			      "rst_core", "rst_hevc_cabac";
> +		power-domains = <&power RK3588_PD_RKVDEC1>;
> +		sram = <&vdec1_sram>;
> +		status = "okay";

Should not be needed. Where did you disable it?



Best regards,
Krzysztof

Re: [PATCH v2 2/4] media: rockchip: Introduce the rkvdec2 driver

[Krzysztof Kozlowski]

On 19/06/2024 16:57, Detlev Casanova wrote:
> +static const char * const rkvdec2_clk_names[] = {
> +	"axi",
> +	"ahb",
> +	"core",
> +	"cabac",
> +	"hevc_cabac",
> +};
> +
> +/*
> + * Some SoCs, like RK3588 have multiple identical vdpu34x cores, but the
> + * kernel is currently missing support for multi-core handling. Exposing
> + * separate devices for each core to userspace is bad, since that does
> + * not allow scheduling tasks properly (and creates ABI). With this workaround
> + * the driver will only probe for the first core and early exit for the other
> + * cores. Once the driver gains multi-core support, the same technique
> + * for detecting the main core can be used to cluster all cores together.
> + */
> +static int rkvdec2_disable_multicore(struct rkvdec2_dev *rkvdec)
> +{
> +	const char *compatible;
> +	struct device_node *node;
> +	int ret;
> +
> +	/* Intentionally ignores the fallback strings */
> +	ret = of_property_read_string(rkvdec->dev->of_node, "compatible", &compatible);
> +	if (ret)
> +		return ret;
> +
> +	/* first compatible node found from the root node is considered the main core */

So you rely on order of nodes? Before you claim "identical cores", but
now "main core" suggests one is different than others.

...

Best regards,
Krzysztof

Re: [PATCH v2 4/4] arm64: dts: rockchip: Add rkvdec2 Video Decoder on rk3588(s)

[Detlev Casanova]

[-- Attachment #1: Type: text/plain, Size: 5378 bytes --]

Hi Jonas, Alex,

On Wednesday, June 19, 2024 2:06:40 P.M. EDT Jonas Karlman wrote:
> Hi Alex,
> 
> On 2024-06-19 19:19, Alex Bee wrote:
> > Am 19.06.24 um 17:28 schrieb Jonas Karlman:
> >> Hi Detlev,
> >> 
> >> On 2024-06-19 16:57, Detlev Casanova wrote:
> >>> Add the rkvdec2 Video Decoder to the RK3588s devicetree.
> >>> 
> >>> Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
> >>> ---
> >>> 
> >>>   arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 50 +++++++++++++++++++++++
> >>>   1 file changed, 50 insertions(+)
> >>> 
> >>> diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> >>> b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi index
> >>> 6ac5ac8b48ab..7690632f57f1 100644
> >>> --- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> >>> +++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> >>> @@ -2596,6 +2596,16 @@ system_sram2: sram@ff001000 {
> >>> 
> >>>   		ranges = <0x0 0x0 0xff001000 0xef000>;
> >>>   		#address-cells = <1>;
> >>>   		#size-cells = <1>;
> >>> 
> >>> +
> >>> +		vdec0_sram: rkvdec-sram@0 {
> >>> +			reg = <0x0 0x78000>;
> >>> +			pool;
> >>> +		};
> >>> +
> >>> +		vdec1_sram: rkvdec-sram@1 {
> >>> +			reg = <0x78000 0x77000>;
> >>> +			pool;
> >>> +		};
> >>> 
> >>>   	};
> >>>   	
> >>>   	pinctrl: pinctrl {
> >>> 
> >>> @@ -2665,6 +2675,46 @@ gpio4: gpio@fec50000 {
> >>> 
> >>>   			#interrupt-cells = <2>;
> >>>   		
> >>>   		};
> >>>   	
> >>>   	};
> >>> 
> >>> +
> >>> +	vdec0: video-decoder@fdc38100 {
> >>> +		compatible = "rockchip,rk3588-vdec";
> >>> +		reg = <0x0 0xfdc38100 0x0 0x500>;
> >>> +		interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH 0>;
> >>> +		clocks = <&cru ACLK_RKVDEC0>, <&cru HCLK_RKVDEC0>, 
<&cru
> >>> CLK_RKVDEC0_CA>, +			 <&cru 
CLK_RKVDEC0_CORE>, <&cru
> >>> CLK_RKVDEC0_HEVC_CA>;
> >>> +		clock-names = "axi", "ahb", "cabac", "core", 
"hevc_cabac";
> >>> +		assigned-clocks = <&cru ACLK_RKVDEC0>, <&cru 
CLK_RKVDEC0_CORE>,
> >>> +				  <&cru CLK_RKVDEC0_CA>, <&cru 
CLK_RKVDEC0_HEVC_CA>;
> >>> +		assigned-clock-rates = <800000000>, <600000000>,
> >>> +				       <600000000>, <1000000000>;
> >>> +		resets = <&cru SRST_A_RKVDEC0>, <&cru SRST_H_RKVDEC0>, 
<&cru
> >>> SRST_RKVDEC0_CA>, +			 <&cru 
SRST_RKVDEC0_CORE>, <&cru
> >>> SRST_RKVDEC0_HEVC_CA>;
> >>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
> >>> +			      "rst_core", "rst_hevc_cabac";
> >>> +		power-domains = <&power RK3588_PD_RKVDEC0>;
> >>> +		sram = <&vdec0_sram>;
> >>> +		status = "okay";
> >>> +	};
> >>> +
> >>> +	vdec1: video-decoder@fdc40100 {
> >>> +		compatible = "rockchip,rk3588-vdec";
> >>> +		reg = <0x0 0xfdc40100 0x0 0x500>;
> >>> +		interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH 0>;
> >>> +		clocks = <&cru ACLK_RKVDEC1>, <&cru HCLK_RKVDEC1>, 
<&cru
> >>> CLK_RKVDEC1_CA>, +			 <&cru 
CLK_RKVDEC1_CORE>, <&cru
> >>> CLK_RKVDEC1_HEVC_CA>;
> >>> +		clock-names = "axi", "ahb", "cabac", "core", 
"hevc_cabac";
> >>> +		assigned-clocks = <&cru ACLK_RKVDEC1>, <&cru 
CLK_RKVDEC1_CORE>,
> >>> +				  <&cru CLK_RKVDEC1_CA>, <&cru 
CLK_RKVDEC1_HEVC_CA>;
> >>> +		assigned-clock-rates = <800000000>, <600000000>,
> >>> +				       <600000000>, <1000000000>;
> >>> +		resets = <&cru SRST_A_RKVDEC1>, <&cru SRST_H_RKVDEC1>, 
<&cru
> >>> SRST_RKVDEC1_CA>, +			 <&cru 
SRST_RKVDEC1_CORE>, <&cru
> >>> SRST_RKVDEC1_HEVC_CA>;
> >>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
> >>> +			      "rst_core", "rst_hevc_cabac";
> >>> +		power-domains = <&power RK3588_PD_RKVDEC1>;
> >>> +		sram = <&vdec1_sram>;
> >>> +		status = "okay";
> >>> +	};
> >> 
> >> This is still missing the iommus, please add the iommus, they should be
> >> 
> >> supported/same as the one used for e.g. VOP2:
> >>    compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> >> 
> >> The VOP2 MMUs does have one extra mmu_cfg_mode flag in AUTO_GATING,
> >> compared to the VDPU381 MMUs, however only the AV1D MMU should be
> >> special on RK3588.
> >> 
> >> Please add the iommus :-)
> > 
> > When looking add the vendor DT/iommu driver I'm seeing serval quirks
> > applied for vdec's iommus. Since it's rightly frowned upon adding such
> > boolean-quirk-properties to upstream devicetrees, we'd at least need
> > additional (fallback-) compatibles, even if it works with the iommu driver
> > as is (what I doubt, but haven't tested). We need to be able to apply
> > those
> > quirks later without changing the devicetree (as usual) and I'm sure RK
> > devs haven't added these quirks for the personal amusement.
> 
> Based on what I investigated the hw should work similar, and the quirks
> mostly seem related to optimizations and sw quirks, like do not zap each
> line, keep it alive even when pm runtime say it is not in use and other
> quirks that seem to be more of sw nature on how to best utilize the hw.

I did some testing with the IOMMU but unfortunately, I'm only getting page 
fault errors. This may be something I'm doing wrong, but it clearly needs more 
investigation.

> > If Detlev says
> > iommu is out of scope for this series (which is valid), I'd say it's fine
> > to leave them out for now (as no binding exists) and the HW works
> > (obviously) fine without them.
> 
> Sure, use of MMU can be added later.

I'd rather go for that for now. I'll add that IMMU support is missing in the 
TODO file.

> Regards,
> Jonas
> 
> >> Regards,
> >> Jonas
> >> 
> >>>   };
> >>>   
> >>>   #include "rk3588s-pinctrl.dtsi"


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 2/4] media: rockchip: Introduce the rkvdec2 driver

[Sebastian Reichel]

[-- Attachment #1: Type: text/plain, Size: 2020 bytes --]

Hi,

On Thu, Jun 20, 2024 at 12:30:00PM GMT, Krzysztof Kozlowski wrote:
> On 19/06/2024 16:57, Detlev Casanova wrote:
> > +static const char * const rkvdec2_clk_names[] = {
> > +	"axi",
> > +	"ahb",
> > +	"core",
> > +	"cabac",
> > +	"hevc_cabac",
> > +};
> > +
> > +/*
> > + * Some SoCs, like RK3588 have multiple identical vdpu34x cores, but the
> > + * kernel is currently missing support for multi-core handling. Exposing
> > + * separate devices for each core to userspace is bad, since that does
> > + * not allow scheduling tasks properly (and creates ABI). With this workaround
> > + * the driver will only probe for the first core and early exit for the other
> > + * cores. Once the driver gains multi-core support, the same technique
> > + * for detecting the main core can be used to cluster all cores together.
> > + */
> > +static int rkvdec2_disable_multicore(struct rkvdec2_dev *rkvdec)
> > +{
> > +	const char *compatible;
> > +	struct device_node *node;
> > +	int ret;
> > +
> > +	/* Intentionally ignores the fallback strings */
> > +	ret = of_property_read_string(rkvdec->dev->of_node, "compatible", &compatible);
> > +	if (ret)
> > +		return ret;
> > +
> > +	/* first compatible node found from the root node is considered the main core */
> 
> So you rely on order of nodes? Before you claim "identical cores", but
> now "main core" suggests one is different than others.

Heh, I wrote that comment for Hantro. By main core I was referencing
the software side of things. With a number of equal cores and no DT
node describing a cluster (from HW point of view it's just equal
cores), they somehow need to be combined into a single entity to allow
scheduling work between them. This solves the issue by making one of
the devices the "main" device. From the HW point it's exactly the same
as the others. The function could also use the last core or the second
one. It does not matter as long as there is only one "main" core.

Greetings,

-- Sebastian

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 2/4] media: rockchip: Introduce the rkvdec2 driver

[Detlev Casanova]

[-- Attachment #1: Type: text/plain, Size: 2164 bytes --]

Hi Jianfeng,

On Wednesday, June 19, 2024 1:46:23 P.M. EDT Jianfeng Liu wrote:
> Hi Detlev,
> 
> On Wed, 19 Jun 2024 10:57:19 -0400, Detlev Casanova wrote:
> >+	if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
> >+		height *= 2;
> >+
> >+	if (width > ctx->coded_fmt.fmt.pix_mp.width ||
> >+	    height > ctx->coded_fmt.fmt.pix_mp.height)
> >+		return -EINVAL;
> 
> I did further invesatigation on chromium. I find that before real video
> is decoded, chromium will call VIDIOC_STREAMON twice with value of
> sps->flags 0:
> 
> At the first time width and height are 16; ctx->coded_fmt.fmt.pix_mp.width
> and coded_fmt.fmt.pix_mp.height are 16, which are the min size of decoder;
> At the second time width and height are still 16; while
> coded_fmt.fmt.pix_mp.width is 1920 and coded_fmt.fmt.pix_mp.height is
> 1088, which are the real size of video.
> 
> So VIDIOC_STREAMON will fall at the first time call because sps->flags is
> 0 so V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set, and then height is
> doubled to 32 which is larger than 16.
> 
> What do you think if we skip doubling height if sps->flags is 0 and at the
> same time V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set? The following hack
> did fix my chromium:
> 
> --- a/drivers/staging/media/rkvdec2/rkvdec2-h264.c
> +++ b/drivers/staging/media/rkvdec2/rkvdec2-h264.c
> @@ -767,7 +767,7 @@ static int rkvdec2_h264_validate_sps(struct rkvdec2_ctx
> *ctx, * which is half the final height (see (7-18) in the
>          * specification)
>          */
> -       if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
> +       if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY) && sps->flags)
> height *= 2;
> 
>         if (width > ctx->coded_fmt.fmt.pix_mp.width ||

I'm not sure what Chromium is trying to do here. But the spec is clear that 
height should be multiplied by 2 (That's actually 7-8, not 7-18):

  FrameHeightInMbs = ( 2 – frame_mbs_only_flag ) * PicHeightInMapUnits

This feels like hacking the driver to please a specific userspace application, 
so I'd like to understand better what chromium is doing.

Regards,
Detlev.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 2/4] media: rockchip: Introduce the rkvdec2 driver

[Nicolas Dufresne]

Hi Jianfeng,

Le jeudi 20 juin 2024 à 01:46 +0800, Jianfeng Liu a écrit :
> Hi Detlev,
> 
> On Wed, 19 Jun 2024 10:57:19 -0400, Detlev Casanova wrote:
> > +	if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
> > +		height *= 2;
> > +
> > +	if (width > ctx->coded_fmt.fmt.pix_mp.width ||
> > +	    height > ctx->coded_fmt.fmt.pix_mp.height)
> > +		return -EINVAL;
> 
> I did further invesatigation on chromium. I find that before real video
> is decoded, chromium will call VIDIOC_STREAMON twice with value of
> sps->flags 0:
> 
> At the first time width and height are 16; ctx->coded_fmt.fmt.pix_mp.width
> and coded_fmt.fmt.pix_mp.height are 16, which are the min size of decoder;

This falls short of a specification to support the uninitialized usage by
Chromium. That being said, we do make an effort to try and have a valid default
SPS control and OUTPUT format combination. So my suggestion would be to set
V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY in the default compound control init. This
way, 0x0 get translate to 16x16 instead of 16x32, thus will work with more
drivers.

Chromium these days is not being tested on anything else then MTK, which has a
64x64 minimum size, this is why they never get into this issue. This driver
validation is entirely correct. Removing in some cases is unsafe, it would need
to be replaced with STREAMON only validation of the CAPTURE sizes (which
currently is validate by implied propagation of valid SPS/OUTPUT).

**not even compiled tested, just to illustrate**

diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-
core/v4l2-ctrls-core.c
index c4d995f32191..a55e165ea9c3 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -111,6 +111,7 @@ static void std_init_compound(const struct v4l2_ctrl *ctrl,
u32 idx,
        struct v4l2_ctrl_vp9_frame *p_vp9_frame;
        struct v4l2_ctrl_fwht_params *p_fwht_params;
        struct v4l2_ctrl_h264_scaling_matrix *p_h264_scaling_matrix;
+       struct v4l2_ctrl_h264_sps *p_h264_sps;
        struct v4l2_ctrl_av1_sequence *p_av1_sequence;
        void *p = ptr.p + idx * ctrl->elem_size;
 
@@ -179,6 +180,18 @@ static void std_init_compound(const struct v4l2_ctrl *ctrl,
u32 idx,
                 */
                memset(p_h264_scaling_matrix, 16,
sizeof(*p_h264_scaling_matrix));
                break;
+       case V4L2_CTRL_TYPE_H264_SPS:
+               p_h264_sps = p;
+               /*
+                * Without V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY,
+                * frame_mbs_only_flag set to 0 will translate to a miniumum
+                * height of 32 (see H.264 specification 7-8). Some driver may
+                * have a minimum size lower then 32, which would fail
+                * validation with the SPS value. Set this flag, so that there
+                * is now doubling in the height, allowing a valid default.
+                */
+               p_h264_sps->flags = V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY;
+               break;
        }
 }

Nicolas

> At the second time width and height are still 16; while
> coded_fmt.fmt.pix_mp.width is 1920 and coded_fmt.fmt.pix_mp.height is
> 1088, which are the real size of video.
> 
> So VIDIOC_STREAMON will fall at the first time call because sps->flags is
> 0 so V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set, and then height is
> doubled to 32 which is larger than 16.
> 
> What do you think if we skip doubling height if sps->flags is 0 and at the
> same time V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set? The following hack
> did fix my chromium:
> 
> --- a/drivers/staging/media/rkvdec2/rkvdec2-h264.c
> +++ b/drivers/staging/media/rkvdec2/rkvdec2-h264.c
> @@ -767,7 +767,7 @@ static int rkvdec2_h264_validate_sps(struct rkvdec2_ctx *ctx,
>          * which is half the final height (see (7-18) in the
>          * specification)
>          */
> -       if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
> +       if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY) && sps->flags)
>                 height *= 2;
>  
>         if (width > ctx->coded_fmt.fmt.pix_mp.width ||
> 
> Best regards
> Jianfeng

Re: [PATCH v2 2/4] media: rockchip: Introduce the rkvdec2 driver

[Jianfeng Liu]

Hi Detlev,

On Thu, 20 Jun 2024 10:07:41 -0400, Detlev Casanova wrote:

>This feels like hacking the driver to please a specific userspace application, 
>so I'd like to understand better what chromium is doing.

Yes that hack is ugly. I have added log print in chromium to see if they
have set frame_mbs_only_flag to zero and found nothing. This sps->flags is
initialized 0 by kernel's v4l2 code. I printed all sps values in function
rkvdec2_h264_validate_sps and they are all initial values when chromiumn
call VIDIOC_STREAMON at the first time.

Hi Nicolas,

On Thu, 20 Jun 2024 11:03:54 -0400, Nicolas Dufresne wrote:

>This falls short of a specification to support the uninitialized usage by
>Chromium. That being said, we do make an effort to try and have a valid default
>SPS control and OUTPUT format combination. So my suggestion would be to set
>V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY in the default compound control init. This
>way, 0x0 get translate to 16x16 instead of 16x32, thus will work with more
>drivers.

Yeah that's the root cause. Vaule of sps->flags is initialized to 0 along
with pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1, so this
check would fall with minimal decoder size 16x16.

>Chromium these days is not being tested on anything else then MTK, which has a
>64x64 minimum size, this is why they never get into this issue. This driver
>validation is entirely correct. Removing in some cases is unsafe, it would need
>to be replaced with STREAMON only validation of the CAPTURE sizes (which
>currently is validate by implied propagation of valid SPS/OUTPUT).
>
>**not even compiled tested, just to illustrate**
>
>diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-
>core/v4l2-ctrls-core.c
>index c4d995f32191..a55e165ea9c3 100644
>--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
>+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
>@@ -111,6 +111,7 @@ static void std_init_compound(const struct v4l2_ctrl *ctrl,
>u32 idx,
>        struct v4l2_ctrl_vp9_frame *p_vp9_frame;
>        struct v4l2_ctrl_fwht_params *p_fwht_params;
>        struct v4l2_ctrl_h264_scaling_matrix *p_h264_scaling_matrix;
>+       struct v4l2_ctrl_h264_sps *p_h264_sps;
>        struct v4l2_ctrl_av1_sequence *p_av1_sequence;
>        void *p = ptr.p + idx * ctrl->elem_size;
> 
>@@ -179,6 +180,18 @@ static void std_init_compound(const struct v4l2_ctrl *ctrl,
>u32 idx,
>                 */
>                memset(p_h264_scaling_matrix, 16,
>sizeof(*p_h264_scaling_matrix));
>                break;
>+       case V4L2_CTRL_TYPE_H264_SPS:
>+               p_h264_sps = p;
>+               /*
>+                * Without V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY,
>+                * frame_mbs_only_flag set to 0 will translate to a miniumum
>+                * height of 32 (see H.264 specification 7-8). Some driver may
>+                * have a minimum size lower then 32, which would fail
>+                * validation with the SPS value. Set this flag, so that there
>+                * is now doubling in the height, allowing a valid default.
>+                */
>+               p_h264_sps->flags = V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY;
>+               break;
>        }
> }
>
>Nicolas

This patch makes sense and I just confirmed that it works with chromium.
Thank you very much!

Best regards,
Jianfeng

Re: [PATCH v2 2/4] media: rockchip: Introduce the rkvdec2 driver

[Krzysztof Kozlowski]

On 20/06/2024 15:41, Sebastian Reichel wrote:
> Hi,
> 
> On Thu, Jun 20, 2024 at 12:30:00PM GMT, Krzysztof Kozlowski wrote:
>> On 19/06/2024 16:57, Detlev Casanova wrote:
>>> +static const char * const rkvdec2_clk_names[] = {
>>> +	"axi",
>>> +	"ahb",
>>> +	"core",
>>> +	"cabac",
>>> +	"hevc_cabac",
>>> +};
>>> +
>>> +/*
>>> + * Some SoCs, like RK3588 have multiple identical vdpu34x cores, but the
>>> + * kernel is currently missing support for multi-core handling. Exposing
>>> + * separate devices for each core to userspace is bad, since that does
>>> + * not allow scheduling tasks properly (and creates ABI). With this workaround
>>> + * the driver will only probe for the first core and early exit for the other
>>> + * cores. Once the driver gains multi-core support, the same technique
>>> + * for detecting the main core can be used to cluster all cores together.
>>> + */
>>> +static int rkvdec2_disable_multicore(struct rkvdec2_dev *rkvdec)
>>> +{
>>> +	const char *compatible;
>>> +	struct device_node *node;
>>> +	int ret;
>>> +
>>> +	/* Intentionally ignores the fallback strings */
>>> +	ret = of_property_read_string(rkvdec->dev->of_node, "compatible", &compatible);
>>> +	if (ret)
>>> +		return ret;
>>> +
>>> +	/* first compatible node found from the root node is considered the main core */
>>
>> So you rely on order of nodes? Before you claim "identical cores", but
>> now "main core" suggests one is different than others.
> 
> Heh, I wrote that comment for Hantro. By main core I was referencing
> the software side of things. With a number of equal cores and no DT
> node describing a cluster (from HW point of view it's just equal
> cores), they somehow need to be combined into a single entity to allow
> scheduling work between them. This solves the issue by making one of
> the devices the "main" device. From the HW point it's exactly the same
> as the others. The function could also use the last core or the second
> one. It does not matter as long as there is only one "main" core.

Sounds good. Maybe comment could be a bit extended with this
explanation, so the term "main" is clarified.

Best regards,
Krzysztof

Re: [PATCH v2 4/4] arm64: dts: rockchip: Add rkvdec2 Video Decoder on rk3588(s)

[Jonas Karlman]

Hi Detlev and Alex,

On 2024-06-20 15:31, Detlev Casanova wrote:
> Hi Jonas, Alex,
> 
> On Wednesday, June 19, 2024 2:06:40 P.M. EDT Jonas Karlman wrote:
>> Hi Alex,
>>
>> On 2024-06-19 19:19, Alex Bee wrote:
>>> Am 19.06.24 um 17:28 schrieb Jonas Karlman:
>>>> Hi Detlev,
>>>>
>>>> On 2024-06-19 16:57, Detlev Casanova wrote:
>>>>> Add the rkvdec2 Video Decoder to the RK3588s devicetree.
>>>>>
>>>>> Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
>>>>> ---
>>>>>
>>>>>   arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 50 +++++++++++++++++++++++
>>>>>   1 file changed, 50 insertions(+)
>>>>>
>>>>> diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
>>>>> b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi index
>>>>> 6ac5ac8b48ab..7690632f57f1 100644
>>>>> --- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
>>>>> +++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
>>>>> @@ -2596,6 +2596,16 @@ system_sram2: sram@ff001000 {
>>>>>
>>>>>   		ranges = <0x0 0x0 0xff001000 0xef000>;
>>>>>   		#address-cells = <1>;
>>>>>   		#size-cells = <1>;
>>>>>
>>>>> +
>>>>> +		vdec0_sram: rkvdec-sram@0 {
>>>>> +			reg = <0x0 0x78000>;
>>>>> +			pool;
>>>>> +		};
>>>>> +
>>>>> +		vdec1_sram: rkvdec-sram@1 {
>>>>> +			reg = <0x78000 0x77000>;
>>>>> +			pool;
>>>>> +		};
>>>>>
>>>>>   	};
>>>>>   	
>>>>>   	pinctrl: pinctrl {
>>>>>
>>>>> @@ -2665,6 +2675,46 @@ gpio4: gpio@fec50000 {
>>>>>
>>>>>   			#interrupt-cells = <2>;
>>>>>   		
>>>>>   		};
>>>>>   	
>>>>>   	};
>>>>>
>>>>> +
>>>>> +	vdec0: video-decoder@fdc38100 {
>>>>> +		compatible = "rockchip,rk3588-vdec";
>>>>> +		reg = <0x0 0xfdc38100 0x0 0x500>;
>>>>> +		interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH 0>;
>>>>> +		clocks = <&cru ACLK_RKVDEC0>, <&cru HCLK_RKVDEC0>, 
> <&cru
>>>>> CLK_RKVDEC0_CA>, +			 <&cru 
> CLK_RKVDEC0_CORE>, <&cru
>>>>> CLK_RKVDEC0_HEVC_CA>;
>>>>> +		clock-names = "axi", "ahb", "cabac", "core", 
> "hevc_cabac";
>>>>> +		assigned-clocks = <&cru ACLK_RKVDEC0>, <&cru 
> CLK_RKVDEC0_CORE>,
>>>>> +				  <&cru CLK_RKVDEC0_CA>, <&cru 
> CLK_RKVDEC0_HEVC_CA>;
>>>>> +		assigned-clock-rates = <800000000>, <600000000>,
>>>>> +				       <600000000>, <1000000000>;
>>>>> +		resets = <&cru SRST_A_RKVDEC0>, <&cru SRST_H_RKVDEC0>, 
> <&cru
>>>>> SRST_RKVDEC0_CA>, +			 <&cru 
> SRST_RKVDEC0_CORE>, <&cru
>>>>> SRST_RKVDEC0_HEVC_CA>;
>>>>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
>>>>> +			      "rst_core", "rst_hevc_cabac";
>>>>> +		power-domains = <&power RK3588_PD_RKVDEC0>;
>>>>> +		sram = <&vdec0_sram>;
>>>>> +		status = "okay";
>>>>> +	};
>>>>> +
>>>>> +	vdec1: video-decoder@fdc40100 {
>>>>> +		compatible = "rockchip,rk3588-vdec";
>>>>> +		reg = <0x0 0xfdc40100 0x0 0x500>;
>>>>> +		interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH 0>;
>>>>> +		clocks = <&cru ACLK_RKVDEC1>, <&cru HCLK_RKVDEC1>, 
> <&cru
>>>>> CLK_RKVDEC1_CA>, +			 <&cru 
> CLK_RKVDEC1_CORE>, <&cru
>>>>> CLK_RKVDEC1_HEVC_CA>;
>>>>> +		clock-names = "axi", "ahb", "cabac", "core", 
> "hevc_cabac";
>>>>> +		assigned-clocks = <&cru ACLK_RKVDEC1>, <&cru 
> CLK_RKVDEC1_CORE>,
>>>>> +				  <&cru CLK_RKVDEC1_CA>, <&cru 
> CLK_RKVDEC1_HEVC_CA>;
>>>>> +		assigned-clock-rates = <800000000>, <600000000>,
>>>>> +				       <600000000>, <1000000000>;
>>>>> +		resets = <&cru SRST_A_RKVDEC1>, <&cru SRST_H_RKVDEC1>, 
> <&cru
>>>>> SRST_RKVDEC1_CA>, +			 <&cru 
> SRST_RKVDEC1_CORE>, <&cru
>>>>> SRST_RKVDEC1_HEVC_CA>;
>>>>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
>>>>> +			      "rst_core", "rst_hevc_cabac";
>>>>> +		power-domains = <&power RK3588_PD_RKVDEC1>;
>>>>> +		sram = <&vdec1_sram>;
>>>>> +		status = "okay";
>>>>> +	};
>>>>
>>>> This is still missing the iommus, please add the iommus, they should be
>>>>
>>>> supported/same as the one used for e.g. VOP2:
>>>>    compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
>>>>
>>>> The VOP2 MMUs does have one extra mmu_cfg_mode flag in AUTO_GATING,
>>>> compared to the VDPU381 MMUs, however only the AV1D MMU should be
>>>> special on RK3588.
>>>>
>>>> Please add the iommus :-)
>>>
>>> When looking add the vendor DT/iommu driver I'm seeing serval quirks
>>> applied for vdec's iommus. Since it's rightly frowned upon adding such
>>> boolean-quirk-properties to upstream devicetrees, we'd at least need
>>> additional (fallback-) compatibles, even if it works with the iommu driver
>>> as is (what I doubt, but haven't tested). We need to be able to apply
>>> those
>>> quirks later without changing the devicetree (as usual) and I'm sure RK
>>> devs haven't added these quirks for the personal amusement.
>>
>> Based on what I investigated the hw should work similar, and the quirks
>> mostly seem related to optimizations and sw quirks, like do not zap each
>> line, keep it alive even when pm runtime say it is not in use and other
>> quirks that seem to be more of sw nature on how to best utilize the hw.
> 
> I did some testing with the IOMMU but unfortunately, I'm only getting page 
> fault errors. This may be something I'm doing wrong, but it clearly needs more 
> investigation.

I re-tested and the addition of sram seem to now cause page faults, the
sram also need to be mapped in the iommu.

However, doing more testing revealed that use of iommu present the same
issue as seen with hevc on rk3399, after a fail fluster tests continue
to fail until a reset.

Seeing how this issue was very similar I re-tested on rk3399 without
iommu and cma=1G and could observe that there was no longer any need to
reset after a failed test. Interestingly the score also went up from
135 to 137/147.

Digging some more revealed that the iommu also is reset during the
internal rkvdec soft reset on error, leaving the iommu with dte_addr=0
and paging in disabled state.

Ensuring that the iommu was reconfigured after a failure fixed the issue
observed on rk3399 and I now also get 137/147 hevc fluster score using
the iommu.

Will send out a rkvdec hevc v2 series after some more testing.

Guessing there is a similar need to reconfigure iommu on rk3588, and my
initial tests also showed promising result, however more tests are
needed.

Regards,
Jonas

> 
>>> If Detlev says
>>> iommu is out of scope for this series (which is valid), I'd say it's fine
>>> to leave them out for now (as no binding exists) and the HW works
>>> (obviously) fine without them.
>>
>> Sure, use of MMU can be added later.
> 
> I'd rather go for that for now. I'll add that IMMU support is missing in the 
> TODO file.
> 
>> Regards,
>> Jonas
>>
>>>> Regards,
>>>> Jonas
>>>>
>>>>>   };
>>>>>   
>>>>>   #include "rk3588s-pinctrl.dtsi"
> 

Re: [PATCH v2 4/4] arm64: dts: rockchip: Add rkvdec2 Video Decoder on rk3588(s)

[Detlev Casanova]

[-- Attachment #1: Type: text/plain, Size: 7857 bytes --]

Hi Jonas,

On Monday, June 24, 2024 5:16:33 A.M. EDT Jonas Karlman wrote:
> Hi Detlev and Alex,
> 
> On 2024-06-20 15:31, Detlev Casanova wrote:
> > Hi Jonas, Alex,
> > 
> > On Wednesday, June 19, 2024 2:06:40 P.M. EDT Jonas Karlman wrote:
> >> Hi Alex,
> >> 
> >> On 2024-06-19 19:19, Alex Bee wrote:
> >>> Am 19.06.24 um 17:28 schrieb Jonas Karlman:
> >>>> Hi Detlev,
> >>>> 
> >>>> On 2024-06-19 16:57, Detlev Casanova wrote:
> >>>>> Add the rkvdec2 Video Decoder to the RK3588s devicetree.
> >>>>> 
> >>>>> Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
> >>>>> ---
> >>>>> 
> >>>>>   arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 50
> >>>>>   +++++++++++++++++++++++
> >>>>>   1 file changed, 50 insertions(+)
> >>>>> 
> >>>>> diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> >>>>> b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi index
> >>>>> 6ac5ac8b48ab..7690632f57f1 100644
> >>>>> --- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> >>>>> +++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> >>>>> @@ -2596,6 +2596,16 @@ system_sram2: sram@ff001000 {
> >>>>> 
> >>>>>   		ranges = <0x0 0x0 0xff001000 0xef000>;
> >>>>>   		#address-cells = <1>;
> >>>>>   		#size-cells = <1>;
> >>>>> 
> >>>>> +
> >>>>> +		vdec0_sram: rkvdec-sram@0 {
> >>>>> +			reg = <0x0 0x78000>;
> >>>>> +			pool;
> >>>>> +		};
> >>>>> +
> >>>>> +		vdec1_sram: rkvdec-sram@1 {
> >>>>> +			reg = <0x78000 0x77000>;
> >>>>> +			pool;
> >>>>> +		};
> >>>>> 
> >>>>>   	};
> >>>>>   	
> >>>>>   	pinctrl: pinctrl {
> >>>>> 
> >>>>> @@ -2665,6 +2675,46 @@ gpio4: gpio@fec50000 {
> >>>>> 
> >>>>>   			#interrupt-cells = <2>;
> >>>>>   		
> >>>>>   		};
> >>>>>   	
> >>>>>   	};
> >>>>> 
> >>>>> +
> >>>>> +	vdec0: video-decoder@fdc38100 {
> >>>>> +		compatible = "rockchip,rk3588-vdec";
> >>>>> +		reg = <0x0 0xfdc38100 0x0 0x500>;
> >>>>> +		interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH 0>;
> >>>>> +		clocks = <&cru ACLK_RKVDEC0>, <&cru HCLK_RKVDEC0>,
> > 
> > <&cru
> > 
> >>>>> CLK_RKVDEC0_CA>, +			 <&cru
> > 
> > CLK_RKVDEC0_CORE>, <&cru
> > 
> >>>>> CLK_RKVDEC0_HEVC_CA>;
> >>>>> +		clock-names = "axi", "ahb", "cabac", "core",
> > 
> > "hevc_cabac";
> > 
> >>>>> +		assigned-clocks = <&cru ACLK_RKVDEC0>, <&cru
> > 
> > CLK_RKVDEC0_CORE>,
> > 
> >>>>> +				  <&cru CLK_RKVDEC0_CA>, <&cru
> > 
> > CLK_RKVDEC0_HEVC_CA>;
> > 
> >>>>> +		assigned-clock-rates = <800000000>, <600000000>,
> >>>>> +				       <600000000>, <1000000000>;
> >>>>> +		resets = <&cru SRST_A_RKVDEC0>, <&cru SRST_H_RKVDEC0>,
> > 
> > <&cru
> > 
> >>>>> SRST_RKVDEC0_CA>, +			 <&cru
> > 
> > SRST_RKVDEC0_CORE>, <&cru
> > 
> >>>>> SRST_RKVDEC0_HEVC_CA>;
> >>>>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
> >>>>> +			      "rst_core", "rst_hevc_cabac";
> >>>>> +		power-domains = <&power RK3588_PD_RKVDEC0>;
> >>>>> +		sram = <&vdec0_sram>;
> >>>>> +		status = "okay";
> >>>>> +	};
> >>>>> +
> >>>>> +	vdec1: video-decoder@fdc40100 {
> >>>>> +		compatible = "rockchip,rk3588-vdec";
> >>>>> +		reg = <0x0 0xfdc40100 0x0 0x500>;
> >>>>> +		interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH 0>;
> >>>>> +		clocks = <&cru ACLK_RKVDEC1>, <&cru HCLK_RKVDEC1>,
> > 
> > <&cru
> > 
> >>>>> CLK_RKVDEC1_CA>, +			 <&cru
> > 
> > CLK_RKVDEC1_CORE>, <&cru
> > 
> >>>>> CLK_RKVDEC1_HEVC_CA>;
> >>>>> +		clock-names = "axi", "ahb", "cabac", "core",
> > 
> > "hevc_cabac";
> > 
> >>>>> +		assigned-clocks = <&cru ACLK_RKVDEC1>, <&cru
> > 
> > CLK_RKVDEC1_CORE>,
> > 
> >>>>> +				  <&cru CLK_RKVDEC1_CA>, <&cru
> > 
> > CLK_RKVDEC1_HEVC_CA>;
> > 
> >>>>> +		assigned-clock-rates = <800000000>, <600000000>,
> >>>>> +				       <600000000>, <1000000000>;
> >>>>> +		resets = <&cru SRST_A_RKVDEC1>, <&cru SRST_H_RKVDEC1>,
> > 
> > <&cru
> > 
> >>>>> SRST_RKVDEC1_CA>, +			 <&cru
> > 
> > SRST_RKVDEC1_CORE>, <&cru
> > 
> >>>>> SRST_RKVDEC1_HEVC_CA>;
> >>>>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
> >>>>> +			      "rst_core", "rst_hevc_cabac";
> >>>>> +		power-domains = <&power RK3588_PD_RKVDEC1>;
> >>>>> +		sram = <&vdec1_sram>;
> >>>>> +		status = "okay";
> >>>>> +	};
> >>>> 
> >>>> This is still missing the iommus, please add the iommus, they should be
> >>>> 
> >>>> supported/same as the one used for e.g. VOP2:
> >>>>    compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> >>>> 
> >>>> The VOP2 MMUs does have one extra mmu_cfg_mode flag in AUTO_GATING,
> >>>> compared to the VDPU381 MMUs, however only the AV1D MMU should be
> >>>> special on RK3588.
> >>>> 
> >>>> Please add the iommus :-)
> >>> 
> >>> When looking add the vendor DT/iommu driver I'm seeing serval quirks
> >>> applied for vdec's iommus. Since it's rightly frowned upon adding such
> >>> boolean-quirk-properties to upstream devicetrees, we'd at least need
> >>> additional (fallback-) compatibles, even if it works with the iommu
> >>> driver
> >>> as is (what I doubt, but haven't tested). We need to be able to apply
> >>> those
> >>> quirks later without changing the devicetree (as usual) and I'm sure RK
> >>> devs haven't added these quirks for the personal amusement.
> >> 
> >> Based on what I investigated the hw should work similar, and the quirks
> >> mostly seem related to optimizations and sw quirks, like do not zap each
> >> line, keep it alive even when pm runtime say it is not in use and other
> >> quirks that seem to be more of sw nature on how to best utilize the hw.
> > 
> > I did some testing with the IOMMU but unfortunately, I'm only getting page
> > fault errors. This may be something I'm doing wrong, but it clearly needs
> > more investigation.
> 
> I re-tested and the addition of sram seem to now cause page faults, the
> sram also need to be mapped in the iommu.
> 
> However, doing more testing revealed that use of iommu present the same
> issue as seen with hevc on rk3399, after a fail fluster tests continue
> to fail until a reset.
> 
> Seeing how this issue was very similar I re-tested on rk3399 without
> iommu and cma=1G and could observe that there was no longer any need to
> reset after a failed test. Interestingly the score also went up from
> 135 to 137/147.
> 
> Digging some more revealed that the iommu also is reset during the
> internal rkvdec soft reset on error, leaving the iommu with dte_addr=0
> and paging in disabled state.
> 
> Ensuring that the iommu was reconfigured after a failure fixed the issue
> observed on rk3399 and I now also get 137/147 hevc fluster score using
> the iommu.
> 
> Will send out a rkvdec hevc v2 series after some more testing.
> 
> Guessing there is a similar need to reconfigure iommu on rk3588, and my
> initial tests also showed promising result, however more tests are
> needed.

I did some testing with the IOMMU. The good news is that it now works with the 
SRAM.
I am also able to hack the iommu driver to force a reset in case of an error 
in the decoder. I'm not sure how to implement that with the IOMMU kernel API 
though.

Another issue is that resetting the iommu will drop all buffer addresses of 
other decoding contexts that may be running in parallel.

I *think* that the downstream mpp remaps the buffers in the iommu for each 
frame, but I'm not sure about that either.

So running fluster with `-j 1` gives me the expected 129/135 passed tests, but 
`-j 8` will start failing all tests after the first fail (well, first fail 
because of decoder error).

> Regards,
> Jonas
> 
> >>> If Detlev says
> >>> iommu is out of scope for this series (which is valid), I'd say it's
> >>> fine
> >>> to leave them out for now (as no binding exists) and the HW works
> >>> (obviously) fine without them.
> >> 
> >> Sure, use of MMU can be added later.
> > 
> > I'd rather go for that for now. I'll add that IMMU support is missing in
> > the TODO file.
> > 
> >> Regards,
> >> Jonas
> >> 
> >>>> Regards,
> >>>> Jonas
> >>>> 
> >>>>>   };
> >>>>>   
> >>>>>   #include "rk3588s-pinctrl.dtsi"


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 4/4] arm64: dts: rockchip: Add rkvdec2 Video Decoder on rk3588(s)

[Jonas Karlman]

Hi Datlev,

On 2024-06-27 22:56, Detlev Casanova wrote:
> Hi Jonas,
> 
> On Monday, June 24, 2024 5:16:33 A.M. EDT Jonas Karlman wrote:
>> Hi Detlev and Alex,
>>
>> On 2024-06-20 15:31, Detlev Casanova wrote:
>>> Hi Jonas, Alex,
>>>
>>> On Wednesday, June 19, 2024 2:06:40 P.M. EDT Jonas Karlman wrote:
>>>> Hi Alex,
>>>>
>>>> On 2024-06-19 19:19, Alex Bee wrote:
>>>>> Am 19.06.24 um 17:28 schrieb Jonas Karlman:
>>>>>> Hi Detlev,
>>>>>>
>>>>>> On 2024-06-19 16:57, Detlev Casanova wrote:
>>>>>>> Add the rkvdec2 Video Decoder to the RK3588s devicetree.
>>>>>>>
>>>>>>> Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
>>>>>>> ---
>>>>>>>
>>>>>>>   arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 50
>>>>>>>   +++++++++++++++++++++++
>>>>>>>   1 file changed, 50 insertions(+)
>>>>>>>
>>>>>>> diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
>>>>>>> b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi index
>>>>>>> 6ac5ac8b48ab..7690632f57f1 100644
>>>>>>> --- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
>>>>>>> +++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
>>>>>>> @@ -2596,6 +2596,16 @@ system_sram2: sram@ff001000 {
>>>>>>>
>>>>>>>   		ranges = <0x0 0x0 0xff001000 0xef000>;
>>>>>>>   		#address-cells = <1>;
>>>>>>>   		#size-cells = <1>;
>>>>>>>
>>>>>>> +
>>>>>>> +		vdec0_sram: rkvdec-sram@0 {
>>>>>>> +			reg = <0x0 0x78000>;
>>>>>>> +			pool;
>>>>>>> +		};
>>>>>>> +
>>>>>>> +		vdec1_sram: rkvdec-sram@1 {
>>>>>>> +			reg = <0x78000 0x77000>;
>>>>>>> +			pool;
>>>>>>> +		};
>>>>>>>
>>>>>>>   	};
>>>>>>>   	
>>>>>>>   	pinctrl: pinctrl {
>>>>>>>
>>>>>>> @@ -2665,6 +2675,46 @@ gpio4: gpio@fec50000 {
>>>>>>>
>>>>>>>   			#interrupt-cells = <2>;
>>>>>>>   		
>>>>>>>   		};
>>>>>>>   	
>>>>>>>   	};
>>>>>>>
>>>>>>> +
>>>>>>> +	vdec0: video-decoder@fdc38100 {
>>>>>>> +		compatible = "rockchip,rk3588-vdec";
>>>>>>> +		reg = <0x0 0xfdc38100 0x0 0x500>;
>>>>>>> +		interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH 0>;
>>>>>>> +		clocks = <&cru ACLK_RKVDEC0>, <&cru HCLK_RKVDEC0>,
>>>
>>> <&cru
>>>
>>>>>>> CLK_RKVDEC0_CA>, +			 <&cru
>>>
>>> CLK_RKVDEC0_CORE>, <&cru
>>>
>>>>>>> CLK_RKVDEC0_HEVC_CA>;
>>>>>>> +		clock-names = "axi", "ahb", "cabac", "core",
>>>
>>> "hevc_cabac";
>>>
>>>>>>> +		assigned-clocks = <&cru ACLK_RKVDEC0>, <&cru
>>>
>>> CLK_RKVDEC0_CORE>,
>>>
>>>>>>> +				  <&cru CLK_RKVDEC0_CA>, <&cru
>>>
>>> CLK_RKVDEC0_HEVC_CA>;
>>>
>>>>>>> +		assigned-clock-rates = <800000000>, <600000000>,
>>>>>>> +				       <600000000>, <1000000000>;
>>>>>>> +		resets = <&cru SRST_A_RKVDEC0>, <&cru SRST_H_RKVDEC0>,
>>>
>>> <&cru
>>>
>>>>>>> SRST_RKVDEC0_CA>, +			 <&cru
>>>
>>> SRST_RKVDEC0_CORE>, <&cru
>>>
>>>>>>> SRST_RKVDEC0_HEVC_CA>;
>>>>>>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
>>>>>>> +			      "rst_core", "rst_hevc_cabac";
>>>>>>> +		power-domains = <&power RK3588_PD_RKVDEC0>;
>>>>>>> +		sram = <&vdec0_sram>;
>>>>>>> +		status = "okay";
>>>>>>> +	};
>>>>>>> +
>>>>>>> +	vdec1: video-decoder@fdc40100 {
>>>>>>> +		compatible = "rockchip,rk3588-vdec";
>>>>>>> +		reg = <0x0 0xfdc40100 0x0 0x500>;
>>>>>>> +		interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH 0>;
>>>>>>> +		clocks = <&cru ACLK_RKVDEC1>, <&cru HCLK_RKVDEC1>,
>>>
>>> <&cru
>>>
>>>>>>> CLK_RKVDEC1_CA>, +			 <&cru
>>>
>>> CLK_RKVDEC1_CORE>, <&cru
>>>
>>>>>>> CLK_RKVDEC1_HEVC_CA>;
>>>>>>> +		clock-names = "axi", "ahb", "cabac", "core",
>>>
>>> "hevc_cabac";
>>>
>>>>>>> +		assigned-clocks = <&cru ACLK_RKVDEC1>, <&cru
>>>
>>> CLK_RKVDEC1_CORE>,
>>>
>>>>>>> +				  <&cru CLK_RKVDEC1_CA>, <&cru
>>>
>>> CLK_RKVDEC1_HEVC_CA>;
>>>
>>>>>>> +		assigned-clock-rates = <800000000>, <600000000>,
>>>>>>> +				       <600000000>, <1000000000>;
>>>>>>> +		resets = <&cru SRST_A_RKVDEC1>, <&cru SRST_H_RKVDEC1>,
>>>
>>> <&cru
>>>
>>>>>>> SRST_RKVDEC1_CA>, +			 <&cru
>>>
>>> SRST_RKVDEC1_CORE>, <&cru
>>>
>>>>>>> SRST_RKVDEC1_HEVC_CA>;
>>>>>>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
>>>>>>> +			      "rst_core", "rst_hevc_cabac";
>>>>>>> +		power-domains = <&power RK3588_PD_RKVDEC1>;
>>>>>>> +		sram = <&vdec1_sram>;
>>>>>>> +		status = "okay";
>>>>>>> +	};
>>>>>>
>>>>>> This is still missing the iommus, please add the iommus, they should be
>>>>>>
>>>>>> supported/same as the one used for e.g. VOP2:
>>>>>>    compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
>>>>>>
>>>>>> The VOP2 MMUs does have one extra mmu_cfg_mode flag in AUTO_GATING,
>>>>>> compared to the VDPU381 MMUs, however only the AV1D MMU should be
>>>>>> special on RK3588.
>>>>>>
>>>>>> Please add the iommus :-)
>>>>>
>>>>> When looking add the vendor DT/iommu driver I'm seeing serval quirks
>>>>> applied for vdec's iommus. Since it's rightly frowned upon adding such
>>>>> boolean-quirk-properties to upstream devicetrees, we'd at least need
>>>>> additional (fallback-) compatibles, even if it works with the iommu
>>>>> driver
>>>>> as is (what I doubt, but haven't tested). We need to be able to apply
>>>>> those
>>>>> quirks later without changing the devicetree (as usual) and I'm sure RK
>>>>> devs haven't added these quirks for the personal amusement.
>>>>
>>>> Based on what I investigated the hw should work similar, and the quirks
>>>> mostly seem related to optimizations and sw quirks, like do not zap each
>>>> line, keep it alive even when pm runtime say it is not in use and other
>>>> quirks that seem to be more of sw nature on how to best utilize the hw.
>>>
>>> I did some testing with the IOMMU but unfortunately, I'm only getting page
>>> fault errors. This may be something I'm doing wrong, but it clearly needs
>>> more investigation.
>>
>> I re-tested and the addition of sram seem to now cause page faults, the
>> sram also need to be mapped in the iommu.
>>
>> However, doing more testing revealed that use of iommu present the same
>> issue as seen with hevc on rk3399, after a fail fluster tests continue
>> to fail until a reset.
>>
>> Seeing how this issue was very similar I re-tested on rk3399 without
>> iommu and cma=1G and could observe that there was no longer any need to
>> reset after a failed test. Interestingly the score also went up from
>> 135 to 137/147.
>>
>> Digging some more revealed that the iommu also is reset during the
>> internal rkvdec soft reset on error, leaving the iommu with dte_addr=0
>> and paging in disabled state.
>>
>> Ensuring that the iommu was reconfigured after a failure fixed the issue
>> observed on rk3399 and I now also get 137/147 hevc fluster score using
>> the iommu.
>>
>> Will send out a rkvdec hevc v2 series after some more testing.
>>
>> Guessing there is a similar need to reconfigure iommu on rk3588, and my
>> initial tests also showed promising result, however more tests are
>> needed.
> 
> I did some testing with the IOMMU. The good news is that it now works with the 
> SRAM.

Great, I did not look into SRAM at all, just replaced sram prop with iommus for
my tests, so great that you found a way to make it work with the iommu :-)

> I am also able to hack the iommu driver to force a reset in case of an error 
> in the decoder. I'm not sure how to implement that with the IOMMU kernel API 
> though.

I am planning on sending something along the way of this as an RFC:

https://github.com/Kwiboo/linux-rockchip/compare/6da640232631...bf332524d880

If we re-configure and re-enable the iommu just before next decoding run
after a decoding has failed seem to resolve any issue I have seen, have
mainly been tested with rkvdec and HEVC on RK3399/RK3328. On RK3588 this
also seemed to work, at least when I tested earlier this week.

> 
> Another issue is that resetting the iommu will drop all buffer addresses of 
> other decoding contexts that may be running in parallel.

I do not think we need/should reset the iommu, we just need to deal with
the fact that the rkvdec will reset and disable use of the mmu when it
reset itself.

> 
> I *think* that the downstream mpp remaps the buffers in the iommu for each 
> frame, but I'm not sure about that either.

As long as a frame can be decoded correctly, the mmu config seem to continue
to be valid and next frame can be decoded.

> 
> So running fluster with `-j 1` gives me the expected 129/135 passed tests, but 
> `-j 8` will start failing all tests after the first fail (well, first fail 
> because of decoder error).

This was the main issue blocking rkvdec hevc, just re-confgure the mmu
after a frame fails to decode seem to resolve this issue.

Biggest issue at the moment is how to properly signal iommu subsystem that
it should re-configure, I may have abused the flush_iotlb_all ops, since
that seemed closest existing hook.

Will send an RFC to linux-iommu to collect input on how to best signal
iommu subsystem that the mmu has been reset by an external event and now
need to be re-configured.

Regards,
Jonas

> 
>> Regards,
>> Jonas
>>
>>>>> If Detlev says
>>>>> iommu is out of scope for this series (which is valid), I'd say it's
>>>>> fine
>>>>> to leave them out for now (as no binding exists) and the HW works
>>>>> (obviously) fine without them.
>>>>
>>>> Sure, use of MMU can be added later.
>>>
>>> I'd rather go for that for now. I'll add that IMMU support is missing in
>>> the TODO file.
>>>
>>>> Regards,
>>>> Jonas
>>>>
>>>>>> Regards,
>>>>>> Jonas
>>>>>>
>>>>>>>   };
>>>>>>>   
>>>>>>>   #include "rk3588s-pinctrl.dtsi"
> 

Re: [PATCH v2 4/4] arm64: dts: rockchip: Add rkvdec2 Video Decoder on rk3588(s)

[Detlev Casanova]

[-- Attachment #1: Type: text/plain, Size: 6395 bytes --]

Hi Jonas,

On Thursday, June 27, 2024 6:39:36 P.M. EDT Jonas Karlman wrote:
[snip]
> >>>>>>> SRST_RKVDEC1_HEVC_CA>;
> >>>>>>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
> >>>>>>> +			      "rst_core", "rst_hevc_cabac";
> >>>>>>> +		power-domains = <&power RK3588_PD_RKVDEC1>;
> >>>>>>> +		sram = <&vdec1_sram>;
> >>>>>>> +		status = "okay";
> >>>>>>> +	};
> >>>>>> 
> >>>>>> This is still missing the iommus, please add the iommus, they should
> >>>>>> be
> >>>>>> 
> >>>>>> supported/same as the one used for e.g. VOP2:
> >>>>>>    compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> >>>>>> 
> >>>>>> The VOP2 MMUs does have one extra mmu_cfg_mode flag in AUTO_GATING,
> >>>>>> compared to the VDPU381 MMUs, however only the AV1D MMU should be
> >>>>>> special on RK3588.
> >>>>>> 
> >>>>>> Please add the iommus :-)
> >>>>> 
> >>>>> When looking add the vendor DT/iommu driver I'm seeing serval quirks
> >>>>> applied for vdec's iommus. Since it's rightly frowned upon adding such
> >>>>> boolean-quirk-properties to upstream devicetrees, we'd at least need
> >>>>> additional (fallback-) compatibles, even if it works with the iommu
> >>>>> driver
> >>>>> as is (what I doubt, but haven't tested). We need to be able to apply
> >>>>> those
> >>>>> quirks later without changing the devicetree (as usual) and I'm sure
> >>>>> RK
> >>>>> devs haven't added these quirks for the personal amusement.
> >>>> 
> >>>> Based on what I investigated the hw should work similar, and the quirks
> >>>> mostly seem related to optimizations and sw quirks, like do not zap
> >>>> each
> >>>> line, keep it alive even when pm runtime say it is not in use and other
> >>>> quirks that seem to be more of sw nature on how to best utilize the hw.
> >>> 
> >>> I did some testing with the IOMMU but unfortunately, I'm only getting
> >>> page
> >>> fault errors. This may be something I'm doing wrong, but it clearly
> >>> needs
> >>> more investigation.
> >> 
> >> I re-tested and the addition of sram seem to now cause page faults, the
> >> sram also need to be mapped in the iommu.
> >> 
> >> However, doing more testing revealed that use of iommu present the same
> >> issue as seen with hevc on rk3399, after a fail fluster tests continue
> >> to fail until a reset.
> >> 
> >> Seeing how this issue was very similar I re-tested on rk3399 without
> >> iommu and cma=1G and could observe that there was no longer any need to
> >> reset after a failed test. Interestingly the score also went up from
> >> 135 to 137/147.
> >> 
> >> Digging some more revealed that the iommu also is reset during the
> >> internal rkvdec soft reset on error, leaving the iommu with dte_addr=0
> >> and paging in disabled state.
> >> 
> >> Ensuring that the iommu was reconfigured after a failure fixed the issue
> >> observed on rk3399 and I now also get 137/147 hevc fluster score using
> >> the iommu.
> >> 
> >> Will send out a rkvdec hevc v2 series after some more testing.
> >> 
> >> Guessing there is a similar need to reconfigure iommu on rk3588, and my
> >> initial tests also showed promising result, however more tests are
> >> needed.
> > 
> > I did some testing with the IOMMU. The good news is that it now works with
> > the SRAM.
> 
> Great, I did not look into SRAM at all, just replaced sram prop with iommus
> for my tests, so great that you found a way to make it work with the iommu
> :-)
> > I am also able to hack the iommu driver to force a reset in case of an
> > error in the decoder. I'm not sure how to implement that with the IOMMU
> > kernel API though.
> 
> I am planning on sending something along the way of this as an RFC:
> 
> https://github.com/Kwiboo/linux-rockchip/compare/6da640232631...bf332524d880
> 
> If we re-configure and re-enable the iommu just before next decoding run
> after a decoding has failed seem to resolve any issue I have seen, have
> mainly been tested with rkvdec and HEVC on RK3399/RK3328. On RK3588 this
> also seemed to work, at least when I tested earlier this week.
>
> > Another issue is that resetting the iommu will drop all buffer addresses
> > of
> > other decoding contexts that may be running in parallel.
> 
> I do not think we need/should reset the iommu, we just need to deal with
> the fact that the rkvdec will reset and disable use of the mmu when it
> reset itself.

Oh I see, it just resets the iommu config in the core's registers, but the 
hardware retains the mapped addresses. That makes things simpler indeed, just 
reconfigure before the next frame in case of status error in the interrupt.

> > I *think* that the downstream mpp remaps the buffers in the iommu for each
> > frame, but I'm not sure about that either.
> 
> As long as a frame can be decoded correctly, the mmu config seem to continue
> to be valid and next frame can be decoded.
> 
> > So running fluster with `-j 1` gives me the expected 129/135 passed tests,
> > but `-j 8` will start failing all tests after the first fail (well, first
> > fail because of decoder error).
> 
> This was the main issue blocking rkvdec hevc, just re-confgure the mmu
> after a frame fails to decode seem to resolve this issue.
> 
> Biggest issue at the moment is how to properly signal iommu subsystem that
> it should re-configure, I may have abused the flush_iotlb_all ops, since
> that seemed closest existing hook.

Oh I see, I was wondering what the flush_iotlb_all ops was doing exactly. So 
does it flush a "cached" copy of the table from the driver back to the hardware 
? As well as reconfigure the registers I guess.

> Will send an RFC to linux-iommu to collect input on how to best signal
> iommu subsystem that the mmu has been reset by an external event and now
> need to be re-configured.

Thank you ! Can you add me in cc so that I can keep track of this ?

> Regards,
> Jonas
> 
> >> Regards,
> >> Jonas
> >> 
> >>>>> If Detlev says
> >>>>> iommu is out of scope for this series (which is valid), I'd say it's
> >>>>> fine
> >>>>> to leave them out for now (as no binding exists) and the HW works
> >>>>> (obviously) fine without them.
> >>>> 
> >>>> Sure, use of MMU can be added later.
> >>> 
> >>> I'd rather go for that for now. I'll add that IMMU support is missing in
> >>> the TODO file.
> >>> 
> >>>> Regards,
> >>>> Jonas
> >>>> 
> >>>>>> Regards,
> >>>>>> Jonas
> >>>>>> 
> >>>>>>>   };
> >>>>>>>   
> >>>>>>>   #include "rk3588s-pinctrl.dtsi"


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 4/4] arm64: dts: rockchip: Add rkvdec2 Video Decoder on rk3588(s)

[Detlev Casanova]

[-- Attachment #1: Type: text/plain, Size: 9607 bytes --]

Hi Jonas !

On Thursday, June 27, 2024 6:39:36 P.M. EDT Jonas Karlman wrote:
> Hi Datlev,
> 
> On 2024-06-27 22:56, Detlev Casanova wrote:
> > Hi Jonas,
> > 
> > On Monday, June 24, 2024 5:16:33 A.M. EDT Jonas Karlman wrote:
> >> Hi Detlev and Alex,
> >> 
> >> On 2024-06-20 15:31, Detlev Casanova wrote:
> >>> Hi Jonas, Alex,
> >>> 
> >>> On Wednesday, June 19, 2024 2:06:40 P.M. EDT Jonas Karlman wrote:
> >>>> Hi Alex,
> >>>> 
> >>>> On 2024-06-19 19:19, Alex Bee wrote:
> >>>>> Am 19.06.24 um 17:28 schrieb Jonas Karlman:
> >>>>>> Hi Detlev,
> >>>>>> 
> >>>>>> On 2024-06-19 16:57, Detlev Casanova wrote:
> >>>>>>> Add the rkvdec2 Video Decoder to the RK3588s devicetree.
> >>>>>>> 
> >>>>>>> Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
> >>>>>>> ---
> >>>>>>> 
> >>>>>>>   arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 50
> >>>>>>>   +++++++++++++++++++++++
> >>>>>>>   1 file changed, 50 insertions(+)
> >>>>>>> 
> >>>>>>> diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> >>>>>>> b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi index
> >>>>>>> 6ac5ac8b48ab..7690632f57f1 100644
> >>>>>>> --- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> >>>>>>> +++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> >>>>>>> @@ -2596,6 +2596,16 @@ system_sram2: sram@ff001000 {
> >>>>>>> 
> >>>>>>>   		ranges = <0x0 0x0 0xff001000 0xef000>;
> >>>>>>>   		#address-cells = <1>;
> >>>>>>>   		#size-cells = <1>;
> >>>>>>> 
> >>>>>>> +
> >>>>>>> +		vdec0_sram: rkvdec-sram@0 {
> >>>>>>> +			reg = <0x0 0x78000>;
> >>>>>>> +			pool;
> >>>>>>> +		};
> >>>>>>> +
> >>>>>>> +		vdec1_sram: rkvdec-sram@1 {
> >>>>>>> +			reg = <0x78000 0x77000>;
> >>>>>>> +			pool;
> >>>>>>> +		};
> >>>>>>> 
> >>>>>>>   	};
> >>>>>>>   	
> >>>>>>>   	pinctrl: pinctrl {
> >>>>>>> 
> >>>>>>> @@ -2665,6 +2675,46 @@ gpio4: gpio@fec50000 {
> >>>>>>> 
> >>>>>>>   			#interrupt-cells = <2>;
> >>>>>>>   		
> >>>>>>>   		};
> >>>>>>>   	
> >>>>>>>   	};
> >>>>>>> 
> >>>>>>> +
> >>>>>>> +	vdec0: video-decoder@fdc38100 {
> >>>>>>> +		compatible = "rockchip,rk3588-vdec";
> >>>>>>> +		reg = <0x0 0xfdc38100 0x0 0x500>;
> >>>>>>> +		interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH 0>;
> >>>>>>> +		clocks = <&cru ACLK_RKVDEC0>, <&cru HCLK_RKVDEC0>,
> >>> 
> >>> <&cru
> >>> 
> >>>>>>> CLK_RKVDEC0_CA>, +			 <&cru
> >>> 
> >>> CLK_RKVDEC0_CORE>, <&cru
> >>> 
> >>>>>>> CLK_RKVDEC0_HEVC_CA>;
> >>>>>>> +		clock-names = "axi", "ahb", "cabac", "core",
> >>> 
> >>> "hevc_cabac";
> >>> 
> >>>>>>> +		assigned-clocks = <&cru ACLK_RKVDEC0>, <&cru
> >>> 
> >>> CLK_RKVDEC0_CORE>,
> >>> 
> >>>>>>> +				  <&cru CLK_RKVDEC0_CA>, <&cru
> >>> 
> >>> CLK_RKVDEC0_HEVC_CA>;
> >>> 
> >>>>>>> +		assigned-clock-rates = <800000000>, <600000000>,
> >>>>>>> +				       <600000000>, <1000000000>;
> >>>>>>> +		resets = <&cru SRST_A_RKVDEC0>, <&cru SRST_H_RKVDEC0>,
> >>> 
> >>> <&cru
> >>> 
> >>>>>>> SRST_RKVDEC0_CA>, +			 <&cru
> >>> 
> >>> SRST_RKVDEC0_CORE>, <&cru
> >>> 
> >>>>>>> SRST_RKVDEC0_HEVC_CA>;
> >>>>>>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
> >>>>>>> +			      "rst_core", "rst_hevc_cabac";
> >>>>>>> +		power-domains = <&power RK3588_PD_RKVDEC0>;
> >>>>>>> +		sram = <&vdec0_sram>;
> >>>>>>> +		status = "okay";
> >>>>>>> +	};
> >>>>>>> +
> >>>>>>> +	vdec1: video-decoder@fdc40100 {
> >>>>>>> +		compatible = "rockchip,rk3588-vdec";
> >>>>>>> +		reg = <0x0 0xfdc40100 0x0 0x500>;
> >>>>>>> +		interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH 0>;
> >>>>>>> +		clocks = <&cru ACLK_RKVDEC1>, <&cru HCLK_RKVDEC1>,
> >>> 
> >>> <&cru
> >>> 
> >>>>>>> CLK_RKVDEC1_CA>, +			 <&cru
> >>> 
> >>> CLK_RKVDEC1_CORE>, <&cru
> >>> 
> >>>>>>> CLK_RKVDEC1_HEVC_CA>;
> >>>>>>> +		clock-names = "axi", "ahb", "cabac", "core",
> >>> 
> >>> "hevc_cabac";
> >>> 
> >>>>>>> +		assigned-clocks = <&cru ACLK_RKVDEC1>, <&cru
> >>> 
> >>> CLK_RKVDEC1_CORE>,
> >>> 
> >>>>>>> +				  <&cru CLK_RKVDEC1_CA>, <&cru
> >>> 
> >>> CLK_RKVDEC1_HEVC_CA>;
> >>> 
> >>>>>>> +		assigned-clock-rates = <800000000>, <600000000>,
> >>>>>>> +				       <600000000>, <1000000000>;
> >>>>>>> +		resets = <&cru SRST_A_RKVDEC1>, <&cru SRST_H_RKVDEC1>,
> >>> 
> >>> <&cru
> >>> 
> >>>>>>> SRST_RKVDEC1_CA>, +			 <&cru
> >>> 
> >>> SRST_RKVDEC1_CORE>, <&cru
> >>> 
> >>>>>>> SRST_RKVDEC1_HEVC_CA>;
> >>>>>>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
> >>>>>>> +			      "rst_core", "rst_hevc_cabac";
> >>>>>>> +		power-domains = <&power RK3588_PD_RKVDEC1>;
> >>>>>>> +		sram = <&vdec1_sram>;
> >>>>>>> +		status = "okay";
> >>>>>>> +	};
> >>>>>> 
> >>>>>> This is still missing the iommus, please add the iommus, they should
> >>>>>> be
> >>>>>> 
> >>>>>> supported/same as the one used for e.g. VOP2:
> >>>>>>    compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> >>>>>> 
> >>>>>> The VOP2 MMUs does have one extra mmu_cfg_mode flag in AUTO_GATING,
> >>>>>> compared to the VDPU381 MMUs, however only the AV1D MMU should be
> >>>>>> special on RK3588.
> >>>>>> 
> >>>>>> Please add the iommus :-)
> >>>>> 
> >>>>> When looking add the vendor DT/iommu driver I'm seeing serval quirks
> >>>>> applied for vdec's iommus. Since it's rightly frowned upon adding such
> >>>>> boolean-quirk-properties to upstream devicetrees, we'd at least need
> >>>>> additional (fallback-) compatibles, even if it works with the iommu
> >>>>> driver
> >>>>> as is (what I doubt, but haven't tested). We need to be able to apply
> >>>>> those
> >>>>> quirks later without changing the devicetree (as usual) and I'm sure
> >>>>> RK
> >>>>> devs haven't added these quirks for the personal amusement.
> >>>> 
> >>>> Based on what I investigated the hw should work similar, and the quirks
> >>>> mostly seem related to optimizations and sw quirks, like do not zap
> >>>> each
> >>>> line, keep it alive even when pm runtime say it is not in use and other
> >>>> quirks that seem to be more of sw nature on how to best utilize the hw.
> >>> 
> >>> I did some testing with the IOMMU but unfortunately, I'm only getting
> >>> page
> >>> fault errors. This may be something I'm doing wrong, but it clearly
> >>> needs
> >>> more investigation.
> >> 
> >> I re-tested and the addition of sram seem to now cause page faults, the
> >> sram also need to be mapped in the iommu.
> >> 
> >> However, doing more testing revealed that use of iommu present the same
> >> issue as seen with hevc on rk3399, after a fail fluster tests continue
> >> to fail until a reset.
> >> 
> >> Seeing how this issue was very similar I re-tested on rk3399 without
> >> iommu and cma=1G and could observe that there was no longer any need to
> >> reset after a failed test. Interestingly the score also went up from
> >> 135 to 137/147.
> >> 
> >> Digging some more revealed that the iommu also is reset during the
> >> internal rkvdec soft reset on error, leaving the iommu with dte_addr=0
> >> and paging in disabled state.
> >> 
> >> Ensuring that the iommu was reconfigured after a failure fixed the issue
> >> observed on rk3399 and I now also get 137/147 hevc fluster score using
> >> the iommu.
> >> 
> >> Will send out a rkvdec hevc v2 series after some more testing.
> >> 
> >> Guessing there is a similar need to reconfigure iommu on rk3588, and my
> >> initial tests also showed promising result, however more tests are
> >> needed.
> > 
> > I did some testing with the IOMMU. The good news is that it now works with
> > the SRAM.
> 
> Great, I did not look into SRAM at all, just replaced sram prop with iommus
> for my tests, so great that you found a way to make it work with the iommu
> :-)
> > I am also able to hack the iommu driver to force a reset in case of an
> > error in the decoder. I'm not sure how to implement that with the IOMMU
> > kernel API though.
> 
> I am planning on sending something along the way of this as an RFC:
> 
> https://github.com/Kwiboo/linux-rockchip/compare/6da640232631...bf332524d880
> 
> If we re-configure and re-enable the iommu just before next decoding run
> after a decoding has failed seem to resolve any issue I have seen, have
> mainly been tested with rkvdec and HEVC on RK3399/RK3328. On RK3588 this
> also seemed to work, at least when I tested earlier this week.
> 
> > Another issue is that resetting the iommu will drop all buffer addresses
> > of
> > other decoding contexts that may be running in parallel.
> 
> I do not think we need/should reset the iommu, we just need to deal with
> the fact that the rkvdec will reset and disable use of the mmu when it
> reset itself.
> 
> > I *think* that the downstream mpp remaps the buffers in the iommu for each
> > frame, but I'm not sure about that either.
> 
> As long as a frame can be decoded correctly, the mmu config seem to continue
> to be valid and next frame can be decoded.
> 
> > So running fluster with `-j 1` gives me the expected 129/135 passed tests,
> > but `-j 8` will start failing all tests after the first fail (well, first
> > fail because of decoder error).
> 
> This was the main issue blocking rkvdec hevc, just re-confgure the mmu
> after a frame fails to decode seem to resolve this issue.
> 
> Biggest issue at the moment is how to properly signal iommu subsystem that
> it should re-configure, I may have abused the flush_iotlb_all ops, since
> that seemed closest existing hook.
> 
> Will send an RFC to linux-iommu to collect input on how to best signal
> iommu subsystem that the mmu has been reset by an external event and now
> need to be re-configured.

Do you mind if I go ahead and send your iommu flush_iotlb_all patch upstream to 
start the discussion ? I'd love for this patch set to move along and that's 
kind of a blocker right now.

Detlev.



[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 4/4] arm64: dts: rockchip: Add rkvdec2 Video Decoder on rk3588(s)

[Jonas Karlman]

Hi Detlev,

On 2024-07-26 17:26, Detlev Casanova wrote:
> Hi Jonas !
> 
> On Thursday, June 27, 2024 6:39:36 P.M. EDT Jonas Karlman wrote:
>> Hi Datlev,
>>
>> On 2024-06-27 22:56, Detlev Casanova wrote:
>>> Hi Jonas,
>>>
>>> On Monday, June 24, 2024 5:16:33 A.M. EDT Jonas Karlman wrote:
>>>> Hi Detlev and Alex,
>>>>
>>>> On 2024-06-20 15:31, Detlev Casanova wrote:
>>>>> Hi Jonas, Alex,
>>>>>
>>>>> On Wednesday, June 19, 2024 2:06:40 P.M. EDT Jonas Karlman wrote:
>>>>>> Hi Alex,
>>>>>>
>>>>>> On 2024-06-19 19:19, Alex Bee wrote:
>>>>>>> Am 19.06.24 um 17:28 schrieb Jonas Karlman:
>>>>>>>> Hi Detlev,
>>>>>>>>
>>>>>>>> On 2024-06-19 16:57, Detlev Casanova wrote:
>>>>>>>>> Add the rkvdec2 Video Decoder to the RK3588s devicetree.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
>>>>>>>>> ---
>>>>>>>>>
>>>>>>>>>   arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 50
>>>>>>>>>   +++++++++++++++++++++++
>>>>>>>>>   1 file changed, 50 insertions(+)
>>>>>>>>>
>>>>>>>>> diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
>>>>>>>>> b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi index
>>>>>>>>> 6ac5ac8b48ab..7690632f57f1 100644
>>>>>>>>> --- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
>>>>>>>>> +++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
>>>>>>>>> @@ -2596,6 +2596,16 @@ system_sram2: sram@ff001000 {
>>>>>>>>>
>>>>>>>>>   		ranges = <0x0 0x0 0xff001000 0xef000>;
>>>>>>>>>   		#address-cells = <1>;
>>>>>>>>>   		#size-cells = <1>;
>>>>>>>>>
>>>>>>>>> +
>>>>>>>>> +		vdec0_sram: rkvdec-sram@0 {
>>>>>>>>> +			reg = <0x0 0x78000>;
>>>>>>>>> +			pool;
>>>>>>>>> +		};
>>>>>>>>> +
>>>>>>>>> +		vdec1_sram: rkvdec-sram@1 {
>>>>>>>>> +			reg = <0x78000 0x77000>;
>>>>>>>>> +			pool;
>>>>>>>>> +		};
>>>>>>>>>
>>>>>>>>>   	};
>>>>>>>>>   	
>>>>>>>>>   	pinctrl: pinctrl {
>>>>>>>>>
>>>>>>>>> @@ -2665,6 +2675,46 @@ gpio4: gpio@fec50000 {
>>>>>>>>>
>>>>>>>>>   			#interrupt-cells = <2>;
>>>>>>>>>   		
>>>>>>>>>   		};
>>>>>>>>>   	
>>>>>>>>>   	};
>>>>>>>>>
>>>>>>>>> +
>>>>>>>>> +	vdec0: video-decoder@fdc38100 {
>>>>>>>>> +		compatible = "rockchip,rk3588-vdec";
>>>>>>>>> +		reg = <0x0 0xfdc38100 0x0 0x500>;
>>>>>>>>> +		interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH 0>;
>>>>>>>>> +		clocks = <&cru ACLK_RKVDEC0>, <&cru HCLK_RKVDEC0>,
>>>>>
>>>>> <&cru
>>>>>
>>>>>>>>> CLK_RKVDEC0_CA>, +			 <&cru
>>>>>
>>>>> CLK_RKVDEC0_CORE>, <&cru
>>>>>
>>>>>>>>> CLK_RKVDEC0_HEVC_CA>;
>>>>>>>>> +		clock-names = "axi", "ahb", "cabac", "core",
>>>>>
>>>>> "hevc_cabac";
>>>>>
>>>>>>>>> +		assigned-clocks = <&cru ACLK_RKVDEC0>, <&cru
>>>>>
>>>>> CLK_RKVDEC0_CORE>,
>>>>>
>>>>>>>>> +				  <&cru CLK_RKVDEC0_CA>, <&cru
>>>>>
>>>>> CLK_RKVDEC0_HEVC_CA>;
>>>>>
>>>>>>>>> +		assigned-clock-rates = <800000000>, <600000000>,
>>>>>>>>> +				       <600000000>, <1000000000>;
>>>>>>>>> +		resets = <&cru SRST_A_RKVDEC0>, <&cru SRST_H_RKVDEC0>,
>>>>>
>>>>> <&cru
>>>>>
>>>>>>>>> SRST_RKVDEC0_CA>, +			 <&cru
>>>>>
>>>>> SRST_RKVDEC0_CORE>, <&cru
>>>>>
>>>>>>>>> SRST_RKVDEC0_HEVC_CA>;
>>>>>>>>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
>>>>>>>>> +			      "rst_core", "rst_hevc_cabac";
>>>>>>>>> +		power-domains = <&power RK3588_PD_RKVDEC0>;
>>>>>>>>> +		sram = <&vdec0_sram>;
>>>>>>>>> +		status = "okay";
>>>>>>>>> +	};
>>>>>>>>> +
>>>>>>>>> +	vdec1: video-decoder@fdc40100 {
>>>>>>>>> +		compatible = "rockchip,rk3588-vdec";
>>>>>>>>> +		reg = <0x0 0xfdc40100 0x0 0x500>;
>>>>>>>>> +		interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH 0>;
>>>>>>>>> +		clocks = <&cru ACLK_RKVDEC1>, <&cru HCLK_RKVDEC1>,
>>>>>
>>>>> <&cru
>>>>>
>>>>>>>>> CLK_RKVDEC1_CA>, +			 <&cru
>>>>>
>>>>> CLK_RKVDEC1_CORE>, <&cru
>>>>>
>>>>>>>>> CLK_RKVDEC1_HEVC_CA>;
>>>>>>>>> +		clock-names = "axi", "ahb", "cabac", "core",
>>>>>
>>>>> "hevc_cabac";
>>>>>
>>>>>>>>> +		assigned-clocks = <&cru ACLK_RKVDEC1>, <&cru
>>>>>
>>>>> CLK_RKVDEC1_CORE>,
>>>>>
>>>>>>>>> +				  <&cru CLK_RKVDEC1_CA>, <&cru
>>>>>
>>>>> CLK_RKVDEC1_HEVC_CA>;
>>>>>
>>>>>>>>> +		assigned-clock-rates = <800000000>, <600000000>,
>>>>>>>>> +				       <600000000>, <1000000000>;
>>>>>>>>> +		resets = <&cru SRST_A_RKVDEC1>, <&cru SRST_H_RKVDEC1>,
>>>>>
>>>>> <&cru
>>>>>
>>>>>>>>> SRST_RKVDEC1_CA>, +			 <&cru
>>>>>
>>>>> SRST_RKVDEC1_CORE>, <&cru
>>>>>
>>>>>>>>> SRST_RKVDEC1_HEVC_CA>;
>>>>>>>>> +		reset-names = "rst_axi", "rst_ahb", "rst_cabac",
>>>>>>>>> +			      "rst_core", "rst_hevc_cabac";
>>>>>>>>> +		power-domains = <&power RK3588_PD_RKVDEC1>;
>>>>>>>>> +		sram = <&vdec1_sram>;
>>>>>>>>> +		status = "okay";
>>>>>>>>> +	};
>>>>>>>>
>>>>>>>> This is still missing the iommus, please add the iommus, they should
>>>>>>>> be
>>>>>>>>
>>>>>>>> supported/same as the one used for e.g. VOP2:
>>>>>>>>    compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
>>>>>>>>
>>>>>>>> The VOP2 MMUs does have one extra mmu_cfg_mode flag in AUTO_GATING,
>>>>>>>> compared to the VDPU381 MMUs, however only the AV1D MMU should be
>>>>>>>> special on RK3588.
>>>>>>>>
>>>>>>>> Please add the iommus :-)
>>>>>>>
>>>>>>> When looking add the vendor DT/iommu driver I'm seeing serval quirks
>>>>>>> applied for vdec's iommus. Since it's rightly frowned upon adding such
>>>>>>> boolean-quirk-properties to upstream devicetrees, we'd at least need
>>>>>>> additional (fallback-) compatibles, even if it works with the iommu
>>>>>>> driver
>>>>>>> as is (what I doubt, but haven't tested). We need to be able to apply
>>>>>>> those
>>>>>>> quirks later without changing the devicetree (as usual) and I'm sure
>>>>>>> RK
>>>>>>> devs haven't added these quirks for the personal amusement.
>>>>>>
>>>>>> Based on what I investigated the hw should work similar, and the quirks
>>>>>> mostly seem related to optimizations and sw quirks, like do not zap
>>>>>> each
>>>>>> line, keep it alive even when pm runtime say it is not in use and other
>>>>>> quirks that seem to be more of sw nature on how to best utilize the hw.
>>>>>
>>>>> I did some testing with the IOMMU but unfortunately, I'm only getting
>>>>> page
>>>>> fault errors. This may be something I'm doing wrong, but it clearly
>>>>> needs
>>>>> more investigation.
>>>>
>>>> I re-tested and the addition of sram seem to now cause page faults, the
>>>> sram also need to be mapped in the iommu.
>>>>
>>>> However, doing more testing revealed that use of iommu present the same
>>>> issue as seen with hevc on rk3399, after a fail fluster tests continue
>>>> to fail until a reset.
>>>>
>>>> Seeing how this issue was very similar I re-tested on rk3399 without
>>>> iommu and cma=1G and could observe that there was no longer any need to
>>>> reset after a failed test. Interestingly the score also went up from
>>>> 135 to 137/147.
>>>>
>>>> Digging some more revealed that the iommu also is reset during the
>>>> internal rkvdec soft reset on error, leaving the iommu with dte_addr=0
>>>> and paging in disabled state.
>>>>
>>>> Ensuring that the iommu was reconfigured after a failure fixed the issue
>>>> observed on rk3399 and I now also get 137/147 hevc fluster score using
>>>> the iommu.
>>>>
>>>> Will send out a rkvdec hevc v2 series after some more testing.
>>>>
>>>> Guessing there is a similar need to reconfigure iommu on rk3588, and my
>>>> initial tests also showed promising result, however more tests are
>>>> needed.
>>>
>>> I did some testing with the IOMMU. The good news is that it now works with
>>> the SRAM.
>>
>> Great, I did not look into SRAM at all, just replaced sram prop with iommus
>> for my tests, so great that you found a way to make it work with the iommu
>> :-)
>>> I am also able to hack the iommu driver to force a reset in case of an
>>> error in the decoder. I'm not sure how to implement that with the IOMMU
>>> kernel API though.
>>
>> I am planning on sending something along the way of this as an RFC:
>>
>> https://github.com/Kwiboo/linux-rockchip/compare/6da640232631...bf332524d880
>>
>> If we re-configure and re-enable the iommu just before next decoding run
>> after a decoding has failed seem to resolve any issue I have seen, have
>> mainly been tested with rkvdec and HEVC on RK3399/RK3328. On RK3588 this
>> also seemed to work, at least when I tested earlier this week.
>>
>>> Another issue is that resetting the iommu will drop all buffer addresses
>>> of
>>> other decoding contexts that may be running in parallel.
>>
>> I do not think we need/should reset the iommu, we just need to deal with
>> the fact that the rkvdec will reset and disable use of the mmu when it
>> reset itself.
>>
>>> I *think* that the downstream mpp remaps the buffers in the iommu for each
>>> frame, but I'm not sure about that either.
>>
>> As long as a frame can be decoded correctly, the mmu config seem to continue
>> to be valid and next frame can be decoded.
>>
>>> So running fluster with `-j 1` gives me the expected 129/135 passed tests,
>>> but `-j 8` will start failing all tests after the first fail (well, first
>>> fail because of decoder error).
>>
>> This was the main issue blocking rkvdec hevc, just re-confgure the mmu
>> after a frame fails to decode seem to resolve this issue.
>>
>> Biggest issue at the moment is how to properly signal iommu subsystem that
>> it should re-configure, I may have abused the flush_iotlb_all ops, since
>> that seemed closest existing hook.
>>
>> Will send an RFC to linux-iommu to collect input on how to best signal
>> iommu subsystem that the mmu has been reset by an external event and now
>> need to be re-configured.
> 
> Do you mind if I go ahead and send your iommu flush_iotlb_all patch upstream to 
> start the discussion ? I'd love for this patch set to move along and that's 
> kind of a blocker right now.

Sorry for the delay, will try to get something on list this weekend or
early next week.

I have reworked our LibreELEC FFmpeg v4l2-request patches [1] to help
test the iommu change for rkvdec1 hevc, and have now patches that should
be more upstreamable.

Testing the iommu flush with hevc on rkvdec1 has shown that the iommu
change seem to work for most part, however there was still situations
when parallel jobs and/or threads was used with fluster that some
unrelated tests could fail, with a single job it improved hevc score to
137 of 145, so it is an improvement but for perfect multi-stream
decoding hard-reset handling may also be needed in future.

Also running fluster VP9 test suite also cause my RK3399 board to freeze
with a kernel panic in iommu irq handler, have not yet tried to dig any
deeper, if it is an old issue or a new issue due to the new iommu flush.

[1] https://github.com/FFmpeg/FFmpeg/compare/master...Kwiboo:FFmpeg:v4l2request-2024-v2

Regards,
Jonas

> 
> Detlev.
> 
>