* Re: [PATCH 0/4] ASoC: meson: aiu: align I2S design to the AXG one
From: Mark Brown @ 2026-05-21 11:36 UTC (permalink / raw)
To: Jerome Brunet, Liam Girdwood, Jaroslav Kysela, Takashi Iwai,
Neil Armstrong, Kevin Hilman, Martin Blumenstingl, Valerio Setti
Cc: linux-kernel, linux-sound, linux-arm-kernel, linux-amlogic
In-Reply-To: <20260515-reshape-aiu-as-axg-v1-0-53b457784ff3@baylibre.com>
On Fri, 15 May 2026 17:10:36 +0200, Valerio Setti wrote:
> ASoC: meson: aiu: align I2S design to the AXG one
>
> This is the first follow-up patch series based on RFC [1]. The goal here
> is simply to reshape Amlogic GX's AIU implementation for I2S to follow
> the same design as in AXG's TDM. Keeping the same design allows for
> unifying the two platform implementations in the future.
>
> [...]
Applied to
https://git.kernel.org/pub/scm/linux/kernel/git/broonie/sound.git for-7.2
Thanks!
[1/4] ASoC: meson: gx: add gx-formatter and gx-interface
https://git.kernel.org/broonie/sound/c/4efe33e7a2f0
[2/4] ASoC: meson: aiu-encoder-i2s: use gx_iface and gx_stream structures
https://git.kernel.org/broonie/sound/c/3383866c1b77
[3/4] ASoC: meson: aiu: introduce I2S output formatter
https://git.kernel.org/broonie/sound/c/df6057a25c52
[4/4] ASoC: meson: aiu: use aiu-formatter-i2s to format I2S output data
https://git.kernel.org/broonie/sound/c/ca3543cf247b
All being well this means that it will be integrated into the linux-next
tree (usually sometime in the next 24 hours) and sent to Linus during
the next merge window (or sooner if it is a bug fix), however if
problems are discovered then the patch may be dropped or reverted.
You may get further e-mails resulting from automated or manual testing
and review of the tree, please engage with people reporting problems and
send followup patches addressing any issues that are reported if needed.
If any updates are required or you are submitting further changes they
should be sent as incremental updates against current git, existing
patches will not be replaced.
Please add any relevant lists and maintainers to the CCs when replying
to this mail.
Thanks,
Mark
^ permalink raw reply
* Re: [PATCH v4] ASoC: dt-bindings: imx-card: Complete the full list of supported DAI formats
From: Mark Brown @ 2026-05-21 23:18 UTC (permalink / raw)
To: lgirdwood, robh, krzk+dt, conor+dt, Frank.Li, shengjiu.wang,
s.hauer, kernel, festevam, linux-sound, devicetree, imx,
linux-arm-kernel, linux-kernel, Chancel Liu
In-Reply-To: <20260331012450.1298115-1-chancel.liu@nxp.com>
On Tue, 31 Mar 2026 10:24:50 +0900, Chancel Liu wrote:
> ASoC: dt-bindings: imx-card: Complete the full list of supported DAI formats
Applied to
https://git.kernel.org/pub/scm/linux/kernel/git/broonie/sound.git for-7.2
Thanks!
[1/1] ASoC: dt-bindings: imx-card: Complete the full list of supported DAI formats
https://git.kernel.org/broonie/sound/c/caa7a3711862
All being well this means that it will be integrated into the linux-next
tree (usually sometime in the next 24 hours) and sent to Linus during
the next merge window (or sooner if it is a bug fix), however if
problems are discovered then the patch may be dropped or reverted.
You may get further e-mails resulting from automated or manual testing
and review of the tree, please engage with people reporting problems and
send followup patches addressing any issues that are reported if needed.
If any updates are required or you are submitting further changes they
should be sent as incremental updates against current git, existing
patches will not be replaced.
Please add any relevant lists and maintainers to the CCs when replying
to this mail.
Thanks,
Mark
^ permalink raw reply
* Re: (subset) [PATCH v4 0/6] regulator: mt6359: cleanup and add supplies
From: Mark Brown @ 2026-05-20 16:17 UTC (permalink / raw)
To: Liam Girdwood, Lee Jones, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, AngeloGioacchino Del Regno,
Chen-Yu Tsai
Cc: linux-arm-kernel, linux-mediatek, devicetree
In-Reply-To: <20260514091520.2718987-1-wenst@chromium.org>
On Thu, 14 May 2026 17:15:13 +0800, Chen-Yu Tsai wrote:
> regulator: mt6359: cleanup and add supplies
>
> Hi,
>
> This is v4 of my "MT6359 PMIC cleanup and add supplies" series. This
> version addresses review comments from Sashiko.
>
> [...]
Applied to
https://git.kernel.org/pub/scm/linux/kernel/git/broonie/regulator.git for-7.2
Thanks!
[2/6] regulator: dt-bindings: mt6359: Drop regulator-name pattern restrictions
https://git.kernel.org/broonie/regulator/c/cdc517688ffa
[3/6] regulator: dt-bindings: mt6359: Deprecate bogus vcn33_[12]_* split regulators
https://git.kernel.org/broonie/regulator/c/beb4fe279989
[4/6] regulator: mt6359: const-ify regulator descriptions
https://git.kernel.org/broonie/regulator/c/eb17a319f1c9
[5/6] regulator: mt6359: Add regulator supply names
https://git.kernel.org/broonie/regulator/c/10be8fc1d534
[6/6] regulator: mt6359: Add proper ldo_vcn33_[12] regulators
https://git.kernel.org/broonie/regulator/c/fb6a6297acfa
All being well this means that it will be integrated into the linux-next
tree (usually sometime in the next 24 hours) and sent to Linus during
the next merge window (or sooner if it is a bug fix), however if
problems are discovered then the patch may be dropped or reverted.
You may get further e-mails resulting from automated or manual testing
and review of the tree, please engage with people reporting problems and
send followup patches addressing any issues that are reported if needed.
If any updates are required or you are submitting further changes they
should be sent as incremental updates against current git, existing
patches will not be replaced.
Please add any relevant lists and maintainers to the CCs when replying
to this mail.
Thanks,
Mark
^ permalink raw reply
* [PATCH 14/14] arm64: dts: imx8mp: add VC8000E encoder node
From: Paul Kocialkowski @ 2026-05-22 10:16 UTC (permalink / raw)
To: devicetree, imx, linux-arm-kernel, linux-kernel, linux-media
Cc: Rob Herring, Krzysztof Kozlowski, Conor Dooley, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Nicolas Dufresne,
Benjamin Gaignard, Philipp Zabel, Mauro Carvalho Chehab,
Hans Verkuil, Marco Felsch, Michael Tretter
In-Reply-To: <20260522101653.2565125-1-paulk@sys-base.io>
From: Marco Felsch <m.felsch@pengutronix.de>
Add support for the Versilicon VC8000E multi-codec stateless encoder.
The IP integrated on the i.MX8MP supports H.264 and H.265 encoding.
Signed-off-by: Marco Felsch <m.felsch@pengutronix.de>
---
arch/arm64/boot/dts/freescale/imx8mp.dtsi | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/arch/arm64/boot/dts/freescale/imx8mp.dtsi b/arch/arm64/boot/dts/freescale/imx8mp.dtsi
index a3de6604e29f..4e63c2b16c1a 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mp.dtsi
@@ -2290,6 +2290,17 @@ vpu_g2: video-codec@38310000 {
power-domains = <&vpumix_blk_ctrl IMX8MP_VPUBLK_PD_G2>;
};
+ vpu_vc8000e: video-codec@38320000 {
+ compatible = "nxp,imx8mp-vpu-vc8000e";
+ reg = <0x38320000 0x10000>;
+ interrupts = <GIC_SPI 30 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&clk IMX8MP_CLK_VPU_VC8KE_ROOT>;
+ assigned-clocks = <&clk IMX8MP_CLK_VPU_VC8000E>;
+ assigned-clock-parents = <&clk IMX8MP_SYS_PLL1_800M>;
+ assigned-clock-rates = <400000000>;
+ power-domains = <&vpumix_blk_ctrl IMX8MP_VPUBLK_PD_VC8000E>;
+ };
+
vpumix_blk_ctrl: blk-ctrl@38330000 {
compatible = "fsl,imx8mp-vpu-blk-ctrl", "syscon";
reg = <0x38330000 0x100>;
--
2.53.0
^ permalink raw reply related
* [PATCH 13/14] media: verilisicon: imx8m: Add support for the VC8000E on i.MX8MP
From: Paul Kocialkowski @ 2026-05-22 10:16 UTC (permalink / raw)
To: devicetree, imx, linux-arm-kernel, linux-kernel, linux-media
Cc: Rob Herring, Krzysztof Kozlowski, Conor Dooley, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Nicolas Dufresne,
Benjamin Gaignard, Philipp Zabel, Mauro Carvalho Chehab,
Hans Verkuil, Marco Felsch, Michael Tretter, Paul Kocialkowski
In-Reply-To: <20260522101653.2565125-1-paulk@sys-base.io>
Add the required platform-specific bits for driving the VC8000E found
on the NXP i.MX8MP SoC.
Signed-off-by: Paul Kocialkowski <paulk@sys-base.io>
Signed-off-by: Marco Felsch <m.felsch@pengutronix.de>
Co-authored-by: Marco Felsch <m.felsch@pengutronix.de>
---
.../media/platform/verisilicon/hantro_drv.c | 1 +
.../media/platform/verisilicon/hantro_hw.h | 1 +
.../media/platform/verisilicon/imx8m_vpu_hw.c | 113 ++++++++++++++++++
3 files changed, 115 insertions(+)
diff --git a/drivers/media/platform/verisilicon/hantro_drv.c b/drivers/media/platform/verisilicon/hantro_drv.c
index 2de27f0a2be0..540e3b647fe4 100644
--- a/drivers/media/platform/verisilicon/hantro_drv.c
+++ b/drivers/media/platform/verisilicon/hantro_drv.c
@@ -874,6 +874,7 @@ static const struct of_device_id of_hantro_match[] = {
#endif
#ifdef CONFIG_VIDEO_HANTRO_IMX8M
{ .compatible = "nxp,imx8mm-vpu-g1", .data = &imx8mm_vpu_g1_variant, },
+ { .compatible = "nxp,imx8mp-vpu-vc8000e", .data = &imx8mp_vpu_vc8000e_variant, },
{ .compatible = "nxp,imx8mq-vpu", .data = &imx8mq_vpu_variant, },
{ .compatible = "nxp,imx8mq-vpu-g1", .data = &imx8mq_vpu_g1_variant },
{ .compatible = "nxp,imx8mq-vpu-g2", .data = &imx8mq_vpu_g2_variant },
diff --git a/drivers/media/platform/verisilicon/hantro_hw.h b/drivers/media/platform/verisilicon/hantro_hw.h
index a0c752ef44dd..5f79fb401da5 100644
--- a/drivers/media/platform/verisilicon/hantro_hw.h
+++ b/drivers/media/platform/verisilicon/hantro_hw.h
@@ -420,6 +420,7 @@ enum hantro_enc_fmt {
ROCKCHIP_VPU_ENC_FMT_UYVY422 = 3,
};
+extern const struct hantro_variant imx8mp_vpu_vc8000e_variant;
extern const struct hantro_variant imx8mm_vpu_g1_variant;
extern const struct hantro_variant imx8mq_vpu_g1_variant;
extern const struct hantro_variant imx8mq_vpu_g2_variant;
diff --git a/drivers/media/platform/verisilicon/imx8m_vpu_hw.c b/drivers/media/platform/verisilicon/imx8m_vpu_hw.c
index f9f276385c11..50ce4a5f979d 100644
--- a/drivers/media/platform/verisilicon/imx8m_vpu_hw.c
+++ b/drivers/media/platform/verisilicon/imx8m_vpu_hw.c
@@ -234,6 +234,96 @@ static const struct hantro_fmt imx8m_vpu_g2_dec_fmts[] = {
},
};
+static const struct hantro_fmt imx8mp_vc8000e_fmts[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_YUV420M,
+ .codec_mode = HANTRO_MODE_NONE,
+ .enc_fmt = ROCKCHIP_VPU_ENC_FMT_YUV420P,
+ .frmsize = {
+ .min_width = FMT_MIN_WIDTH,
+ .max_width = FMT_FHD_WIDTH,
+ .step_width = MB_DIM,
+ .min_height = FMT_MIN_HEIGHT,
+ .max_height = FMT_FHD_HEIGHT,
+ /* TODO: implement dummy reads to relax size restrictions */
+ .step_height = 64,
+ },
+ }, {
+ .fourcc = V4L2_PIX_FMT_YUV420,
+ .codec_mode = HANTRO_MODE_NONE,
+ .enc_fmt = ROCKCHIP_VPU_ENC_FMT_YUV420P,
+ .frmsize = {
+ .min_width = FMT_MIN_WIDTH,
+ .max_width = FMT_FHD_WIDTH,
+ .step_width = MB_DIM,
+ .min_height = FMT_MIN_HEIGHT,
+ .max_height = FMT_FHD_HEIGHT,
+ /* TODO: implement dummy reads to relax size restrictions */
+ .step_height = 64,
+ },
+ }, {
+ .fourcc = V4L2_PIX_FMT_NV12M,
+ .codec_mode = HANTRO_MODE_NONE,
+ .enc_fmt = ROCKCHIP_VPU_ENC_FMT_YUV420SP,
+ .frmsize = {
+ .min_width = FMT_MIN_WIDTH,
+ .max_width = FMT_FHD_WIDTH,
+ .step_width = MB_DIM,
+ .min_height = FMT_MIN_HEIGHT,
+ .max_height = FMT_FHD_HEIGHT,
+ .step_height = MB_DIM,
+ },
+ }, {
+ .fourcc = V4L2_PIX_FMT_NV12,
+ .codec_mode = HANTRO_MODE_NONE,
+ .enc_fmt = ROCKCHIP_VPU_ENC_FMT_YUV420SP,
+ .frmsize = {
+ .min_width = FMT_MIN_WIDTH,
+ .max_width = FMT_FHD_WIDTH,
+ .step_width = MB_DIM,
+ .min_height = FMT_MIN_HEIGHT,
+ .max_height = FMT_FHD_HEIGHT,
+ .step_height = MB_DIM,
+ },
+ }, {
+ .fourcc = V4L2_PIX_FMT_YUYV,
+ .codec_mode = HANTRO_MODE_NONE,
+ .enc_fmt = ROCKCHIP_VPU_ENC_FMT_YUYV422,
+ .frmsize = {
+ .min_width = FMT_MIN_WIDTH,
+ .max_width = FMT_FHD_WIDTH,
+ .step_width = MB_DIM,
+ .min_height = FMT_MIN_HEIGHT,
+ .max_height = FMT_FHD_HEIGHT,
+ .step_height = MB_DIM,
+ },
+ }, {
+ .fourcc = V4L2_PIX_FMT_UYVY,
+ .codec_mode = HANTRO_MODE_NONE,
+ .enc_fmt = ROCKCHIP_VPU_ENC_FMT_UYVY422,
+ .frmsize = {
+ .min_width = FMT_MIN_WIDTH,
+ .max_width = FMT_FHD_WIDTH,
+ .step_width = MB_DIM,
+ .min_height = FMT_MIN_HEIGHT,
+ .max_height = FMT_FHD_HEIGHT,
+ .step_height = MB_DIM,
+ },
+ }, {
+ .fourcc = V4L2_PIX_FMT_H264_SLICE,
+ .codec_mode = HANTRO_MODE_H264_ENC,
+ .max_depth = 2,
+ .frmsize = {
+ .min_width = FMT_MIN_WIDTH,
+ .max_width = FMT_FHD_WIDTH,
+ .step_width = MB_DIM,
+ .min_height = FMT_MIN_HEIGHT,
+ .max_height = FMT_FHD_HEIGHT,
+ .step_height = MB_DIM,
+ },
+ },
+};
+
static int imx8mq_vpu_hw_init(struct hantro_dev *vpu)
{
vpu->ctrl_base = vpu->reg_bases[vpu->variant->num_regs - 1];
@@ -305,6 +395,15 @@ static const struct hantro_codec_ops imx8mq_vpu_g2_codec_ops[] = {
},
};
+static const struct hantro_codec_ops imx8mp_vpu_vc8000e_codec_ops[] = {
+ [HANTRO_MODE_H264_ENC] = {
+ .run = hantro_vc8000e_h264_enc_run,
+ .done = hantro_vc8000e_h264_enc_done,
+ .init = hantro_vc8000e_h264_enc_init,
+ .exit = hantro_vc8000e_h264_enc_exit,
+ },
+};
+
/*
* VPU variants.
*/
@@ -317,6 +416,10 @@ static const struct hantro_irq imx8mq_g2_irqs[] = {
{ "g2", hantro_g2_irq },
};
+static const struct hantro_irq imx8mp_vc8000e_irqs[] = {
+ { "vc8000e", hantro_vc8000e_irq },
+};
+
static const char * const imx8mq_clk_names[] = { "g1", "g2", "bus" };
static const char * const imx8mq_reg_names[] = { "g1", "g2", "ctrl" };
static const char * const imx8mq_g1_clk_names[] = { "g1" };
@@ -382,3 +485,13 @@ const struct hantro_variant imx8mm_vpu_g1_variant = {
.clk_names = imx8mq_g1_clk_names,
.num_clocks = ARRAY_SIZE(imx8mq_g1_clk_names),
};
+
+const struct hantro_variant imx8mp_vpu_vc8000e_variant = {
+ .enc_fmts = imx8mp_vc8000e_fmts,
+ .num_enc_fmts = ARRAY_SIZE(imx8mp_vc8000e_fmts),
+ .codec = HANTRO_H264_ENCODER,
+ .codec_ops = imx8mp_vpu_vc8000e_codec_ops,
+ .irqs = imx8mp_vc8000e_irqs,
+ .num_irqs = ARRAY_SIZE(imx8mp_vc8000e_irqs),
+ .num_clocks = 1,
+};
--
2.53.0
^ permalink raw reply related
* [PATCH 12/14] media: verisilicon: Add support for the VC8000E H.264 encoder
From: Paul Kocialkowski @ 2026-05-22 10:16 UTC (permalink / raw)
To: devicetree, imx, linux-arm-kernel, linux-kernel, linux-media
Cc: Rob Herring, Krzysztof Kozlowski, Conor Dooley, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Nicolas Dufresne,
Benjamin Gaignard, Philipp Zabel, Mauro Carvalho Chehab,
Hans Verkuil, Marco Felsch, Michael Tretter, Paul Kocialkowski
In-Reply-To: <20260522101653.2565125-1-paulk@sys-base.io>
This introduces support for the Hantro VC8000E H.264 encoder,
a stateless hardware encoder from Verisilicon.
This driver is the first user of the V4L2 H.264 stateless encoding
core and uAPI.
The hardware needs to be programmed in a sequential way (registers
written in ascending offset order) so the registers are programmed
in a memory buffer first and copied in order at the end.
They are described as packed structured for convenience and should
also contain all required definitions for HEVC and JPEG support.
Only a single reference frame is currently supported, even though
the registers seems to indicate that two could be supported.
More research and investigation is needed to operate two.
The lambda tables are currently hardcoded to generally reasonable
values but could be recalculated for each target QP to give more
accurate results.
Most other features of the encoder are supported, except ROI and
intra areas support which are currently disabled.
Signed-off-by: Paul Kocialkowski <paulk@sys-base.io>
---
drivers/media/platform/verisilicon/Kconfig | 1 +
drivers/media/platform/verisilicon/Makefile | 2 +
drivers/media/platform/verisilicon/hantro.h | 14 +
.../media/platform/verisilicon/hantro_drv.c | 152 +-
.../media/platform/verisilicon/hantro_h264.c | 6 +-
.../media/platform/verisilicon/hantro_hw.h | 27 +
.../media/platform/verisilicon/hantro_v4l2.c | 6 +
.../platform/verisilicon/hantro_vc8000e.c | 68 +
.../verisilicon/hantro_vc8000e_h264_enc.c | 883 +++++++
.../verisilicon/hantro_vc8000e_regs.h | 2129 +++++++++++++++++
10 files changed, 3279 insertions(+), 9 deletions(-)
create mode 100644 drivers/media/platform/verisilicon/hantro_vc8000e.c
create mode 100644 drivers/media/platform/verisilicon/hantro_vc8000e_h264_enc.c
create mode 100644 drivers/media/platform/verisilicon/hantro_vc8000e_regs.h
diff --git a/drivers/media/platform/verisilicon/Kconfig b/drivers/media/platform/verisilicon/Kconfig
index 3272a24db71d..ae186a848e24 100644
--- a/drivers/media/platform/verisilicon/Kconfig
+++ b/drivers/media/platform/verisilicon/Kconfig
@@ -12,6 +12,7 @@ config VIDEO_HANTRO
select VIDEOBUF2_VMALLOC
select V4L2_MEM2MEM_DEV
select V4L2_H264
+ select V4L2_H264_ENC
select V4L2_JPEG_HELPER
select V4L2_VP9
help
diff --git a/drivers/media/platform/verisilicon/Makefile b/drivers/media/platform/verisilicon/Makefile
index f6f019d04ff0..caa32de29ab7 100644
--- a/drivers/media/platform/verisilicon/Makefile
+++ b/drivers/media/platform/verisilicon/Makefile
@@ -7,6 +7,8 @@ hantro-vpu-y += \
hantro_v4l2.o \
hantro_postproc.o \
hantro_h1_jpeg_enc.o \
+ hantro_vc8000e.o \
+ hantro_vc8000e_h264_enc.o \
hantro_g1.o \
hantro_g1_h264_dec.o \
hantro_g1_mpeg2_dec.o \
diff --git a/drivers/media/platform/verisilicon/hantro.h b/drivers/media/platform/verisilicon/hantro.h
index badd0b13988c..b21c16dd6c1b 100644
--- a/drivers/media/platform/verisilicon/hantro.h
+++ b/drivers/media/platform/verisilicon/hantro.h
@@ -32,6 +32,7 @@ struct hantro_codec_ops;
struct hantro_postproc_ops;
#define HANTRO_JPEG_ENCODER BIT(0)
+#define HANTRO_H264_ENCODER BIT(1)
#define HANTRO_ENCODERS 0x0000ffff
#define HANTRO_MPEG2_DECODER BIT(16)
#define HANTRO_VP8_DECODER BIT(17)
@@ -107,6 +108,7 @@ struct hantro_variant {
* enum hantro_codec_mode - codec operating mode.
* @HANTRO_MODE_NONE: No operating mode. Used for RAW video formats.
* @HANTRO_MODE_JPEG_ENC: JPEG encoder.
+ * @HANTRO_MODE_H264_ENC: H264 encoder.
* @HANTRO_MODE_H264_DEC: H264 decoder.
* @HANTRO_MODE_MPEG2_DEC: MPEG-2 decoder.
* @HANTRO_MODE_VP8_DEC: VP8 decoder.
@@ -117,6 +119,7 @@ struct hantro_variant {
enum hantro_codec_mode {
HANTRO_MODE_NONE = -1,
HANTRO_MODE_JPEG_ENC,
+ HANTRO_MODE_H264_ENC,
HANTRO_MODE_H264_DEC,
HANTRO_MODE_MPEG2_DEC,
HANTRO_MODE_VP8_DEC,
@@ -272,6 +275,7 @@ struct hantro_ctx {
/* Specific for particular codec modes. */
union {
struct hantro_h264_dec_hw_ctx h264_dec;
+ struct hantro_h264_enc_hw_ctx h264_enc;
struct hantro_mpeg2_dec_hw_ctx mpeg2_dec;
struct hantro_vp8_dec_hw_ctx vp8_dec;
struct hantro_hevc_dec_hw_ctx hevc_dec;
@@ -466,6 +470,16 @@ static __always_inline void hantro_reg_write_relaxed(struct hantro_dev *vpu,
vdpu_write_relaxed(vpu, vdpu_read_mask(vpu, reg, val), reg->base);
}
+static __always_inline void hantro_io_copy(void __iomem *dst, void *src,
+ size_t size)
+{
+#ifdef CONFIG_ARM64
+ __iowrite32_copy(dst, src, size / 4);
+#else
+ memcpy_toio(dst, src, size);
+#endif
+}
+
void *hantro_get_ctrl(struct hantro_ctx *ctx, u32 id);
dma_addr_t hantro_get_ref(struct hantro_ctx *ctx, u64 ts);
diff --git a/drivers/media/platform/verisilicon/hantro_drv.c b/drivers/media/platform/verisilicon/hantro_drv.c
index d798ba361b25..2de27f0a2be0 100644
--- a/drivers/media/platform/verisilicon/hantro_drv.c
+++ b/drivers/media/platform/verisilicon/hantro_drv.c
@@ -420,6 +420,8 @@ static const struct v4l2_ctrl_ops hantro_av1_ctrl_ops = {
V4L2_JPEG_ACTIVE_MARKER_DHT)
static const struct hantro_ctrl controls[] = {
+ /* JPEG Encoder */
+
{
.codec = HANTRO_JPEG_ENCODER,
.cfg = {
@@ -446,7 +448,127 @@ static const struct hantro_ctrl controls[] = {
*/
.flags = V4L2_CTRL_FLAG_READ_ONLY,
},
+ },
+
+ /* H.264 Encoder */
+
+ {
+ .codec = HANTRO_H264_ENCODER,
+ .cfg = {
+ .id = V4L2_CID_STATELESS_H264_ENCODE_PARAMS,
+ },
+ }, {
+ .codec = HANTRO_H264_ENCODER,
+ .cfg = {
+ .id = V4L2_CID_STATELESS_H264_SPS,
+ },
+ }, {
+ .codec = HANTRO_H264_ENCODER,
+ .cfg = {
+ .id = V4L2_CID_STATELESS_H264_PPS,
+ },
+ }, {
+ .codec = HANTRO_H264_ENCODER,
+ .cfg = {
+ .id = V4L2_CID_STATELESS_H264_START_CODE,
+ .min = V4L2_STATELESS_H264_START_CODE_NONE,
+ .max = V4L2_STATELESS_H264_START_CODE_ANNEX_B,
+ .def = V4L2_STATELESS_H264_START_CODE_ANNEX_B,
+ },
+ }, {
+ .codec = HANTRO_H264_ENCODER,
+ .cfg = {
+ .id = V4L2_CID_MPEG_VIDEO_AU_DELIMITER,
+ .step = 1,
+ .min = 0,
+ .max = 1,
+ .def = 0,
+ },
+ }, {
+ .codec = HANTRO_H264_ENCODER,
+ .cfg = {
+ .id = V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR,
+ .step = 1,
+ .min = 0,
+ .max = 1,
+ .def = 0,
+ },
+ }, {
+ .codec = HANTRO_H264_ENCODER,
+ .cfg = {
+ .id = V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE,
+ .step = 1,
+ .min = 0,
+ .max = 1,
+ .def = 0,
+ },
+ }, {
+ .codec = HANTRO_H264_ENCODER,
+ .cfg = {
+ .id = V4L2_CID_MPEG_VIDEO_H264_MIN_QP,
+ .step = 1,
+ .min = 0,
+ .max = 51,
+ .def = 0,
+ },
+ }, {
+ .codec = HANTRO_H264_ENCODER,
+ .cfg = {
+ .id = V4L2_CID_MPEG_VIDEO_H264_MAX_QP,
+ .step = 1,
+ .min = 0,
+ .max = 51,
+ .def = 51,
+ },
+ }, {
+ .codec = HANTRO_H264_ENCODER,
+ .cfg = {
+ .id = V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP,
+ .step = 1,
+ .min = 0,
+ .max = 51,
+ .def = 24,
+ },
}, {
+ .codec = HANTRO_H264_ENCODER,
+ .cfg = {
+ .id = V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP,
+ .step = 1,
+ .min = 0,
+ .max = 51,
+ .def = 28,
+ },
+ }, {
+ .codec = HANTRO_H264_ENCODER,
+ .cfg = {
+ .id = V4L2_CID_MPEG_VIDEO_BITRATE_MODE,
+ .min = V4L2_MPEG_VIDEO_BITRATE_MODE_CBR,
+ .max = V4L2_MPEG_VIDEO_BITRATE_MODE_CQ,
+ .def = V4L2_MPEG_VIDEO_BITRATE_MODE_CBR,
+ },
+ }, {
+ .codec = HANTRO_H264_ENCODER,
+ .cfg = {
+ .id = V4L2_CID_MPEG_VIDEO_CONSTANT_QUALITY,
+ .step = 1,
+ .min = 0,
+ .max = 100,
+ .def = 80,
+ },
+ }, {
+ .codec = HANTRO_H264_ENCODER,
+ .cfg = {
+ .id = V4L2_CID_MPEG_VIDEO_BITRATE,
+ .step = 1,
+ .min = 1000,
+ .max = 96000000,
+ .def = 2000000,
+ },
+ },
+
+ /* MPEG-2 Decoder */
+
+ {
.codec = HANTRO_MPEG2_DECODER,
.cfg = {
.id = V4L2_CID_STATELESS_MPEG2_SEQUENCE,
@@ -461,7 +583,11 @@ static const struct hantro_ctrl controls[] = {
.cfg = {
.id = V4L2_CID_STATELESS_MPEG2_QUANTISATION,
},
- }, {
+ },
+
+ /* VP8 Decoder */
+
+ {
.codec = HANTRO_VP8_DECODER,
.cfg = {
.id = V4L2_CID_STATELESS_VP8_FRAME,
@@ -471,7 +597,11 @@ static const struct hantro_ctrl controls[] = {
.cfg = {
.id = V4L2_CID_STATELESS_H264_DECODE_PARAMS,
},
- }, {
+ },
+
+ /* H.264 Decoder */
+
+ {
.codec = HANTRO_H264_DECODER,
.cfg = {
.id = V4L2_CID_STATELESS_H264_SPS,
@@ -513,7 +643,11 @@ static const struct hantro_ctrl controls[] = {
BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED),
.def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
}
- }, {
+ },
+
+ /* HEVC Decoder */
+
+ {
.codec = HANTRO_HEVC_DECODER,
.cfg = {
.id = V4L2_CID_STATELESS_HEVC_DECODE_MODE,
@@ -565,7 +699,11 @@ static const struct hantro_ctrl controls[] = {
.cfg = {
.id = V4L2_CID_STATELESS_HEVC_SCALING_MATRIX,
},
- }, {
+ },
+
+ /* VP9 Decoder */
+
+ {
.codec = HANTRO_VP9_DECODER,
.cfg = {
.id = V4L2_CID_STATELESS_VP9_FRAME,
@@ -576,7 +714,11 @@ static const struct hantro_ctrl controls[] = {
.cfg = {
.id = V4L2_CID_STATELESS_VP9_COMPRESSED_HDR,
},
- }, {
+ },
+
+ /* AV1 Decoder */
+
+ {
.codec = HANTRO_AV1_DECODER,
.cfg = {
.id = V4L2_CID_STATELESS_AV1_FRAME,
diff --git a/drivers/media/platform/verisilicon/hantro_h264.c b/drivers/media/platform/verisilicon/hantro_h264.c
index 2414782f1eb6..d71e041010ad 100644
--- a/drivers/media/platform/verisilicon/hantro_h264.c
+++ b/drivers/media/platform/verisilicon/hantro_h264.c
@@ -453,13 +453,11 @@ int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx)
if (WARN_ON(!ctrls->decode))
return -EINVAL;
- ctrls->sps =
- hantro_get_ctrl(ctx, V4L2_CID_STATELESS_H264_SPS);
+ ctrls->sps = hantro_get_ctrl(ctx, V4L2_CID_STATELESS_H264_SPS);
if (WARN_ON(!ctrls->sps))
return -EINVAL;
- ctrls->pps =
- hantro_get_ctrl(ctx, V4L2_CID_STATELESS_H264_PPS);
+ ctrls->pps = hantro_get_ctrl(ctx, V4L2_CID_STATELESS_H264_PPS);
if (WARN_ON(!ctrls->pps))
return -EINVAL;
diff --git a/drivers/media/platform/verisilicon/hantro_hw.h b/drivers/media/platform/verisilicon/hantro_hw.h
index c9b6556f8b2b..a0c752ef44dd 100644
--- a/drivers/media/platform/verisilicon/hantro_hw.h
+++ b/drivers/media/platform/verisilicon/hantro_hw.h
@@ -12,11 +12,13 @@
#include <linux/interrupt.h>
#include <linux/v4l2-controls.h>
#include <media/v4l2-ctrls.h>
+#include <media/v4l2-h264-enc.h>
#include <media/v4l2-vp9.h>
#include <media/videobuf2-core.h>
#include "rockchip_av1_entropymode.h"
#include "rockchip_av1_filmgrain.h"
+#include "hantro_vc8000e_regs.h"
#define DEC_8190_ALIGN_MASK 0x07U
@@ -69,6 +71,22 @@ struct hantro_aux_buf {
unsigned long attrs;
};
+struct hantro_vc8000e_rec_buf {
+ struct hantro_aux_buf luma;
+ struct hantro_aux_buf chroma;
+ struct hantro_aux_buf luma_4n;
+ struct hantro_aux_buf colctbs;
+};
+
+struct hantro_h264_enc_hw_ctx {
+ struct v4l2_h264_enc enc;
+ struct hantro_aux_buf nal_tbl;
+
+ union {
+ struct hantro_vc8000e_regs vc8000e_regs;
+ };
+};
+
/* Max. number of entries in the DPB (HW limitation). */
#define HANTRO_H264_DPB_SIZE 16
@@ -451,6 +469,15 @@ int hantro_g1_h264_dec_run(struct hantro_ctx *ctx);
int hantro_h264_dec_init(struct hantro_ctx *ctx);
void hantro_h264_dec_exit(struct hantro_ctx *ctx);
+irqreturn_t hantro_vc8000e_irq(int irq, void *dev_id);
+int hantro_vc8000e_h264_enc_init(struct hantro_ctx *ctx);
+void hantro_vc8000e_h264_enc_exit(struct hantro_ctx *ctx);
+void hantro_vc8000e_h264_enc_done(struct hantro_ctx *ctx);
+int hantro_vc8000e_h264_enc_run(struct hantro_ctx *ctx);
+int hantro_h264_enc_prepare_run(struct hantro_ctx *ctx);
+int hantro_h264_enc_init(struct hantro_ctx *ctx);
+void hantro_h264_enc_exit(struct hantro_ctx *ctx);
+
int hantro_hevc_dec_init(struct hantro_ctx *ctx);
void hantro_hevc_dec_exit(struct hantro_ctx *ctx);
int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx);
diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index 2125f2913d9a..96b72c1f37db 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -629,6 +629,12 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx,
hantro_get_format_depth(pix_mp->pixelformat),
need_postproc);
+ /* Propagate dimensions for encoders. */
+ if (ctx->is_encoder) {
+ ctx->dst_fmt.width = pix_mp->width;
+ ctx->dst_fmt.height = pix_mp->height;
+ }
+
/* Colorimetry information are always propagated. */
ctx->dst_fmt.colorspace = pix_mp->colorspace;
ctx->dst_fmt.ycbcr_enc = pix_mp->ycbcr_enc;
diff --git a/drivers/media/platform/verisilicon/hantro_vc8000e.c b/drivers/media/platform/verisilicon/hantro_vc8000e.c
new file mode 100644
index 000000000000..ebd726e13460
--- /dev/null
+++ b/drivers/media/platform/verisilicon/hantro_vc8000e.c
@@ -0,0 +1,68 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Hantro VPU codec driver
+ *
+ * Copyright (C) 2025-2026 Paul Kocialkowski <paulk@sys-base.io>
+ */
+
+#include "hantro.h"
+#include "hantro_vc8000e_regs.h"
+
+irqreturn_t hantro_vc8000e_irq(int irq, void *dev_id)
+{
+ struct hantro_dev *vpu = dev_id;
+ u32 regs_buffer[HANTRO_VC8000E_SWREG_OFFSET(swreg6) / 4];
+ struct hantro_vc8000e_regs *regs =
+ (struct hantro_vc8000e_regs *)regs_buffer;
+ enum vb2_buffer_state state;
+
+ hantro_vc8000e_swreg_read(vpu, regs, swreg1);
+
+ pr_debug("+ hantro-vc8000e-irq: %#x\n",
+ regs_buffer[HANTRO_VC8000E_SWREG_OFFSET(swreg1) / 4]);
+
+ if (regs->swreg1.irq)
+ pr_debug(" - irq\n");
+ if (regs->swreg1.frame_rdy_status)
+ pr_debug(" - frame ready\n");
+ if (regs->swreg1.bus_error_status)
+ pr_debug(" - bus error\n");
+ if (regs->swreg1.sw_reset)
+ pr_debug(" - sw reset\n");
+ if (regs->swreg1.buffer_full)
+ pr_debug(" - buffer full\n");
+ if (regs->swreg1.timeout)
+ pr_debug(" - timeout\n");
+ if (regs->swreg1.slice_rdy_status)
+ pr_debug(" - slice ready\n");
+ if (regs->swreg1.irq_fuse_error)
+ pr_debug(" - fuse error\n");
+ if (regs->swreg1.strm_segment_rdy_int)
+ pr_debug(" - segment ready\n");
+
+ hantro_vc8000e_swreg_read(vpu, regs, swreg4);
+ hantro_vc8000e_swreg_read(vpu, regs, swreg5);
+
+ /* Make sure to disble the encoder on error for safety. */
+ if ((regs->swreg1.bus_error_status ||
+ regs->swreg1.buffer_full ||
+ regs->swreg1.timeout ||
+ regs->swreg1.irq_fuse_error) &&
+ (regs->swreg4.mode == HANTRO_VC8000E_SWREG4_MODE_H264 ||
+ regs->swreg4.mode == HANTRO_VC8000E_SWREG4_MODE_HEVC)) {
+ regs->swreg5.enable = 0;
+ hantro_vc8000e_swreg_write(vpu, regs, swreg5);
+ }
+
+ state = regs->swreg1.frame_rdy_status ? VB2_BUF_STATE_DONE :
+ VB2_BUF_STATE_ERROR;
+
+ regs->swreg1.irq_dis = 1;
+ regs->swreg1.timeout_int = 0;
+
+ hantro_vc8000e_swreg_write(vpu, regs, swreg1);
+
+ hantro_irq_done(vpu, state);
+
+ return IRQ_HANDLED;
+}
diff --git a/drivers/media/platform/verisilicon/hantro_vc8000e_h264_enc.c b/drivers/media/platform/verisilicon/hantro_vc8000e_h264_enc.c
new file mode 100644
index 000000000000..d3522d61443d
--- /dev/null
+++ b/drivers/media/platform/verisilicon/hantro_vc8000e_h264_enc.c
@@ -0,0 +1,883 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Hantro VPU codec driver
+ *
+ * Copyright (C) 2024 Pengutronix, Marco Felsch <kernel@pengutronix.de>
+ * Copyright (C) 2025-2026 Paul Kocialkowski <paulk@sys-base.io>
+ */
+
+#include <linux/unaligned.h>
+#include <linux/delay.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/v4l2-h264-enc-rbsp.h>
+
+#include "hantro.h"
+#include "hantro_hw.h"
+#include "hantro_vc8000e_regs.h"
+
+static int
+hantro_vc8000e_h264_enc_state_constrain(struct v4l2_h264_enc *enc,
+ struct v4l2_h264_enc_state *state)
+{
+ struct v4l2_ctrl_h264_sps *sps = &state->sps;
+ struct v4l2_ctrl_h264_pps *pps = &state->pps;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+
+ /* SPS */
+
+ if (V4L2_H264_SPS_HAS_CHROMA_FORMAT(sps)) {
+ sps->chroma_format_idc = 1;
+ sps->flags &= ~V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE;
+ sps->bit_depth_luma_minus8 = 0;
+ sps->bit_depth_chroma_minus8 = 0;
+ sps->flags &= ~V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS;
+ }
+
+ /* Only one reference frame is currently supported. */
+ if (sps->max_num_ref_frames > 1)
+ sps->max_num_ref_frames = 1;
+
+ if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY)) {
+ sps->flags |= V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY;
+ sps->flags &= ~V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD;
+ }
+
+ sps->flags |= V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE;
+
+ /* PPS */
+
+ /* Only one reference frame is currently supported. */
+ pps->num_ref_idx_l0_default_active_minus1 = 0;
+ pps->num_ref_idx_l1_default_active_minus1 = 0;
+
+ /* Encode */
+
+ /* Only a single bit is available for idr_pic_id. */
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_IDR_PIC)
+ encode->idr_pic_id &= 1;
+
+ /* The hardware doesn't allow overriding the active list numbers. */
+ if (encode->flags & V4L2_H264_SLICE_FLAG_NUM_REF_IDX_ACTIVE_OVERRIDE) {
+ encode->flags &= ~V4L2_H264_SLICE_FLAG_NUM_REF_IDX_ACTIVE_OVERRIDE;
+
+ encode->num_ref_idx_l0_active_minus1 = 0;
+ encode->num_ref_idx_l1_active_minus1 = 0;
+ }
+
+ /* Only a single bit is available for cabac_init_idc. */
+ if (encode->cabac_init_idc > 1)
+ encode->cabac_init_idc = 1;
+
+ /* Only a single bit is available for disable_deblocking_filter_idc. */
+ if (encode->disable_deblocking_filter_idc > 1)
+ encode->disable_deblocking_filter_idc = 1;
+
+ return 0;
+}
+
+static int
+hantro_vc8000e_h264_enc_rec_buffer_alloc(struct v4l2_h264_enc *enc,
+ struct v4l2_h264_enc_rec_buffer *buffer)
+{
+ struct hantro_ctx *ctx = enc->private_data;
+ struct device *dev = ctx->dev->dev;
+ struct hantro_vc8000e_rec_buf *rec_buf;
+ unsigned int width_mbs, height_mbs;
+
+ width_mbs = MB_WIDTH(ctx->src_fmt.width);
+ height_mbs = MB_HEIGHT(ctx->src_fmt.height);
+
+ rec_buf = kzalloc(sizeof(*rec_buf), GFP_KERNEL);
+ if (!rec_buf)
+ goto error;
+
+ rec_buf->luma.size = ctx->src_fmt.width * ctx->src_fmt.height;
+ rec_buf->luma.cpu = dma_alloc_coherent(dev, rec_buf->luma.size,
+ &rec_buf->luma.dma, GFP_KERNEL);
+ if (!rec_buf->luma.cpu)
+ goto error;
+
+ rec_buf->chroma.size = rec_buf->luma.size / 2;
+ rec_buf->chroma.cpu = dma_alloc_coherent(dev, rec_buf->chroma.size,
+ &rec_buf->chroma.dma,
+ GFP_KERNEL);
+ if (!rec_buf->chroma.cpu)
+ goto error;
+
+ rec_buf->luma_4n.size = width_mbs * 4 * height_mbs * 4;
+ rec_buf->luma_4n.cpu = dma_alloc_coherent(dev, rec_buf->luma_4n.size,
+ &rec_buf->luma_4n.dma,
+ GFP_KERNEL);
+ if (!rec_buf->luma_4n.cpu)
+ goto error;
+
+ rec_buf->colctbs.size = DIV_ROUND_UP(width_mbs * height_mbs, 2);
+ rec_buf->colctbs.cpu = dma_alloc_coherent(dev, rec_buf->colctbs.size,
+ &rec_buf->colctbs.dma,
+ GFP_KERNEL);
+ if (!rec_buf->colctbs.cpu)
+ goto error;
+
+ buffer->private_data = rec_buf;
+
+ return 0;
+
+error:
+ if (rec_buf)
+ kfree(rec_buf);
+
+ return -ENOMEM;
+}
+
+static int
+hantro_vc8000e_h264_enc_rec_buffer_free(struct v4l2_h264_enc *enc,
+ struct v4l2_h264_enc_rec_buffer *buffer)
+{
+ struct hantro_ctx *ctx = enc->private_data;
+ struct device *dev = ctx->dev->dev;
+ struct hantro_vc8000e_rec_buf *rec_buf = buffer->private_data;
+
+ if (!rec_buf)
+ return -EINVAL;
+
+ dma_free_coherent(dev, rec_buf->luma.size, rec_buf->luma.cpu,
+ rec_buf->luma.dma);
+
+ dma_free_coherent(dev, rec_buf->luma_4n.size, rec_buf->luma_4n.cpu,
+ rec_buf->luma_4n.dma);
+
+ dma_free_coherent(dev, rec_buf->chroma.size, rec_buf->chroma.cpu,
+ rec_buf->chroma.dma);
+
+ dma_free_coherent(dev, rec_buf->colctbs.size, rec_buf->colctbs.cpu,
+ rec_buf->colctbs.dma);
+
+ kfree(rec_buf);
+ buffer->private_data = NULL;
+
+ return 0;
+}
+
+static const struct v4l2_h264_enc_ops hantro_vc8000e_h264_enc_ops = {
+ .state_constrain = hantro_vc8000e_h264_enc_state_constrain,
+ .rec_buffer_alloc = hantro_vc8000e_h264_enc_rec_buffer_alloc,
+ .rec_buffer_free = hantro_vc8000e_h264_enc_rec_buffer_free,
+};
+
+int hantro_vc8000e_h264_enc_init(struct hantro_ctx *ctx)
+{
+ struct hantro_h264_enc_hw_ctx *h264_ctx = &ctx->h264_enc;
+ struct hantro_aux_buf *nal_tbl = &h264_ctx->nal_tbl;
+ struct v4l2_h264_enc *enc = &h264_ctx->enc;
+ struct device *dev = ctx->dev->dev;
+
+ nal_tbl->size = ALIGN(MB_HEIGHT(ctx->src_fmt.height), 8);
+ nal_tbl->cpu = dma_alloc_coherent(dev, nal_tbl->size,
+ &nal_tbl->dma, GFP_KERNEL);
+ if (!nal_tbl->cpu)
+ return -ENOMEM;
+
+ enc->ops = &hantro_vc8000e_h264_enc_ops;
+ enc->private_data = ctx;
+ enc->format_mplane = &ctx->dst_fmt;
+ enc->timeperframe = &ctx->dst_timeperframe;
+ enc->ctrl_handler = &ctx->ctrl_handler;
+ enc->ref_slots_count_init = 2;
+ enc->flags = V4L2_H264_ENC_FLAG_INTER_PRED |
+ V4L2_H264_ENC_FLAG_HW_SLICE_HEADER;
+
+ return v4l2_h264_enc_init(enc);
+}
+
+void hantro_vc8000e_h264_enc_exit(struct hantro_ctx *ctx)
+{
+ struct hantro_h264_enc_hw_ctx *h264_ctx = &ctx->h264_enc;
+ struct hantro_aux_buf *nal_tbl = &h264_ctx->nal_tbl;
+ struct device *dev = ctx->dev->dev;
+
+ if (nal_tbl->cpu)
+ dma_free_coherent(dev, nal_tbl->size, nal_tbl->cpu,
+ nal_tbl->dma);
+
+ v4l2_h264_enc_exit(&h264_ctx->enc);
+}
+
+static int ref_setup(struct hantro_ctx *ctx)
+{
+ struct hantro_h264_enc_hw_ctx *h264_ctx = &ctx->h264_enc;
+ struct hantro_vc8000e_regs *regs = &h264_ctx->vc8000e_regs;
+ struct v4l2_h264_enc *enc = &h264_ctx->enc;
+ struct v4l2_h264_enc_ref *ref = &enc->ref;
+ struct v4l2_h264_enc_state *state = &enc->state_active;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ struct v4l2_h264_dpb_entry *dpb_entry;
+ struct v4l2_h264_enc_rec_buffer *buffer;
+ struct hantro_vc8000e_rec_buf *rec_buf;
+ bool ltr_present = false;
+ unsigned int index;
+
+ regs->swreg193.nal_ref_idc = encode->nal_ref_idc != 0;
+
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_LONG_TERM_REFERENCE) {
+ regs->swreg198.mark_current_longterm = 1;
+ /* We only get a single long-term reference without MMCO. */
+ regs->swreg194.cur_longtermidx = 0;
+ }
+
+ regs->swreg193.l0_used_by_next_pic0 = 1;
+ regs->swreg193.l0_used_by_next_pic1 = 1;
+ regs->swreg194.l1_used_by_next_pic0 = 1;
+ regs->swreg194.l1_used_by_next_pic1 = 1;
+
+ regs->swreg17.active_l0_cnt = ref->l0_active_count;
+ regs->swreg91.active_l1_cnt = ref->l1_active_count;
+ regs->swreg4.active_override_flag = 1;
+
+ if (ref->l0_active_count > 0) {
+ index = ref->l0[0].index;
+ dpb_entry = &ref->dpb[index];
+ buffer = &ref->buffers[index];
+ rec_buf = buffer->private_data;
+
+ regs->swreg18.refpic_recon_l0_y0 = rec_buf->luma.dma;
+ regs->swreg19.refpic_recon_l0_chroma0 = rec_buf->chroma.dma;
+ regs->swreg74.refpic_recon_l0_4n0_base = rec_buf->luma_4n.dma;
+
+ regs->swreg17.l0_used_by_curr_pic0 = 1;
+ /* TODO: diff with wrap. */
+ regs->swreg17.l0_delta_poc0 = 1;
+ regs->swreg193.l0_delta_framenum0 = 1;
+
+ if (dpb_entry->flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) {
+ regs->swreg17.l0_long_term_flag0 = 1;
+ regs->swreg198.l0_longtermidx0 = 0;
+ ltr_present = true;
+ }
+ }
+
+ if (ref->l0_active_count > 1) {
+ index = ref->l0[1].index;
+ dpb_entry = &ref->dpb[index];
+ buffer = &ref->buffers[index];
+ rec_buf = buffer->private_data;
+
+ regs->swreg20.refpic_recon_l0_y1 = rec_buf->luma.dma;
+ regs->swreg21.refpic_recon_l0_chroma1 = rec_buf->chroma.dma;
+ regs->swreg76.refpic_recon_l0_4n1_base = rec_buf->luma_4n.dma;
+
+ regs->swreg17.l0_used_by_curr_pic1 = 1;
+ /* TODO: diff with wrap. */
+ regs->swreg17.l0_delta_poc1 = 1;
+ regs->swreg193.l0_delta_framenum1 = 1;
+
+ if (dpb_entry->flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) {
+ regs->swreg17.l0_long_term_flag1 = 1;
+ regs->swreg198.l0_longtermidx1 = 0;
+ ltr_present = true;
+ }
+ }
+
+ if (ltr_present)
+ regs->swreg91.long_term_ref_pics_present_flag = 1;
+
+ return 0;
+}
+
+static int lambda_setup(struct hantro_ctx *ctx)
+{
+ struct hantro_h264_enc_hw_ctx *h264_ctx = &ctx->h264_enc;
+ struct hantro_vc8000e_regs *regs = &h264_ctx->vc8000e_regs;
+
+ /* Intra SATD */
+
+ regs->swreg125.intra_satd_lambda_0 = 0x47;
+ regs->swreg125.intra_satd_lambda_1 = 0x3f;
+ regs->swreg126.intra_satd_lambda_2 = 0x38;
+ regs->swreg126.intra_satd_lambda_3 = 0x32;
+ regs->swreg127.intra_satd_lambda_4 = 0;
+ regs->swreg127.intra_satd_lambda_5 = 0;
+ regs->swreg128.intra_satd_lambda_6 = 0;
+ regs->swreg128.intra_satd_lambda_7 = 0;
+ regs->swreg129.intra_satd_lambda_8 = 0;
+ regs->swreg129.intra_satd_lambda_9 = 0;
+ regs->swreg130.intra_satd_lambda_10 = 0;
+ regs->swreg130.intra_satd_lambda_11 = 0;
+ regs->swreg131.intra_satd_lambda_12 = 0;
+ regs->swreg131.intra_satd_lambda_13 = 0;
+ regs->swreg132.intra_satd_lambda_14 = 0;
+ regs->swreg132.intra_satd_lambda_15 = 0;
+ regs->swreg174.intra_satd_lambda_16 = 0x1c4;
+ regs->swreg174.intra_satd_lambda_17 = 0x192;
+ regs->swreg175.intra_satd_lambda_18 = 0x166;
+ regs->swreg175.intra_satd_lambda_19 = 0x13f;
+ regs->swreg176.intra_satd_lambda_20 = 0x11c;
+ regs->swreg176.intra_satd_lambda_21 = 0xfd;
+ regs->swreg177.intra_satd_lambda_22 = 0xe2;
+ regs->swreg177.intra_satd_lambda_23 = 0xc9;
+ regs->swreg178.intra_satd_lambda_24 = 0xb3;
+ regs->swreg178.intra_satd_lambda_25 = 0xa0;
+ regs->swreg179.intra_satd_lambda_26 = 0x8e;
+ regs->swreg179.intra_satd_lambda_27 = 0x7f;
+ regs->swreg180.intra_satd_lambda_28 = 0x71;
+ regs->swreg180.intra_satd_lambda_29 = 0x65;
+ regs->swreg181.intra_satd_lambda_30 = 0x5a;
+ regs->swreg181.intra_satd_lambda_31 = 0x50;
+
+ /* Inter SATD */
+
+ regs->swreg28.lambda_satd_me_0 = 0x24;
+ regs->swreg28.lambda_satd_me_1 = 0x20;
+ regs->swreg29.lambda_satd_me_2 = 0x1c;
+ regs->swreg29.lambda_satd_me_3 = 0x19;
+ regs->swreg30.lambda_satd_me_4 = 0x16;
+ regs->swreg30.lambda_satd_me_5 = 0x14;
+ regs->swreg31.lambda_satd_me_6 = 0x12;
+ regs->swreg31.lambda_satd_me_7 = 0x10;
+ regs->swreg32.lambda_satd_me_8 = 0xe;
+ regs->swreg32.lambda_satd_me_9 = 0xd;
+ regs->swreg33.lambda_satd_me_10 = 0xb;
+ regs->swreg33.lambda_satd_me_11 = 0xa;
+ regs->swreg34.lambda_satd_me_12 = 0x9;
+ regs->swreg34.lambda_satd_me_13 = 0x8;
+ regs->swreg78.lambda_satd_me_14 = 0x7;
+ regs->swreg78.lambda_satd_me_15 = 0x6;
+ regs->swreg150.lambda_satd_me_16 = 0;
+ regs->swreg150.lambda_satd_me_17 = 0;
+ regs->swreg151.lambda_satd_me_18 = 0;
+ regs->swreg151.lambda_satd_me_19 = 0;
+ regs->swreg152.lambda_satd_me_20 = 0x88;
+ regs->swreg152.lambda_satd_me_21 = 0;
+ regs->swreg153.lambda_satd_me_22 = 0;
+ regs->swreg153.lambda_satd_me_23 = 0;
+ regs->swreg154.lambda_satd_me_24 = 0;
+ regs->swreg154.lambda_satd_me_25 = 0;
+ regs->swreg155.lambda_satd_me_26 = 0;
+ regs->swreg155.lambda_satd_me_27 = 0;
+ regs->swreg156.lambda_satd_me_28 = 0;
+ regs->swreg156.lambda_satd_me_29 = 0;
+ regs->swreg157.lambda_satd_me_30 = 0;
+ regs->swreg157.lambda_satd_me_31 = 0;
+
+ /* Inter SSE */
+
+ regs->swreg79.lambda_sse_me_0 = 0x4f;
+ regs->swreg122.lambda_sse_me_1 = 0x3f;
+ regs->swreg123.lambda_sse_me_2 = 0x32;
+ regs->swreg124.lambda_sse_me_3 = 0x28;
+ regs->swreg138.lambda_sse_me_4 = 0;
+ regs->swreg139.lambda_sse_me_5 = 0;
+ regs->swreg140.lambda_sse_me_6 = 0;
+ regs->swreg141.lambda_sse_me_7 = 0;
+ regs->swreg142.lambda_sse_me_8 = 0;
+ regs->swreg143.lambda_sse_me_9 = 0;
+ regs->swreg144.lambda_sse_me_10 = 0;
+ regs->swreg145.lambda_sse_me_11 = 0;
+ regs->swreg146.lambda_sse_me_12 = 0;
+ regs->swreg147.lambda_sse_me_13 = 0;
+ regs->swreg148.lambda_sse_me_14 = 0;
+ regs->swreg149.lambda_sse_me_15 = 0;
+ regs->swreg158.lambda_sse_me_16 = 0;
+ regs->swreg159.lambda_sse_me_17 = 0x800;
+ regs->swreg160.lambda_sse_me_18 = 0;
+ regs->swreg161.lambda_sse_me_19 = 0;
+ regs->swreg162.lambda_sse_me_20 = 0;
+ regs->swreg163.lambda_sse_me_21 = 0;
+ regs->swreg164.lambda_sse_me_22 = 0;
+ regs->swreg165.lambda_sse_me_23 = 0;
+ regs->swreg166.lambda_sse_me_24 = 0;
+ regs->swreg167.lambda_sse_me_25 = 0;
+ regs->swreg168.lambda_sse_me_26 = 0x13c;
+ regs->swreg169.lambda_sse_me_27 = 0xfb;
+ regs->swreg172.lambda_sse_me_30 = 0x7d;
+ regs->swreg173.lambda_sse_me_31 = 0x64;
+
+ regs->swreg35.lambda_motion_sse = 0;
+
+ if (regs->swreg214.hwabsqpsupport) {
+ /* Used for lambda calculation, different intra/inter value. */
+ regs->swreg170_qp_absolute.sse_qp_factor = 0x1f5c;
+ regs->swreg171_qp_absolute.sad_qp_factor = 0x2ccd;
+ }
+
+ return 0;
+}
+
+static int areas_setup(struct hantro_ctx *ctx)
+{
+ struct hantro_h264_enc_hw_ctx *h264_ctx = &ctx->h264_enc;
+ struct hantro_vc8000e_regs *regs = &h264_ctx->vc8000e_regs;
+
+ /* Intra */
+
+ regs->swreg23.intra_area_left = 0xff;
+ regs->swreg195.intra_area_left_msb = 1;
+ regs->swreg249.intra_area_left_msb2 = 1;
+
+ regs->swreg23.intra_area_right = 0xff;
+ regs->swreg195.intra_area_right_msb = 1;
+ regs->swreg249.intra_area_right_msb2 = 1;
+
+ regs->swreg23.intra_area_top = 0xff;
+ regs->swreg195.intra_area_top_msb = 1;
+ regs->swreg249.intra_area_top_msb2 = 1;
+
+ regs->swreg23.intra_area_bottom = 0xff;
+ regs->swreg195.intra_area_bottom_msb = 1;
+ regs->swreg249.intra_area_bottom_msb2 = 1;
+
+ /* IPCM1 */
+
+ regs->swreg208_h264.ipcm1_left = 0x1ff;
+ regs->swreg249.ipcm1_left_msb = 1;
+
+ regs->swreg209.ipcm1_right = 0x1ff;
+ regs->swreg249.ipcm1_right_msb = 1;
+
+ regs->swreg209.ipcm1_top = 0x1ff;
+ regs->swreg209.ipcm1_bottom = 0x1ff;
+
+ regs->swreg249.ipcm1_top_msb = 1;
+ regs->swreg249.ipcm1_bottom_msb = 1;
+
+ /* IPCM2 */
+
+ regs->swreg210.ipcm2_left = 0x1ff;
+ regs->swreg249.ipcm2_left_msb = 1;
+
+ regs->swreg211.ipcm2_right = 0x1ff;
+ regs->swreg249.ipcm2_right_msb = 1;
+
+ regs->swreg212.ipcm2_top = 0x1ff;
+ regs->swreg249.ipcm2_top_msb = 1;
+
+ regs->swreg213.ipcm2_bottom = 0x1ff;
+ regs->swreg249.ipcm2_bottom_msb = 1;
+
+ /* ROI1 */
+
+ regs->swreg24.roi1_left = 0xff;
+ regs->swreg195.roi1_left_msb = 1;
+ regs->swreg249.roi1_left_msb2 = 1;
+
+ regs->swreg24.roi1_right = 0xff;
+ regs->swreg195.roi1_right_msb = 1;
+ regs->swreg249.roi1_right_msb2 = 1;
+
+ regs->swreg24.roi1_top = 0xff;
+ regs->swreg195.roi1_top_msb = 1;
+ regs->swreg249.roi1_top_msb2 = 1;
+
+ regs->swreg24.roi1_bottom = 0xff;
+ regs->swreg195.roi1_bottom_msb = 1;
+ regs->swreg249.roi1_bottom_msb2 = 1;
+
+ /* ROI2 */
+
+ regs->swreg25.roi2_left = 0xff;
+ regs->swreg195.roi2_left_msb = 1;
+ regs->swreg249.roi2_left_msb2 = 1;
+
+ regs->swreg25.roi2_right = 0xff;
+ regs->swreg195.roi2_right_msb = 1;
+ regs->swreg249.roi2_right_msb2 = 1;
+
+ regs->swreg25.roi2_top = 0xff;
+ regs->swreg195.roi2_top_msb = 1;
+ regs->swreg249.roi2_top_msb2 = 1;
+
+ regs->swreg25.roi2_bottom = 0xff;
+ regs->swreg195.roi2_bottom_msb = 1;
+ regs->swreg249.roi2_bottom_msb2 = 1;
+
+ if (regs->swreg226.hwroi8support) {
+ /* ROI3 */
+
+ regs->swreg252.roi3_left = 0x3ff;
+ regs->swreg252.roi3_right = 0x3ff;
+ regs->swreg252.roi3_top = 0x3ff;
+ regs->swreg253.roi3_bottom = 0x3ff;
+
+ /* ROI4 */
+
+ regs->swreg253.roi4_left = 0x3ff;
+ regs->swreg254.roi4_right = 0x3ff;
+ regs->swreg253.roi4_top = 0x3ff;
+ regs->swreg254.roi4_bottom = 0x3ff;
+
+ /* ROI5 */
+
+ regs->swreg254.roi5_left = 0x3ff;
+ regs->swreg255.roi5_right = 0x3ff;
+ regs->swreg255.roi5_top = 0x3ff;
+ regs->swreg255.roi5_bottom = 0x3ff;
+
+ /* ROI6 */
+
+ regs->swreg256.roi6_left = 0x3ff;
+ regs->swreg256.roi6_right = 0x3ff;
+ regs->swreg256.roi6_top = 0x3ff;
+ regs->swreg257.roi6_bottom = 0x3ff;
+
+ /* ROI7 */
+
+ regs->swreg257.roi7_left = 0x3ff;
+ regs->swreg258.roi7_right = 0x3ff;
+ regs->swreg257.roi7_top = 0x3ff;
+ regs->swreg258.roi7_bottom = 0x3ff;
+
+ /* ROI8 */
+
+ regs->swreg258.roi8_left = 0x3ff;
+ regs->swreg259.roi8_right = 0x3ff;
+ regs->swreg259.roi8_top = 0x3ff;
+ regs->swreg259.roi8_bottom = 0x3ff;
+ }
+
+ return 0;
+}
+
+int hantro_vc8000e_h264_enc_run(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ struct v4l2_pix_format_mplane *src_fmt = &ctx->src_fmt;
+ struct hantro_h264_enc_hw_ctx *h264_ctx = &ctx->h264_enc;
+ struct hantro_vc8000e_regs *regs = &h264_ctx->vc8000e_regs;
+ struct v4l2_h264_enc *enc = &h264_ctx->enc;
+ struct v4l2_h264_enc_state *state = &enc->state_active;
+ const struct v4l2_ctrl_h264_sps *sps;
+ const struct v4l2_ctrl_h264_pps *pps;
+ const struct v4l2_ctrl_h264_encode_params *encode;
+ struct vb2_v4l2_buffer *src_buf, *dst_buf;
+ struct hantro_vc8000e_rec_buf *rec_buf;
+ const struct v4l2_format_info *info;
+ unsigned int luma_stride;
+ unsigned int chroma_stride;
+ int ret;
+
+ hantro_start_prepare_run(ctx);
+
+ info = v4l2_format_info(src_fmt->pixelformat);
+ if (!info)
+ return -EINVAL;
+
+ src_buf = hantro_get_src_buf(ctx);
+ dst_buf = hantro_get_dst_buf(ctx);
+
+ ret = v4l2_h264_enc_step(enc, dst_buf);
+ if (ret)
+ return ret;
+
+ sps = &state->sps;
+ pps = &state->pps;
+ encode = &state->encode;
+
+ memset(regs, 0, sizeof(*regs));
+
+ /* Read relevant read-only registers. */
+ hantro_vc8000e_swreg_read(vpu, regs, swreg0);
+ hantro_vc8000e_swreg_read(vpu, regs, swreg80);
+ hantro_vc8000e_swreg_read(vpu, regs, swreg214);
+ hantro_vc8000e_swreg_read(vpu, regs, swreg226);
+ hantro_vc8000e_swreg_read(vpu, regs, swreg287);
+
+ /* Mode */
+
+ if (!regs->swreg80.hwh264support)
+ return -ENODEV;
+
+ regs->swreg4.mode = HANTRO_VC8000E_SWREG4_MODE_H264;
+
+ /* Input */
+
+ regs->swreg38.input_format = ctx->vpu_src_fmt->enc_fmt;
+ regs->swreg38.input_rotation = HANTRO_VC8000E_SWREG38_INPUT_ROTATION_0;
+
+ luma_stride = src_fmt->plane_fmt[0].bytesperline;
+
+ /*
+ * The hardware seems to expect the luma stride to represent pixels per
+ * line for packed cases, instead of the usual bytes per line.
+ */
+ if (info->comp_planes == 1)
+ luma_stride /= info->bpp[0];
+
+ regs->swreg210.input_lu_stride = luma_stride;
+
+ if (info->comp_planes > 1) {
+ if (src_fmt->num_planes > 1)
+ chroma_stride = src_fmt->plane_fmt[1].bytesperline;
+ else if (info->comp_planes > 2)
+ chroma_stride = luma_stride / 2;
+ else
+ chroma_stride = luma_stride;
+
+ regs->swreg211.input_ch_stride = chroma_stride;
+ }
+
+ regs->swreg12.input_y_base =
+ vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf, 0);
+
+ if (info->comp_planes > 1) {
+ if (src_fmt->num_planes > 1)
+ regs->swreg13.input_cb_base =
+ vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf,
+ 1);
+ else
+ regs->swreg13.input_cb_base =
+ vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf,
+ 0) +
+ luma_stride * src_fmt->height;
+ }
+
+ if (info->comp_planes > 2) {
+ if (src_fmt->num_planes > 1)
+ regs->swreg14.input_cr_base =
+ vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf,
+ 2);
+ else
+ regs->swreg14.input_cr_base =
+ vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf,
+ 0) +
+ luma_stride * src_fmt->height +
+ chroma_stride * src_fmt->height;
+ }
+
+ /* Output */
+
+ if (enc->rbsp_update & V4L2_H264_ENC_RBSP_UPDATE_START_CODE)
+ regs->swreg4.output_strm_mode =
+ HANTRO_VC8000E_SWREG4_OUTPUT_STRM_MODE_BYTE_STREAM;
+ else
+ regs->swreg4.output_strm_mode =
+ HANTRO_VC8000E_SWREG4_OUTPUT_STRM_MODE_NAL_STREAM;
+
+ regs->swreg8.output_strm_base =
+ vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0) +
+ v4l2_h264_enc_rbsp_bytes_count(&enc->rbsp);
+ regs->swreg9.output_strm_buffer_limit =
+ vb2_plane_size(&dst_buf->vb2_buf, 0) -
+ v4l2_h264_enc_rbsp_bytes_count(&enc->rbsp);
+
+ if (!regs->swreg9.output_strm_buffer_limit)
+ return -ENOMEM;
+
+ regs->swreg10.size_tbl_base = h264_ctx->nal_tbl.dma;
+ regs->swreg6.nal_size_write = 1;
+
+ regs->swreg196.num_ctb_rows_per_sync = 1;
+
+ regs->swreg199.hash_type = HANTRO_VC8000E_SWREG199_HASH_TYPE_NONE;
+
+ /* Picture */
+
+ regs->swreg5.pic_width = src_fmt->width / 8;
+ regs->swreg5.pic_height = src_fmt->height / 8;
+ regs->swreg38.rowlength = src_fmt->width;
+
+ regs->swreg281.chroma_format_idc = sps->chroma_format_idc;
+ regs->swreg38.output_bitwidth_lum =
+ HANTRO_VC8000E_SWREG38_OUTPUT_BITWIDTH_LUM_8_BIT;
+
+ regs->swreg11.poc = enc->ref.pic_order_cnt;
+ regs->swreg277.pic_order_cnt_type = sps->pic_order_cnt_type;
+ if (!sps->pic_order_cnt_type)
+ regs->swreg277.log2_max_pic_order_cnt_lsb =
+ sps->log2_max_pic_order_cnt_lsb_minus4 + 4;
+
+ regs->swreg192.framenum = encode->frame_num;
+ regs->swreg277.log2_max_frame_num = sps->log2_max_frame_num_minus4 + 4;
+
+ /* Reconstruction */
+
+ regs->swreg212.ref_lu_stride = src_fmt->width * 4;
+ regs->swreg237.ref_ch_stride = src_fmt->width * 4;
+ regs->swreg213.ref_ds_lu_stride = src_fmt->width;
+
+ rec_buf = enc->ref.buffer_current.private_data;
+
+ regs->swreg15.recon_y_base = rec_buf->luma.dma;
+ regs->swreg16.recon_chroma_base = rec_buf->chroma.dma;
+ regs->swreg72.recon_luma_4n_base = rec_buf->luma_4n.dma;
+ regs->swreg114.colctbs_store_base = rec_buf->colctbs.dma;
+
+ /* Reference */
+
+ ret = ref_setup(ctx);
+ if (ret)
+ return ret;
+
+ /* Slice */
+
+ if (encode->slice_type == V4L2_H264_SLICE_TYPE_I)
+ regs->swreg5.frame_coding_type =
+ HANTRO_VC8000E_SWREG5_FRAME_CODING_TYPE_I;
+ else if (encode->slice_type == V4L2_H264_SLICE_TYPE_P)
+ regs->swreg5.frame_coding_type =
+ HANTRO_VC8000E_SWREG5_FRAME_CODING_TYPE_P;
+ else
+ return -EINVAL;
+
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_IDR_PIC) {
+ regs->swreg191.nal_unit_type = V4L2_H264_NALU_TYPE_SLICE_IDR;
+ regs->swreg193.idr_pic_id = encode->idr_pic_id;
+ } else {
+ regs->swreg191.nal_unit_type =
+ V4L2_H264_NALU_TYPE_SLICE_NON_IDR;
+ }
+
+ regs->swreg191.pps_id = encode->pic_parameter_set_id;
+
+ /* Quantization */
+
+ regs->swreg172.qp_min = state->qp_min;
+ regs->swreg173.qp_max = state->qp_max;
+ regs->swreg7.pic_init_qp = pps->pic_init_qp_minus26 + 26;
+ regs->swreg7.pic_qp = enc->rc.qp;
+ regs->swreg4.chroma_qp_offset = pps->chroma_qp_index_offset;
+
+ /* Coding */
+
+ if (pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE) {
+ regs->swreg193.entropy_coding_mode = 1;
+ regs->swreg7.cabac_init_flag = encode->cabac_init_idc;
+ }
+
+ if (pps->flags & V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE)
+ regs->swreg193.transform8x8_enable = 1;
+
+ if (pps->flags & V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT) {
+ regs->swreg5.pps_deblocking_filter_override_enabled_flag = 1;
+ regs->swreg5.slice_deblocking_filter_override_flag = 1;
+
+ regs->swreg6.deblocking_filter_dis =
+ encode->disable_deblocking_filter_idc;
+ regs->swreg6.deblocking_tc_offset =
+ encode->slice_alpha_c0_offset_div2;
+ regs->swreg6.deblocking_beta_offset =
+ encode->slice_beta_offset_div2;
+ }
+
+ regs->swreg4.min_trb_size = HANTRO_VC8000E_SWREG4_TRB_SIZE_4X4;
+ regs->swreg4.max_trb_size = HANTRO_VC8000E_SWREG4_TRB_SIZE_16X16;
+ regs->swreg4.min_cb_size = HANTRO_VC8000E_SWREG4_CB_SIZE_8X8;
+ regs->swreg4.max_cb_size = HANTRO_VC8000E_SWREG4_CB_SIZE_16X16;
+ regs->swreg4.max_trans_hierarchy_depth_inter = 2;
+ regs->swreg4.max_trans_hierarchy_depth_intra = 1;
+
+ regs->swreg36.bits_est_1n_cu_penalty = 15;
+ regs->swreg35.bits_est_tu_split_penalty = 3;
+
+ regs->swreg35.bits_est_bias_intra_cu_8 = 22;
+ regs->swreg35.bits_est_bias_intra_cu_16 = 40;
+ regs->swreg36.bits_est_bias_intra_cu_32 = 86;
+ regs->swreg36.bits_est_bias_intra_cu_64 = 304;
+ regs->swreg36.inter_skip_bias = 124;
+
+ regs->swreg201.mean_thr0 = 5;
+ regs->swreg201.mean_thr1 = 5;
+ regs->swreg201.mean_thr2 = 5;
+ regs->swreg201.mean_thr3 = 5;
+
+ regs->swreg203_h264.lum_dc_sum_thr = 5;
+ regs->swreg203_h264.cb_dc_sum_thr = 1;
+ regs->swreg203_h264.cr_dc_sum_thr = 1;
+
+ regs->swreg26.intra_size_factor_0 = 506;
+ regs->swreg26.intra_size_factor_1 = 506;
+ regs->swreg26.intra_size_factor_2 = 709;
+ regs->swreg27.intra_size_factor_3 = 709;
+
+ regs->swreg27.intra_mode_factor_0 = 24;
+ regs->swreg27.intra_mode_factor_1 = 12;
+ regs->swreg27.intra_mode_factor_2 = 48;
+
+ regs->swreg182.qp_delta_gain = 313;
+
+ /* Lambda */
+
+ ret = lambda_setup(ctx);
+ if (ret)
+ return ret;
+
+ /* Areas */
+
+ ret = areas_setup(ctx);
+ if (ret)
+ return ret;
+
+ /* Urgent thresholds */
+
+ regs->swreg272.wr_urgent_disable_threshold =
+ HANTRO_VC8000E_SWREG272_URGENT_THRESHOLD_DISABLE;
+ regs->swreg272.wr_urgent_enable_threshold =
+ HANTRO_VC8000E_SWREG272_URGENT_THRESHOLD_DISABLE;
+ regs->swreg272.rd_urgent_disable_threshold =
+ HANTRO_VC8000E_SWREG272_URGENT_THRESHOLD_DISABLE;
+ regs->swreg272.rd_urgent_enable_threshold =
+ HANTRO_VC8000E_SWREG272_URGENT_THRESHOLD_DISABLE;
+
+ /* AXI bus */
+
+ regs->swreg81.max_burst = 16;
+ regs->swreg261.axi_read_outstanding_num = 64;
+ regs->swreg246.axi_write_outstanding_num = 64;
+ regs->swreg320.axi_burst_align_rd_lu_ref_prefetch = 1;
+
+ /* Automatic clock gating */
+
+ regs->swreg3.clock_gate_inter_h264_e = 1;
+ regs->swreg3.clock_gate_inter_h265_e = 1;
+ regs->swreg3.clock_gate_inter_e = 1;
+ regs->swreg3.clock_gate_encoder_h264_e = 1;
+ regs->swreg3.clock_gate_encoder_h265_e = 1;
+ regs->swreg3.clock_gate_encoder_e = 1;
+
+ /* IRQ status */
+
+ regs->swreg1.irq = 1;
+ regs->swreg1.frame_rdy_status = 1;
+ regs->swreg1.bus_error_status = 1;
+ regs->swreg1.sw_reset = 1;
+ regs->swreg1.buffer_full = 1;
+ regs->swreg1.timeout = 1;
+ regs->swreg1.irq_line_buffer = 1;
+ regs->swreg1.slice_rdy_status = 1;
+
+ /* IRQ */
+
+ regs->swreg1.irq_dis = 0;
+ regs->swreg1.timeout_int = 1;
+
+ /* Register write */
+
+ hantro_io_copy(vpu->enc_base, regs, sizeof(*regs));
+
+ hantro_end_prepare_run(ctx);
+
+ /* Enable */
+
+ regs->swreg5.enable = 1;
+
+ hantro_vc8000e_swreg_write(vpu, regs, swreg5);
+
+ return 0;
+}
+
+void hantro_vc8000e_h264_enc_done(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ struct hantro_h264_enc_hw_ctx *h264_ctx = &ctx->h264_enc;
+ struct hantro_vc8000e_regs *regs = &h264_ctx->vc8000e_regs;
+ struct v4l2_h264_enc *enc = &h264_ctx->enc;
+ struct vb2_v4l2_buffer *dst_buf = hantro_get_dst_buf(ctx);
+ u32 bytesused;
+
+ hantro_vc8000e_swreg_read(vpu, regs, swreg9);
+
+ bytesused = regs->swreg9.output_strm_buffer_limit +
+ v4l2_h264_enc_rbsp_bytes_count(&enc->rbsp);
+
+ vb2_set_plane_payload(&dst_buf->vb2_buf, 0, bytesused);
+
+ v4l2_h264_enc_complete(enc, dst_buf);
+}
diff --git a/drivers/media/platform/verisilicon/hantro_vc8000e_regs.h b/drivers/media/platform/verisilicon/hantro_vc8000e_regs.h
new file mode 100644
index 000000000000..fad100572708
--- /dev/null
+++ b/drivers/media/platform/verisilicon/hantro_vc8000e_regs.h
@@ -0,0 +1,2129 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Hantro VPU codec driver
+ *
+ * Copyright (C) 2025-2026 Paul Kocialkowski <paulk@sys-base.io>
+ */
+
+#ifndef HANTRO_VC8000E_REGS_H_
+#define HANTRO_VC8000E_REGS_H_
+
+#include <linux/types.h>
+
+#define HANTRO_VC8000E_SWREG_OFFSET(s) \
+ offsetof(struct hantro_vc8000e_regs, s)
+
+#define hantro_vc8000e_swreg_read(v, r, s) \
+ ({ \
+ u32 *pointer = (u32 *)&(r)->s; \
+ *pointer = vepu_read(v, HANTRO_VC8000E_SWREG_OFFSET(s)); \
+ })
+
+#define hantro_vc8000e_swreg_write(v, r, s) \
+ ({ \
+ u32 *pointer = (u32 *)&(r)->s; \
+ vepu_write(v, *pointer, HANTRO_VC8000E_SWREG_OFFSET(s)); \
+ })
+
+#define HANTRO_VC8000E_SWREG0_MAJOR_NUMBER_V6_0 0x60
+#define HANTRO_VC8000E_SWREG0_MAJOR_NUMBER_V6_1 0x61
+#define HANTRO_VC8000E_SWREG0_MAJOR_NUMBER_V6_2 0x62
+#define HANTRO_VC8000E_SWREG0_PRODUCT_ID_VC8000E 0x8000
+
+#define HANTRO_VC8000E_SWREG4_OUTPUT_STRM_MODE_BYTE_STREAM 0x0
+#define HANTRO_VC8000E_SWREG4_OUTPUT_STRM_MODE_NAL_STREAM 0x1
+#define HANTRO_VC8000E_SWREG4_TRB_SIZE_4X4 0x0
+#define HANTRO_VC8000E_SWREG4_TRB_SIZE_8X8 0x1
+#define HANTRO_VC8000E_SWREG4_TRB_SIZE_16X16 0x2
+#define HANTRO_VC8000E_SWREG4_TRB_SIZE_32X32 0x3
+#define HANTRO_VC8000E_SWREG4_CB_SIZE_8X8 0x0
+#define HANTRO_VC8000E_SWREG4_CB_SIZE_16X16 0x1
+#define HANTRO_VC8000E_SWREG4_CB_SIZE_32X32 0x2
+#define HANTRO_VC8000E_SWREG4_CB_SIZE_64X64 0x3
+#define HANTRO_VC8000E_SWREG4_MODE_HEVC 0x1
+#define HANTRO_VC8000E_SWREG4_MODE_H264 0x2
+#define HANTRO_VC8000E_SWREG4_MODE_JPEG 0x4
+
+#define HANTRO_VC8000E_SWREG5_FRAME_CODING_TYPE_P 0x0
+#define HANTRO_VC8000E_SWREG5_FRAME_CODING_TYPE_I 0x1
+#define HANTRO_VC8000E_SWREG5_FRAME_CODING_TYPE_B 0x2
+
+#define HANTRO_VC8000E_SWREG36_OUTPUT_BITWIDTH_CHROMA_8_BIT 0x0
+#define HANTRO_VC8000E_SWREG36_OUTPUT_BITWIDTH_CHROMA_9_BIT 0x1
+#define HANTRO_VC8000E_SWREG36_OUTPUT_BITWIDTH_CHROMA_10_BIT 0x2
+
+#define HANTRO_VC8000E_SWREG38_OUTPUT_BITWIDTH_LUM_8_BIT 0x0
+#define HANTRO_VC8000E_SWREG38_OUTPUT_BITWIDTH_LUM_9_BIT 0x1
+#define HANTRO_VC8000E_SWREG38_OUTPUT_BITWIDTH_LUM_10_BIT 0x2
+#define HANTRO_VC8000E_SWREG38_INPUT_ROTATION_0 0x0
+#define HANTRO_VC8000E_SWREG38_INPUT_ROTATION_90 0x1
+#define HANTRO_VC8000E_SWREG38_INPUT_ROTATION_270 0x2
+#define HANTRO_VC8000E_SWREG38_INPUT_ROTATION_180 0x3
+#define HANTRO_VC8000E_SWREG38_INPUT_FORMAT_YUV420P 0x0
+#define HANTRO_VC8000E_SWREG38_INPUT_FORMAT_YUV420SP 0x1
+#define HANTRO_VC8000E_SWREG38_INPUT_FORMAT_YUYV422 0x2
+#define HANTRO_VC8000E_SWREG38_INPUT_FORMAT_UYVY422 0x3
+#define HANTRO_VC8000E_SWREG38_INPUT_FORMAT_RGB565 0x4
+#define HANTRO_VC8000E_SWREG38_INPUT_FORMAT_RGB555 0x5
+#define HANTRO_VC8000E_SWREG38_INPUT_FORMAT_RGB444 0x6
+#define HANTRO_VC8000E_SWREG38_INPUT_FORMAT_RGB888 0x7
+#define HANTRO_VC8000E_SWREG38_INPUT_FORMAT_RGB101010 0x8
+#define HANTRO_VC8000E_SWREG38_INPUT_FORMAT_I010 0x9
+#define HANTRO_VC8000E_SWREG38_INPUT_FORMAT_P010 0xa
+#define HANTRO_VC8000E_SWREG38_INPUT_FORMAT_PACKED10BITPLANAR 0xb
+#define HANTRO_VC8000E_SWREG38_INPUT_FORMAT_Y0L2 0xc
+#define HANTRO_VC8000E_SWREG38_INPUT_FORMAT_DAHUAHEVC 0xd
+#define HANTRO_VC8000E_SWREG38_INPUT_FORMAT_DAHUAH264 0xe
+
+#define HANTRO_VC8000E_SWREG80_HWBUSWIDTH_32_BIT 0x0
+#define HANTRO_VC8000E_SWREG80_HWBUSWIDTH_64_BIT 0x1
+#define HANTRO_VC8000E_SWREG80_HWBUSWIDTH_128_BIT 0x2
+#define HANTRO_VC8000E_SWREG80_HWBUS_AHB 0x1
+#define HANTRO_VC8000E_SWREG80_HWBUS_OCP 0x2
+#define HANTRO_VC8000E_SWREG80_HWBUS_AXI 0x3
+#define HANTRO_VC8000E_SWREG80_HWBUS_PCI 0x4
+#define HANTRO_VC8000E_SWREG80_HWBUS_AXIAHB 0x5
+#define HANTRO_VC8000E_SWREG80_HWBUS_AXIAPB 0x6
+
+#define HANTRO_VC8000E_SWREG_ROI_QP_TYPE_DELTA 0x0
+#define HANTRO_VC8000E_SWREG_ROI_QP_TYPE_ABSOLUTE 0x1
+
+#define HANTRO_VC8000E_SWREG181_RC_BLOCK_SIZE_64X64 0x0
+#define HANTRO_VC8000E_SWREG181_RC_BLOCK_SIZE_32X32 0x1
+#define HANTRO_VC8000E_SWREG181_RC_BLOCK_SIZE_16X16 0x2
+
+#define HANTRO_VC8000E_SWREG199_HASH_TYPE_NONE 0x0
+#define HANTRO_VC8000E_SWREG199_HASH_TYPE_CRC32 0x1
+#define HANTRO_VC8000E_SWREG199_HASH_TYPE_CHECKSUM32 0x2
+
+#define HANTRO_VC8000E_SWREG214_HWROIMAPVERSION_4_BPP 0x0
+#define HANTRO_VC8000E_SWREG214_HWROIMAPVERSION_8_BPP 0x1
+
+#define HANTRO_VC8000E_SWREG226_HWINLOOPDSRATIO_1_1 0x0
+#define HANTRO_VC8000E_SWREG226_HWINLOOPDSRATIO_1_2 0x1
+#define HANTRO_VC8000E_SWREG226_BFRAME_ME4N_HOR_SEARCHRANGE_64 0x0
+#define HANTRO_VC8000E_SWREG226_BFRAME_ME4N_HOR_SEARCHRANGE_128 0x1
+#define HANTRO_VC8000E_SWREG226_BFRAME_ME4N_HOR_SEARCHRANGE_192 0x2
+#define HANTRO_VC8000E_SWREG226_BFRAME_ME4N_HOR_SEARCHRANGE_256 0x3
+#define HANTRO_VC8000E_SWREG226_HWP010REFSUPPORT_NORMAL 0x0
+#define HANTRO_VC8000E_SWREG226_HWP010REFSUPPORT_TILED_P010 0x1
+
+#define HANTRO_VC8000E_SWREG272_URGENT_THRESHOLD_DISABLE 0xff
+
+#define HANTRO_VC8000E_SWREG281_CHROMA_FORMAT_IDC_400 0x0
+#define HANTRO_VC8000E_SWREG281_CHROMA_FORMAT_IDC_420 0x1
+
+struct hantro_vc8000e_regs {
+ struct {
+ u32 minor_number : 8;
+ u32 major_number : 8;
+ u32 product_id : 16;
+ } swreg0;
+
+ struct {
+ u32 irq : 1;
+ u32 irq_dis : 1;
+ u32 frame_rdy_status : 1;
+ u32 bus_error_status : 1;
+ u32 sw_reset : 1;
+ u32 buffer_full : 1;
+ u32 timeout : 1;
+ u32 irq_line_buffer : 1;
+ u32 slice_rdy_status : 1;
+ u32 irq_fuse_error : 1;
+ u32 reserved0 : 1;
+ u32 timeout_int : 1;
+ u32 strm_segment_rdy_int : 1;
+ u32 reserved1 : 19;
+ } swreg1;
+
+ struct {
+ u32 ctb_rc_mem_out_swap : 4;
+ u32 roi_map_qp_delta_map_swap : 4;
+ u32 pic_swap : 4;
+ u32 strm_swap : 4;
+ u32 axi_read_id : 8;
+ u32 axi_write_id : 8;
+ } swreg2;
+
+ struct {
+ u32 reserved0 : 1;
+ u32 strm_segment_int : 1;
+ u32 line_buffer_int : 1;
+ u32 slice_int : 1;
+ u32 reserved1 : 16;
+ u32 cu_info_mem_out_swap : 4;
+ u32 axi_rd_id_e : 1;
+ u32 axi_wr_id_e : 1;
+ u32 clock_gate_inter_h264_e : 1;
+ u32 clock_gate_inter_h265_e : 1;
+ u32 clock_gate_inter_e : 1;
+ u32 clock_gate_encoder_h264_e : 1;
+ u32 clock_gate_encoder_h265_e : 1;
+ u32 clock_gate_encoder_e : 1;
+ } swreg3;
+
+ struct {
+ u32 max_trans_hierarchy_depth_inter : 3;
+ u32 max_trans_hierarchy_depth_intra : 3;
+ u32 sao_enable : 1;
+ u32 active_override_flag : 1;
+ u32 scaling_list_enabled_flag : 1;
+ u32 reserved0 : 2;
+ u32 bw_linebuf_disable : 1;
+ u32 strong_intra_smoothing_enabled_flag : 1;
+ u32 chroma_qp_offset : 5;
+ u32 output_strm_mode : 1;
+ u32 max_trb_size : 2;
+ u32 min_trb_size : 2;
+ u32 max_cb_size : 2;
+ u32 min_cb_size : 2;
+ u32 reserved1 : 2;
+ u32 mode : 3;
+ } swreg4;
+
+ union {
+ struct {
+ u32 enable : 1;
+ u32 frame_coding_type : 2;
+ u32 ref_frames : 2;
+ u32 buffer_full_continue : 1;
+ u32 output_cu_info_enabled : 1;
+ u32 reserved0 : 1;
+ u32 slice_deblocking_filter_override_flag : 1;
+ u32 pps_deblocking_filter_override_enabled_flag : 1;
+ u32 reserved1 : 1;
+ u32 pic_height : 11;
+ u32 pic_width : 10;
+ } swreg5;
+
+ struct {
+ u32 reserved : 8;
+ u32 jpeg_pic_height : 12;
+ u32 jpeg_pic_width : 12;
+ } swreg5_jpeg;
+ };
+
+ struct {
+ u32 cu_qp_delta_enabled : 1;
+ u32 nal_size_write : 1;
+ u32 rps_id : 5;
+ u32 deblocking_beta_offset : 4;
+ u32 deblocking_tc_offset : 4;
+ u32 deblocking_filter_dis : 1;
+ u32 num_positive_pics : 2;
+ u32 num_negative_pics : 2;
+ u32 num_short_term_ref_pic_sets : 5;
+ u32 slice_size : 7;
+ } swreg6;
+
+ struct {
+ u32 roi2_delta_qp : 4;
+ u32 roi1_delta_qp : 4;
+ u32 pic_qp : 6;
+ u32 diff_cu_qp_delta_depth : 2;
+ u32 reserved : 1;
+ u32 num_slices_ready : 8;
+ u32 cabac_init_flag : 1;
+ u32 pic_init_qp : 6;
+ } swreg7;
+
+ struct {
+ u32 output_strm_base : 32;
+ } swreg8;
+
+ struct {
+ u32 output_strm_buffer_limit : 32;
+ } swreg9;
+
+ struct {
+ u32 size_tbl_base : 32;
+ } swreg10;
+
+ struct {
+ u32 poc : 32;
+ } swreg11;
+
+ struct {
+ u32 input_y_base : 32;
+ } swreg12;
+
+ struct {
+ u32 input_cb_base : 32;
+ } swreg13;
+
+ struct {
+ u32 input_cr_base : 32;
+ } swreg14;
+
+ struct {
+ u32 recon_y_base : 32;
+ } swreg15;
+
+ struct {
+ u32 recon_chroma_base : 32;
+ } swreg16;
+
+ struct {
+ u32 l0_ref1_chroma_compressor_enable : 1;
+ u32 l0_ref1_luma_compressor_enable : 1;
+ u32 l0_ref0_chroma_compressor_enable : 1;
+ u32 l0_ref0_luma_compressor_enable : 1;
+ u32 recon_chroma_compressor_enable : 1;
+ u32 recon_luma_compressor_enable : 1;
+ u32 active_l0_cnt : 2;
+ u32 l0_used_by_curr_pic1 : 1;
+ u32 l0_long_term_flag1 : 1;
+ u32 l0_delta_poc1 : 10;
+ u32 l0_used_by_curr_pic0 : 1;
+ u32 l0_long_term_flag0 : 1;
+ u32 l0_delta_poc0 : 10;
+ } swreg17;
+
+ union {
+ struct {
+ u32 refpic_recon_l0_y0 : 32;
+ } swreg18;
+
+ struct {
+ u32 jpeg_rst : 16;
+ u32 jpeg_rstint : 8;
+ u32 jpeg_mode : 1;
+ u32 jpeg_slice : 1;
+ u32 strm_startoffset : 6;
+ } swreg18_jpeg;
+ };
+
+ union {
+ struct {
+ u32 refpic_recon_l0_chroma0 : 32;
+ } swreg19;
+
+ struct {
+ u32 strm_hdrrem1 : 32;
+ } swreg19_jpeg;
+ };
+
+ union {
+ struct {
+ u32 refpic_recon_l0_y1 : 32;
+ } swreg20;
+
+ struct {
+ u32 reserved : 8;
+ u32 ljpeg_pt : 3;
+ u32 ljpeg_psv : 3;
+ u32 ljpeg_format : 2;
+ u32 ljpeg_en : 1;
+ u32 jpeg_rowlength : 15;
+ } swreg20_jpeg;
+ };
+
+ union {
+ struct {
+ u32 refpic_recon_l0_chroma1 : 32;
+ } swreg21;
+
+ struct {
+ u32 strm_hdrrem2 : 32;
+ } swreg21_jpeg;
+ };
+
+ struct {
+ u32 roi_area_enable : 1;
+ u32 roi_map_enable : 1;
+ u32 qadj_enable : 1;
+ u32 rc_enable : 1;
+ u32 cir_interval : 14;
+ u32 cir_start : 14;
+ } swreg22;
+
+ struct {
+ u32 intra_area_bottom : 8;
+ u32 intra_area_top : 8;
+ u32 intra_area_right : 8;
+ u32 intra_area_left : 8;
+ } swreg23;
+
+ struct {
+ u32 roi1_bottom : 8;
+ u32 roi1_top : 8;
+ u32 roi1_right : 8;
+ u32 roi1_left : 8;
+ } swreg24;
+
+ struct {
+ u32 roi2_bottom : 8;
+ u32 roi2_top : 8;
+ u32 roi2_right : 8;
+ u32 roi2_left : 8;
+ } swreg25;
+
+ struct {
+ u32 reserved : 2;
+ u32 intra_size_factor_2 : 10;
+ u32 intra_size_factor_1 : 10;
+ u32 intra_size_factor_0 : 10;
+ } swreg26;
+
+ struct {
+ u32 reserved : 4;
+ u32 intra_mode_factor_2 : 7;
+ u32 intra_mode_factor_1 : 6;
+ u32 intra_mode_factor_0 : 5;
+ u32 intra_size_factor_3 : 10;
+ } swreg27;
+
+ struct {
+ u32 reserved : 6;
+ u32 lambda_satd_me_1 : 13;
+ u32 lambda_satd_me_0 : 13;
+ } swreg28;
+
+ struct {
+ u32 reserved : 6;
+ u32 lambda_satd_me_3 : 13;
+ u32 lambda_satd_me_2 : 13;
+ } swreg29;
+
+ struct {
+ u32 reserved : 6;
+ u32 lambda_satd_me_5 : 13;
+ u32 lambda_satd_me_4 : 13;
+ } swreg30;
+
+ struct {
+ u32 reserved : 6;
+ u32 lambda_satd_me_7 : 13;
+ u32 lambda_satd_me_6 : 13;
+ } swreg31;
+
+ struct {
+ u32 reserved : 6;
+ u32 lambda_satd_me_9 : 13;
+ u32 lambda_satd_me_8 : 13;
+ } swreg32;
+
+ struct {
+ u32 reserved : 6;
+ u32 lambda_satd_me_11 : 13;
+ u32 lambda_satd_me_10 : 13;
+ } swreg33;
+
+ struct {
+ u32 reserved : 6;
+ u32 lambda_satd_me_13 : 13;
+ u32 lambda_satd_me_12 : 13;
+ } swreg34;
+
+ struct {
+ u32 bits_est_bias_intra_cu_16 : 8;
+ u32 bits_est_bias_intra_cu_8 : 7;
+ u32 bits_est_tu_split_penalty : 3;
+ u32 lambda_motion_sse : 14;
+ } swreg35;
+
+ struct {
+ u32 output_bitwidth_chroma : 2;
+ u32 bits_est_1n_cu_penalty : 4;
+ u32 inter_skip_bias : 7;
+ u32 bits_est_bias_intra_cu_64 : 10;
+ u32 bits_est_bias_intra_cu_32 : 9;
+ } swreg36;
+
+ struct {
+ u32 chroffset : 4;
+ u32 lambda_sao_luma : 14;
+ u32 lambda_sao_chroma : 14;
+ } swreg37;
+
+ struct {
+ u32 mirror : 1;
+ u32 yfill : 3;
+ u32 xfill : 2;
+ u32 rowlength : 14;
+ u32 lumoffset : 4;
+ u32 output_bitwidth_lum : 2;
+ u32 input_rotation : 2;
+ u32 input_format : 4;
+ } swreg38;
+
+ struct {
+ u32 rgbcoeffb : 16;
+ u32 rgbcoeffa : 16;
+ } swreg39;
+
+ struct {
+ u32 rgbcoeffe : 16;
+ u32 rgbcoeffc : 16;
+ } swreg40;
+
+ struct {
+ u32 reserved : 1;
+ u32 bmaskmsb : 5;
+ u32 gmaskmsb : 5;
+ u32 rmaskmsb : 5;
+ u32 rgbcoefff : 16;
+ } swreg41;
+
+ struct {
+ u32 basescaledoutlum : 32;
+ } swreg42;
+
+ struct {
+ u32 scale_mode : 2;
+ u32 scaledoutwidthmsb : 1;
+ u32 scaledoutwidthratio : 16;
+ u32 scaledoutwidth : 13;
+ } swreg43;
+
+ struct {
+ u32 input_format_msb : 2;
+ u32 scaledoutheightratio : 16;
+ u32 scaledoutheight : 14;
+ } swreg44;
+
+ struct {
+ u32 reserved : 2;
+ u32 scaledout_format : 1;
+ u32 nalunitsize_swap : 4;
+ u32 scaledverticalcopy : 1;
+ u32 scaledhorizontalcopy : 1;
+ u32 vscale_weight_en : 1;
+ u32 scaledskiptoppixelrow : 2;
+ u32 scaledskipleftpixelcolumn : 2;
+ u32 encoded_ctb_number : 13;
+ u32 chroma_swap : 1;
+ u32 scaledout_swap : 4;
+ } swreg45;
+
+ struct {
+ u32 compressedcoeff_base : 32;
+ } swreg46;
+
+ struct {
+ u32 compressedcoeff_base_msb : 32;
+ } swreg47;
+
+ struct {
+ u32 basescaledoutlum_msb : 32;
+ } swreg48;
+
+ struct {
+ u32 refpic_recon_l0_y0_msb : 32;
+ } swreg49;
+
+ struct {
+ u32 refpic_recon_l0_chroma0_msb : 32;
+ } swreg50;
+
+ struct {
+ u32 refpic_recon_l0_y1_msb : 32;
+ } swreg51;
+
+ struct {
+ u32 refpic_recon_l0_chroma1_msb : 32;
+ } swreg52;
+
+ struct {
+ u32 input_y_base_msb : 32;
+ } swreg53;
+
+ struct {
+ u32 input_cb_base_msb : 32;
+ } swreg54;
+
+ struct {
+ u32 input_cr_base_msb : 32;
+ } swreg55;
+
+ struct {
+ u32 recon_y_base_msb : 32;
+ } swreg56;
+
+ struct {
+ u32 recon_chroma_base_msb : 32;
+ } swreg57;
+
+ struct {
+ u32 size_tbl_base_msb : 32;
+ } swreg58;
+
+ struct {
+ u32 output_strm_base_msb : 32;
+ } swreg59;
+
+ struct {
+ u32 recon_luma_compress_table_base : 32;
+ } swreg60;
+
+ struct {
+ u32 recon_luma_compress_table_base_msb : 32;
+ } swreg61;
+
+ struct {
+ u32 recon_chroma_compress_table_base : 32;
+ } swreg62;
+
+ struct {
+ u32 recon_chroma_compress_table_base_msb : 32;
+ } swreg63;
+
+ struct {
+ u32 l0_ref0_luma_compress_table_base : 32;
+ } swreg64;
+
+ struct {
+ u32 l0_ref0_luma_compress_table_base_msb : 32;
+ } swreg65;
+
+ struct {
+ u32 l0_ref0_chroma_compress_table_base : 32;
+ } swreg66;
+
+ struct {
+ u32 l0_ref0_chroma_compress_table_base_msb : 32;
+ } swreg67;
+
+ struct {
+ u32 l0_ref1_luma_compress_table_base : 32;
+ } swreg68;
+
+ struct {
+ u32 l0_ref1_luma_compress_table_base_msb : 32;
+ } swreg69;
+
+ struct {
+ u32 l0_ref1_chroma_compress_table_base : 32;
+ } swreg70;
+
+ struct {
+ u32 l0_ref1_chroma_compress_table_base_msb : 32;
+ } swreg71;
+
+ struct {
+ u32 recon_luma_4n_base : 32;
+ } swreg72;
+
+ struct {
+ u32 recon_luma_4n_base_msb : 32;
+ } swreg73;
+
+ struct {
+ u32 refpic_recon_l0_4n0_base : 32;
+ } swreg74;
+
+ struct {
+ u32 refpic_recon_l0_4n0_base_msb : 32;
+ } swreg75;
+
+ struct {
+ u32 refpic_recon_l0_4n1_base : 32;
+ } swreg76;
+
+ struct {
+ u32 refpic_recon_l0_4n1_base_msb : 32;
+ } swreg77;
+
+ struct {
+ u32 reserved : 6;
+ u32 lambda_satd_me_15 : 13;
+ u32 lambda_satd_me_14 : 13;
+ } swreg78;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_0 : 21;
+ } swreg79;
+
+ struct {
+ u32 hwmaxvideowidth : 13;
+ u32 hwbuswidth : 2;
+ u32 hwjpegsupport : 1;
+ u32 hwtu32support : 1;
+ u32 hwrfcsupport : 1;
+ u32 hwprogrdosupport : 1;
+ u32 hwlinebufsupport : 1;
+ u32 hwcavlcsupport : 1;
+ u32 hwbus : 3;
+ u32 hwmain10support : 1;
+ u32 hwdenoisesupport : 1;
+ u32 hwvp9support : 1;
+ u32 hwhevcsupport : 1;
+ u32 hwrgbsupport : 1;
+ u32 hwbframesupport : 1;
+ u32 hwscalingsupport : 1;
+ u32 hwh264support : 1;
+ } swreg80;
+
+ struct {
+ u32 timeout_cycles : 23;
+ u32 timeout_override_e : 1;
+ u32 max_burst : 8;
+ } swreg81;
+
+ struct {
+ u32 hw_performance : 32;
+ } swreg82;
+
+ struct {
+ u32 refpic_recon_l1_y0 : 32;
+ } swreg83;
+
+ struct {
+ u32 refpic_recon_l1_chroma0 : 32;
+ } swreg84;
+
+ struct {
+ u32 refpic_recon_l1_y1 : 32;
+ } swreg85;
+
+ struct {
+ u32 refpic_recon_l1_chroma1 : 32;
+ } swreg86;
+
+ struct {
+ u32 refpic_recon_l1_y0_msb : 32;
+ } swreg87;
+
+ struct {
+ u32 refpic_recon_l1_chroma0_msb : 32;
+ } swreg88;
+
+ struct {
+ u32 refpic_recon_l1_y1_msb : 32;
+ } swreg89;
+
+ struct {
+ u32 refpic_recon_l1_chroma1_msb : 32;
+ } swreg90;
+
+ struct {
+ u32 l1_ref1_chroma_compressor_enable : 1;
+ u32 l1_ref1_luma_compressor_enable : 1;
+ u32 l1_ref0_chroma_compressor_enable : 1;
+ u32 l1_ref0_luma_compressor_enable : 1;
+ u32 long_term_ref_pics_present_flag : 1;
+ u32 reserved : 1;
+ u32 active_l1_cnt : 2;
+ u32 l1_used_by_curr_pic1 : 1;
+ u32 l1_long_term_flag1 : 1;
+ u32 l1_delta_poc1 : 10;
+ u32 l1_used_by_curr_pic0 : 1;
+ u32 l1_long_term_flag0 : 1;
+ u32 l1_delta_poc0 : 10;
+ } swreg91;
+
+ struct {
+ u32 refpic_recon_l1_4n0_base : 32;
+ } swreg92;
+
+ struct {
+ u32 refpic_recon_l1_4n0_base_msb : 32;
+ } swreg93;
+
+ struct {
+ u32 refpic_recon_l1_4n1_base : 32;
+ } swreg94;
+
+ struct {
+ u32 refpic_recon_l1_4n1_base_msb : 32;
+ } swreg95;
+
+ struct {
+ u32 l1_ref0_luma_compress_table_base : 32;
+ } swreg96;
+
+ struct {
+ u32 l1_ref0_luma_compress_table_base_msb : 32;
+ } swreg97;
+
+ struct {
+ u32 l1_ref0_chroma_compress_table_base : 32;
+ } swreg98;
+
+ struct {
+ u32 l1_ref0_chroma_compress_table_base_msb : 32;
+ } swreg99;
+
+ struct {
+ u32 l1_ref1_luma_compress_table_base : 32;
+ } swreg100;
+
+ struct {
+ u32 l1_ref1_luma_compress_table_base_msb : 32;
+ } swreg101;
+
+ struct {
+ u32 l1_ref1_chroma_compress_table_base : 32;
+ } swreg102;
+
+ struct {
+ u32 l1_ref1_chroma_compress_table_base_msb : 32;
+ } swreg103;
+
+ struct {
+ u32 ref_pic_list_modi_flag_l0 : 1;
+ u32 list_entry_l0_pic0 : 4;
+ u32 list_entry_l0_pic1 : 4;
+ u32 reserved0 : 7;
+ u32 ref_pic_list_modi_flag_l1 : 1;
+ u32 list_entry_l1_pic0 : 4;
+ u32 list_entry_l1_pic1 : 4;
+ u32 reserved1 : 4;
+ u32 rdo_level : 2;
+ u32 lists_modi_present_flag : 1;
+ } swreg104;
+
+ struct {
+ u32 targetpicsize : 32;
+ } swreg105;
+
+ struct {
+ u32 minpicsize : 32;
+ } swreg106;
+
+ struct {
+ u32 maxpicsize : 32;
+ } swreg107;
+
+ struct {
+ u32 nonzerocount : 24;
+ u32 averageqp : 8;
+ } swreg108;
+
+ struct {
+ u32 roimapdeltaqpaddr : 32;
+ } swreg109;
+
+ struct {
+ u32 roimapdeltaqpaddr_msb : 32;
+ } swreg110;
+
+ struct {
+ u32 reserved : 12;
+ u32 intracu8num : 20;
+ } swreg111;
+
+ struct {
+ u32 reserved : 12;
+ u32 skipcu8num : 20;
+ } swreg112;
+
+ struct {
+ u32 pbframe4nrdcost : 32;
+ } swreg113;
+
+ struct {
+ u32 colctbs_store_base : 32;
+ } swreg114;
+
+ struct {
+ u32 colctbs_store_base_msb : 32;
+ } swreg115;
+
+ struct {
+ u32 colctbs_load_base : 32;
+ } swreg116;
+
+ struct {
+ u32 colctbs_load_base_msb : 32;
+ } swreg117;
+
+ struct {
+ u32 ctbrcthrdmax : 16;
+ u32 ctbrcthrdmin : 16;
+ } swreg118;
+
+ struct {
+ u32 ctbbitsmax : 16;
+ u32 ctbbitsmin : 16;
+ } swreg119;
+
+ struct {
+ u32 totallcubits : 32;
+ } swreg120;
+
+ struct {
+ u32 bitsratio : 32;
+ } swreg121;
+
+ union {
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_1 : 21;
+ } swreg122;
+
+ struct {
+ u32 av1_enable_order_hint : 1;
+ u32 av1_allow_update_cdf : 1;
+ u32 av1_interp_filter : 3;
+ u32 av1_switchable_motion_mode : 1;
+ u32 av1_cur_frame_force_integer_mv : 1;
+ u32 av1_enable_dual_filter : 1;
+ u32 av1_enable_interintra_compound : 1;
+ u32 av1_list1_ref_frame : 4;
+ u32 av1_list0_ref_frame : 4;
+ u32 av1_reference_mode : 2;
+ u32 av1_skip_mode_flag : 1;
+ u32 av1_allow_high_precision_mv : 1;
+ u32 av1_seg_enable : 1;
+ u32 av1_reduced_tx_set_used : 1;
+ u32 av1_tx_mode : 2;
+ u32 av1_enable_filter_intra : 1;
+ u32 av1_delta_q_res : 4;
+ u32 av1_coded_lossless : 1;
+ u32 av1_allow_intrabc : 1;
+ } swreg122_av1;
+ };
+
+ union {
+ struct {
+ u32 ctbrc_qpdelta_flag_reverse : 1;
+ u32 reserved : 10;
+ u32 lambda_sse_me_2 : 21;
+ } swreg123;
+
+ struct {
+ u32 reserved : 1;
+ u32 av1_btxtypesearch : 1;
+ u32 av1_primary_ref_frame : 3;
+ u32 av1_sharpness_lvl : 3;
+ u32 av1_db_filter_lvl_v : 6;
+ u32 av1_db_filter_lvl_u : 6;
+ u32 av1_db_filter_lvl1 : 6;
+ u32 av1_db_filter_lvl0 : 6;
+ } swreg123_av1;
+ };
+
+ union {
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_3 : 21;
+ } swreg124;
+
+ struct {
+ u32 reserved : 15;
+ u32 av1_cdef_bits : 2;
+ u32 av1_cdef_uv_strengths : 6;
+ u32 av1_cdef_strengths : 6;
+ u32 av1_cdef_damping : 3;
+ } swreg124_av1;
+ };
+
+ union {
+ struct {
+ u32 reserved : 4;
+ u32 intra_satd_lambda_1 : 14;
+ u32 intra_satd_lambda_0 : 14;
+ } swreg125;
+
+ struct {
+ u32 av1_framectx_base : 32;
+ } swreg125_av1;
+ };
+
+ union {
+ struct {
+ u32 reserved : 4;
+ u32 intra_satd_lambda_3 : 14;
+ u32 intra_satd_lambda_2 : 14;
+ } swreg126;
+
+ struct {
+ u32 av1_framectx_base_msb : 32;
+ } swreg126_av1;
+ };
+
+ struct {
+ u32 reserved : 4;
+ u32 intra_satd_lambda_5 : 14;
+ u32 intra_satd_lambda_4 : 14;
+ } swreg127;
+
+ struct {
+ u32 reserved : 4;
+ u32 intra_satd_lambda_7 : 14;
+ u32 intra_satd_lambda_6 : 14;
+ } swreg128;
+
+ struct {
+ u32 reserved : 4;
+ u32 intra_satd_lambda_9 : 14;
+ u32 intra_satd_lambda_8 : 14;
+ } swreg129;
+
+ struct {
+ u32 reserved : 4;
+ u32 intra_satd_lambda_11 : 14;
+ u32 intra_satd_lambda_10 : 14;
+ } swreg130;
+
+ struct {
+ u32 reserved : 4;
+ u32 intra_satd_lambda_13 : 14;
+ u32 intra_satd_lambda_12 : 14;
+ } swreg131;
+
+ struct {
+ u32 reserved : 4;
+ u32 intra_satd_lambda_15 : 14;
+ u32 intra_satd_lambda_14 : 14;
+ } swreg132;
+
+ struct {
+ u32 sse_div_256 : 32;
+ } swreg133;
+
+ struct {
+ u32 nr_mbnum_invert_reg : 16;
+ u32 reserved : 8;
+ u32 noise_low : 6;
+ u32 noise_reduction_enable : 2;
+ } swreg134;
+
+ struct {
+ u32 reserved : 5;
+ u32 thresh_sigma_cur : 21;
+ u32 sliceqp_prev : 6;
+ } swreg135;
+
+ struct {
+ u32 frame_sigma_calced : 16;
+ u32 sigma_cur : 16;
+ } swreg136;
+
+ struct {
+ u32 reserved : 11;
+ u32 thresh_sigma_calced : 21;
+ } swreg137;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_4 : 21;
+ } swreg138;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_5 : 21;
+ } swreg139;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_6 : 21;
+ } swreg140;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_7 : 21;
+ } swreg141;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_8 : 21;
+ } swreg142;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_9 : 21;
+ } swreg143;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_10 : 21;
+ } swreg144;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_11 : 21;
+ } swreg145;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_12 : 21;
+ } swreg146;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_13 : 21;
+ } swreg147;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_14 : 21;
+ } swreg148;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_15 : 21;
+ } swreg149;
+
+ struct {
+ u32 reserved : 6;
+ u32 lambda_satd_me_17 : 13;
+ u32 lambda_satd_me_16 : 13;
+ } swreg150;
+
+ struct {
+ u32 reserved : 6;
+ u32 lambda_satd_me_19 : 13;
+ u32 lambda_satd_me_18 : 13;
+ } swreg151;
+
+ struct {
+ u32 reserved : 6;
+ u32 lambda_satd_me_21 : 13;
+ u32 lambda_satd_me_20 : 13;
+ } swreg152;
+
+ struct {
+ u32 reserved : 6;
+ u32 lambda_satd_me_23 : 13;
+ u32 lambda_satd_me_22 : 13;
+ } swreg153;
+
+ struct {
+ u32 reserved : 6;
+ u32 lambda_satd_me_25 : 13;
+ u32 lambda_satd_me_24 : 13;
+ } swreg154;
+
+ struct {
+ u32 reserved : 6;
+ u32 lambda_satd_me_27 : 13;
+ u32 lambda_satd_me_26 : 13;
+ } swreg155;
+
+ struct {
+ u32 reserved : 6;
+ u32 lambda_satd_me_29 : 13;
+ u32 lambda_satd_me_28 : 13;
+ } swreg156;
+
+ struct {
+ u32 reserved : 6;
+ u32 lambda_satd_me_31 : 13;
+ u32 lambda_satd_me_30 : 13;
+ } swreg157;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_16 : 21;
+ } swreg158;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_17 : 21;
+ } swreg159;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_18 : 21;
+ } swreg160;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_19 : 21;
+ } swreg161;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_20 : 21;
+ } swreg162;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_21 : 21;
+ } swreg163;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_22 : 21;
+ } swreg164;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_23 : 21;
+ } swreg165;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_24 : 21;
+ } swreg166;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_25 : 21;
+ } swreg167;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_26 : 21;
+ } swreg168;
+
+ struct {
+ u32 reserved : 11;
+ u32 lambda_sse_me_27 : 21;
+ } swreg169;
+
+ union {
+ struct {
+ u32 roi1_delta_qp_rc : 5;
+ u32 rc_ctbrc_sliceqpoffset : 6;
+ u32 lambda_sse_me_28 : 21;
+ } swreg170_qp_delta;
+
+ struct {
+ u32 roi1_qp_type : 1;
+ u32 roi1_qp_value : 7;
+ u32 sse_qp_factor : 15;
+ u32 reserved : 8;
+ u32 lambda_depth : 1;
+ } swreg170_qp_absolute;
+ };
+
+ union {
+ struct {
+ u32 roi2_delta_qp_rc : 5;
+ u32 reserved : 6;
+ u32 lambda_sse_me_29 : 21;
+ } swreg171_qp_delta;
+
+ struct {
+ u32 roi2_qp_type : 1;
+ u32 roi2_qp_value : 7;
+ u32 sad_qp_factor : 15;
+ u32 reserved : 9;
+ } swreg171_qp_absolute;
+ };
+
+ struct {
+ u32 complexity_offset : 5;
+ u32 qp_min : 6;
+ u32 lambda_sse_me_30 : 21;
+ } swreg172;
+
+ struct {
+ u32 rc_qpdelta_range : 4;
+ u32 reserved : 1;
+ u32 qp_max : 6;
+ u32 lambda_sse_me_31 : 21;
+ } swreg173;
+
+ struct {
+ u32 reserved : 4;
+ u32 intra_satd_lambda_17 : 14;
+ u32 intra_satd_lambda_16 : 14;
+ } swreg174;
+
+ struct {
+ u32 reserved : 4;
+ u32 intra_satd_lambda_19 : 14;
+ u32 intra_satd_lambda_18 : 14;
+ } swreg175;
+
+ struct {
+ u32 reserved : 4;
+ u32 intra_satd_lambda_21 : 14;
+ u32 intra_satd_lambda_20 : 14;
+ } swreg176;
+
+ struct {
+ u32 reserved : 4;
+ u32 intra_satd_lambda_23 : 14;
+ u32 intra_satd_lambda_22 : 14;
+ } swreg177;
+
+ struct {
+ u32 reserved : 4;
+ u32 intra_satd_lambda_25 : 14;
+ u32 intra_satd_lambda_24 : 14;
+ } swreg178;
+
+ struct {
+ u32 reserved : 4;
+ u32 intra_satd_lambda_27 : 14;
+ u32 intra_satd_lambda_26 : 14;
+ } swreg179;
+
+ struct {
+ u32 reserved : 4;
+ u32 intra_satd_lambda_29 : 14;
+ u32 intra_satd_lambda_28 : 14;
+ } swreg180;
+
+ struct {
+ u32 reserved : 2;
+ u32 rc_block_size : 2;
+ u32 intra_satd_lambda_31 : 14;
+ u32 intra_satd_lambda_30 : 14;
+ } swreg181;
+
+ struct {
+ u32 qp_delta_gain : 16;
+ u32 qp_fractional : 16;
+ } swreg182;
+
+ struct {
+ u32 reserved : 6;
+ u32 qp_sum : 26;
+ } swreg183;
+
+ struct {
+ u32 reserved : 12;
+ u32 qp_num : 20;
+ } swreg184;
+
+ struct {
+ u32 timeout_cycles_msb : 9;
+ u32 pic_complexity : 23;
+ } swreg185;
+
+ struct {
+ u32 cu_information_table_base : 32;
+ } swreg186;
+
+ struct {
+ u32 cu_information_table_base_msb : 32;
+ } swreg187;
+
+ struct {
+ u32 cu_information_base : 32;
+ } swreg188;
+
+ struct {
+ u32 cu_information_base_msb : 32;
+ } swreg189;
+
+ struct {
+ u32 reserved : 30;
+ u32 num_long_term_pics : 2;
+ } swreg190;
+
+ struct {
+ u32 slice_header_size : 16;
+ u32 prefixnal_svc_ext : 1;
+ u32 pps_id : 6;
+ u32 nuh_temporal_id : 3;
+ u32 nal_unit_type : 6;
+ } swreg191;
+
+ struct {
+ u32 framenum : 32;
+ } swreg192;
+
+ struct {
+ u32 entropy_coding_mode : 1;
+ u32 transform8x8_enable : 1;
+ u32 idr_pic_id : 1;
+ u32 nal_ref_idc : 1;
+ u32 yfill_msb : 2;
+ u32 xfill_msb : 2;
+ u32 l0_used_by_next_pic1 : 1;
+ u32 l0_delta_framenum1 : 11;
+ u32 l0_used_by_next_pic0 : 1;
+ u32 l0_delta_framenum0 : 11;
+ } swreg193;
+
+ struct {
+ u32 reserved : 2;
+ u32 cur_longtermidx : 3;
+ u32 max_longtermidx_plus1 : 3;
+ u32 l1_used_by_next_pic1 : 1;
+ u32 l1_delta_framenum1 : 11;
+ u32 l1_used_by_next_pic0 : 1;
+ u32 l1_delta_framenum0 : 11;
+ } swreg194;
+
+ struct {
+ u32 reserved : 2;
+ u32 pic_width_msb : 2;
+ u32 roi2_bottom_msb : 1;
+ u32 roi2_top_msb : 1;
+ u32 roi2_right_msb : 1;
+ u32 roi2_left_msb : 1;
+ u32 roi1_bottom_msb : 1;
+ u32 roi1_top_msb : 1;
+ u32 roi1_right_msb : 1;
+ u32 roi1_left_msb : 1;
+ u32 intra_area_bottom_msb : 1;
+ u32 intra_area_top_msb : 1;
+ u32 intra_area_right_msb : 1;
+ u32 intra_area_left_msb : 1;
+ u32 cir_interval_msb : 4;
+ u32 cir_start_msb : 4;
+ u32 slice_size_msb : 2;
+ u32 num_slices_ready_msb : 2;
+ u32 encoded_ctb_number_msb : 4;
+ } swreg195;
+
+ struct {
+ u32 ctb_row_wr_ptr : 10;
+ u32 ctb_row_rd_ptr : 10;
+ u32 num_ctb_rows_per_sync : 9;
+ u32 input_buf_loopback_en : 1;
+ u32 low_latency_en : 1;
+ u32 low_latency_hw_sync_en : 1;
+ } swreg196;
+
+ union {
+ struct {
+ u32 reserved : 2;
+ u32 l1_delta_poc0_msb : 10;
+ u32 l0_delta_poc1_msb : 10;
+ u32 l0_delta_poc0_msb : 10;
+ } swreg197;
+
+ struct {
+ u32 ctb_row_rd_ptr_jpeg_msb : 5;
+ u32 ctb_row_wr_ptr_jpeg_msb : 5;
+ u32 reserved : 22;
+ } swreg197_jpeg;
+ };
+
+ struct {
+ u32 l1_longtermidx1 : 3;
+ u32 l1_longtermidx0 : 3;
+ u32 l0_longtermidx1 : 3;
+ u32 l0_longtermidx0 : 3;
+ u32 mark_current_longterm : 1;
+ u32 l0_delta_framenum0_msb : 9;
+ u32 l1_delta_poc1_msb : 10;
+ } swreg198;
+
+ struct {
+ u32 osd_alphablend_enable : 1;
+ u32 hash_offset : 2;
+ u32 hash_type : 2;
+ u32 l1_delta_framenum1_msb : 9;
+ u32 l1_delta_framenum0_msb : 9;
+ u32 l0_delta_framenum1_msb : 9;
+ } swreg199;
+
+ struct {
+ u32 hash_val : 32;
+ } swreg200;
+
+ struct {
+ u32 mean_thr3 : 8;
+ u32 mean_thr2 : 8;
+ u32 mean_thr1 : 8;
+ u32 mean_thr0 : 8;
+ } swreg201;
+
+ struct {
+ u32 thr_dc_chroma_8x8 : 16;
+ u32 thr_dc_lum_8x8 : 16;
+ } swreg202;
+
+ union {
+ struct {
+ u32 cr_dc_sum_thr : 8;
+ u32 cb_dc_sum_thr : 8;
+ u32 reserved : 8;
+ u32 lum_dc_sum_thr : 8;
+ } swreg203_h264;
+
+ struct {
+ u32 thr_dc_chroma_16x16 : 16;
+ u32 thr_dc_lum_16x16 : 16;
+ } swreg203_hevc;
+ };
+
+ struct {
+ u32 thr_dc_chroma_32x32 : 16;
+ u32 thr_dc_lum_32x32 : 16;
+ } swreg204;
+
+ struct {
+ u32 thr_ac_num_chroma_8x8 : 16;
+ u32 thr_ac_num_lum_8x8 : 16;
+ } swreg205;
+
+ struct {
+ u32 thr_ac_num_chroma_16x16 : 16;
+ u32 thr_ac_num_lum_16x16 : 16;
+ } swreg206;
+
+ struct {
+ u32 thr_ac_num_chroma_32x32 : 16;
+ u32 thr_ac_num_lum_32x32 : 16;
+ } swreg207;
+
+ union {
+ struct {
+ u32 reserved0 : 3;
+ u32 skip_map_enable : 1;
+ u32 ipcm1_left : 9;
+ u32 enable_smart : 1;
+ u32 foreground_pixel_thx : 6;
+ u32 reserved1 : 6;
+ u32 smart_qp : 6;
+ } swreg208_h264;
+
+ struct {
+ u32 reserved : 3;
+ u32 skip_map_enable : 1;
+ u32 ipcm1_left : 9;
+ u32 enable_smart : 1;
+ u32 foreground_pixel_thx : 6;
+ u32 mdqpc : 6;
+ u32 mdqpy : 6;
+ } swreg208_hevc;
+ };
+
+ struct {
+ u32 reserved : 3;
+ u32 ipcm_map_enable : 1;
+ u32 pcm_filter_disable : 1;
+ u32 ipcm1_bottom : 9;
+ u32 ipcm1_top : 9;
+ u32 ipcm1_right : 9;
+ } swreg209;
+
+ struct {
+ u32 reserved : 3;
+ u32 ipcm2_left : 9;
+ u32 input_lu_stride : 20;
+ } swreg210;
+
+ struct {
+ u32 reserved : 3;
+ u32 ipcm2_right : 9;
+ u32 input_ch_stride : 20;
+ } swreg211;
+
+ struct {
+ u32 reserved : 3;
+ u32 ipcm2_top : 9;
+ u32 ref_lu_stride : 20;
+ } swreg212;
+
+ struct {
+ u32 reserved : 5;
+ u32 ipcm2_bottom : 9;
+ u32 ref_ds_lu_stride : 18;
+ } swreg213;
+
+ struct {
+ u32 hwmaxvideowidthjpeg : 13;
+ u32 hwmaxvideowidthh264 : 13;
+ u32 hwroimapversion : 3;
+ u32 hwintratu32support : 1;
+ u32 hwabsqpsupport : 1;
+ u32 hwljpegsupport : 1;
+ } swreg214;
+
+ struct {
+ u32 totalarlen : 32;
+ } swreg215;
+
+ struct {
+ u32 totalr : 32;
+ } swreg216;
+
+ struct {
+ u32 totalar : 32;
+ } swreg217;
+
+ struct {
+ u32 totalrlast : 32;
+ } swreg218;
+
+ struct {
+ u32 totalawlen : 32;
+ } swreg219;
+
+ struct {
+ u32 totalw : 32;
+ } swreg220;
+
+ struct {
+ u32 totalaw : 32;
+ } swreg221;
+
+ struct {
+ u32 totalwlast : 32;
+ } swreg222;
+
+ struct {
+ u32 totalb : 32;
+ } swreg223;
+
+ struct {
+ u32 cb_const_pixel : 10;
+ u32 cr_const_pixel : 10;
+ u32 skipframe_en : 1;
+ u32 ssim_en : 1;
+ u32 reserved : 9;
+ u32 chroma_const_en : 1;
+ } swreg224;
+
+ struct {
+ u32 reserved : 6;
+ u32 roimap_qpdelta_ver : 3;
+ u32 roimap_cuctrl_ver : 3;
+ u32 roimap_cuctrl_enable : 1;
+ u32 roimap_cuctrl_index_enable : 1;
+ u32 loop_filter_across_tiles_enabled_flag : 1;
+ u32 tiles_enabled_flag : 1;
+ u32 num_tile_rows : 8;
+ u32 num_tile_columns : 8;
+ } swreg225;
+
+ struct {
+ u32 hwdynamicmaxtusize : 1;
+ u32 hwiframeonly : 1;
+ u32 hwstreamsegmentsupport : 1;
+ u32 hwstreambufchain : 1;
+ u32 hwinloopdsratio : 1;
+ u32 hwmultipasssupport : 1;
+ u32 hwrdoqsupport : 1;
+ u32 bframe_me4n_hor_searchrange : 2;
+ u32 hwroi8support : 1;
+ u32 hwgmvsupport : 1;
+ u32 hwjpeg422support : 1;
+ u32 hwctbrcversion : 3;
+ u32 me_vert_searchrange_h264 : 6;
+ u32 me_vert_searchrange_hevc : 6;
+ u32 hwcuinforversion : 3;
+ u32 hwp010refsupport : 1;
+ u32 hwssimsupport : 1;
+ } swreg226;
+
+ struct {
+ u32 ssim_y_numerator_lsb : 32;
+ } swreg227;
+
+ struct {
+ u32 ssim_y_numerator_msb : 32;
+ } swreg228;
+
+ struct {
+ u32 ssim_u_numerator_lsb : 32;
+ } swreg229;
+
+ struct {
+ u32 ssim_u_numerator_msb : 32;
+ } swreg230;
+
+ struct {
+ u32 ssim_v_numerator_lsb : 32;
+ } swreg231;
+
+ struct {
+ u32 ssim_v_numerator_msb : 32;
+ } swreg232;
+
+ struct {
+ u32 ssim_y_denominator : 32;
+ } swreg233;
+
+ struct {
+ u32 ssim_uv_denominator : 32;
+ } swreg234;
+
+ struct {
+ u32 rps_used_by_cur_1 : 1;
+ u32 rps_used_by_cur_0 : 1;
+ u32 rps_delta_poc_2 : 10;
+ u32 rps_delta_poc_1 : 10;
+ u32 rps_delta_poc_0 : 10;
+ } swreg235;
+
+ struct {
+ u32 reserved : 12;
+ u32 p010_ref_enable : 1;
+ u32 short_term_ref_pic_set_sps_flag : 1;
+ u32 rps_pos_pic_num : 3;
+ u32 rps_neg_pic_num : 3;
+ u32 rps_used_by_cur_3 : 1;
+ u32 rps_used_by_cur_2 : 1;
+ u32 rps_delta_poc_3 : 10;
+ } swreg236;
+
+ struct {
+ u32 reserved : 11;
+ u32 dummyreaden : 1;
+ u32 ref_ch_stride : 20;
+ } swreg237;
+
+ struct {
+ u32 dummyreadaddr : 32;
+ } swreg238;
+
+ struct {
+ u32 current_ctb_mad_base : 32;
+ } swreg239;
+
+ struct {
+ u32 current_ctb_mad_base_msb : 32;
+ } swreg240;
+
+ struct {
+ u32 previous_ctb_mad_base : 32;
+ } swreg241;
+
+ struct {
+ u32 previous_ctb_mad_base_msb : 32;
+ } swreg242;
+
+ struct {
+ u32 reserved : 11;
+ u32 ctb_rc_model_param0 : 21;
+ } swreg243;
+
+ struct {
+ u32 reserved : 2;
+ u32 roi3_qp_type : 1;
+ u32 roi3_qp_value : 7;
+ u32 ctb_rc_model_param1 : 22;
+ } swreg244;
+
+ struct {
+ u32 rc_qpdelta_range_msb : 2;
+ u32 ctb_rc_row_factor : 16;
+ u32 ctb_rc_model_param_min : 14;
+ } swreg245;
+
+ struct {
+ u32 reserved : 3;
+ u32 ctb_rc_delay : 3;
+ u32 axi_write_outstanding_num : 8;
+ u32 ctb_rc_qp_step : 18;
+ } swreg246;
+
+ struct {
+ u32 reserved0 : 1;
+ u32 ctb_rc_prev_mad_valid : 1;
+ u32 reserved1 : 4;
+ u32 prev_pic_lum_mad : 26;
+ } swreg247;
+
+ struct {
+ u32 roi4_qp_type : 1;
+ u32 roi4_qp_value : 7;
+ u32 ctb_qp_sum_for_rc : 24;
+ } swreg248;
+
+ union {
+ struct {
+ u32 reserved : 3;
+ u32 ipcm2_bottom_msb : 1;
+ u32 ipcm2_top_msb : 1;
+ u32 ipcm2_right_msb : 1;
+ u32 ipcm2_left_msb : 1;
+ u32 ipcm1_bottom_msb : 1;
+ u32 ipcm1_top_msb : 1;
+ u32 ipcm1_right_msb : 1;
+ u32 ipcm1_left_msb : 1;
+ u32 pic_width_msb2 : 1;
+ u32 roi2_bottom_msb2 : 1;
+ u32 roi2_top_msb2 : 1;
+ u32 roi2_right_msb2 : 1;
+ u32 roi2_left_msb2 : 1;
+ u32 roi1_bottom_msb2 : 1;
+ u32 roi1_top_msb2 : 1;
+ u32 roi1_right_msb2 : 1;
+ u32 roi1_left_msb2 : 1;
+ u32 intra_area_bottom_msb2 : 1;
+ u32 intra_area_top_msb2 : 1;
+ u32 intra_area_right_msb2 : 1;
+ u32 intra_area_left_msb2 : 1;
+ u32 cir_interval_msb2 : 2;
+ u32 cir_start_msb2 : 2;
+ u32 slice_size_msb2 : 1;
+ u32 num_slices_ready_msb2 : 1;
+ u32 encoded_ctb_number_msb2 : 2;
+ } swreg249;
+
+ struct {
+ u32 reserved0 : 5;
+ u32 jpeg_rowlength_msb : 2;
+ u32 jpeg_pic_height_msb : 2;
+ u32 jpeg_pic_width_msb : 2;
+ u32 reserved1 : 21;
+ } swreg249_jpeg;
+ };
+
+ struct {
+ u32 reserved : 4;
+ u32 global_vertical_mv_l0 : 14;
+ u32 global_horizontal_mv_l0 : 14;
+ } swreg250;
+
+ struct {
+ u32 reserved : 4;
+ u32 global_vertical_mv_l1 : 14;
+ u32 global_horizontal_mv_l1 : 14;
+ } swreg251;
+
+ struct {
+ u32 reserved : 2;
+ u32 roi3_right : 10;
+ u32 roi3_top : 10;
+ u32 roi3_left : 10;
+ } swreg252;
+
+ struct {
+ u32 reserved : 2;
+ u32 roi4_top : 10;
+ u32 roi4_left : 10;
+ u32 roi3_bottom : 10;
+ } swreg253;
+
+ struct {
+ u32 reserved : 2;
+ u32 roi5_left : 10;
+ u32 roi4_bottom : 10;
+ u32 roi4_right : 10;
+ } swreg254;
+
+ struct {
+ u32 reserved : 2;
+ u32 roi5_bottom : 10;
+ u32 roi5_right : 10;
+ u32 roi5_top : 10;
+ } swreg255;
+
+ struct {
+ u32 reserved : 2;
+ u32 roi6_right : 10;
+ u32 roi6_top : 10;
+ u32 roi6_left : 10;
+ } swreg256;
+
+ struct {
+ u32 reserved : 2;
+ u32 roi7_top : 10;
+ u32 roi7_left : 10;
+ u32 roi6_bottom : 10;
+ } swreg257;
+
+ struct {
+ u32 reserved : 2;
+ u32 roi8_left : 10;
+ u32 roi7_bottom : 10;
+ u32 roi7_right : 10;
+ } swreg258;
+
+ struct {
+ u32 reserved : 1;
+ u32 current_max_tu_size_decrease : 1;
+ u32 roi8_bottom : 10;
+ u32 roi8_right : 10;
+ u32 roi8_top : 10;
+ } swreg259;
+
+ struct {
+ u32 roi5_qp_type : 1;
+ u32 roi5_qp_value : 7;
+ u32 roi6_qp_type : 1;
+ u32 roi6_qp_value : 7;
+ u32 roi7_qp_type : 1;
+ u32 roi7_qp_value : 7;
+ u32 roi8_qp_type : 1;
+ u32 roi8_qp_value : 7;
+ } swreg260;
+
+ struct {
+ u32 motion_score_enable : 1;
+ u32 pass1_skip_cabac : 1;
+ u32 rdoq_enable : 1;
+ u32 multi_core_en : 1;
+ u32 axi_read_outstanding_num : 8;
+ u32 prp_in_loop_ds_ratio : 1;
+ u32 rgblumaoffset : 5;
+ u32 reserved : 14;
+ } swreg261;
+
+ struct {
+ u32 lum_sse_div_256 : 32;
+ } swreg262;
+
+ struct {
+ u32 cb_sse_div_64 : 32;
+ } swreg263;
+
+ struct {
+ u32 cr_sse_div_64 : 32;
+ } swreg264;
+
+ struct {
+ u32 ddr_polling_interval : 16;
+ u32 ref_ready_threshold : 16;
+ } swreg265;
+
+ struct {
+ u32 multicore_sync_l0_addr : 32;
+ } swreg266;
+
+ struct {
+ u32 multicore_sync_l0_addr_msb : 32;
+ } swreg267;
+
+ struct {
+ u32 multicore_sync_l1_addr : 32;
+ } swreg268;
+
+ struct {
+ u32 multicore_sync_l1_addr_msb : 32;
+ } swreg269;
+
+ struct {
+ u32 multicore_sync_rec_addr : 32;
+ } swreg270;
+
+ struct {
+ u32 multicore_sync_rec_addr_msb : 32;
+ } swreg271;
+
+ struct {
+ u32 wr_urgent_disable_threshold : 8;
+ u32 wr_urgent_enable_threshold : 8;
+ u32 rd_urgent_disable_threshold : 8;
+ u32 rd_urgent_enable_threshold : 8;
+ } swreg272;
+
+ struct {
+ u32 roimap_cuctrl_index_addr : 32;
+ } swreg273;
+
+ struct {
+ u32 roimap_cuctrl_index_addr_msb : 32;
+ } swreg274;
+
+ struct {
+ u32 roimap_cuctrl_addr : 32;
+ } swreg275;
+
+ struct {
+ u32 roimap_cuctrl_addr_msb : 32;
+ } swreg276;
+
+ struct {
+ u32 reserved : 5;
+ u32 syn_amount_per_loopback : 15;
+ u32 pic_order_cnt_type : 2;
+ u32 log2_max_frame_num : 5;
+ u32 log2_max_pic_order_cnt_lsb : 5;
+ } swreg277;
+
+ struct {
+ u32 output_strm_buf1_base : 32;
+ } swreg278;
+
+ struct {
+ u32 output_strm_buf1_base_msb : 32;
+ } swreg279;
+
+ struct {
+ u32 output_strm_buffer1_limit : 32;
+ } swreg280;
+
+ struct {
+ u32 reserved : 2;
+ u32 chroma_format_idc : 2;
+ u32 num_ctb_rows_per_sync_msb : 6;
+ u32 strm_segment_wr_ptr : 10;
+ u32 strm_segment_rd_ptr : 10;
+ u32 strm_segment_en : 1;
+ u32 strm_segment_sw_sync_en : 1;
+ } swreg281;
+
+ struct {
+ u32 strm_segment_size : 32;
+ } swreg282;
+
+ struct {
+ u32 motion_score_l0_0 : 32;
+ } swreg283;
+
+ struct {
+ u32 motion_score_l0_1 : 32;
+ } swreg284;
+
+ struct {
+ u32 motion_score_l1_0 : 32;
+ } swreg285;
+
+ struct {
+ u32 motion_score_l1_1 : 32;
+ } swreg286;
+
+ struct {
+ u32 reserved0 : 21;
+ u32 hwmonochromesupport : 1;
+ u32 hwmevertrangeprogramable : 1;
+ u32 hwctbrcmoremode : 1;
+ u32 reserved1 : 4;
+ u32 hwcutreesupport : 1;
+ u32 hwscaler420support : 1;
+ u32 hwcscextensionsupport : 1;
+ u32 hwvideoheightext : 1;
+ } swreg287;
+
+ struct {
+ u32 reserved : 12;
+ u32 cuinfoversion : 3;
+ u32 ctb_qp_sum_for_rc_msb : 2;
+ u32 pic_complexity_msb : 4;
+ u32 qp_num_msb : 3;
+ u32 qp_sum_msb : 2;
+ u32 skipcu8num_msb : 3;
+ u32 intracu8num_msb : 3;
+ } swreg288;
+
+ struct {
+ u32 rgbcoeffh : 16;
+ u32 rgbcoeffg : 16;
+ } swreg289;
+
+ struct {
+ u32 totalarlen2 : 32;
+ } swreg290;
+
+ struct {
+ u32 totalr2 : 32;
+ } swreg291;
+
+ struct {
+ u32 totalar2 : 32;
+ } swreg292;
+
+ struct {
+ u32 totalrlast2 : 32;
+ } swreg293;
+
+ struct {
+ u32 totalawlen2 : 32;
+ } swreg294;
+
+ struct {
+ u32 totalw2 : 32;
+ } swreg295;
+
+ struct {
+ u32 totalaw2 : 32;
+ } swreg296;
+
+ struct {
+ u32 totalwlast2 : 32;
+ } swreg297;
+
+ struct {
+ u32 totalb2 : 32;
+ } swreg298;
+
+ struct {
+ u32 ext_sram_lum_fwd_base : 32;
+ } swreg299;
+
+ struct {
+ u32 ext_sram_lum_fwd_base_msb : 32;
+ } swreg300;
+
+ struct {
+ u32 ext_sram_lum_bwd_base : 32;
+ } swreg301;
+
+ struct {
+ u32 ext_sram_lum_bwd_base_msb : 32;
+ } swreg302;
+
+ struct {
+ u32 ext_sram_chr_fwd_base : 32;
+ } swreg303;
+
+ struct {
+ u32 ext_sram_chr_fwd_base_msb : 32;
+ } swreg304;
+
+ struct {
+ u32 ext_sram_chr_bwd_base : 32;
+ } swreg305;
+
+ struct {
+ u32 ext_sram_chr_bwd_base_msb : 32;
+ } swreg306;
+
+ struct {
+ u32 extlinebuffer_linecnt_chr_bwd : 8;
+ u32 extlinebuffer_linecnt_chr_fwd : 8;
+ u32 extlinebuffer_linecnt_lum_bwd : 8;
+ u32 extlinebuffer_linecnt_lum_fwd : 8;
+ } swreg307;
+
+ struct {
+ u32 axi_strm_write_pending : 32;
+ } swreg308;
+
+ struct {
+ u32 axi_recon_write_pending : 32;
+ } swreg309;
+
+ struct {
+ u32 axi_rec4n_write_pending : 32;
+ } swreg310;
+
+ struct {
+ u32 axi_prp_read_pending : 32;
+ } swreg311;
+
+ struct {
+ u32 axi_ref_read_pending : 32;
+ } swreg312;
+
+ struct {
+ u32 axi_ref4n_read_pending : 32;
+ } swreg313;
+
+ struct {
+ u32 axi_rcroi_read_pending : 32;
+ } swreg314;
+
+ struct {
+ u32 axi_read_channel_pending : 32;
+ } swreg315;
+
+ struct {
+ u32 axi_write_channel_pending : 32;
+ } swreg316;
+
+ struct {
+ u32 axi_total_pending : 32;
+ } swreg317;
+
+ struct {
+ u32 hw_debug : 32;
+ } swreg318;
+
+ struct {
+ u32 axi_burst_align_fuse_rd_lu_ref_prefetch : 4;
+ u32 axi_burst_align_fuse_rd_ch_ref_prefetch : 4;
+ u32 axi_burst_align_fuse_rd_prp : 4;
+ u32 axi_burst_align_fuse_rd_common : 4;
+ u32 axi_burst_align_fuse_wr_luma_ref : 4;
+ u32 axi_burst_align_fuse_wr_chroma_ref : 4;
+ u32 axi_burst_align_fuse_wr_stream : 4;
+ u32 axi_burst_align_fuse_wr_common : 4;
+ } swreg319;
+
+ struct {
+ u32 axi_burst_align_rd_lu_ref_prefetch : 4;
+ u32 axi_burst_align_rd_ch_ref_prefetch : 4;
+ u32 axi_burst_align_rd_prp : 4;
+ u32 axi_burst_align_rd_common : 4;
+ u32 axi_burst_align_wr_luma_ref : 4;
+ u32 axi_burst_align_wr_chroma_ref : 4;
+ u32 axi_burst_align_wr_stream : 4;
+ u32 axi_burst_align_wr_common : 4;
+ } swreg320;
+
+ struct {
+ u32 reserved : 26;
+ u32 me_assigned_vert_search_range : 6;
+ } swreg321;
+} __packed;
+
+#endif /* HANTRO_VC8000E_REGS_H_ */
--
2.53.0
^ permalink raw reply related
* [PATCH 11/14] media: verisilicon: Add common encoder parm and frameintervals ioctls
From: Paul Kocialkowski @ 2026-05-22 10:16 UTC (permalink / raw)
To: devicetree, imx, linux-arm-kernel, linux-kernel, linux-media
Cc: Rob Herring, Krzysztof Kozlowski, Conor Dooley, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Nicolas Dufresne,
Benjamin Gaignard, Philipp Zabel, Mauro Carvalho Chehab,
Hans Verkuil, Marco Felsch, Michael Tretter, Paul Kocialkowski
In-Reply-To: <20260522101653.2565125-1-paulk@sys-base.io>
This introduces the required encoder ioctls for configuring the
frame rate (via the parm timeperframe field) and enumerating available
frame rates (via enum_frameintervals) for the encoder.
Signed-off-by: Paul Kocialkowski <paulk@sys-base.io>
---
drivers/media/platform/verisilicon/hantro.h | 3 +
.../media/platform/verisilicon/hantro_drv.c | 3 +
.../media/platform/verisilicon/hantro_v4l2.c | 99 +++++++++++++++++++
3 files changed, 105 insertions(+)
diff --git a/drivers/media/platform/verisilicon/hantro.h b/drivers/media/platform/verisilicon/hantro.h
index e0fdc4535b2d..badd0b13988c 100644
--- a/drivers/media/platform/verisilicon/hantro.h
+++ b/drivers/media/platform/verisilicon/hantro.h
@@ -258,6 +258,9 @@ struct hantro_ctx {
struct v4l2_pix_format_mplane dst_fmt;
struct v4l2_pix_format_mplane ref_fmt;
+ struct v4l2_fract src_timeperframe;
+ struct v4l2_fract dst_timeperframe;
+
struct v4l2_ctrl_handler ctrl_handler;
int jpeg_quality;
int bit_depth;
diff --git a/drivers/media/platform/verisilicon/hantro_drv.c b/drivers/media/platform/verisilicon/hantro_drv.c
index 6f72e25fa88c..d798ba361b25 100644
--- a/drivers/media/platform/verisilicon/hantro_drv.c
+++ b/drivers/media/platform/verisilicon/hantro_drv.c
@@ -923,8 +923,11 @@ static int hantro_add_func(struct hantro_dev *vpu, unsigned int funcid)
vpu->decoder = func;
v4l2_disable_ioctl(vfd, VIDIOC_TRY_ENCODER_CMD);
v4l2_disable_ioctl(vfd, VIDIOC_ENCODER_CMD);
+ v4l2_disable_ioctl(vfd, VIDIOC_ENUM_FRAMEINTERVALS);
v4l2_disable_ioctl(vfd, VIDIOC_G_SELECTION);
v4l2_disable_ioctl(vfd, VIDIOC_S_SELECTION);
+ v4l2_disable_ioctl(vfd, VIDIOC_G_PARM);
+ v4l2_disable_ioctl(vfd, VIDIOC_S_PARM);
}
video_set_drvdata(vfd, vpu);
diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index 1001feee5c07..2125f2913d9a 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -213,6 +213,41 @@ static int vidioc_enum_framesizes(struct file *file, void *priv,
return 0;
}
+static int vidioc_enum_frameintervals(struct file *file, void *priv,
+ struct v4l2_frmivalenum *fival)
+{
+ struct v4l2_frmsizeenum fsize = { 0 };
+ unsigned int width = fival->width;
+ unsigned int height = fival->height;
+ int ret;
+
+ if (fival->index > 0)
+ return -EINVAL;
+
+ /* First check that the provided format and dimensions are valid. */
+ fsize.pixel_format = fival->pixel_format;
+ ret = vidioc_enum_framesizes(file, priv, &fsize);
+ if (ret)
+ return ret;
+
+ if (width < fsize.stepwise.min_width ||
+ width > fsize.stepwise.max_width ||
+ height < fsize.stepwise.min_height ||
+ height > fsize.stepwise.max_height)
+ return -EINVAL;
+
+ /* Any possible frame interval is acceptable. */
+ fival->type = V4L2_FRMIVAL_TYPE_CONTINUOUS;
+ fival->stepwise.min.numerator = 1;
+ fival->stepwise.min.denominator = USHRT_MAX;
+ fival->stepwise.max.numerator = USHRT_MAX;
+ fival->stepwise.max.denominator = 1;
+ fival->stepwise.step.numerator = 1;
+ fival->stepwise.step.denominator = 1;
+
+ return 0;
+}
+
static int vidioc_enum_fmt(struct file *file, void *priv,
struct v4l2_fmtdesc *f, bool capture)
@@ -484,10 +519,23 @@ hantro_reset_raw_fmt(struct hantro_ctx *ctx, int bit_depth, bool need_postproc)
return ret;
}
+static void
+hantro_reset_timeperframe(struct hantro_ctx *ctx)
+{
+ struct v4l2_fract *timeperframe = &ctx->src_timeperframe;
+ struct v4l2_fract *timeperframe_propagate = &ctx->dst_timeperframe;
+
+ timeperframe->numerator = 1;
+ timeperframe->denominator = 25;
+
+ *timeperframe_propagate = *timeperframe;
+}
+
void hantro_reset_fmts(struct hantro_ctx *ctx)
{
hantro_reset_encoded_fmt(ctx);
hantro_reset_raw_fmt(ctx, HANTRO_DEFAULT_BIT_DEPTH, HANTRO_AUTO_POSTPROC);
+ hantro_reset_timeperframe(ctx);
}
static void
@@ -739,6 +787,54 @@ static int vidioc_s_selection(struct file *file, void *priv,
return 0;
}
+static int vidioc_g_parm(struct file *file, void *priv,
+ struct v4l2_streamparm *parm)
+{
+ struct hantro_ctx *ctx = file_to_ctx(file);
+ struct v4l2_fract *timeperframe;
+
+ if (V4L2_TYPE_IS_OUTPUT(parm->type)) {
+ timeperframe = &ctx->src_timeperframe;
+ parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
+ parm->parm.output.timeperframe = *timeperframe;
+ } else {
+ timeperframe = &ctx->dst_timeperframe;
+ parm->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
+ parm->parm.capture.timeperframe = *timeperframe;
+ }
+
+ return 0;
+}
+
+static int vidioc_s_parm(struct file *file, void *priv,
+ struct v4l2_streamparm *parm)
+{
+ struct hantro_ctx *ctx = file_to_ctx(file);
+ struct v4l2_fract *timeperframe_propagate;
+ struct v4l2_fract *timeperframe_ctx;
+ struct v4l2_fract *timeperframe;
+
+ if (V4L2_TYPE_IS_OUTPUT(parm->type)) {
+ parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
+ timeperframe = &parm->parm.output.timeperframe;
+ timeperframe_ctx = &ctx->src_timeperframe;
+ timeperframe_propagate = &ctx->dst_timeperframe;
+ } else {
+ parm->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
+ timeperframe = &parm->parm.capture.timeperframe;
+ timeperframe_ctx = &ctx->dst_timeperframe;
+ timeperframe_propagate = NULL;
+ }
+
+ *timeperframe_ctx = *timeperframe;
+
+ /* Propagate from source to destination. */
+ if (timeperframe_propagate)
+ *timeperframe_propagate = *timeperframe;
+
+ return 0;
+}
+
static const struct v4l2_event hantro_eos_event = {
.type = V4L2_EVENT_EOS
};
@@ -774,6 +870,7 @@ static int vidioc_encoder_cmd(struct file *file, void *priv,
const struct v4l2_ioctl_ops hantro_ioctl_ops = {
.vidioc_querycap = vidioc_querycap,
.vidioc_enum_framesizes = vidioc_enum_framesizes,
+ .vidioc_enum_frameintervals = vidioc_enum_frameintervals,
.vidioc_try_fmt_vid_cap_mplane = vidioc_try_fmt_cap_mplane,
.vidioc_try_fmt_vid_out_mplane = vidioc_try_fmt_out_mplane,
@@ -801,6 +898,8 @@ const struct v4l2_ioctl_ops hantro_ioctl_ops = {
.vidioc_g_selection = vidioc_g_selection,
.vidioc_s_selection = vidioc_s_selection,
+ .vidioc_g_parm = vidioc_g_parm,
+ .vidioc_s_parm = vidioc_s_parm,
.vidioc_decoder_cmd = v4l2_m2m_ioctl_stateless_decoder_cmd,
.vidioc_try_decoder_cmd = v4l2_m2m_ioctl_stateless_try_decoder_cmd,
--
2.53.0
^ permalink raw reply related
* [PATCH 10/14] media: hantro: use hantro_decoded_buffer only for dst_vq
From: Paul Kocialkowski @ 2026-05-22 10:16 UTC (permalink / raw)
To: devicetree, imx, linux-arm-kernel, linux-kernel, linux-media
Cc: Rob Herring, Krzysztof Kozlowski, Conor Dooley, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Nicolas Dufresne,
Benjamin Gaignard, Philipp Zabel, Mauro Carvalho Chehab,
Hans Verkuil, Marco Felsch, Michael Tretter
In-Reply-To: <20260522101653.2565125-1-paulk@sys-base.io>
From: Marco Felsch <m.felsch@pengutronix.de>
The dst_vq buffer size for encoders should not use the size of the
'hantro_decoded_buffer'. Make use of 'v4l2_m2m_buffer' instead till some
encoder requires pre buffer extra data.
Signed-off-by: Marco Felsch <m.felsch@pengutronix.de>
---
drivers/media/platform/verisilicon/hantro_drv.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/media/platform/verisilicon/hantro_drv.c b/drivers/media/platform/verisilicon/hantro_drv.c
index e21306f2bf2e..6f72e25fa88c 100644
--- a/drivers/media/platform/verisilicon/hantro_drv.c
+++ b/drivers/media/platform/verisilicon/hantro_drv.c
@@ -248,7 +248,10 @@ queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
dst_vq->io_modes = VB2_MMAP | VB2_DMABUF;
dst_vq->drv_priv = ctx;
dst_vq->ops = &hantro_queue_ops;
- dst_vq->buf_struct_size = sizeof(struct hantro_decoded_buffer);
+ if (ctx->is_encoder)
+ dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+ else
+ dst_vq->buf_struct_size = sizeof(struct hantro_decoded_buffer);
dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
dst_vq->lock = &ctx->dev->vpu_mutex;
dst_vq->dev = ctx->dev->v4l2_dev.dev;
--
2.53.0
^ permalink raw reply related
* [PATCH 09/14] media: verisilicon: Cancel job with runtime pm put/clk disable on failure
From: Paul Kocialkowski @ 2026-05-22 10:16 UTC (permalink / raw)
To: devicetree, imx, linux-arm-kernel, linux-kernel, linux-media
Cc: Rob Herring, Krzysztof Kozlowski, Conor Dooley, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Nicolas Dufresne,
Benjamin Gaignard, Philipp Zabel, Mauro Carvalho Chehab,
Hans Verkuil, Marco Felsch, Michael Tretter, Paul Kocialkowski
In-Reply-To: <20260522101653.2565125-1-paulk@sys-base.io>
The no-pm variant of hantro_job_finish is only good to use directly if
runtime pm get failed. In other cases, we need to do a runtime pm put
and bulk clk disable to correctly undo what was set up.
Fixes: 892bb6ecead9 ("media: hantro: do a PM resume earlier")
Signed-off-by: Paul Kocialkowski <paulk@sys-base.io>
---
.../media/platform/verisilicon/hantro_drv.c | 19 ++++++++++++-------
1 file changed, 12 insertions(+), 7 deletions(-)
diff --git a/drivers/media/platform/verisilicon/hantro_drv.c b/drivers/media/platform/verisilicon/hantro_drv.c
index e0c11fe8b55c..e21306f2bf2e 100644
--- a/drivers/media/platform/verisilicon/hantro_drv.c
+++ b/drivers/media/platform/verisilicon/hantro_drv.c
@@ -169,29 +169,34 @@ void hantro_end_prepare_run(struct hantro_ctx *ctx)
static void device_run(void *priv)
{
struct hantro_ctx *ctx = priv;
+ struct hantro_dev *vpu = ctx->dev;
struct vb2_v4l2_buffer *src, *dst;
int ret;
src = hantro_get_src_buf(ctx);
dst = hantro_get_dst_buf(ctx);
- ret = pm_runtime_resume_and_get(ctx->dev->dev);
+ ret = pm_runtime_resume_and_get(vpu->dev);
if (ret < 0)
- goto err_cancel_job;
+ goto err_cancel_job_no_pm;
- ret = clk_bulk_enable(ctx->dev->variant->num_clocks, ctx->dev->clocks);
+ ret = clk_bulk_enable(vpu->variant->num_clocks, vpu->clocks);
if (ret)
- goto err_cancel_job;
+ goto err_cancel_job_pm;
v4l2_m2m_buf_copy_metadata(src, dst, true);
if (ctx->codec_ops->run(ctx))
- goto err_cancel_job;
+ goto err_cancel_job_clk;
return;
-err_cancel_job:
- hantro_job_finish_no_pm(ctx->dev, ctx, VB2_BUF_STATE_ERROR);
+err_cancel_job_clk:
+ clk_bulk_disable(vpu->variant->num_clocks, vpu->clocks);
+err_cancel_job_pm:
+ pm_runtime_put_autosuspend(vpu->dev);
+err_cancel_job_no_pm:
+ hantro_job_finish_no_pm(vpu, ctx, VB2_BUF_STATE_ERROR);
}
static const struct v4l2_m2m_ops vpu_m2m_ops = {
--
2.53.0
^ permalink raw reply related
* [PATCH 08/14] media: verisilicon: Report default pixel coding for non-JPEG and fix JPEG case
From: Paul Kocialkowski @ 2026-05-22 10:16 UTC (permalink / raw)
To: devicetree, imx, linux-arm-kernel, linux-kernel, linux-media
Cc: Rob Herring, Krzysztof Kozlowski, Conor Dooley, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Nicolas Dufresne,
Benjamin Gaignard, Philipp Zabel, Mauro Carvalho Chehab,
Hans Verkuil, Marco Felsch, Michael Tretter, Paul Kocialkowski
In-Reply-To: <20260522101653.2565125-1-paulk@sys-base.io>
The JPEG colorspace is very specific to the JPEG coded format and is
not relevant as a default for other types of coded (or non-coded) formats.
These would typically use ITU-R Rec. BT.709 but it could be a number of other
ones as well so reporting the default value is best.
Furthermore other pixel coding attributes are best set accordingly instead
of keeping default values in the JPEG case.
Signed-off-by: Paul Kocialkowski <paulk@sys-base.io>
---
.../media/platform/verisilicon/hantro_v4l2.c | 18 ++++++++++++++----
1 file changed, 14 insertions(+), 4 deletions(-)
diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index fcf3bd9bcda2..1001feee5c07 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -415,10 +415,20 @@ hantro_reset_fmt(struct v4l2_pix_format_mplane *fmt,
fmt->pixelformat = vpu_fmt->fourcc;
fmt->field = V4L2_FIELD_NONE;
- fmt->colorspace = V4L2_COLORSPACE_JPEG;
- fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
- fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
- fmt->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+
+ if (fmt->pixelformat == V4L2_PIX_FMT_JPEG) {
+ fmt->colorspace = V4L2_COLORSPACE_JPEG;
+ fmt->ycbcr_enc = V4L2_MAP_YCBCR_ENC_DEFAULT(fmt->colorspace);
+ fmt->quantization =
+ V4L2_MAP_QUANTIZATION_DEFAULT(false, fmt->colorspace,
+ fmt->ycbcr_enc);
+ fmt->xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(fmt->colorspace);
+ } else {
+ fmt->colorspace = V4L2_COLORSPACE_DEFAULT;
+ fmt->ycbcr_enc = V4L2_XFER_FUNC_DEFAULT;
+ fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
+ fmt->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+ }
}
static void
--
2.53.0
^ permalink raw reply related
* [PATCH 07/14] media: h264: Add stateless encode rate control
From: Paul Kocialkowski @ 2026-05-22 10:16 UTC (permalink / raw)
To: devicetree, imx, linux-arm-kernel, linux-kernel, linux-media
Cc: Rob Herring, Krzysztof Kozlowski, Conor Dooley, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Nicolas Dufresne,
Benjamin Gaignard, Philipp Zabel, Mauro Carvalho Chehab,
Hans Verkuil, Marco Felsch, Michael Tretter, Paul Kocialkowski
In-Reply-To: <20260522101653.2565125-1-paulk@sys-base.io>
The H.264 stateless encode rate control implementation currently supports:
- A direct QP mode;
- A constant quality (CQ) mode, with a linear quality to QP mapping;
- A constant bitrate (CBR) mode.
Related controls from the v4l2 stateful uAPI are reused directly.
The CBR mode is implemented using a size budget approach which allocates
a given size to the current frame in order to match the bitrate.
The size is then used to derive a QP estimation, through a static table for
the initial case and using collected statistics from the past few frames of
the same type in the general case. A size difference ratio from previous
frames is calculated (using up to 3 frames) and a QP diff is estimated from
a static table depending on that ratio. The newly obtained QPs from the
statistics of the previous frames are then averaged with a different weight
(penalty) based on their temporal difference and size difference with the
target.
Signed-off-by: Paul Kocialkowski <paulk@sys-base.io>
---
drivers/media/v4l2-core/Makefile | 3 +-
drivers/media/v4l2-core/v4l2-h264-enc-rc.c | 558 +++++++++++++++++++++
drivers/media/v4l2-core/v4l2-h264-enc.c | 196 ++++++++
include/media/v4l2-h264-enc-rc.h | 108 ++++
include/media/v4l2-h264-enc.h | 18 +
include/media/v4l2-h264.h | 24 +
6 files changed, 906 insertions(+), 1 deletion(-)
create mode 100644 drivers/media/v4l2-core/v4l2-h264-enc-rc.c
create mode 100644 include/media/v4l2-h264-enc-rc.h
diff --git a/drivers/media/v4l2-core/Makefile b/drivers/media/v4l2-core/Makefile
index aba9e310f2e5..1f83e1ee554f 100644
--- a/drivers/media/v4l2-core/Makefile
+++ b/drivers/media/v4l2-core/Makefile
@@ -29,7 +29,8 @@ obj-$(CONFIG_V4L2_CCI) += v4l2-cci.o
obj-$(CONFIG_V4L2_FLASH_LED_CLASS) += v4l2-flash-led-class.o
obj-$(CONFIG_V4L2_FWNODE) += v4l2-fwnode.o
obj-$(CONFIG_V4L2_H264) += v4l2-h264.o
-obj-$(CONFIG_V4L2_H264_ENC) += v4l2-h264-enc.o v4l2-h264-enc-rbsp.o
+obj-$(CONFIG_V4L2_H264_ENC) += v4l2-h264-enc.o v4l2-h264-enc-rbsp.o \
+ v4l2-h264-enc-rc.o
obj-$(CONFIG_V4L2_JPEG_HELPER) += v4l2-jpeg.o
obj-$(CONFIG_V4L2_MEM2MEM_DEV) += v4l2-mem2mem.o
obj-$(CONFIG_V4L2_VP9) += v4l2-vp9.o
diff --git a/drivers/media/v4l2-core/v4l2-h264-enc-rc.c b/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
new file mode 100644
index 000000000000..8f9622395eb6
--- /dev/null
+++ b/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
@@ -0,0 +1,558 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * V4L2 H.264 Encode Rate Control
+ *
+ * Copyright (C) 2025-2026 Paul Kocialkowski <paulk@sys-base.io>
+ */
+
+#include <linux/module.h>
+#include <media/v4l2-h264-enc.h>
+
+int v4l2_h264_enc_rc_init(struct v4l2_h264_enc_rc *rc)
+{
+ rc->qp = 0;
+ rc->mode = NULL;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_rc_init);
+
+void v4l2_h264_enc_rc_exit(struct v4l2_h264_enc_rc *rc)
+{
+ if (rc->mode) {
+ v4l2_h264_enc_rc_op(rc, exit);
+ v4l2_h264_enc_rc_mode_op(rc, exit);
+ rc->mode = NULL;
+ }
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_rc_exit);
+
+int v4l2_h264_enc_rc_stats_collect(struct v4l2_h264_enc_rc *rc,
+ unsigned long bytesused)
+{
+ struct v4l2_h264_enc_state *state = rc->state;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ struct v4l2_fract *timeperframe = &state->timeperframe;
+ struct v4l2_h264_enc_rc_stats *stats = &rc->stats;
+ struct v4l2_h264_enc_rc_stats_type *type;
+ unsigned long bitrate;
+ long size_error;
+ long size_drift;
+ long bitrate_error;
+ long bitrate_drift;
+ unsigned int index;
+
+ /* Collect individual and total frame sizes. */
+
+ index = stats->index;
+
+ if (stats->count == V4L2_H264_ENC_RC_STATS_SIZE_COUNT)
+ stats->size_total -= stats->size[index];
+
+ stats->slice_type[index] = encode->slice_type;
+ stats->size[index] = bytesused;
+ stats->size_total += bytesused;
+
+ if (stats->count < V4L2_H264_ENC_RC_STATS_SIZE_COUNT)
+ stats->count++;
+
+ stats->index = (index + 1) % V4L2_H264_ENC_RC_STATS_SIZE_COUNT;
+
+ bitrate = 8 * stats->size_total * timeperframe->denominator /
+ stats->count / timeperframe->numerator;
+
+ size_error = (long)bytesused - (long)rc->size;
+ size_drift = 100 * size_error / (long)rc->size;
+
+ bitrate_error = (long)bitrate - (long)state->bitrate;
+ bitrate_drift = 100 * bitrate_error / (long)state->bitrate;
+
+ pr_debug("+ v4l2-h264-rc: stats");
+ pr_debug(" size: %lu B, target: %lu B, error: %ld B, drift: %ld pc",
+ bytesused, rc->size, size_error, size_drift);
+ pr_debug(" bitrate: %lu b/s, target: %u b/s, error: %ld b/s, drift: %ld pc",
+ bitrate, state->bitrate, bitrate_error, bitrate_drift);
+
+ /* Collect per-type size and qp information. */
+
+ if (encode->slice_type == V4L2_H264_SLICE_TYPE_I)
+ type = &stats->intra;
+ else if (encode->slice_type == V4L2_H264_SLICE_TYPE_P)
+ type = &stats->pred;
+ else if (encode->slice_type == V4L2_H264_SLICE_TYPE_B)
+ type = &stats->bipred;
+ else
+ return -EINVAL;
+
+ index = type->index;
+
+ type->target[index] = rc->size;
+ type->size[index] = bytesused;
+ type->qp[index] = rc->qp;
+
+ if (type->count < V4L2_H264_ENC_RC_STATS_TYPE_COUNT)
+ type->count++;
+
+ type->index = (index + 1) % V4L2_H264_ENC_RC_STATS_TYPE_COUNT;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_rc_stats_collect);
+
+static int direct_estimate(struct v4l2_h264_enc_rc *rc, unsigned int *qp)
+{
+ struct v4l2_h264_enc_state *state = rc->state;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+
+ if (encode->slice_type == V4L2_H264_SLICE_TYPE_I)
+ *qp = state->qp_i;
+ else if (encode->slice_type == V4L2_H264_SLICE_TYPE_P)
+ *qp = state->qp_p;
+ else if (encode->slice_type == V4L2_H264_SLICE_TYPE_B)
+ *qp = state->qp_b;
+ else
+ return -EINVAL;
+
+ return 0;
+}
+
+static const struct v4l2_h264_enc_rc_mode direct_mode = {
+ .name = "direct qp",
+ .type = V4L2_H264_ENC_RC_TYPE_DIRECT,
+ .estimate = direct_estimate,
+};
+
+static int cq_estimate(struct v4l2_h264_enc_rc *rc, unsigned int *qp)
+{
+ struct v4l2_h264_enc_state *state = rc->state;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ /* TODO: Use some reasonable PSNR to QP mapping. */
+ unsigned int qp_range_intra[2] = { 12, 44 };
+ unsigned int qp_range_inter[2] = { 16, 48 };
+ unsigned int qp_min, qp_max;
+ unsigned int qp_diff_range;
+ unsigned int quality_diff;
+ unsigned int quality_diff_range;
+
+ if (encode->slice_type == V4L2_H264_SLICE_TYPE_I) {
+ qp_min = qp_range_intra[0];
+ qp_max = qp_range_intra[1];
+ } else {
+ qp_min = qp_range_inter[0];
+ qp_max = qp_range_inter[1];
+ }
+
+ qp_diff_range = qp_max - qp_min;
+
+ quality_diff = state->quality - state->quality_min;
+ quality_diff_range = state->quality_max - state->quality_min;
+
+ *qp = qp_min + qp_diff_range * quality_diff / quality_diff_range;
+
+ return 0;
+}
+
+static const struct v4l2_h264_enc_rc_mode cq_mode = {
+ .name = "constant quality",
+ .type = V4L2_H264_ENC_RC_TYPE_CQ,
+ .estimate = cq_estimate,
+};
+
+static int cbr_measure(struct v4l2_h264_enc_rc *rc, unsigned long *size)
+{
+ struct v4l2_h264_enc_state *state = rc->state;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ struct v4l2_fract *timeperframe = &state->timeperframe;
+ struct v4l2_h264_enc_rc_stats *stats = &rc->stats;
+ unsigned long long budget;
+ unsigned long target;
+ unsigned long target_nominal;
+
+ pr_debug("+ v4l2-h264-enc-rc: cbr measure");
+
+ /* Nominal average size target for exact bitrate. */
+ target_nominal = state->bitrate * timeperframe->numerator /
+ timeperframe->denominator / 8;
+
+ /* Total size budget for twice the size window. */
+ budget = 2 * V4L2_H264_ENC_RC_STATS_SIZE_COUNT * state->bitrate *
+ timeperframe->numerator / timeperframe->denominator / 8;
+
+ if (budget < stats->size_total) {
+ target = target_nominal;
+ goto complete;
+ }
+
+ /* Remaining size budget after deducing total size in window. */
+ budget -= stats->size_total;
+ /* Average size target from remaining size budget. */
+ target = budget / (2 * V4L2_H264_ENC_RC_STATS_SIZE_COUNT - stats->count);
+
+complete:
+ pr_debug(" nominal target: %lu bytes", target_nominal);
+ pr_debug(" uniform target: %lu bytes", target);
+
+ /* Bump size for intra frames and reduce for pred frames.
+ * We are better off with good quality initial references.
+ * Maintain average assuming 1 intra frame for 24 inter frames.
+ */
+ if (encode->slice_type == V4L2_H264_SLICE_TYPE_I)
+ target = target * 6;
+ else
+ target = target * 80 / 100;
+
+ /* Reduce target size on bitrate overshoot to calm things down. */
+ if (9 * stats->size_total > 10 * stats->count * target_nominal) {
+ pr_debug(" overshoot, reducing target size");
+ target = 80 * target / 100;
+ }
+
+ pr_debug(" final target: %lu bytes", target);
+
+ *size = target;
+
+ return 0;
+}
+
+/* Intra bit size per macroblock estimation for QP starting at 10. */
+static unsigned int cbr_qp_initial_bits_mb[] = {
+ 437, 411, 376, 355, 318, 303, 291, 271, 252, 238, 218, 204, 188, 171,
+ 158, 149, 133, 124, 115, 105, 98, 92, 83, 77, 70, 63, 57, 52, 45, 39,
+ 34, 31, 30, 29, 26, 23, 22, 20, 18, 17, 16, 15
+};
+
+static int cbr_qp_initial(struct v4l2_h264_enc_rc *rc, unsigned int *qp)
+{
+ struct v4l2_h264_enc_state *state = rc->state;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ unsigned long size_mb;
+ unsigned int qp_initial;
+ unsigned int threshold;
+ unsigned int i;
+
+ size_mb = 8 * rc->size / (state->width_mbs * state->height_mbs);
+
+ pr_debug(" initial estimate from %lu bits/mb", size_mb);
+
+ for (i = 0; i < ARRAY_SIZE(cbr_qp_initial_bits_mb); i++) {
+ threshold = cbr_qp_initial_bits_mb[i];
+
+ /* Expect inter frames to be 6 times smaller. */
+ if (encode->slice_type != V4L2_H264_SLICE_TYPE_I)
+ threshold = cbr_qp_initial_bits_mb[i] / 6;
+
+ if (size_mb >= threshold) {
+ qp_initial = 10 + i;
+ break;
+ }
+ }
+
+ /* Fallback to the lowest QP (last in the list). */
+ if (i == ARRAY_SIZE(cbr_qp_initial_bits_mb))
+ qp_initial = 10 + i - 1;
+
+ *qp = qp_initial;
+
+ return 0;
+}
+
+/*
+ * This table gives the QP decrease/increase (coded as index, with QP = 0 in
+ * the middle) for each size ratio (in base 1000).
+ *
+ * Intra is a bit agressive since we generally have few intra slices to adapt
+ * to a scene change. The QP diff range goes from -4 to +4.
+ */
+static unsigned long cbr_ratio_qp_diff_steps_intra[] = {
+ 3000, 2000, 1500, 1250, 800, 666, 500, 333
+};
+
+/*
+ * Inter is less agressive since we generally have more inter slices to adapt
+ * to a scene change and want to give more stability to QP. The QP diff range
+ * goes from -3 to +3.
+ */
+static unsigned long cbr_ratio_qp_diff_steps_inter[] = {
+ 2000, 1500, 1250, 800, 666, 500
+};
+
+static int cbr_ratio_qp_diff(unsigned long ratio, unsigned char slice_type)
+{
+ unsigned long *steps;
+ unsigned int count;
+ unsigned int i;
+
+ if (slice_type == V4L2_H264_SLICE_TYPE_I) {
+ steps = cbr_ratio_qp_diff_steps_intra;
+ count = ARRAY_SIZE(cbr_ratio_qp_diff_steps_intra);
+ } else {
+ steps = cbr_ratio_qp_diff_steps_inter;
+ count = ARRAY_SIZE(cbr_ratio_qp_diff_steps_inter);
+ }
+
+ WARN_ON(count % 2);
+
+ for (i = 0; i < count; i++)
+ if (ratio > steps[i])
+ return -1 * (int)count / 2 + i;
+
+ return count / 2;
+}
+
+static int cbr_estimate(struct v4l2_h264_enc_rc *rc, unsigned int *qp)
+{
+ struct v4l2_h264_enc_state *state = rc->state;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ struct v4l2_h264_enc_rc_stats *stats = &rc->stats;
+ struct v4l2_h264_enc_rc_stats_type *type;
+ unsigned int type_index[3];
+ unsigned int ratio_qp[3];
+ unsigned int penalty[3];
+ unsigned int penalty_total;
+ unsigned int weight[3];
+ unsigned int weight_total;
+ int ratio_qp_diff;
+ unsigned long diff;
+ unsigned long diff_closest;
+ unsigned int slot_closest;
+ unsigned int index;
+ unsigned int count;
+ unsigned int total;
+ unsigned long ratio;
+ unsigned int i;
+
+ if (!stats->count)
+ return cbr_qp_initial(rc, qp);
+
+ if (encode->slice_type == V4L2_H264_SLICE_TYPE_I)
+ type = &stats->intra;
+ else if (encode->slice_type == V4L2_H264_SLICE_TYPE_P)
+ type = &stats->pred;
+ else if (encode->slice_type == V4L2_H264_SLICE_TYPE_B)
+ type = &stats->bipred;
+ else
+ return -EINVAL;
+
+ pr_debug("+ v4l2-h264-enc-rc: cbr estimate");
+
+ if (!type->count)
+ return cbr_qp_initial(rc, qp);
+
+ for (i = 0; i < 3; i++) {
+ if (i == type->count)
+ break;
+
+ index = type->index;
+ if (index <= i)
+ index = type->count + index - (i + 1);
+ else
+ index -= i + 1;
+
+ /* Set initial penalty based on frame age. */
+ penalty[i] = 4 * i;
+ type_index[i] = index;
+
+ /* Track frame with size closest to target. */
+ diff = abs((long)rc->size - type->size[index]);
+
+ if (!i || diff < diff_closest) {
+ diff_closest = diff;
+ slot_closest = i;
+ }
+ }
+
+ count = i;
+ penalty_total = 0;
+
+ for (i = 0; i < count; i++) {
+ /* Add penalty for frames with larger size difference. */
+ if (i != slot_closest)
+ penalty[i] += 10;
+
+ penalty_total += penalty[i];
+
+ index = type_index[i];
+
+ /* Calculate QP from target to observed size ratio. */
+ ratio = 1000UL * rc->size / type->size[index];
+
+ ratio_qp_diff = cbr_ratio_qp_diff(ratio, encode->slice_type);
+
+ if ((int)type->qp[index] + ratio_qp_diff < 0)
+ ratio_qp[i] = 0;
+ else if ((int)type->qp[index] + ratio_qp_diff > 51)
+ ratio_qp[i] = 51;
+ else
+ ratio_qp[i] = type->qp[index] + ratio_qp_diff;
+
+ diff = abs((long)rc->size - type->size[index]);
+
+ pr_debug(" - backlog %d size: %lu (diff: %lu), qp: %u, penalty %u, ratio qp: %u",
+ -(i + 1), type->size[index], diff, type->qp[index],
+ penalty[i], ratio_qp[i]);
+ }
+
+ total = 0;
+ weight_total = 0;
+
+ for (i = 0; i < count; i++) {
+ /* Weight each calculated QP using the penalty ratio. */
+ if (penalty_total)
+ weight[i] = 1000 * (penalty_total - penalty[i]) /
+ penalty_total;
+ else
+ weight[i] = 1000;
+
+ /* Give non-zero low weight to full penalty cases. */
+ if (weight[i] < 200)
+ weight[i] = 200;
+
+ total += ratio_qp[i] * weight[i];
+ weight_total += weight[i];
+
+ pr_debug(" - backlog %d weight: %u", -(i + 1),
+ weight[i]);
+ }
+
+ *qp = total / weight_total;
+
+ return 0;
+}
+
+static int cbr_complete(struct v4l2_h264_enc_rc *rc, unsigned long bytesused)
+{
+ return v4l2_h264_enc_rc_stats_collect(rc, bytesused);
+}
+
+static const struct v4l2_h264_enc_rc_mode cbr_mode = {
+ .name = "constant bitrate",
+ .type = V4L2_H264_ENC_RC_TYPE_CBR,
+ .measure = cbr_measure,
+ .estimate = cbr_estimate,
+ .complete = cbr_complete,
+};
+
+static const struct v4l2_h264_enc_rc_mode *modes[] = {
+ &direct_mode,
+ &cq_mode,
+ &cbr_mode,
+};
+
+static int mode_prepare(struct v4l2_h264_enc_rc *rc)
+{
+ struct v4l2_h264_enc_state *state = rc->state;
+ const struct v4l2_h264_enc_rc_mode *mode = NULL;
+ unsigned int i;
+ int type;
+ int ret;
+
+ if (!state->frame_rc_enable)
+ type = V4L2_H264_ENC_RC_TYPE_DIRECT;
+ else if (state->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CQ)
+ type = V4L2_H264_ENC_RC_TYPE_CQ;
+ else if (state->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CBR)
+ type = V4L2_H264_ENC_RC_TYPE_CBR;
+ else if (state->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_VBR)
+ type = V4L2_H264_ENC_RC_TYPE_VBR;
+ else
+ return -EINVAL;
+
+ for (i = 0; i < ARRAY_SIZE(modes); i++) {
+ if (modes[i] && modes[i]->type == type) {
+ mode = modes[i];
+ break;
+ }
+ }
+
+ if (!mode)
+ return -EINVAL;
+
+ memset(&rc->stats, 0, sizeof(rc->stats));
+
+ rc->mode = mode;
+
+ ret = v4l2_h264_enc_rc_op(rc, init);
+ if (ret && ret != -EOPNOTSUPP)
+ return ret;
+
+ ret = v4l2_h264_enc_rc_mode_op(rc, init);
+ if (ret && ret != -EOPNOTSUPP)
+ return ret;
+
+ return 0;
+}
+
+int v4l2_h264_enc_rc_mode_update(struct v4l2_h264_enc_rc *rc)
+{
+ if (!rc->mode)
+ return 0;
+
+ v4l2_h264_enc_rc_op(rc, exit);
+ v4l2_h264_enc_rc_mode_op(rc, exit);
+
+ rc->mode = NULL;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_rc_mode_update);
+
+int v4l2_h264_enc_rc_step(struct v4l2_h264_enc_rc *rc,
+ struct v4l2_h264_enc_state *state)
+{
+ unsigned int qp;
+ int ret;
+
+ rc->state = state;
+
+ if (!rc->mode) {
+ ret = mode_prepare(rc);
+ if (ret)
+ return ret;
+ }
+
+ ret = v4l2_h264_enc_rc_mode_op(rc, measure, &rc->size);
+ if (ret && ret != -EOPNOTSUPP)
+ return ret;
+
+ /* Estimate using driver implementation first. */
+ ret = v4l2_h264_enc_rc_op(rc, estimate, &qp);
+ if (ret && ret != -EOPNOTSUPP)
+ return ret;
+
+ /* Fallback to common implementation otherwise. */
+ if (ret == -EOPNOTSUPP) {
+ ret = v4l2_h264_enc_rc_mode_op(rc, estimate, &qp);
+ if (ret)
+ return ret;
+ }
+
+ rc->qp = clamp(qp, state->qp_min, state->qp_max);
+
+ return 0;
+
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_rc_step);
+
+int v4l2_h264_enc_rc_complete(struct v4l2_h264_enc_rc *rc,
+ unsigned long bytesused)
+{
+ int ret;
+
+ if (!rc->mode)
+ return -EINVAL;
+
+ ret = v4l2_h264_enc_rc_op(rc, complete, bytesused);
+ if (ret && ret != -EOPNOTSUPP)
+ return ret;
+
+ ret = v4l2_h264_enc_rc_mode_op(rc, complete, bytesused);
+ if (ret && ret != -EOPNOTSUPP)
+ return ret;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_rc_complete);
+
+MODULE_DESCRIPTION("V4L2 H.264 Encode Rate Control");
+MODULE_AUTHOR("Paul Kocialkowski <paulk@sys-base.io>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/v4l2-core/v4l2-h264-enc.c b/drivers/media/v4l2-core/v4l2-h264-enc.c
index 3b7bca117818..67ea82cebe7f 100644
--- a/drivers/media/v4l2-core/v4l2-h264-enc.c
+++ b/drivers/media/v4l2-core/v4l2-h264-enc.c
@@ -10,6 +10,7 @@
#include <media/v4l2-h264.h>
#include <media/v4l2-h264-enc.h>
#include <media/v4l2-h264-enc-rbsp.h>
+#include <media/v4l2-h264-enc-rc.h>
#include <media/videobuf2-v4l2.h>
static int rec_buffer_alloc(struct v4l2_h264_enc *enc,
@@ -94,6 +95,7 @@ static void rec_buffers_free(struct v4l2_h264_enc *enc)
int v4l2_h264_enc_init(struct v4l2_h264_enc *enc)
{
+ struct v4l2_h264_enc_rc *rc = &enc->rc;
struct v4l2_h264_enc_rbsp *rbsp = &enc->rbsp;
unsigned int slots_count = 0;
int ret;
@@ -123,15 +125,30 @@ int v4l2_h264_enc_init(struct v4l2_h264_enc *enc)
if (ret)
return ret;
+ rc->ops = enc->rc_ops;
+ rc->private_data = enc->private_data;
+
+ ret = v4l2_h264_enc_rc_init(rc);
+ if (ret)
+ goto error_buffers;
+
rbsp->ops = enc->rbsp_ops;
rbsp->private_data = enc->private_data;
return 0;
+
+error_buffers:
+ rec_buffers_free(enc);
+
+ return ret;
}
EXPORT_SYMBOL_GPL(v4l2_h264_enc_init);
void v4l2_h264_enc_exit(struct v4l2_h264_enc *enc)
{
+ struct v4l2_h264_enc_rc *rc = &enc->rc;
+
+ v4l2_h264_enc_rc_exit(rc);
rec_buffers_free(enc);
}
EXPORT_SYMBOL_GPL(v4l2_h264_enc_exit);
@@ -496,6 +513,100 @@ static int state_prepare_params(struct v4l2_h264_enc *enc)
return 0;
}
+static int state_prepare_rc(struct v4l2_h264_enc *enc)
+{
+ struct v4l2_h264_enc_state *state = &enc->state_next;
+ struct v4l2_ctrl_handler *handler = enc->ctrl_handler;
+ struct v4l2_ctrl *ctrl;
+
+ ctrl = v4l2_ctrl_find(handler, V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE);
+ if (ctrl && ctrl->cur.val)
+ state->frame_rc_enable = true;
+ else
+ state->frame_rc_enable = false;
+
+ ctrl = v4l2_ctrl_find(handler, V4L2_CID_MPEG_VIDEO_H264_MIN_QP);
+ if (ctrl)
+ state->qp_min = ctrl->cur.val;
+ else
+ state->qp_min = 0;
+
+ ctrl = v4l2_ctrl_find(handler, V4L2_CID_MPEG_VIDEO_H264_MAX_QP);
+ if (ctrl)
+ state->qp_max = ctrl->cur.val;
+ else
+ state->qp_max = 51;
+
+ if (!state->frame_rc_enable) {
+ ctrl = v4l2_ctrl_find(handler,
+ V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP);
+ if (!ctrl)
+ return -EINVAL;
+
+ state->qp_i = ctrl->cur.val;
+ }
+
+ if (!state->frame_rc_enable &&
+ (enc->flags & V4L2_H264_ENC_FLAG_INTER_PRED)) {
+ ctrl = v4l2_ctrl_find(handler,
+ V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP);
+ if (!ctrl)
+ return -EINVAL;
+
+ state->qp_p = ctrl->cur.val;
+ }
+
+ if (!state->frame_rc_enable &&
+ (enc->flags & V4L2_H264_ENC_FLAG_INTER_BIPRED)) {
+ ctrl = v4l2_ctrl_find(handler,
+ V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP);
+ if (!ctrl)
+ return -EINVAL;
+
+ state->qp_b = ctrl->cur.val;
+ }
+
+ if (!state->frame_rc_enable)
+ return 0;
+
+ ctrl = v4l2_ctrl_find(handler, V4L2_CID_MPEG_VIDEO_BITRATE_MODE);
+ if (!ctrl)
+ return -EINVAL;
+
+ state->bitrate_mode = ctrl->cur.val;
+
+ if (state->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CQ) {
+ ctrl = v4l2_ctrl_find(handler,
+ V4L2_CID_MPEG_VIDEO_CONSTANT_QUALITY);
+ if (!ctrl)
+ return -EINVAL;
+
+ state->quality = ctrl->cur.val;
+ state->quality_min = ctrl->minimum;
+ state->quality_max = ctrl->maximum;
+ }
+
+ if (state->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CBR ||
+ state->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_VBR) {
+ ctrl = v4l2_ctrl_find(handler, V4L2_CID_MPEG_VIDEO_BITRATE);
+ if (!ctrl)
+ return -EINVAL;
+
+ state->bitrate = ctrl->cur.val;
+ }
+
+ if (state->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_VBR) {
+ ctrl = v4l2_ctrl_find(handler,
+ V4L2_CID_MPEG_VIDEO_BITRATE_PEAK);
+ if (!ctrl)
+ return -EINVAL;
+
+ state->bitrate_peak = ctrl->cur.val;
+ }
+
+ return 0;
+}
+
static int state_prepare(struct v4l2_h264_enc *enc)
{
struct v4l2_h264_enc_state *state = &enc->state_next;
@@ -507,6 +618,10 @@ static int state_prepare(struct v4l2_h264_enc *enc)
if (ret)
return ret;
+ ret = state_prepare_rc(enc);
+ if (ret)
+ return ret;
+
ret = v4l2_h264_enc_op(enc, state_constrain, state);
if (ret && ret != -EOPNOTSUPP)
return ret;
@@ -1075,6 +1190,75 @@ static int ref_complete(struct v4l2_h264_enc *enc,
return 0;
}
+static int rc_update(struct v4l2_h264_enc *enc)
+{
+ struct v4l2_h264_enc_state *state = &enc->state_next;
+ struct v4l2_h264_enc_state *state_active = &enc->state_active;
+ int ret;
+
+ if (!enc->state_serial ||
+ state_active->frame_rc_enable != state->frame_rc_enable ||
+ (state_active->frame_rc_enable &&
+ state_active->bitrate_mode != state->bitrate_mode)) {
+ ret = v4l2_h264_enc_rc_mode_update(&enc->rc);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int rc_step(struct v4l2_h264_enc *enc)
+{
+ struct v4l2_h264_enc_state *state = &enc->state_active;
+ struct v4l2_ctrl_h264_pps *pps = &state->pps;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ struct v4l2_h264_enc_rc *rc = &enc->rc;
+ int ret;
+
+ ret = v4l2_h264_enc_rc_step(rc, state);
+ if (ret)
+ return ret;
+
+ encode->slice_qp_delta = rc->qp - (pps->pic_init_qp_minus26 + 26);
+
+ pr_debug("+ v4l2-h264-enc: rc");
+
+ if (rc->mode) {
+ pr_debug(" rc mode: %s", rc->mode->name);
+
+ if (rc->mode->type == V4L2_H264_ENC_RC_TYPE_CBR) {
+ pr_debug(" rc bitrate: %u bits/s", state->bitrate);
+ pr_debug(" rc target: %lu bytes", rc->size);
+ }
+ }
+
+ pr_debug(" rc qp: %u", rc->qp);
+
+ return 0;
+}
+
+static int rc_complete(struct v4l2_h264_enc *enc,
+ struct vb2_v4l2_buffer *buffer)
+{
+ struct v4l2_h264_enc_rc *rc = &enc->rc;
+ unsigned long bytesused;
+ int ret;
+
+ bytesused = vb2_get_plane_payload(&buffer->vb2_buf, 0);
+ if (WARN_ON(!bytesused))
+ return -EINVAL;
+
+ ret = v4l2_h264_enc_rc_complete(rc, bytesused);
+ if (ret)
+ return ret;
+
+ pr_debug("+ v4l2-h264-enc: complete");
+ pr_debug(" size: %lu bytes", bytesused);
+
+ return 0;
+}
+
int v4l2_h264_enc_step(struct v4l2_h264_enc *enc,
struct vb2_v4l2_buffer *buffer)
{
@@ -1088,6 +1272,10 @@ int v4l2_h264_enc_step(struct v4l2_h264_enc *enc,
if (ret)
return ret;
+ ret = rc_update(enc);
+ if (ret)
+ return ret;
+
ret = state_commit(enc);
if (ret)
return ret;
@@ -1096,6 +1284,10 @@ int v4l2_h264_enc_step(struct v4l2_h264_enc *enc,
if (ret)
return ret;
+ ret = rc_step(enc);
+ if (ret)
+ return ret;
+
ret = rbsp_step(enc, buffer);
if (ret)
return ret;
@@ -1117,6 +1309,10 @@ int v4l2_h264_enc_complete(struct v4l2_h264_enc *enc,
if (ret)
return ret;
+ ret = rc_complete(enc, buffer);
+ if (ret)
+ return ret;
+
return 0;
}
EXPORT_SYMBOL_GPL(v4l2_h264_enc_complete);
diff --git a/include/media/v4l2-h264-enc-rc.h b/include/media/v4l2-h264-enc-rc.h
new file mode 100644
index 000000000000..8b453c3a212e
--- /dev/null
+++ b/include/media/v4l2-h264-enc-rc.h
@@ -0,0 +1,108 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * V4L2 H.264 Encode Rate Control
+ *
+ * Copyright (C) 2025-2026 Paul Kocialkowski <paulk@sys-base.io>
+ */
+
+#ifndef _MEDIA_V4L2_H264_ENC_RC_H
+#define _MEDIA_V4L2_H264_ENC_RC_H
+
+#include <linux/v4l2-controls.h>
+#include <media/v4l2-h264.h>
+
+#define V4L2_H264_ENC_RC_STATS_TYPE_COUNT 10
+#define V4L2_H264_ENC_RC_STATS_SIZE_COUNT 25
+
+#define v4l2_h264_enc_rc_op(r, o, a...) \
+ ({ \
+ int ret; \
+ if ((r)->ops && (r)->ops->o) \
+ ret = (r)->ops->o(r, ##a); \
+ else \
+ ret = -EOPNOTSUPP; \
+ ret; \
+ })
+
+#define v4l2_h264_enc_rc_mode_op(r, o, a...) \
+ ({ \
+ int ret; \
+ if ((r)->mode && (r)->mode->o) \
+ ret = (r)->mode->o(r, ##a); \
+ else \
+ ret = -EOPNOTSUPP; \
+ ret; \
+ })
+
+struct v4l2_h264_enc_rc;
+struct v4l2_h264_enc_state;
+struct vb2_v4l2_buffer;
+
+enum v4l2_h264_enc_rc_type {
+ V4L2_H264_ENC_RC_TYPE_DIRECT,
+ V4L2_H264_ENC_RC_TYPE_CQ,
+ V4L2_H264_ENC_RC_TYPE_CBR,
+ V4L2_H264_ENC_RC_TYPE_VBR,
+};
+
+struct v4l2_h264_enc_rc_mode {
+ char name[32];
+ int type;
+ int (*init)(struct v4l2_h264_enc_rc *rc);
+ int (*exit)(struct v4l2_h264_enc_rc *rc);
+ int (*measure)(struct v4l2_h264_enc_rc *rc, unsigned long *size);
+ int (*estimate)(struct v4l2_h264_enc_rc *rc, unsigned int *qp);
+ int (*complete)(struct v4l2_h264_enc_rc *rc, unsigned long bytesused);
+};
+
+struct v4l2_h264_enc_rc_ops {
+ int (*init)(struct v4l2_h264_enc_rc *rc);
+ int (*exit)(struct v4l2_h264_enc_rc *rc);
+ int (*estimate)(struct v4l2_h264_enc_rc *rc, unsigned int *qp);
+ int (*complete)(struct v4l2_h264_enc_rc *rc, unsigned long bytesused);
+};
+
+struct v4l2_h264_enc_rc_stats_type {
+ unsigned long target[V4L2_H264_ENC_RC_STATS_TYPE_COUNT];
+ unsigned long size[V4L2_H264_ENC_RC_STATS_TYPE_COUNT];
+ unsigned int qp[V4L2_H264_ENC_RC_STATS_TYPE_COUNT];
+ unsigned int count;
+ unsigned int index;
+};
+
+struct v4l2_h264_enc_rc_stats {
+ struct v4l2_h264_enc_rc_stats_type intra;
+ struct v4l2_h264_enc_rc_stats_type pred;
+ struct v4l2_h264_enc_rc_stats_type bipred;
+
+ unsigned char slice_type[V4L2_H264_ENC_RC_STATS_SIZE_COUNT];
+ unsigned long size[V4L2_H264_ENC_RC_STATS_SIZE_COUNT];
+ unsigned long long size_total;
+ unsigned int count;
+ unsigned int index;
+};
+
+struct v4l2_h264_enc_rc {
+ const struct v4l2_h264_enc_rc_ops *ops;
+ void *private_data;
+
+ struct v4l2_h264_enc_state *state;
+ const struct v4l2_h264_enc_rc_mode *mode;
+ void *mode_data;
+
+ struct v4l2_h264_enc_rc_stats stats;
+ unsigned long size;
+ unsigned int qp;
+};
+
+int v4l2_h264_enc_rc_init(struct v4l2_h264_enc_rc *rc);
+void v4l2_h264_enc_rc_exit(struct v4l2_h264_enc_rc *rc);
+int v4l2_h264_enc_rc_stats_collect(struct v4l2_h264_enc_rc *rc,
+ unsigned long bytesused);
+int v4l2_h264_enc_rc_mode_update(struct v4l2_h264_enc_rc *rc);
+int v4l2_h264_enc_rc_step(struct v4l2_h264_enc_rc *rc,
+ struct v4l2_h264_enc_state *state);
+int v4l2_h264_enc_rc_complete(struct v4l2_h264_enc_rc *rc,
+ unsigned long bytesused);
+
+#endif
diff --git a/include/media/v4l2-h264-enc.h b/include/media/v4l2-h264-enc.h
index 817a9ca2f169..6debdc95232c 100644
--- a/include/media/v4l2-h264-enc.h
+++ b/include/media/v4l2-h264-enc.h
@@ -11,6 +11,7 @@
#include <linux/v4l2-controls.h>
#include <linux/videodev2.h>
#include <media/v4l2-h264-enc-rbsp.h>
+#include <media/v4l2-h264-enc-rc.h>
#include <media/videobuf2-v4l2.h>
#define V4L2_H264_ENC_MB_UNIT 16
@@ -74,6 +75,21 @@ struct v4l2_h264_enc_state {
unsigned int width_aligned;
unsigned int height_mbs;
unsigned int height_aligned;
+
+ bool frame_rc_enable;
+
+ unsigned int qp_min;
+ unsigned int qp_max;
+ unsigned int qp_i;
+ unsigned int qp_p;
+ unsigned int qp_b;
+
+ int bitrate_mode;
+ unsigned int quality;
+ unsigned int quality_min;
+ unsigned int quality_max;
+ unsigned int bitrate;
+ unsigned int bitrate_peak;
};
struct v4l2_h264_enc_ops {
@@ -87,6 +103,7 @@ struct v4l2_h264_enc_ops {
struct v4l2_h264_enc {
const struct v4l2_h264_enc_ops *ops;
+ const struct v4l2_h264_enc_rc_ops *rc_ops;
const struct v4l2_h264_enc_rbsp_ops *rbsp_ops;
void *private_data;
@@ -100,6 +117,7 @@ struct v4l2_h264_enc {
struct v4l2_h264_enc_state state_next;
unsigned int state_serial;
+ struct v4l2_h264_enc_rc rc;
struct v4l2_h264_enc_ref ref;
struct v4l2_h264_enc_rbsp rbsp;
unsigned int rbsp_update;
diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
index 3b00a1b67fe5..5e77f690902b 100644
--- a/include/media/v4l2-h264.h
+++ b/include/media/v4l2-h264.h
@@ -161,6 +161,30 @@ struct v4l2_h264_reflist_builder {
u8 num_valid;
};
+static inline char v4l2_h264_slice_type_char(unsigned char slice_type)
+{
+ if (slice_type == V4L2_H264_SLICE_TYPE_I)
+ return 'I';
+ else if (slice_type == V4L2_H264_SLICE_TYPE_P)
+ return 'P';
+ else if (slice_type == V4L2_H264_SLICE_TYPE_B)
+ return 'B';
+ else
+ return 'X';
+}
+
+static inline const char *v4l2_h264_slice_type_name(unsigned char slice_type)
+{
+ if (slice_type == V4L2_H264_SLICE_TYPE_I)
+ return "intra";
+ else if (slice_type == V4L2_H264_SLICE_TYPE_P)
+ return "inter-pred";
+ else if (slice_type == V4L2_H264_SLICE_TYPE_B)
+ return "inter-bipred";
+ else
+ return "invalid";
+}
+
void
v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
const struct v4l2_ctrl_h264_decode_params *dec_params,
--
2.53.0
^ permalink raw reply related
* [PATCH 06/14] media: h264: Add stateless encode reference management
From: Paul Kocialkowski @ 2026-05-22 10:16 UTC (permalink / raw)
To: devicetree, imx, linux-arm-kernel, linux-kernel, linux-media
Cc: Rob Herring, Krzysztof Kozlowski, Conor Dooley, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Nicolas Dufresne,
Benjamin Gaignard, Philipp Zabel, Mauro Carvalho Chehab,
Hans Verkuil, Marco Felsch, Michael Tretter, Paul Kocialkowski
In-Reply-To: <20260522101653.2565125-1-paulk@sys-base.io>
The H.264 stateless encode reference management implementation is
responsible for allocating and tracking reconstruction buffers that
need to be used by hardware as well as track references in the DPB
following the sliding window decoded reference picture marking process.
It is also responsible for building the L0 and L1 reference lists
from the DPB, using the common v4l2 h264 reflist builder.
Signed-off-by: Paul Kocialkowski <paulk@sys-base.io>
---
drivers/media/v4l2-core/v4l2-h264-enc.c | 418 ++++++++++++++++++++++++
include/media/v4l2-h264-enc.h | 33 ++
2 files changed, 451 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-h264-enc.c b/drivers/media/v4l2-core/v4l2-h264-enc.c
index 0b46922d1d7a..3b7bca117818 100644
--- a/drivers/media/v4l2-core/v4l2-h264-enc.c
+++ b/drivers/media/v4l2-core/v4l2-h264-enc.c
@@ -12,9 +12,90 @@
#include <media/v4l2-h264-enc-rbsp.h>
#include <media/videobuf2-v4l2.h>
+static int rec_buffer_alloc(struct v4l2_h264_enc *enc,
+ struct v4l2_h264_enc_rec_buffer *buffer)
+{
+ int ret;
+
+ ret = v4l2_h264_enc_op(enc, rec_buffer_alloc, buffer);
+ if (ret)
+ return ret;
+
+ buffer->allocated = true;
+ enc->ref.slots_count++;
+
+ return 0;
+}
+
+static void rec_buffer_free(struct v4l2_h264_enc *enc,
+ struct v4l2_h264_enc_rec_buffer *buffer)
+{
+ if (WARN_ON(!enc->ref.slots_count))
+ return;
+
+ v4l2_h264_enc_op(enc, rec_buffer_free, buffer);
+
+ buffer->allocated = false;
+ enc->ref.slots_count--;
+}
+
+static int rec_buffers_alloc(struct v4l2_h264_enc *enc,
+ unsigned int slots_count)
+{
+ struct v4l2_h264_enc_ref *ref = &enc->ref;
+ unsigned int i;
+ int ret;
+
+ ret = rec_buffer_alloc(enc, &ref->buffer_current);
+ if (ret)
+ return ret;
+
+ if (!(enc->flags & (V4L2_H264_ENC_FLAG_INTER_PRED |
+ V4L2_H264_ENC_FLAG_INTER_BIPRED)))
+ return 0;
+
+ for (i = 0; i < slots_count; i++) {
+ ret = rec_buffer_alloc(enc, &ref->buffers[i]);
+ if (ret)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ while (i > 0) {
+ i--;
+ rec_buffer_free(enc, &ref->buffers[i]);
+ }
+
+ rec_buffer_free(enc, &ref->buffer_current);
+
+ return ret;
+}
+
+static void rec_buffers_free(struct v4l2_h264_enc *enc)
+{
+ struct v4l2_h264_enc_ref *ref = &enc->ref;
+ unsigned int i;
+
+ rec_buffer_free(enc, &ref->buffer_current);
+
+ if (!(enc->flags & (V4L2_H264_ENC_FLAG_INTER_PRED |
+ V4L2_H264_ENC_FLAG_INTER_BIPRED)))
+ return;
+
+ for (i = 0; i < V4L2_H264_NUM_DPB_ENTRIES; i++) {
+ if (!ref->buffers[i].allocated)
+ continue;
+
+ rec_buffer_free(enc, &ref->buffers[i]);
+ }
+}
+
int v4l2_h264_enc_init(struct v4l2_h264_enc *enc)
{
struct v4l2_h264_enc_rbsp *rbsp = &enc->rbsp;
+ unsigned int slots_count = 0;
int ret;
if ((!enc->format && !enc->format_mplane) || !enc->timeperframe ||
@@ -25,6 +106,23 @@ int v4l2_h264_enc_init(struct v4l2_h264_enc *enc)
memset(&enc->state_next, 0, sizeof(enc->state_next));
enc->state_serial = 0;
+ memset(&enc->ref, 0, sizeof(enc->ref));
+
+ if (enc->flags & (V4L2_H264_ENC_FLAG_INTER_PRED |
+ V4L2_H264_ENC_FLAG_INTER_BIPRED)) {
+ if (!enc->ref_slots_count_init)
+ slots_count = V4L2_H264_NUM_DPB_ENTRIES / 2;
+ else if (WARN_ON(enc->ref_slots_count_init >
+ V4L2_H264_NUM_DPB_ENTRIES))
+ slots_count = V4L2_H264_NUM_DPB_ENTRIES;
+ else
+ slots_count = enc->ref_slots_count_init;
+ }
+
+ ret = rec_buffers_alloc(enc, slots_count);
+ if (ret)
+ return ret;
+
rbsp->ops = enc->rbsp_ops;
rbsp->private_data = enc->private_data;
@@ -34,6 +132,7 @@ EXPORT_SYMBOL_GPL(v4l2_h264_enc_init);
void v4l2_h264_enc_exit(struct v4l2_h264_enc *enc)
{
+ rec_buffers_free(enc);
}
EXPORT_SYMBOL_GPL(v4l2_h264_enc_exit);
@@ -665,6 +764,317 @@ static int rbsp_step(struct v4l2_h264_enc *enc,
return 0;
}
+static struct v4l2_h264_enc_rec_buffer *
+ref_step_buffers_slot_find(struct v4l2_h264_enc *enc)
+{
+ struct v4l2_h264_enc_rec_buffer *buffer;
+ unsigned int i;
+
+ for (i = 0; i < V4L2_H264_NUM_DPB_ENTRIES; i++) {
+ buffer = &enc->ref.buffers[i];
+
+ if (buffer->allocated)
+ continue;
+
+ return buffer;
+ }
+
+ return NULL;
+}
+
+static int ref_step_buffers(struct v4l2_h264_enc *enc)
+{
+ struct v4l2_h264_enc_state *state = &enc->state_active;
+ struct v4l2_ctrl_h264_sps *sps = &state->sps;
+ struct v4l2_h264_enc_ref *ref = &enc->ref;
+ struct v4l2_h264_enc_rec_buffer *buffer;
+ unsigned int count;
+ unsigned int i;
+ int ret;
+
+ /*
+ * Only increase the number of slots. It avoids the cost of free
+ * (including on the first frame) and the cost of possible future
+ * allocations, at the expense of memory.
+ */
+ if (sps->max_num_ref_frames <= ref->slots_count)
+ return 0;
+
+ count = sps->max_num_ref_frames - ref->slots_count;
+
+ for (i = 0; i < count; i++) {
+ buffer = ref_step_buffers_slot_find(enc);
+ if (WARN_ON(!buffer))
+ return -ENOMEM;
+
+ ret = rec_buffer_alloc(enc, buffer);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int ref_step_poc(struct v4l2_h264_enc *enc)
+{
+ struct v4l2_h264_enc_state *state = &enc->state_active;
+ struct v4l2_ctrl_h264_sps *sps = &state->sps;
+ struct v4l2_ctrl_h264_pps *pps = &state->pps;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ struct v4l2_h264_enc_ref *ref = &enc->ref;
+ unsigned int pic_order_cnt_lsb;
+ unsigned int pic_order_cnt_msb;
+ unsigned int prev_pic_order_cnt_msb;
+ unsigned int prev_pic_order_cnt_lsb;
+ unsigned int max_pic_order_cnt_lsb;
+ unsigned int delta_pic_order_cnt_bottom;
+ unsigned int top_field_order_cnt;
+ unsigned int bottom_field_order_cnt;
+
+ /* Only pic_order_cnt_type = 0 is currently supported. */
+ if (sps->pic_order_cnt_type)
+ return -EINVAL;
+
+ max_pic_order_cnt_lsb = BIT(sps->log2_max_pic_order_cnt_lsb_minus4 + 4);
+ pic_order_cnt_lsb = encode->pic_order_cnt_lsb;
+
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_IDR_PIC) {
+ prev_pic_order_cnt_msb = 0;
+ prev_pic_order_cnt_lsb = 0;
+ } else {
+ prev_pic_order_cnt_msb = ref->prev_pic_order_cnt_msb;
+ prev_pic_order_cnt_lsb = ref->prev_pic_order_cnt_lsb;
+ }
+
+ if ((pic_order_cnt_lsb < prev_pic_order_cnt_lsb) &&
+ ((prev_pic_order_cnt_lsb - pic_order_cnt_lsb) >=
+ (max_pic_order_cnt_lsb / 2)))
+ pic_order_cnt_msb = prev_pic_order_cnt_msb +
+ max_pic_order_cnt_lsb;
+ else if ((pic_order_cnt_lsb > prev_pic_order_cnt_lsb) &&
+ ((pic_order_cnt_lsb - prev_pic_order_cnt_lsb) >
+ (max_pic_order_cnt_lsb / 2)))
+ pic_order_cnt_msb = prev_pic_order_cnt_msb -
+ max_pic_order_cnt_lsb;
+ else
+ pic_order_cnt_msb = prev_pic_order_cnt_msb;
+
+ top_field_order_cnt = pic_order_cnt_msb + pic_order_cnt_lsb;
+
+ if (pps->flags & V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT)
+ delta_pic_order_cnt_bottom = encode->delta_pic_order_cnt_bottom;
+ else
+ delta_pic_order_cnt_bottom = 0;
+
+ if (!(encode->flags & V4L2_H264_ENCODE_FLAG_FIELD_PIC))
+ bottom_field_order_cnt = top_field_order_cnt +
+ delta_pic_order_cnt_bottom;
+ else
+ bottom_field_order_cnt = top_field_order_cnt;
+
+ ref->pic_order_cnt_msb = pic_order_cnt_msb;
+ ref->pic_order_cnt_lsb = pic_order_cnt_lsb;
+ ref->top_field_order_cnt = top_field_order_cnt;
+ ref->bottom_field_order_cnt = bottom_field_order_cnt;
+ ref->pic_order_cnt = min(top_field_order_cnt, bottom_field_order_cnt);
+
+ return 0;
+}
+
+static int ref_step(struct v4l2_h264_enc *enc)
+{
+ struct v4l2_h264_enc_state *state = &enc->state_active;
+ struct v4l2_ctrl_h264_sps *sps = &state->sps;
+ struct v4l2_ctrl_h264_pps *pps = &state->pps;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ struct v4l2_h264_enc_ref *ref = &enc->ref;
+ unsigned int l0_active_count_max;
+ unsigned int l1_active_count_max;
+ int ret;
+
+ ret = ref_step_buffers(enc);
+ if (ret)
+ return ret;
+
+ ret = ref_step_poc(enc);
+ if (ret)
+ return ret;
+
+ ref->l0_active_count = 0;
+ ref->l1_active_count = 0;
+
+ if (encode->slice_type == V4L2_H264_SLICE_TYPE_I) {
+ /* Flush the DPB on IDR pictures. */
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_IDR_PIC)
+ memset(ref->dpb, 0, sizeof(ref->dpb));
+
+ return 0;
+ }
+
+ /* Generate reference lists. */
+
+ v4l2_h264_init_reflist_builder_gen(&ref->builder, sps, ref->dpb,
+ ref->pic_order_cnt,
+ encode->frame_num,
+ V4L2_H264_FRAME_REF);
+
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_NUM_REF_IDX_ACTIVE_OVERRIDE) {
+ l0_active_count_max = encode->num_ref_idx_l0_active_minus1 + 1;
+ l1_active_count_max = encode->num_ref_idx_l1_active_minus1 + 1;
+ } else {
+ l0_active_count_max =
+ pps->num_ref_idx_l0_default_active_minus1 + 1;
+ l1_active_count_max =
+ pps->num_ref_idx_l1_default_active_minus1 + 1;
+ }
+
+ switch (encode->slice_type) {
+ case V4L2_H264_SLICE_TYPE_P:
+ v4l2_h264_build_p_ref_list(&ref->builder, ref->l0);
+
+ if (ref->builder.num_valid > l0_active_count_max)
+ ref->l0_active_count = l0_active_count_max;
+ else
+ ref->l0_active_count = ref->builder.num_valid;
+
+ WARN_ON(!ref->l0_active_count);
+
+ break;
+ case V4L2_H264_SLICE_TYPE_B:
+ v4l2_h264_build_b_ref_lists(&ref->builder, ref->l0, ref->l1);
+
+ if (ref->builder.num_valid > l0_active_count_max)
+ ref->l0_active_count = l0_active_count_max;
+ else
+ ref->l0_active_count = ref->builder.num_valid;
+
+ WARN_ON(!ref->l0_active_count);
+
+ if (ref->builder.num_valid > l1_active_count_max)
+ ref->l1_active_count = l1_active_count_max;
+ else
+ ref->l1_active_count = ref->builder.num_valid;
+
+ WARN_ON(!ref->l1_active_count);
+
+ break;
+ }
+
+ pr_debug("+ v4l2-h264-enc: ref");
+ pr_debug(" ref active l0: %u, l1: %u", ref->l0_active_count,
+ ref->l1_active_count);
+
+ return 0;
+}
+
+static int ref_complete_slot_find(struct v4l2_h264_enc *enc,
+ unsigned int *index)
+{
+ struct v4l2_h264_enc_state *state = &enc->state_active;
+ struct v4l2_ctrl_h264_sps *sps = &state->sps;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ struct v4l2_h264_enc_ref *ref = &enc->ref;
+ unsigned int max_frame_num = BIT(sps->log2_max_frame_num_minus4 + 4);
+ unsigned int frame_num_wrap_smallest = encode->frame_num;
+ unsigned int frame_num_wrap_smallest_index;
+ unsigned int frame_num_wrap;
+ struct v4l2_h264_enc_rec_buffer *buffer;
+ struct v4l2_h264_dpb_entry *dpb_entry;
+ unsigned int i;
+
+ for (i = 0; i < V4L2_H264_NUM_DPB_ENTRIES; i++) {
+ buffer = &ref->buffers[i];
+ dpb_entry = &ref->dpb[i];
+
+ if (!buffer->allocated)
+ continue;
+
+ /* Return an unused slot. */
+ if (!(dpb_entry->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) {
+ *index = i;
+ return 0;
+ }
+
+ /* Track the smallest FrameNumWrap value. */
+ if (dpb_entry->frame_num > encode->frame_num)
+ frame_num_wrap = dpb_entry->frame_num - max_frame_num;
+ else
+ frame_num_wrap = dpb_entry->frame_num;
+
+ if (frame_num_wrap < frame_num_wrap_smallest) {
+ frame_num_wrap_smallest = frame_num_wrap;
+ frame_num_wrap_smallest_index = i;
+ }
+ }
+
+ /* Clear the evicted DPB entry. */
+ dpb_entry = &ref->dpb[frame_num_wrap_smallest_index];
+ memset(dpb_entry, 0, sizeof(*dpb_entry));
+
+ *index = frame_num_wrap_smallest_index;
+
+ return 0;
+}
+
+static int ref_complete_swap(struct v4l2_h264_enc *enc, unsigned int index,
+ u64 ts)
+{
+ struct v4l2_h264_enc_state *state = &enc->state_active;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ struct v4l2_h264_enc_ref *ref = &enc->ref;
+ struct v4l2_h264_dpb_entry *dpb_entry = &ref->dpb[index];
+ struct v4l2_h264_enc_rec_buffer *buffer = &ref->buffers[index];
+ struct v4l2_h264_enc_rec_buffer *buffer_current =
+ &ref->buffer_current;
+
+ /* Set the DPB entry of the available slot. */
+ memset(dpb_entry, 0, sizeof(*dpb_entry));
+ dpb_entry->reference_ts = ts;
+ dpb_entry->pic_num = encode->frame_num;
+ dpb_entry->frame_num = encode->frame_num;
+ dpb_entry->fields = V4L2_H264_FRAME_REF;
+ dpb_entry->top_field_order_cnt = ref->top_field_order_cnt;
+ dpb_entry->bottom_field_order_cnt = ref->bottom_field_order_cnt;
+ dpb_entry->flags = V4L2_H264_DPB_ENTRY_FLAG_VALID |
+ V4L2_H264_DPB_ENTRY_FLAG_ACTIVE;
+
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_LONG_TERM_REFERENCE)
+ dpb_entry->flags |= V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM;
+
+ /* Swap the current buffer with the available slot. */
+ swap(*buffer_current, *buffer);
+
+ return 0;
+}
+
+static int ref_complete(struct v4l2_h264_enc *enc,
+ struct vb2_v4l2_buffer *buffer)
+{
+ struct v4l2_h264_enc_state *state = &enc->state_active;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ struct v4l2_h264_enc_ref *ref = &enc->ref;
+ unsigned int index;
+ int ret;
+
+ if (!encode->nal_ref_idc)
+ return 0;
+
+ /* Keep the POC as last reference. */
+ ref->prev_pic_order_cnt_msb = ref->pic_order_cnt_msb;
+ ref->prev_pic_order_cnt_lsb = ref->pic_order_cnt_lsb;
+
+ ret = ref_complete_slot_find(enc, &index);
+ if (ret)
+ return ret;
+
+ /* Move our current picture to the DPB. */
+ ret = ref_complete_swap(enc, index, buffer->vb2_buf.timestamp);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
int v4l2_h264_enc_step(struct v4l2_h264_enc *enc,
struct vb2_v4l2_buffer *buffer)
{
@@ -682,6 +1092,10 @@ int v4l2_h264_enc_step(struct v4l2_h264_enc *enc,
if (ret)
return ret;
+ ret = ref_step(enc);
+ if (ret)
+ return ret;
+
ret = rbsp_step(enc, buffer);
if (ret)
return ret;
@@ -699,6 +1113,10 @@ int v4l2_h264_enc_complete(struct v4l2_h264_enc *enc,
if (ret)
return ret;
+ ret = ref_complete(enc, buffer);
+ if (ret)
+ return ret;
+
return 0;
}
EXPORT_SYMBOL_GPL(v4l2_h264_enc_complete);
diff --git a/include/media/v4l2-h264-enc.h b/include/media/v4l2-h264-enc.h
index 3d6b97408707..817a9ca2f169 100644
--- a/include/media/v4l2-h264-enc.h
+++ b/include/media/v4l2-h264-enc.h
@@ -35,6 +35,33 @@
struct v4l2_h264_enc;
+struct v4l2_h264_enc_rec_buffer {
+ void *private_data;
+ bool allocated;
+};
+
+struct v4l2_h264_enc_ref {
+ struct v4l2_h264_enc_rec_buffer buffer_current;
+ struct v4l2_h264_enc_rec_buffer buffers[V4L2_H264_NUM_DPB_ENTRIES];
+ struct v4l2_h264_dpb_entry dpb[V4L2_H264_NUM_DPB_ENTRIES];
+ unsigned int slots_count;
+
+ struct v4l2_h264_reference l0[V4L2_H264_REF_LIST_LEN];
+ unsigned int l0_active_count;
+ struct v4l2_h264_reference l1[V4L2_H264_REF_LIST_LEN];
+ unsigned int l1_active_count;
+
+ struct v4l2_h264_reflist_builder builder;
+
+ unsigned int prev_pic_order_cnt_msb;
+ unsigned int prev_pic_order_cnt_lsb;
+ unsigned int pic_order_cnt_msb;
+ unsigned int pic_order_cnt_lsb;
+ unsigned int top_field_order_cnt;
+ unsigned int bottom_field_order_cnt;
+ unsigned int pic_order_cnt;
+};
+
struct v4l2_h264_enc_state {
struct v4l2_ctrl_h264_sps sps;
struct v4l2_h264_sps_video sps_video;
@@ -52,6 +79,10 @@ struct v4l2_h264_enc_state {
struct v4l2_h264_enc_ops {
int (*state_constrain)(struct v4l2_h264_enc *enc,
struct v4l2_h264_enc_state *state);
+ int (*rec_buffer_alloc)(struct v4l2_h264_enc *enc,
+ struct v4l2_h264_enc_rec_buffer *rec_buffer);
+ int (*rec_buffer_free)(struct v4l2_h264_enc *enc,
+ struct v4l2_h264_enc_rec_buffer *rec_buffer);
};
struct v4l2_h264_enc {
@@ -63,11 +94,13 @@ struct v4l2_h264_enc {
struct v4l2_pix_format_mplane *format_mplane;
struct v4l2_fract *timeperframe;
struct v4l2_ctrl_handler *ctrl_handler;
+ unsigned int ref_slots_count_init;
struct v4l2_h264_enc_state state_active;
struct v4l2_h264_enc_state state_next;
unsigned int state_serial;
+ struct v4l2_h264_enc_ref ref;
struct v4l2_h264_enc_rbsp rbsp;
unsigned int rbsp_update;
--
2.53.0
^ permalink raw reply related
* [PATCH 05/14] media: h264: Add stateless encode rbsp
From: Paul Kocialkowski @ 2026-05-22 10:16 UTC (permalink / raw)
To: devicetree, imx, linux-arm-kernel, linux-kernel, linux-media
Cc: Rob Herring, Krzysztof Kozlowski, Conor Dooley, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Nicolas Dufresne,
Benjamin Gaignard, Philipp Zabel, Mauro Carvalho Chehab,
Hans Verkuil, Marco Felsch, Michael Tretter, Paul Kocialkowski
In-Reply-To: <20260522101653.2565125-1-paulk@sys-base.io>
The H.264 stateless encode rbsp implementation allows generating H.264
bitstream headers to accommodate hardware that does not do it (which is
rather common with stateless encoders).
It currently supports the following NAL unit types: AUD, SPS, PPS,
slice header, with or without start codes.
Dedicated low-level operations are supported in order to accommodate
hardware with dedicated hardware for bitstream storage. It is generally
better to use such mechanisms to ensure slice NAL unit continuation,
especially in unaligned cases.
It is deliberately not too tied to the H.264 stateless encode core so
that it can be reused by stateful drivers that may need to do the same.
Signed-off-by: Paul Kocialkowski <paulk@sys-base.io>
---
drivers/media/v4l2-core/Makefile | 2 +-
drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c | 1173 ++++++++++++++++++
drivers/media/v4l2-core/v4l2-h264-enc.c | 153 +++
include/media/v4l2-h264-enc-rbsp.h | 72 ++
include/media/v4l2-h264-enc.h | 5 +
5 files changed, 1404 insertions(+), 1 deletion(-)
create mode 100644 drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c
create mode 100644 include/media/v4l2-h264-enc-rbsp.h
diff --git a/drivers/media/v4l2-core/Makefile b/drivers/media/v4l2-core/Makefile
index bd319e363c8e..aba9e310f2e5 100644
--- a/drivers/media/v4l2-core/Makefile
+++ b/drivers/media/v4l2-core/Makefile
@@ -29,7 +29,7 @@ obj-$(CONFIG_V4L2_CCI) += v4l2-cci.o
obj-$(CONFIG_V4L2_FLASH_LED_CLASS) += v4l2-flash-led-class.o
obj-$(CONFIG_V4L2_FWNODE) += v4l2-fwnode.o
obj-$(CONFIG_V4L2_H264) += v4l2-h264.o
-obj-$(CONFIG_V4L2_H264_ENC) += v4l2-h264-enc.o
+obj-$(CONFIG_V4L2_H264_ENC) += v4l2-h264-enc.o v4l2-h264-enc-rbsp.o
obj-$(CONFIG_V4L2_JPEG_HELPER) += v4l2-jpeg.o
obj-$(CONFIG_V4L2_MEM2MEM_DEV) += v4l2-mem2mem.o
obj-$(CONFIG_V4L2_VP9) += v4l2-vp9.o
diff --git a/drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c b/drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c
new file mode 100644
index 000000000000..a165f8114ce9
--- /dev/null
+++ b/drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c
@@ -0,0 +1,1173 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * V4L2 H.264 Encode RBSP
+ *
+ * Copyright (C) 2025-2026 Paul Kocialkowski <paulk@sys-base.io>
+ */
+
+#include <linux/bitops.h>
+#include <linux/bits.h>
+#include <linux/bug.h>
+#include <linux/errno.h>
+#include <linux/minmax.h>
+#include <linux/string.h>
+#include <linux/types.h>
+#include <linux/v4l2-controls.h>
+#include <media/v4l2-h264.h>
+#include <media/v4l2-h264-enc-rbsp.h>
+
+int v4l2_h264_enc_rbsp_init(struct v4l2_h264_enc_rbsp *rbsp, u8 *pointer,
+ unsigned int size)
+{
+ rbsp->pointer = pointer;
+ rbsp->size = size;
+
+ rbsp->bit_offset = 0;
+ rbsp->bits_count = 0;
+ rbsp->zero_count = 0;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_rbsp_init);
+
+unsigned int v4l2_h264_enc_rbsp_bits_count(struct v4l2_h264_enc_rbsp *rbsp)
+{
+ return rbsp->bits_count;
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_rbsp_bits_count);
+
+unsigned int v4l2_h264_enc_rbsp_bytes_count(struct v4l2_h264_enc_rbsp *rbsp)
+{
+ WARN_ON(rbsp->bits_count % 8);
+
+ return rbsp->bits_count / 8;
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_rbsp_bytes_count);
+
+static int v4l2_h264_enc_rbsp_bits_raw(struct v4l2_h264_enc_rbsp *rbsp,
+ u32 value, unsigned char bits_count)
+{
+ unsigned int bits_left = bits_count;
+ unsigned int bits_chunk;
+ unsigned int bit_start;
+ unsigned int bit_stop;
+ u32 value_extract;
+ int ret;
+
+ if (bits_count > 32)
+ return -EINVAL;
+ else if (!bits_count)
+ return 0;
+
+ value &= GENMASK(bits_count - 1, 0);
+
+ /* XXX: Has to manage rbsp->bit_offset and rbsp->bits_count. */
+ ret = v4l2_h264_enc_rbsp_op(rbsp, bits_raw, value, bits_count);
+ if (!ret)
+ return 0;
+ else if (ret != -EOPNOTSUPP)
+ return ret;
+
+ while (bits_left > 0) {
+ if (WARN_ON(rbsp->bit_offset >= 8))
+ return -1;
+
+ bits_chunk = min(8u - rbsp->bit_offset, bits_left);
+
+ bit_stop = bits_left - 1;
+ bit_start = bit_stop + 1 - bits_chunk;
+ value_extract = (value & GENMASK(bit_stop, bit_start)) >>
+ bit_start;
+
+ bit_stop = 7u - rbsp->bit_offset;
+ bit_start = bit_stop + 1 - bits_chunk;
+
+ *rbsp->pointer &= ~GENMASK(bit_stop, bit_start);
+ *rbsp->pointer |= value_extract << bit_start;
+
+ rbsp->bit_offset += bits_chunk;
+
+ if (rbsp->bit_offset == 8) {
+ rbsp->pointer++;
+ rbsp->bit_offset = 0;
+ }
+
+ rbsp->bits_count += bits_chunk;
+
+ if (rbsp->bits_count / 8 >= rbsp->size)
+ return -ENOMEM;
+
+ bits_left -= bits_chunk;
+ }
+
+ WARN_ON(bits_left);
+
+ return 0;
+}
+
+static int eptb_bits(struct v4l2_h264_enc_rbsp *rbsp, unsigned int *bits_left,
+ unsigned int zero_step)
+{
+ unsigned int zero_discard;
+ unsigned int zero_count;
+ unsigned int zero_chunk;
+ int ret;
+
+ if (!zero_step)
+ return 0;
+
+ /* Trailing zeros in a non-zero byte are discarded for EPTB. */
+ if (!rbsp->zero_count && rbsp->bit_offset)
+ zero_discard = min(8u - rbsp->bit_offset, zero_step);
+ else
+ zero_discard = 0;
+
+ /* Append discarded zeros before byte alignment. */
+ if (zero_discard) {
+ ret = v4l2_h264_enc_rbsp_bits_raw(rbsp, 0, zero_discard);
+ if (ret)
+ return ret;
+
+ *bits_left -= zero_discard;
+ zero_step -= zero_discard;
+ }
+
+ /* Count relevant zeros for EPTB (starting at byte alignment). */
+ zero_count = rbsp->zero_count + zero_step;
+
+ /* Append EPTB as many times as necessary. */
+ while (zero_count >= 22) {
+ zero_chunk = 22 - rbsp->zero_count;
+
+ ret = v4l2_h264_enc_rbsp_bits_raw(rbsp, 0, zero_chunk);
+ if (ret)
+ return ret;
+
+ /* Two high bits for 0x3 and 6 other bits for stolen zeros. */
+ ret = v4l2_h264_enc_rbsp_bits_raw(rbsp,
+ V4L2_H264_ENC_RBSP_EPTB << 6,
+ 8);
+ if (ret)
+ return ret;
+
+ /* We get 6 zeros after byte alignment. */
+ rbsp->zero_count = 6;
+
+ *bits_left -= zero_chunk;
+ zero_step -= zero_chunk;
+ zero_count = rbsp->zero_count + zero_step;
+ }
+
+ /* Append remaining zero bits. */
+ if (zero_step) {
+ ret = v4l2_h264_enc_rbsp_bits_raw(rbsp, 0, zero_step);
+ if (ret)
+ return ret;
+
+ *bits_left -= zero_step;
+ }
+
+ rbsp->zero_count = zero_count;
+
+ return ret;
+}
+
+static int v4l2_h264_enc_rbsp_bits(struct v4l2_h264_enc_rbsp *rbsp, u32 value,
+ unsigned int bits_count)
+{
+ unsigned int bits_left = bits_count;
+ unsigned int zero_head;
+ unsigned int zero_tail;
+ int ret;
+
+ if (bits_count > 32)
+ return -EINVAL;
+ else if (!bits_count)
+ return 0;
+
+ ret = v4l2_h264_enc_rbsp_op(rbsp, bits, value, bits_count);
+ if (!ret)
+ return 0;
+ else if (ret != -EOPNOTSUPP)
+ return ret;
+
+ value &= GENMASK(bits_count - 1, 0);
+
+ /* Count heading zeros. */
+ if (value)
+ zero_head = bits_count - __fls(value) - 1;
+ else
+ zero_head = bits_count;
+
+ /* Append heading zeros with EPTB. */
+ ret = eptb_bits(rbsp, &bits_left, zero_head);
+ if (ret)
+ return ret;
+
+ /* A zero value is entirely handled as heading zeros. */
+ if (!bits_left || WARN_ON(!value))
+ return 0;
+
+ /* Count trailing zeros. */
+ zero_tail = __ffs(value);
+
+ /* Append non-tail bits. */
+ ret = v4l2_h264_enc_rbsp_bits_raw(rbsp, value >> zero_tail,
+ bits_left - zero_tail);
+ if (ret)
+ return ret;
+
+ /* Reset zero count after appending non-zero bits. */
+ rbsp->zero_count = 0;
+
+ bits_left = zero_tail;
+ if (!bits_left)
+ return 0;
+
+ /* Append trailing zeros with EPTB. */
+ ret = eptb_bits(rbsp, &bits_left, zero_tail);
+ if (ret)
+ return ret;
+
+ WARN_ON(bits_left);
+
+ return 0;
+}
+
+static int v4l2_h264_enc_rbsp_ue(struct v4l2_h264_enc_rbsp *rbsp, u32 value)
+{
+ unsigned int bits_count;
+ int ret;
+
+ ret = v4l2_h264_enc_rbsp_op(rbsp, ue, value);
+ if (!ret)
+ return 0;
+ else if (ret != -EOPNOTSUPP)
+ return ret;
+
+ /*
+ * Exponential-Golomb coding of x stores the value of v + 1.
+ * This takes fls(v + 1) + 1 bits for the non-zero bits and fls(v + 1)
+ * heading zero bits.
+ */
+ value += 1;
+ bits_count = 2 * __fls(value) + 1;
+
+ return v4l2_h264_enc_rbsp_bits(rbsp, value, bits_count);
+}
+
+static int v4l2_h264_enc_rbsp_se(struct v4l2_h264_enc_rbsp *rbsp, s32 value)
+{
+ u32 value_ue;
+ int ret;
+
+ ret = v4l2_h264_enc_rbsp_op(rbsp, se, value);
+ if (!ret)
+ return 0;
+ else if (ret != -EOPNOTSUPP)
+ return ret;
+
+ /*
+ * The signed extension represents numbers in Exponential-Golomb
+ * with each positive value followed by its corresponding negative
+ * value in sequence order.
+ */
+
+ if (value > 0)
+ value_ue = 2 * value - 1;
+ else
+ value_ue = -2 * value;
+
+ return v4l2_h264_enc_rbsp_ue(rbsp, value_ue);
+}
+
+static int v4l2_h264_enc_rbsp_align(struct v4l2_h264_enc_rbsp *rbsp)
+{
+ unsigned int zero_count;
+ int ret;
+
+ ret = v4l2_h264_enc_rbsp_op(rbsp, align);
+ if (!ret)
+ return 0;
+ else if (ret != -EOPNOTSUPP)
+ return ret;
+
+ zero_count = 8 - rbsp->bit_offset;
+ if (!zero_count)
+ return 0;
+
+ return v4l2_h264_enc_rbsp_bits(rbsp, 0, zero_count);
+}
+
+static int v4l2_h264_enc_rbsp_u32(struct v4l2_h264_enc_rbsp *rbsp, u32 value)
+{
+ return v4l2_h264_enc_rbsp_bits(rbsp, value, 32);
+}
+
+static int v4l2_h264_enc_rbsp_u16(struct v4l2_h264_enc_rbsp *rbsp, u16 value)
+{
+ return v4l2_h264_enc_rbsp_bits(rbsp, value, 16);
+}
+
+static int v4l2_h264_enc_rbsp_u8(struct v4l2_h264_enc_rbsp *rbsp, u8 value)
+{
+ return v4l2_h264_enc_rbsp_bits(rbsp, value, 8);
+}
+
+static int v4l2_h264_enc_rbsp_bit(struct v4l2_h264_enc_rbsp *rbsp, u8 value)
+{
+ return v4l2_h264_enc_rbsp_bits(rbsp, value, 1);
+}
+
+static int v4l2_h264_enc_rbsp_flag(struct v4l2_h264_enc_rbsp *rbsp, u32 flags,
+ u32 flag)
+{
+ return v4l2_h264_enc_rbsp_bit(rbsp, (flags & flag) == flag);
+}
+
+int v4l2_h264_enc_rbsp_start_code(struct v4l2_h264_enc_rbsp *rbsp)
+{
+ /* Start code must be inserted at a byte-aligned position. */
+ if (WARN_ON(rbsp->bit_offset))
+ return -EINVAL;
+
+ return v4l2_h264_enc_rbsp_bits_raw(rbsp, V4L2_H264_START_CODE_ANNEX_B,
+ 32);
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_rbsp_start_code);
+
+static int v4l2_h264_enc_rbsp_nalu_begin(struct v4l2_h264_enc_rbsp *rbsp,
+ u8 nal_ref_idc, u8 nal_unit_type)
+{
+ u8 forbidden_zero_bit = 0;
+ int ret;
+
+ /* NALU must be inserted at a byte-aligned position. */
+ if (WARN_ON(rbsp->bit_offset))
+ return -EINVAL;
+
+ /* NALU header bits must not be counted in EPTB. */
+
+ ret = v4l2_h264_enc_rbsp_bits_raw(rbsp, forbidden_zero_bit, 1);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_bits_raw(rbsp, nal_ref_idc, 2);
+ if (ret)
+ return ret;
+
+ return v4l2_h264_enc_rbsp_bits_raw(rbsp, nal_unit_type, 5);
+}
+
+static int v4l2_h264_enc_rbsp_nalu_end(struct v4l2_h264_enc_rbsp *rbsp)
+{
+ u8 rbsp_stop_one_bit = 1;
+ int ret;
+
+ ret = v4l2_h264_enc_rbsp_bit(rbsp, rbsp_stop_one_bit);
+ if (ret)
+ return ret;
+
+ return v4l2_h264_enc_rbsp_align(rbsp);
+}
+
+int v4l2_h264_enc_rbsp_aud(struct v4l2_h264_enc_rbsp *rbsp, u8 primary_pic_type)
+{
+ int ret;
+
+ ret = v4l2_h264_enc_rbsp_nalu_begin(rbsp, 0, V4L2_H264_NALU_TYPE_AUD);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_bits(rbsp, primary_pic_type, 3);
+ if (ret)
+ return ret;
+
+ return v4l2_h264_enc_rbsp_nalu_end(rbsp);
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_rbsp_aud);
+
+static int v4l2_h264_enc_rbsp_sps_hrd(struct v4l2_h264_enc_rbsp *rbsp,
+ const struct v4l2_h264_sps_video_hrd *hrd)
+{
+ unsigned int i;
+ int ret;
+
+ if (hrd->cpb_cnt_minus1 > 31)
+ return -EINVAL;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, hrd->cpb_cnt_minus1);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_bits(rbsp, hrd->bit_rate_scale, 4);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_bits(rbsp, hrd->cpb_size_scale, 4);
+ if (ret)
+ return ret;
+
+ for (i = 0; i <= hrd->cpb_cnt_minus1; i++) {
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ hrd->bit_rate_value_minus1[i]);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ hrd->cpb_size_value_minus1[i]);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_bit(rbsp, hrd->cbr_flag[i]);
+ if (ret)
+ return ret;
+ }
+
+ ret = v4l2_h264_enc_rbsp_bits(rbsp,
+ hrd->initial_cpb_removal_delay_length_minus1,
+ 5);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_bits(rbsp,
+ hrd->cpb_removal_delay_length_minus1, 5);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_bits(rbsp, hrd->dpb_output_delay_length_minus1,
+ 5);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_bits(rbsp, hrd->time_offset_length, 5);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int v4l2_h264_enc_rbsp_sps_vui(struct v4l2_h264_enc_rbsp *rbsp,
+ const struct v4l2_h264_sps_video *sps_video)
+{
+ int ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_ASPECT_RATIO_INFO_PRESENT);
+ if (ret)
+ return ret;
+
+ if (sps_video->flags & V4L2_H264_SPS_VIDEO_FLAG_VUI_ASPECT_RATIO_INFO_PRESENT) {
+ ret = v4l2_h264_enc_rbsp_u8(rbsp, sps_video->aspect_ratio_idc);
+ if (ret)
+ return ret;
+
+ if (sps_video->aspect_ratio_idc ==
+ V4L2_H264_VUI_ASPECT_RATIO_IDC_EXTENDED) {
+ ret = v4l2_h264_enc_rbsp_u16(rbsp,
+ sps_video->sar_width);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_u16(rbsp,
+ sps_video->sar_height);
+ if (ret)
+ return ret;
+ }
+ }
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_OVERSCAN_INFO_PRESENT);
+ if (ret)
+ return ret;
+
+ if (sps_video->flags & V4L2_H264_SPS_VIDEO_FLAG_VUI_OVERSCAN_INFO_PRESENT) {
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_OVERSCAN_APPROPRIATE);
+ if (ret)
+ return ret;
+ }
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_VIDEO_SIGNAL_TYPE_PRESENT);
+ if (ret)
+ return ret;
+
+ if (sps_video->flags & V4L2_H264_SPS_VIDEO_FLAG_VUI_VIDEO_SIGNAL_TYPE_PRESENT) {
+ ret = v4l2_h264_enc_rbsp_bits(rbsp, sps_video->video_format, 3);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_VIDEO_FULL_RANGE);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_COLOUR_DESCRIPTION_PRESENT);
+ if (ret)
+ return ret;
+
+ if (sps_video->flags & V4L2_H264_SPS_VIDEO_FLAG_VUI_COLOUR_DESCRIPTION_PRESENT) {
+ ret = v4l2_h264_enc_rbsp_u8(rbsp,
+ sps_video->colour_primaries);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_u8(rbsp,
+ sps_video->transfer_characteristics);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_u8(rbsp,
+ sps_video->matrix_coefficients);
+ if (ret)
+ return ret;
+ }
+ }
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_CHROMA_LOC_INFO_PRESENT);
+ if (ret)
+ return ret;
+
+ if (sps_video->flags & V4L2_H264_SPS_VIDEO_FLAG_VUI_CHROMA_LOC_INFO_PRESENT) {
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ sps_video->chroma_sample_loc_type_top_field);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ sps_video->chroma_sample_loc_type_bottom_field);
+ if (ret)
+ return ret;
+ }
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_TIMING_INFO_PRESENT);
+ if (ret)
+ return ret;
+
+ if (sps_video->flags & V4L2_H264_SPS_VIDEO_FLAG_VUI_TIMING_INFO_PRESENT) {
+ ret = v4l2_h264_enc_rbsp_u32(rbsp,
+ sps_video->num_units_in_tick);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_u32(rbsp,
+ sps_video->time_scale);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_FIXED_FRAME_RATE);
+ if (ret)
+ return ret;
+ }
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_NAL_HRD_PARAMETERS_PRESENT);
+ if (ret)
+ return ret;
+
+ if (sps_video->flags & V4L2_H264_SPS_VIDEO_FLAG_VUI_NAL_HRD_PARAMETERS_PRESENT) {
+ ret = v4l2_h264_enc_rbsp_sps_hrd(rbsp, &sps_video->nal_hrd);
+ if (ret)
+ return ret;
+ }
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_VCL_HRD_PARAMETERS_PRESENT);
+ if (ret)
+ return ret;
+
+ if (sps_video->flags & V4L2_H264_SPS_VIDEO_FLAG_VUI_VCL_HRD_PARAMETERS_PRESENT) {
+ ret = v4l2_h264_enc_rbsp_sps_hrd(rbsp, &sps_video->vcl_hrd);
+ if (ret)
+ return ret;
+ }
+
+ if (sps_video->flags & V4L2_H264_SPS_VIDEO_FLAG_VUI_NAL_HRD_PARAMETERS_PRESENT ||
+ sps_video->flags & V4L2_H264_SPS_VIDEO_FLAG_VUI_VCL_HRD_PARAMETERS_PRESENT) {
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_LOW_DELAY_HRD);
+ if (ret)
+ return ret;
+ }
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_PIC_STRUCT_PRESENT);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_BITSTREAM_RESTRICTION);
+ if (ret)
+ return ret;
+
+ if (sps_video->flags & V4L2_H264_SPS_VIDEO_FLAG_VUI_BITSTREAM_RESTRICTION) {
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_MOTION_VECTORS_OVER_PIC_BOUNDARIES);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ sps_video->max_bytes_per_pic_denom);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ sps_video->max_bits_per_mb_denom);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ sps_video->log2_max_mv_length_horizontal);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ sps_video->log2_max_mv_length_vertical);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ sps_video->max_num_reorder_frames);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ sps_video->max_dec_frame_buffering);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+int v4l2_h264_enc_rbsp_sps(struct v4l2_h264_enc_rbsp *rbsp,
+ const struct v4l2_ctrl_h264_sps *sps,
+ const struct v4l2_h264_sps_video *sps_video)
+{
+ u8 constraint_set_flags = 0;
+ u8 seq_scaling_matrix_present_flag = 0;
+ unsigned int i;
+ int ret;
+
+ ret = v4l2_h264_enc_rbsp_nalu_begin(rbsp, 0, V4L2_H264_NALU_TYPE_SPS);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_u8(rbsp, sps->profile_idc);
+ if (ret)
+ return ret;
+
+ if (sps->constraint_set_flags & V4L2_H264_SPS_CONSTRAINT_SET0_FLAG)
+ constraint_set_flags |= BIT(7);
+ if (sps->constraint_set_flags & V4L2_H264_SPS_CONSTRAINT_SET1_FLAG)
+ constraint_set_flags |= BIT(6);
+ if (sps->constraint_set_flags & V4L2_H264_SPS_CONSTRAINT_SET2_FLAG)
+ constraint_set_flags |= BIT(5);
+ if (sps->constraint_set_flags & V4L2_H264_SPS_CONSTRAINT_SET3_FLAG)
+ constraint_set_flags |= BIT(4);
+ if (sps->constraint_set_flags & V4L2_H264_SPS_CONSTRAINT_SET4_FLAG)
+ constraint_set_flags |= BIT(3);
+ if (sps->constraint_set_flags & V4L2_H264_SPS_CONSTRAINT_SET5_FLAG)
+ constraint_set_flags |= BIT(2);
+
+ ret = v4l2_h264_enc_rbsp_u8(rbsp, constraint_set_flags);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_u8(rbsp, sps->level_idc);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, sps->seq_parameter_set_id);
+ if (ret)
+ return ret;
+
+ if (V4L2_H264_SPS_HAS_CHROMA_FORMAT(sps)) {
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, sps->chroma_format_idc);
+ if (ret)
+ return ret;
+
+ if (sps->chroma_format_idc == 3) {
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps->flags,
+ V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE);
+ if (ret)
+ return ret;
+ }
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, sps->bit_depth_luma_minus8);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, sps->bit_depth_chroma_minus8);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps->flags,
+ V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS);
+ if (ret)
+ return ret;
+
+ /* Scaling matrix is not supported. */
+ ret = v4l2_h264_enc_rbsp_bit(rbsp,
+ seq_scaling_matrix_present_flag);
+ if (ret)
+ return ret;
+ }
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, sps->log2_max_frame_num_minus4);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, sps->pic_order_cnt_type);
+ if (ret)
+ return ret;
+
+ if (!sps->pic_order_cnt_type) {
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ sps->log2_max_pic_order_cnt_lsb_minus4);
+ if (ret)
+ return ret;
+ } else if (sps->pic_order_cnt_type == 1) {
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps->flags,
+ V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_se(rbsp, sps->offset_for_non_ref_pic);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_se(rbsp,
+ sps->offset_for_top_to_bottom_field);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ sps->num_ref_frames_in_pic_order_cnt_cycle);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < sps->num_ref_frames_in_pic_order_cnt_cycle; i++) {
+ ret = v4l2_h264_enc_rbsp_se(rbsp,
+ sps->offset_for_ref_frame[i]);
+ if (ret)
+ return ret;
+ }
+ } else {
+ return -EINVAL;
+ }
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, sps->max_num_ref_frames);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps->flags,
+ V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, sps->pic_width_in_mbs_minus1);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, sps->pic_height_in_map_units_minus1);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps->flags,
+ V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY);
+ if (ret)
+ return ret;
+
+ if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY)) {
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps->flags,
+ V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD);
+ if (ret)
+ return ret;
+ }
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps->flags,
+ V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_FRAME_CROPPING);
+ if (ret)
+ return ret;
+
+ if (sps_video->flags & V4L2_H264_SPS_VIDEO_FLAG_FRAME_CROPPING) {
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ sps_video->frame_crop_left_offset);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ sps_video->frame_crop_right_offset);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ sps_video->frame_crop_top_offset);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ sps_video->frame_crop_bottom_offset);
+ if (ret)
+ return ret;
+ }
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, sps_video->flags,
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_PARAMETERS_PRESENT);
+ if (ret)
+ return ret;
+
+ if (sps_video->flags & V4L2_H264_SPS_VIDEO_FLAG_VUI_PARAMETERS_PRESENT) {
+ ret = v4l2_h264_enc_rbsp_sps_vui(rbsp, sps_video);
+ if (ret)
+ return ret;
+ }
+
+ return v4l2_h264_enc_rbsp_nalu_end(rbsp);
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_rbsp_sps);
+
+int v4l2_h264_enc_rbsp_pps(struct v4l2_h264_enc_rbsp *rbsp,
+ const struct v4l2_ctrl_h264_pps *pps)
+{
+ u8 pic_scaling_matrix_present_flag = 0;
+ int ret;
+
+ ret = v4l2_h264_enc_rbsp_nalu_begin(rbsp, 0, V4L2_H264_NALU_TYPE_PPS);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, pps->pic_parameter_set_id);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, pps->seq_parameter_set_id);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, pps->flags,
+ V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, pps->flags,
+ V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, pps->num_slice_groups_minus1);
+ if (ret)
+ return ret;
+
+ /* Multiple slice groups are not supported. */
+ if (pps->num_slice_groups_minus1 > 1)
+ return -EINVAL;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ pps->num_ref_idx_l0_default_active_minus1);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ pps->num_ref_idx_l1_default_active_minus1);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, pps->flags,
+ V4L2_H264_PPS_FLAG_WEIGHTED_PRED);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_bits(rbsp, pps->weighted_bipred_idc, 2);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_se(rbsp, pps->pic_init_qp_minus26);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_se(rbsp, pps->pic_init_qs_minus26);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_se(rbsp, pps->chroma_qp_index_offset);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, pps->flags,
+ V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, pps->flags,
+ V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, pps->flags,
+ V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, pps->flags,
+ V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE);
+ if (ret)
+ return ret;
+
+ /* Scaling matrix is not supported. */
+ ret = v4l2_h264_enc_rbsp_bit(rbsp, pic_scaling_matrix_present_flag);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_se(rbsp, pps->second_chroma_qp_index_offset);
+ if (ret)
+ return ret;
+
+ return v4l2_h264_enc_rbsp_nalu_end(rbsp);
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_rbsp_pps);
+
+int v4l2_h264_enc_rbsp_slice_header(struct v4l2_h264_enc_rbsp *rbsp,
+ const struct v4l2_ctrl_h264_sps *sps,
+ const struct v4l2_ctrl_h264_pps *pps,
+ const struct v4l2_ctrl_h264_encode_params *encode)
+{
+ u8 ref_pic_list_modification_flag_l0 = 0;
+ u8 ref_pic_list_modification_flag_l1 = 0;
+ u8 adaptive_ref_pic_marking_mode_flag = 0;
+ u32 first_mb_in_slice = 0;
+ u32 redundant_pic_cnt = 0;
+ u8 sp_for_switch_flag = 0;
+ s32 slice_qs_delta = 0;
+ u8 nal_unit_type;
+ int ret;
+
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_IDR_PIC)
+ nal_unit_type = V4L2_H264_NALU_TYPE_SLICE_IDR;
+ else
+ nal_unit_type = V4L2_H264_NALU_TYPE_SLICE_NON_IDR;
+
+ ret = v4l2_h264_enc_rbsp_nalu_begin(rbsp, encode->nal_ref_idc,
+ nal_unit_type);
+ if (ret)
+ return ret;
+
+ /* Multiple slices are not supported. */
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, first_mb_in_slice);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, encode->slice_type);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, encode->pic_parameter_set_id);
+ if (ret)
+ return ret;
+
+ if (sps->flags & V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE) {
+ ret = v4l2_h264_enc_rbsp_bits(rbsp, encode->colour_plane_id, 2);
+ if (ret)
+ return ret;
+ }
+
+ ret = v4l2_h264_enc_rbsp_bits(rbsp, encode->frame_num,
+ sps->log2_max_frame_num_minus4 + 4);
+ if (ret)
+ return ret;
+
+ if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY)) {
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, encode->flags,
+ V4L2_H264_ENCODE_FLAG_FIELD_PIC);
+ if (ret)
+ return ret;
+
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_FIELD_PIC) {
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, encode->flags,
+ V4L2_H264_ENCODE_FLAG_BOTTOM_FIELD);
+ if (ret)
+ return ret;
+ }
+ }
+
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_IDR_PIC) {
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, encode->idr_pic_id);
+ if (ret)
+ return ret;
+ }
+
+ if (!sps->pic_order_cnt_type) {
+ ret = v4l2_h264_enc_rbsp_bits(rbsp, encode->pic_order_cnt_lsb,
+ sps->log2_max_pic_order_cnt_lsb_minus4 + 4);
+ if (ret)
+ return ret;
+
+ if (pps->flags & V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT &&
+ !(encode->flags & V4L2_H264_ENCODE_FLAG_FIELD_PIC)) {
+ ret = v4l2_h264_enc_rbsp_se(rbsp,
+ encode->delta_pic_order_cnt_bottom);
+ if (ret)
+ return ret;
+ }
+ }
+
+ if (sps->pic_order_cnt_type == 1 &&
+ !(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO)) {
+ ret = v4l2_h264_enc_rbsp_se(rbsp,
+ encode->delta_pic_order_cnt0);
+ if (ret)
+ return ret;
+
+ if (pps->flags & V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT &&
+ !(encode->flags & V4L2_H264_ENCODE_FLAG_FIELD_PIC)) {
+ ret = v4l2_h264_enc_rbsp_se(rbsp,
+ encode->delta_pic_order_cnt1);
+ if (ret)
+ return ret;
+ }
+ }
+
+ if (pps->flags & V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT) {
+ /* Redundant pictures are not supported. */
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, redundant_pic_cnt);
+ if (ret)
+ return ret;
+ }
+
+ if (encode->slice_type == V4L2_H264_SLICE_TYPE_B) {
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, encode->flags,
+ V4L2_H264_ENCODE_FLAG_DIRECT_SPATIAL_MV_PRED);
+ if (ret)
+ return ret;
+ }
+
+ if (encode->slice_type == V4L2_H264_SLICE_TYPE_P ||
+ encode->slice_type == V4L2_H264_SLICE_TYPE_SP ||
+ encode->slice_type == V4L2_H264_SLICE_TYPE_B) {
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, encode->flags,
+ V4L2_H264_ENCODE_FLAG_NUM_REF_IDX_ACTIVE_OVERRIDE);
+ if (ret)
+ return ret;
+
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_NUM_REF_IDX_ACTIVE_OVERRIDE) {
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ encode->num_ref_idx_l0_active_minus1);
+ if (ret)
+ return ret;
+ }
+
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_NUM_REF_IDX_ACTIVE_OVERRIDE &&
+ encode->slice_type == V4L2_H264_SLICE_TYPE_B) {
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ encode->num_ref_idx_l1_active_minus1);
+ if (ret)
+ return ret;
+ }
+ }
+
+ if (encode->slice_type != V4L2_H264_SLICE_TYPE_I &&
+ encode->slice_type != V4L2_H264_SLICE_TYPE_SI) {
+ /* Ref pic list modification is not supported. */
+ ret = v4l2_h264_enc_rbsp_bit(rbsp,
+ ref_pic_list_modification_flag_l0);
+ if (ret)
+ return ret;
+ }
+
+ if (encode->slice_type == V4L2_H264_SLICE_TYPE_B) {
+ /* Ref pic list modification is not supported. */
+ ret = v4l2_h264_enc_rbsp_bit(rbsp,
+ ref_pic_list_modification_flag_l1);
+ if (ret)
+ return ret;
+ }
+
+ /* Prediction weights are not supported. */
+ if (V4L2_H264_CTRL_PRED_WEIGHTS_REQUIRED(pps, encode))
+ return -EINVAL;
+
+ if (encode->nal_ref_idc) {
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_IDR_PIC) {
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, encode->flags,
+ V4L2_H264_ENCODE_FLAG_NO_OUTPUT_OF_PRIOR_PICS);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_flag(rbsp, encode->flags,
+ V4L2_H264_ENCODE_FLAG_LONG_TERM_REFERENCE);
+ if (ret)
+ return ret;
+ } else {
+ /* Adaptive ref pic marking mode is not supported. */
+ ret = v4l2_h264_enc_rbsp_bit(rbsp,
+ adaptive_ref_pic_marking_mode_flag);
+ if (ret)
+ return ret;
+ }
+ }
+
+ if (pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE &&
+ encode->slice_type != V4L2_H264_SLICE_TYPE_I &&
+ encode->slice_type != V4L2_H264_SLICE_TYPE_SI) {
+ ret = v4l2_h264_enc_rbsp_ue(rbsp, encode->cabac_init_idc);
+ if (ret)
+ return ret;
+ }
+
+ ret = v4l2_h264_enc_rbsp_se(rbsp, encode->slice_qp_delta);
+ if (ret)
+ return ret;
+
+ if (encode->slice_type == V4L2_H264_SLICE_TYPE_SP) {
+ /* Switching slices are not supported. */
+ ret = v4l2_h264_enc_rbsp_bit(rbsp, sp_for_switch_flag);
+ if (ret)
+ return ret;
+ }
+
+ if (encode->slice_type == V4L2_H264_SLICE_TYPE_SP ||
+ encode->slice_type == V4L2_H264_SLICE_TYPE_SI) {
+ ret = v4l2_h264_enc_rbsp_se(rbsp, slice_qs_delta);
+ if (ret)
+ return ret;
+ }
+
+ if (pps->flags & V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT) {
+ ret = v4l2_h264_enc_rbsp_ue(rbsp,
+ encode->disable_deblocking_filter_idc);
+ if (ret)
+ return ret;
+
+ if (encode->disable_deblocking_filter_idc != 1) {
+ ret = v4l2_h264_enc_rbsp_se(rbsp,
+ encode->slice_alpha_c0_offset_div2);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_rbsp_se(rbsp,
+ encode->slice_beta_offset_div2);
+ if (ret)
+ return ret;
+ }
+ }
+
+ /* The slice NALU is not finished, the hardware has to write the rest! */
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_rbsp_slice_header);
+
+MODULE_DESCRIPTION("V4L2 H.264 Encode RBSP");
+MODULE_AUTHOR("Paul Kocialkowski <paulk@sys-base.io>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/v4l2-core/v4l2-h264-enc.c b/drivers/media/v4l2-core/v4l2-h264-enc.c
index d4e1450691fb..0b46922d1d7a 100644
--- a/drivers/media/v4l2-core/v4l2-h264-enc.c
+++ b/drivers/media/v4l2-core/v4l2-h264-enc.c
@@ -9,10 +9,12 @@
#include <linux/v4l2-controls.h>
#include <media/v4l2-h264.h>
#include <media/v4l2-h264-enc.h>
+#include <media/v4l2-h264-enc-rbsp.h>
#include <media/videobuf2-v4l2.h>
int v4l2_h264_enc_init(struct v4l2_h264_enc *enc)
{
+ struct v4l2_h264_enc_rbsp *rbsp = &enc->rbsp;
int ret;
if ((!enc->format && !enc->format_mplane) || !enc->timeperframe ||
@@ -23,6 +25,9 @@ int v4l2_h264_enc_init(struct v4l2_h264_enc *enc)
memset(&enc->state_next, 0, sizeof(enc->state_next));
enc->state_serial = 0;
+ rbsp->ops = enc->rbsp_ops;
+ rbsp->private_data = enc->private_data;
+
return 0;
}
EXPORT_SYMBOL_GPL(v4l2_h264_enc_init);
@@ -520,6 +525,146 @@ static int state_complete(struct v4l2_h264_enc *enc,
return 0;
}
+static int rbsp_update(struct v4l2_h264_enc *enc)
+{
+ struct v4l2_h264_enc_state *state = &enc->state_next;
+ struct v4l2_h264_enc_state *state_active = &enc->state_active;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ struct v4l2_ctrl_handler *handler = enc->ctrl_handler;
+ struct v4l2_ctrl *ctrl;
+
+ enc->rbsp_update = 0;
+
+ /* Start Code */
+
+ ctrl = v4l2_ctrl_find(handler, V4L2_CID_STATELESS_H264_START_CODE);
+ if ((ctrl && ctrl->cur.val == V4L2_STATELESS_H264_START_CODE_ANNEX_B) ||
+ !ctrl)
+ enc->rbsp_update |= V4L2_H264_ENC_RBSP_UPDATE_START_CODE;
+
+ /* AUD */
+
+ ctrl = v4l2_ctrl_find(handler, V4L2_CID_MPEG_VIDEO_AU_DELIMITER);
+ if (ctrl && ctrl->cur.val)
+ enc->rbsp_update |= V4L2_H264_ENC_RBSP_UPDATE_AUD;
+
+ /* SPS */
+
+ if (!enc->state_serial) {
+ if (memcmp(&state_active->sps, &state->sps,
+ sizeof(state_active->sps)) ||
+ memcmp(&state_active->sps_video, &state->sps_video,
+ sizeof(state_active->sps_video)))
+ enc->rbsp_update |= V4L2_H264_ENC_RBSP_UPDATE_SPS;
+ } else {
+ enc->rbsp_update |= V4L2_H264_ENC_RBSP_UPDATE_SPS;
+ }
+
+ /* PPS */
+
+ if (!enc->state_serial) {
+ if (memcmp(&state_active->pps, &state->pps,
+ sizeof(state_active->pps)))
+ enc->rbsp_update |= V4L2_H264_ENC_RBSP_UPDATE_PPS;
+ } else {
+ enc->rbsp_update |= V4L2_H264_ENC_RBSP_UPDATE_PPS;
+ }
+
+ /* IDR Prepend */
+
+ ctrl = v4l2_ctrl_find(handler,
+ V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR);
+ if (ctrl && ctrl->cur.val &&
+ encode->flags & V4L2_H264_ENCODE_FLAG_IDR_PIC)
+ enc->rbsp_update |= V4L2_H264_ENC_RBSP_UPDATE_SPS |
+ V4L2_H264_ENC_RBSP_UPDATE_PPS;
+
+ /* Slice */
+
+ enc->rbsp_update |= V4L2_H264_ENC_RBSP_UPDATE_SLICE_HEADER;
+
+ return 0;
+}
+
+static int rbsp_step_unit(struct v4l2_h264_enc *enc,
+ unsigned int rbsp_update, unsigned int hw_flag)
+{
+ struct v4l2_h264_enc_state *state = &enc->state_active;
+ struct v4l2_ctrl_h264_sps *sps = &state->sps;
+ struct v4l2_h264_sps_video *sps_video = &state->sps_video;
+ struct v4l2_ctrl_h264_pps *pps = &state->pps;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ struct v4l2_h264_enc_rbsp *rbsp = &enc->rbsp;
+ u8 primary_pic_type;
+ int ret;
+
+ /* Return if no update is needed or if hardware generates the unit. */
+ if (!(enc->rbsp_update & rbsp_update) || enc->flags & hw_flag)
+ return 0;
+
+ if (enc->rbsp_update & V4L2_H264_ENC_RBSP_UPDATE_START_CODE) {
+ ret = v4l2_h264_enc_rbsp_start_code(rbsp);
+ if (ret)
+ return ret;
+ }
+
+ if (rbsp_update == V4L2_H264_ENC_RBSP_UPDATE_AUD) {
+ if (enc->flags & V4L2_H264_ENC_FLAG_INTER_BIPRED &&
+ enc->flags & V4L2_H264_ENC_FLAG_INTER_PRED)
+ primary_pic_type = V4L2_H264_PRIMARY_PIC_TYPE_IPB;
+ else if (enc->flags & V4L2_H264_ENC_FLAG_INTER_PRED)
+ primary_pic_type = V4L2_H264_PRIMARY_PIC_TYPE_IP;
+ else
+ primary_pic_type = V4L2_H264_PRIMARY_PIC_TYPE_I;
+ }
+
+ if (rbsp_update == V4L2_H264_ENC_RBSP_UPDATE_AUD)
+ return v4l2_h264_enc_rbsp_aud(rbsp, primary_pic_type);
+ else if (rbsp_update == V4L2_H264_ENC_RBSP_UPDATE_SPS)
+ return v4l2_h264_enc_rbsp_sps(rbsp, sps, sps_video);
+ else if (rbsp_update == V4L2_H264_ENC_RBSP_UPDATE_PPS)
+ return v4l2_h264_enc_rbsp_pps(rbsp, pps);
+ else if (rbsp_update == V4L2_H264_ENC_RBSP_UPDATE_SLICE_HEADER)
+ return v4l2_h264_enc_rbsp_slice_header(rbsp, sps, pps, encode);
+
+ return -EINVAL;
+}
+
+static int rbsp_step(struct v4l2_h264_enc *enc,
+ struct vb2_v4l2_buffer *buffer)
+{
+ struct v4l2_h264_enc_rbsp *rbsp = &enc->rbsp;
+ void *pointer = vb2_plane_vaddr(&buffer->vb2_buf, 0);
+ unsigned int size = vb2_plane_size(&buffer->vb2_buf, 0);
+ int ret;
+
+ ret = v4l2_h264_enc_rbsp_init(rbsp, pointer, size);
+ if (ret)
+ return ret;
+
+ ret = rbsp_step_unit(enc, V4L2_H264_ENC_RBSP_UPDATE_AUD,
+ V4L2_H264_ENC_FLAG_HW_AUD);
+ if (ret)
+ return ret;
+
+ ret = rbsp_step_unit(enc, V4L2_H264_ENC_RBSP_UPDATE_SPS,
+ V4L2_H264_ENC_FLAG_HW_SPS);
+ if (ret)
+ return ret;
+
+ ret = rbsp_step_unit(enc, V4L2_H264_ENC_RBSP_UPDATE_PPS,
+ V4L2_H264_ENC_FLAG_HW_PPS);
+ if (ret)
+ return ret;
+
+ ret = rbsp_step_unit(enc, V4L2_H264_ENC_RBSP_UPDATE_SLICE_HEADER,
+ V4L2_H264_ENC_FLAG_HW_SLICE_HEADER);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
int v4l2_h264_enc_step(struct v4l2_h264_enc *enc,
struct vb2_v4l2_buffer *buffer)
{
@@ -529,10 +674,18 @@ int v4l2_h264_enc_step(struct v4l2_h264_enc *enc,
if (ret)
return ret;
+ ret = rbsp_update(enc);
+ if (ret)
+ return ret;
+
ret = state_commit(enc);
if (ret)
return ret;
+ ret = rbsp_step(enc, buffer);
+ if (ret)
+ return ret;
+
return 0;
}
EXPORT_SYMBOL_GPL(v4l2_h264_enc_step);
diff --git a/include/media/v4l2-h264-enc-rbsp.h b/include/media/v4l2-h264-enc-rbsp.h
new file mode 100644
index 000000000000..2ce9b4051436
--- /dev/null
+++ b/include/media/v4l2-h264-enc-rbsp.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * V4L2 H.264 Encode RBSP
+ *
+ * Copyright (C) 2025-2026 Paul Kocialkowski <paulk@sys-base.io>
+ */
+
+#ifndef _MEDIA_V4L2_H264_ENC_RBSP_H
+#define _MEDIA_V4L2_H264_ENC_RBSP_H
+
+#include <linux/v4l2-controls.h>
+#include <media/v4l2-h264.h>
+
+#define V4L2_H264_ENC_RBSP_EPTB 0x3
+
+#define V4L2_H264_ENC_RBSP_UPDATE_START_CODE 0x1
+#define V4L2_H264_ENC_RBSP_UPDATE_AUD 0x2
+#define V4L2_H264_ENC_RBSP_UPDATE_SPS 0x4
+#define V4L2_H264_ENC_RBSP_UPDATE_PPS 0x8
+#define V4L2_H264_ENC_RBSP_UPDATE_SLICE_HEADER 0x10
+
+#define v4l2_h264_enc_rbsp_op(r, o, a...) \
+ ({ \
+ int ret; \
+ if ((r)->ops && (r)->ops->o) \
+ ret = (r)->ops->o(r, ##a); \
+ else \
+ ret = -EOPNOTSUPP; \
+ ret; \
+ })
+
+struct v4l2_h264_enc_rbsp;
+
+struct v4l2_h264_enc_rbsp_ops {
+ int (*bits_raw)(struct v4l2_h264_enc_rbsp *rbsp, u32 value,
+ unsigned char bits_count);
+ int (*bits)(struct v4l2_h264_enc_rbsp *rbsp, u32 value,
+ unsigned char bits_count);
+ int (*ue)(struct v4l2_h264_enc_rbsp *rbsp, u32 value);
+ int (*se)(struct v4l2_h264_enc_rbsp *rbsp, s32 value);
+ int (*align)(struct v4l2_h264_enc_rbsp *rbsp);
+};
+
+struct v4l2_h264_enc_rbsp {
+ const struct v4l2_h264_enc_rbsp_ops *ops;
+ void *private_data;
+
+ u8 *pointer;
+ unsigned int size;
+ unsigned char bit_offset;
+ unsigned int bits_count;
+ unsigned int zero_count;
+};
+
+int v4l2_h264_enc_rbsp_init(struct v4l2_h264_enc_rbsp *rbsp, u8 *pointer,
+ unsigned int size);
+unsigned int v4l2_h264_enc_rbsp_bits_count(struct v4l2_h264_enc_rbsp *rbsp);
+unsigned int v4l2_h264_enc_rbsp_bytes_count(struct v4l2_h264_enc_rbsp *rbsp);
+int v4l2_h264_enc_rbsp_start_code(struct v4l2_h264_enc_rbsp *rbsp);
+int v4l2_h264_enc_rbsp_aud(struct v4l2_h264_enc_rbsp *rbsp,
+ u8 primary_pic_type);
+int v4l2_h264_enc_rbsp_sps(struct v4l2_h264_enc_rbsp *rbsp,
+ const struct v4l2_ctrl_h264_sps *sps,
+ const struct v4l2_h264_sps_video *sps_video);
+int v4l2_h264_enc_rbsp_pps(struct v4l2_h264_enc_rbsp *rbsp,
+ const struct v4l2_ctrl_h264_pps *pps);
+int v4l2_h264_enc_rbsp_slice_header(struct v4l2_h264_enc_rbsp *rbsp,
+ const struct v4l2_ctrl_h264_sps *sps,
+ const struct v4l2_ctrl_h264_pps *pps,
+ const struct v4l2_ctrl_h264_encode_params *encode);
+
+#endif
diff --git a/include/media/v4l2-h264-enc.h b/include/media/v4l2-h264-enc.h
index 2978a73baacd..3d6b97408707 100644
--- a/include/media/v4l2-h264-enc.h
+++ b/include/media/v4l2-h264-enc.h
@@ -10,6 +10,7 @@
#include <linux/v4l2-controls.h>
#include <linux/videodev2.h>
+#include <media/v4l2-h264-enc-rbsp.h>
#include <media/videobuf2-v4l2.h>
#define V4L2_H264_ENC_MB_UNIT 16
@@ -55,6 +56,7 @@ struct v4l2_h264_enc_ops {
struct v4l2_h264_enc {
const struct v4l2_h264_enc_ops *ops;
+ const struct v4l2_h264_enc_rbsp_ops *rbsp_ops;
void *private_data;
struct v4l2_pix_format *format;
@@ -66,6 +68,9 @@ struct v4l2_h264_enc {
struct v4l2_h264_enc_state state_next;
unsigned int state_serial;
+ struct v4l2_h264_enc_rbsp rbsp;
+ unsigned int rbsp_update;
+
unsigned int flags;
};
--
2.53.0
^ permalink raw reply related
* [PATCH 04/14] media: h264: Add stateless encode core
From: Paul Kocialkowski @ 2026-05-22 10:16 UTC (permalink / raw)
To: devicetree, imx, linux-arm-kernel, linux-kernel, linux-media
Cc: Rob Herring, Krzysztof Kozlowski, Conor Dooley, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Nicolas Dufresne,
Benjamin Gaignard, Philipp Zabel, Mauro Carvalho Chehab,
Hans Verkuil, Marco Felsch, Michael Tretter, Paul Kocialkowski
In-Reply-To: <20260522101653.2565125-1-paulk@sys-base.io>
The H.264 stateless encode core is the common implementation called by
drivers to coordinate stateless encoding. It provides all the relevant
configuration parameters to drivers, through a state mechanism.
States are built from controls (some of which are optional) with various
inter-control checks along the way. A state can be constrained by the
driver through a dedicated operation callback in order to disable
unsupported features that may have been requested by userspace.
The general philosophy is to adapt parameters provided by userspace to
reach a state that can be handled by the hardware and bail out if no
such accommodation can be made.
Adapted parameters are returned to userspace by updating the controls.
This is currently not attached to the inbound media request but should
be when support for it becomes available.
Signed-off-by: Paul Kocialkowski <paulk@sys-base.io>
---
drivers/media/v4l2-core/Kconfig | 4 +
drivers/media/v4l2-core/Makefile | 1 +
drivers/media/v4l2-core/v4l2-h264-enc.c | 555 ++++++++++++++++++++++++
include/media/v4l2-h264-enc.h | 79 ++++
4 files changed, 639 insertions(+)
create mode 100644 drivers/media/v4l2-core/v4l2-h264-enc.c
create mode 100644 include/media/v4l2-h264-enc.h
diff --git a/drivers/media/v4l2-core/Kconfig b/drivers/media/v4l2-core/Kconfig
index 331b8e535e5b..121ab82a9631 100644
--- a/drivers/media/v4l2-core/Kconfig
+++ b/drivers/media/v4l2-core/Kconfig
@@ -44,6 +44,10 @@ config V4L2_JPEG_HELPER
config V4L2_H264
tristate
+# Used by drivers that need v4l2-h264-enc.ko
+config V4L2_H264_ENC
+ tristate
+
# Used by drivers that need v4l2-vp9.ko
config V4L2_VP9
tristate
diff --git a/drivers/media/v4l2-core/Makefile b/drivers/media/v4l2-core/Makefile
index 2177b9d63a8f..bd319e363c8e 100644
--- a/drivers/media/v4l2-core/Makefile
+++ b/drivers/media/v4l2-core/Makefile
@@ -29,6 +29,7 @@ obj-$(CONFIG_V4L2_CCI) += v4l2-cci.o
obj-$(CONFIG_V4L2_FLASH_LED_CLASS) += v4l2-flash-led-class.o
obj-$(CONFIG_V4L2_FWNODE) += v4l2-fwnode.o
obj-$(CONFIG_V4L2_H264) += v4l2-h264.o
+obj-$(CONFIG_V4L2_H264_ENC) += v4l2-h264-enc.o
obj-$(CONFIG_V4L2_JPEG_HELPER) += v4l2-jpeg.o
obj-$(CONFIG_V4L2_MEM2MEM_DEV) += v4l2-mem2mem.o
obj-$(CONFIG_V4L2_VP9) += v4l2-vp9.o
diff --git a/drivers/media/v4l2-core/v4l2-h264-enc.c b/drivers/media/v4l2-core/v4l2-h264-enc.c
new file mode 100644
index 000000000000..d4e1450691fb
--- /dev/null
+++ b/drivers/media/v4l2-core/v4l2-h264-enc.c
@@ -0,0 +1,555 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * V4L2 H.264 Encode Core
+ *
+ * Copyright (C) 2025-2026 Paul Kocialkowski <paulk@sys-base.io>
+ */
+
+#include <linux/module.h>
+#include <linux/v4l2-controls.h>
+#include <media/v4l2-h264.h>
+#include <media/v4l2-h264-enc.h>
+#include <media/videobuf2-v4l2.h>
+
+int v4l2_h264_enc_init(struct v4l2_h264_enc *enc)
+{
+ int ret;
+
+ if ((!enc->format && !enc->format_mplane) || !enc->timeperframe ||
+ !enc->ctrl_handler)
+ return -EINVAL;
+
+ memset(&enc->state_active, 0, sizeof(enc->state_active));
+ memset(&enc->state_next, 0, sizeof(enc->state_next));
+ enc->state_serial = 0;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_init);
+
+void v4l2_h264_enc_exit(struct v4l2_h264_enc *enc)
+{
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_exit);
+
+static int state_prepare_params(struct v4l2_h264_enc *enc)
+{
+ struct v4l2_h264_enc_state *state = &enc->state_next;
+ struct v4l2_ctrl_h264_sps *sps = &state->sps;
+ struct v4l2_h264_sps_video *sps_video = &state->sps_video;
+ struct v4l2_ctrl_h264_pps *pps = &state->pps;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ struct v4l2_ctrl_handler *handler = enc->ctrl_handler;
+ struct v4l2_fract *timeperframe = enc->timeperframe;
+ unsigned int width, height;
+ unsigned int colorspace, quantization, xfer_func, ycbcr_enc;
+ struct v4l2_ctrl *ctrl;
+
+ /* Time per frame */
+
+ state->timeperframe = *timeperframe;
+
+ /* Format */
+
+ if (enc->format_mplane)
+ width = enc->format_mplane->width;
+ else
+ width = enc->format->width;
+
+ state->width_mbs = DIV_ROUND_UP(width, V4L2_H264_ENC_MB_UNIT);
+ state->width_aligned = ALIGN(width, V4L2_H264_ENC_MB_UNIT);
+
+ if (enc->format_mplane)
+ height = enc->format_mplane->height;
+ else
+ height = enc->format->height;
+
+ state->height_mbs = DIV_ROUND_UP(height, V4L2_H264_ENC_MB_UNIT);
+ state->height_aligned = ALIGN(height, V4L2_H264_ENC_MB_UNIT);
+
+ /* SPS */
+
+ ctrl = v4l2_ctrl_find(handler, V4L2_CID_STATELESS_H264_SPS);
+ if (!ctrl)
+ return -EINVAL;
+
+ memcpy(sps, ctrl->p_cur.p_h264_sps, sizeof(*sps));
+
+ sps->pic_width_in_mbs_minus1 = state->width_mbs - 1;
+ sps->pic_height_in_map_units_minus1 = state->height_mbs - 1;
+
+ /*
+ * Only pic_order_cnt_type = 0 is currently supported.
+ * Error out since no pic_order_cnt_lsb was provided.
+ */
+ if (sps->pic_order_cnt_type)
+ return -EINVAL;
+
+ /* SPS Video */
+
+ if (width < state->width_aligned || height < state->height_aligned) {
+ sps_video->flags |= V4L2_H264_SPS_VIDEO_FLAG_FRAME_CROPPING;
+
+ sps_video->frame_crop_left_offset = 0;
+ sps_video->frame_crop_right_offset = (state->width_aligned -
+ width) / 2;
+ sps_video->frame_crop_top_offset = 0;
+ sps_video->frame_crop_bottom_offset = (state->height_aligned -
+ height) / 2;
+ }
+
+ if (timeperframe->numerator && timeperframe->denominator) {
+ sps_video->flags |=
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_PARAMETERS_PRESENT |
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_TIMING_INFO_PRESENT |
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_FIXED_FRAME_RATE;
+
+ /* Timing info is always provided as a field rate. */
+ sps_video->num_units_in_tick = timeperframe->numerator;
+ sps_video->time_scale = timeperframe->denominator * 2;
+ }
+
+ ctrl = v4l2_ctrl_find(handler, V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE);
+ if (ctrl && ctrl->cur.val) {
+ sps_video->flags |=
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_ASPECT_RATIO_INFO_PRESENT;
+
+ ctrl = v4l2_ctrl_find(handler,
+ V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_IDC);
+ if (!ctrl)
+ return -EINVAL;
+
+ if (ctrl->cur.val == V4L2_MPEG_VIDEO_H264_VUI_SAR_IDC_EXTENDED)
+ sps_video->aspect_ratio_idc =
+ V4L2_H264_VUI_ASPECT_RATIO_IDC_EXTENDED;
+ else
+ sps_video->aspect_ratio_idc = ctrl->cur.val;
+
+ if (sps_video->aspect_ratio_idc ==
+ V4L2_H264_VUI_ASPECT_RATIO_IDC_EXTENDED) {
+ ctrl = v4l2_ctrl_find(handler,
+ V4L2_CID_MPEG_VIDEO_H264_VUI_EXT_SAR_WIDTH);
+ if (!ctrl)
+ return -EINVAL;
+
+ sps_video->sar_width = ctrl->cur.val;
+
+ ctrl = v4l2_ctrl_find(handler,
+ V4L2_CID_MPEG_VIDEO_H264_VUI_EXT_SAR_HEIGHT);
+ if (!ctrl)
+ return -EINVAL;
+
+ sps_video->sar_height = ctrl->cur.val;
+ }
+ }
+
+ if (enc->format_mplane) {
+ colorspace = enc->format_mplane->colorspace;
+ quantization = enc->format_mplane->quantization;
+ xfer_func = enc->format_mplane->xfer_func;
+ ycbcr_enc = enc->format_mplane->ycbcr_enc;
+ } else {
+ colorspace = enc->format->colorspace;
+ quantization = enc->format->quantization;
+ xfer_func = enc->format->xfer_func;
+ ycbcr_enc = enc->format->ycbcr_enc;
+ }
+
+ if (colorspace != V4L2_COLORSPACE_DEFAULT ||
+ quantization != V4L2_QUANTIZATION_DEFAULT ||
+ xfer_func != V4L2_XFER_FUNC_DEFAULT ||
+ ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT) {
+ sps_video->flags |=
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_VIDEO_SIGNAL_TYPE_PRESENT |
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_COLOUR_DESCRIPTION_PRESENT;
+
+ sps_video->video_format = V4L2_H264_VUI_VIDEO_UNSPECIFIED;
+
+ switch (colorspace) {
+ case V4L2_COLORSPACE_SMPTE170M:
+ sps_video->colour_primaries =
+ V4L2_H264_VUI_COLOUR_SMPTE170M;
+ break;
+ case V4L2_COLORSPACE_SMPTE240M:
+ sps_video->colour_primaries =
+ V4L2_H264_VUI_COLOUR_SMPTE240M;
+ break;
+ case V4L2_COLORSPACE_REC709:
+ sps_video->colour_primaries =
+ V4L2_H264_VUI_COLOUR_BT709;
+ break;
+ case V4L2_COLORSPACE_470_SYSTEM_M:
+ sps_video->colour_primaries =
+ V4L2_H264_VUI_COLOUR_BT470_SYSTEM_M;
+ break;
+ case V4L2_COLORSPACE_470_SYSTEM_BG:
+ sps_video->colour_primaries =
+ V4L2_H264_VUI_COLOUR_BT470_SYSTEM_BG;
+ break;
+ case V4L2_COLORSPACE_BT2020:
+ sps_video->colour_primaries =
+ V4L2_H264_VUI_COLOUR_BT2020;
+ break;
+ case V4L2_COLORSPACE_JPEG:
+ sps_video->colour_primaries =
+ V4L2_H264_VUI_COLOUR_BT709;
+ break;
+ case V4L2_COLORSPACE_SRGB:
+ sps_video->colour_primaries =
+ V4L2_H264_VUI_COLOUR_BT709;
+ break;
+ default:
+ sps_video->colour_primaries =
+ V4L2_H264_VUI_COLOUR_UNSPECIFIED;
+ break;
+ }
+
+ if (quantization == V4L2_QUANTIZATION_FULL_RANGE)
+ sps_video->flags |=
+ V4L2_H264_SPS_VIDEO_FLAG_VUI_VIDEO_FULL_RANGE;
+
+ switch (xfer_func) {
+ case V4L2_XFER_FUNC_709:
+ /*
+ * All of these are equivalent but the H.264
+ * specification allows being more specific.
+ */
+ switch (colorspace) {
+ case V4L2_COLORSPACE_SMPTE170M:
+ sps_video->transfer_characteristics =
+ V4L2_H264_VUI_TRANSFER_SMPTE170M;
+ break;
+ case V4L2_COLORSPACE_470_SYSTEM_M:
+ sps_video->transfer_characteristics =
+ V4L2_H264_VUI_TRANSFER_BT470_SYSTEM_M;
+ break;
+ case V4L2_COLORSPACE_470_SYSTEM_BG:
+ sps_video->transfer_characteristics =
+ V4L2_H264_VUI_TRANSFER_BT470_SYSTEM_BG;
+ break;
+ default:
+ sps_video->transfer_characteristics =
+ V4L2_H264_VUI_TRANSFER_BT709;
+ break;
+ }
+ break;
+ case V4L2_XFER_FUNC_SRGB:
+ sps_video->transfer_characteristics =
+ V4L2_H264_VUI_TRANSFER_SRGB;
+ break;
+ case V4L2_XFER_FUNC_SMPTE240M:
+ sps_video->transfer_characteristics =
+ V4L2_H264_VUI_TRANSFER_SMPTE240M;
+ break;
+ case V4L2_XFER_FUNC_NONE:
+ sps_video->transfer_characteristics =
+ V4L2_H264_VUI_TRANSFER_LINEAR;
+ break;
+ default:
+ sps_video->transfer_characteristics =
+ V4L2_H264_VUI_TRANSFER_UNSPECIFIED;
+ break;
+ }
+
+ switch (ycbcr_enc) {
+ case V4L2_YCBCR_ENC_601:
+ /*
+ * All of these are equivalent but the H.264
+ * specification allows being more specific.
+ */
+ switch (colorspace) {
+ case V4L2_COLORSPACE_SMPTE170M:
+ sps_video->matrix_coefficients =
+ V4L2_H264_VUI_MATRIX_SMPTE170M;
+ break;
+ case V4L2_COLORSPACE_470_SYSTEM_M:
+ sps_video->matrix_coefficients =
+ V4L2_H264_VUI_MATRIX_BT470_SYSTEM_M;
+ break;
+ default:
+ sps_video->matrix_coefficients =
+ V4L2_H264_VUI_MATRIX_BT470_SYSTEM_BG;
+ break;
+ }
+ break;
+ case V4L2_YCBCR_ENC_709:
+ sps_video->matrix_coefficients =
+ V4L2_H264_VUI_MATRIX_BT709;
+ break;
+ case V4L2_YCBCR_ENC_SMPTE240M:
+ sps_video->matrix_coefficients =
+ V4L2_H264_VUI_MATRIX_SMPTE240M;
+ break;
+ case V4L2_YCBCR_ENC_BT2020:
+ sps_video->matrix_coefficients =
+ V4L2_H264_VUI_MATRIX_BT2020;
+ break;
+ case V4L2_YCBCR_ENC_BT2020_CONST_LUM:
+ sps_video->matrix_coefficients =
+ V4L2_H264_VUI_MATRIX_BT2020_CONST_LUM;
+ break;
+ default:
+ sps_video->matrix_coefficients =
+ V4L2_H264_VUI_MATRIX_UNSPECIFIED;
+ break;
+ }
+ }
+
+ /* PPS */
+
+ ctrl = v4l2_ctrl_find(handler, V4L2_CID_STATELESS_H264_PPS);
+ if (!ctrl)
+ return -EINVAL;
+
+ memcpy(pps, ctrl->p_cur.p_h264_pps, sizeof(*pps));
+
+ /* Only single instances of PPS and SPS are supported. */
+ if (pps->seq_parameter_set_id != sps->seq_parameter_set_id)
+ pps->seq_parameter_set_id = sps->seq_parameter_set_id;
+
+ pps->flags &= ~V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT;
+
+ /* Slice groups are not supported. */
+ pps->num_slice_groups_minus1 = 0;
+
+ if (pps->num_ref_idx_l0_default_active_minus1 >
+ (sps->max_num_ref_frames - 1))
+ pps->num_ref_idx_l0_default_active_minus1 =
+ sps->max_num_ref_frames - 1;
+
+ if (pps->num_ref_idx_l1_default_active_minus1 >
+ (sps->max_num_ref_frames - 1))
+ pps->num_ref_idx_l1_default_active_minus1 =
+ sps->max_num_ref_frames - 1;
+
+ pps->flags &= ~V4L2_H264_PPS_FLAG_WEIGHTED_PRED;
+ pps->weighted_bipred_idc = 0;
+
+ /* Switching slices are not supported. */
+ pps->pic_init_qs_minus26 = 0;
+
+ /* Redundant pictures are not supported. */
+ pps->flags &= ~V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT;
+
+ /* Scaling matrix is not supported. */
+ pps->flags &= ~V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT;
+
+ /*
+ * RBSP always writes the optional second_chroma_qp_index_offset,
+ * which has to take the chroma_qp_index_offset value if unsupported.
+ */
+ if (!(enc->flags & V4L2_H264_ENC_FLAG_CHROMA_QP_CR_OFFSET))
+ pps->second_chroma_qp_index_offset =
+ pps->chroma_qp_index_offset;
+
+ /* Encode */
+
+ ctrl = v4l2_ctrl_find(handler, V4L2_CID_STATELESS_H264_ENCODE_PARAMS);
+ if (!ctrl)
+ return -EINVAL;
+
+ memcpy(encode, ctrl->p_cur.p_h264_encode_params, sizeof(*encode));
+
+ if (encode->slice_type == V4L2_H264_SLICE_TYPE_SI)
+ encode->slice_type = V4L2_H264_SLICE_TYPE_I;
+ else if (encode->slice_type == V4L2_H264_SLICE_TYPE_SP)
+ encode->slice_type = V4L2_H264_SLICE_TYPE_P;
+
+ /* We cannot safely reuse parameters of a B frane for a P frame. */
+ if (encode->slice_type == V4L2_H264_SLICE_TYPE_B &&
+ !(enc->flags & V4L2_H264_ENC_FLAG_INTER_BIPRED))
+ return -EINVAL;
+
+ /* We can safely reuse parameters of a P frame for an I frame. */
+ if (encode->slice_type == V4L2_H264_SLICE_TYPE_P &&
+ !(enc->flags & V4L2_H264_ENC_FLAG_INTER_PRED))
+ encode->slice_type = V4L2_H264_SLICE_TYPE_I;
+
+ if (encode->slice_type != V4L2_H264_SLICE_TYPE_I &&
+ !sps->max_num_ref_frames)
+ return -EINVAL;
+
+ /* Ensure the stream starts with an I frame. */
+ if (!enc->state_serial && encode->slice_type != V4L2_H264_SLICE_TYPE_I)
+ return -EINVAL;
+
+ /* Only single instances of PPS and SPS are supported. */
+ if (encode->pic_parameter_set_id != pps->pic_parameter_set_id)
+ encode->pic_parameter_set_id = pps->pic_parameter_set_id;
+
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_NUM_REF_IDX_ACTIVE_OVERRIDE) {
+ if (encode->num_ref_idx_l0_active_minus1 >
+ (sps->max_num_ref_frames - 1))
+ encode->num_ref_idx_l0_active_minus1 =
+ sps->max_num_ref_frames - 1;
+
+ if (encode->num_ref_idx_l1_active_minus1 >
+ (sps->max_num_ref_frames - 1))
+ encode->num_ref_idx_l1_active_minus1 =
+ sps->max_num_ref_frames - 1;
+ }
+
+ return 0;
+}
+
+static int state_prepare(struct v4l2_h264_enc *enc)
+{
+ struct v4l2_h264_enc_state *state = &enc->state_next;
+ int ret;
+
+ memset(state, 0, sizeof(*state));
+
+ ret = state_prepare_params(enc);
+ if (ret)
+ return ret;
+
+ ret = v4l2_h264_enc_op(enc, state_constrain, state);
+ if (ret && ret != -EOPNOTSUPP)
+ return ret;
+
+ return 0;
+}
+
+static void state_debug(struct v4l2_h264_enc *enc)
+{
+ struct v4l2_h264_enc_state *state = &enc->state_active;
+ struct v4l2_ctrl_h264_sps *sps = &state->sps;
+ struct v4l2_ctrl_h264_pps *pps = &state->pps;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+
+ pr_debug("+ v4l2-h264-enc: state");
+
+ pr_debug(" type: %c%s, ref: %s%s",
+ v4l2_h264_slice_type_char(encode->slice_type),
+ encode->flags & V4L2_H264_ENCODE_FLAG_IDR_PIC ? " (IDR)" : "",
+ encode->nal_ref_idc ? "marked" : "unmarked",
+ encode->flags & V4L2_H264_ENCODE_FLAG_LONG_TERM_REFERENCE ?
+ " (long-term)" : "");
+ pr_debug(" width: %u mbs, height: %u mbs",
+ sps->pic_width_in_mbs_minus1,
+ sps->pic_height_in_map_units_minus1);
+ pr_debug(" profile: %u, level: %u", sps->profile_idc,
+ sps->level_idc);
+
+ pr_debug(" entropy coding: %s",
+ pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE ? "cabac" :
+ "cavlc");
+
+ if (pps->flags & (V4L2_H264_PPS_FLAG_WEIGHTED_PRED |
+ V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED |
+ V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE |
+ V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT) ||
+ encode->flags & V4L2_H264_ENCODE_FLAG_DIRECT_SPATIAL_MV_PRED)
+ pr_debug(" coding features:");
+
+ if (pps->flags & V4L2_H264_PPS_FLAG_WEIGHTED_PRED)
+ pr_debug(" - weighted-pred");
+ if (pps->flags & V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED)
+ pr_debug(" - constrained-intra-pred");
+ if (pps->flags & V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE)
+ pr_debug(" - transform-8x8");
+ if (pps->flags & V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT)
+ pr_debug(" - scaling-matrix");
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_DIRECT_SPATIAL_MV_PRED)
+ pr_debug(" - direct-spatial-mv-pred");
+}
+
+static int state_commit(struct v4l2_h264_enc *enc)
+{
+ struct v4l2_h264_enc_state *state = &enc->state_next;
+ struct v4l2_ctrl_handler *handler = enc->ctrl_handler;
+ struct v4l2_ctrl *ctrl;
+ int ret;
+
+ /* The presence of required controls was checked already. */
+ /* TODO: Attach to media request. */
+
+ /* SPS */
+
+ ctrl = v4l2_ctrl_find(handler, V4L2_CID_STATELESS_H264_SPS);
+ ret = v4l2_ctrl_s_ctrl_compound(ctrl, V4L2_CTRL_TYPE_H264_SPS,
+ &state->sps);
+ if (ret)
+ return ret;
+
+ /* PPS */
+
+ ctrl = v4l2_ctrl_find(handler, V4L2_CID_STATELESS_H264_PPS);
+ ret = v4l2_ctrl_s_ctrl_compound(ctrl, V4L2_CTRL_TYPE_H264_PPS,
+ &state->pps);
+ if (ret)
+ return ret;
+
+ /* Encode */
+
+ ctrl = v4l2_ctrl_find(handler, V4L2_CID_STATELESS_H264_ENCODE_PARAMS);
+ ret = v4l2_ctrl_s_ctrl_compound(ctrl, V4L2_CTRL_TYPE_H264_ENCODE_PARAMS,
+ &state->encode);
+ if (ret)
+ return ret;
+
+ /* State */
+
+ memcpy(&enc->state_active, &enc->state_next, sizeof(enc->state_active));
+
+ state_debug(enc);
+
+ return 0;
+}
+
+static int state_complete(struct v4l2_h264_enc *enc,
+ struct vb2_v4l2_buffer *buffer)
+{
+
+ struct v4l2_h264_enc_state *state = &enc->state_active;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+
+ if (encode->slice_type == V4L2_H264_SLICE_TYPE_I)
+ buffer->flags |= V4L2_BUF_FLAG_KEYFRAME;
+ else if (encode->slice_type == V4L2_H264_SLICE_TYPE_P)
+ buffer->flags |= V4L2_BUF_FLAG_PFRAME;
+ else if (encode->slice_type == V4L2_H264_SLICE_TYPE_B)
+ buffer->flags |= V4L2_BUF_FLAG_BFRAME;
+
+ /*
+ * This is only incremented after a successful encode so we can still
+ * detect a stream start case after the first frame failed to encode.
+ */
+ enc->state_serial++;
+
+ return 0;
+}
+
+int v4l2_h264_enc_step(struct v4l2_h264_enc *enc,
+ struct vb2_v4l2_buffer *buffer)
+{
+ int ret;
+
+ ret = state_prepare(enc);
+ if (ret)
+ return ret;
+
+ ret = state_commit(enc);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_step);
+
+int v4l2_h264_enc_complete(struct v4l2_h264_enc *enc,
+ struct vb2_v4l2_buffer *buffer)
+{
+ int ret;
+
+ ret = state_complete(enc, buffer);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_enc_complete);
+
+MODULE_DESCRIPTION("V4L2 H.264 Encode Core");
+MODULE_AUTHOR("Paul Kocialkowski <paulk@sys-base.io>");
+MODULE_LICENSE("GPL");
diff --git a/include/media/v4l2-h264-enc.h b/include/media/v4l2-h264-enc.h
new file mode 100644
index 000000000000..2978a73baacd
--- /dev/null
+++ b/include/media/v4l2-h264-enc.h
@@ -0,0 +1,79 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * V4L2 H.264 Encode Core
+ *
+ * Copyright (C) 2025-2026 Paul Kocialkowski <paulk@sys-base.io>
+ */
+
+#ifndef _MEDIA_V4L2_H264_ENC_H
+#define _MEDIA_V4L2_H264_ENC_H
+
+#include <linux/v4l2-controls.h>
+#include <linux/videodev2.h>
+#include <media/videobuf2-v4l2.h>
+
+#define V4L2_H264_ENC_MB_UNIT 16
+
+#define V4L2_H264_ENC_FLAG_INTER_PRED 0x1
+#define V4L2_H264_ENC_FLAG_INTER_BIPRED 0x2
+#define V4L2_H264_ENC_FLAG_HW_AUD 0x4
+#define V4L2_H264_ENC_FLAG_HW_SPS 0x8
+#define V4L2_H264_ENC_FLAG_HW_PPS 0x10
+#define V4L2_H264_ENC_FLAG_HW_SLICE_HEADER 0x20
+#define V4L2_H264_ENC_FLAG_CHROMA_QP_CR_OFFSET 0x40
+
+#define v4l2_h264_enc_op(e, o, a...) \
+ ({ \
+ int ret; \
+ if ((e)->ops && (e)->ops->o) \
+ ret = (e)->ops->o(e, ##a); \
+ else \
+ ret = -EOPNOTSUPP; \
+ ret; \
+ })
+
+struct v4l2_h264_enc;
+
+struct v4l2_h264_enc_state {
+ struct v4l2_ctrl_h264_sps sps;
+ struct v4l2_h264_sps_video sps_video;
+ struct v4l2_ctrl_h264_pps pps;
+ struct v4l2_ctrl_h264_encode_params encode;
+
+ struct v4l2_fract timeperframe;
+
+ unsigned int width_mbs;
+ unsigned int width_aligned;
+ unsigned int height_mbs;
+ unsigned int height_aligned;
+};
+
+struct v4l2_h264_enc_ops {
+ int (*state_constrain)(struct v4l2_h264_enc *enc,
+ struct v4l2_h264_enc_state *state);
+};
+
+struct v4l2_h264_enc {
+ const struct v4l2_h264_enc_ops *ops;
+ void *private_data;
+
+ struct v4l2_pix_format *format;
+ struct v4l2_pix_format_mplane *format_mplane;
+ struct v4l2_fract *timeperframe;
+ struct v4l2_ctrl_handler *ctrl_handler;
+
+ struct v4l2_h264_enc_state state_active;
+ struct v4l2_h264_enc_state state_next;
+ unsigned int state_serial;
+
+ unsigned int flags;
+};
+
+int v4l2_h264_enc_init(struct v4l2_h264_enc *enc);
+void v4l2_h264_enc_exit(struct v4l2_h264_enc *enc);
+int v4l2_h264_enc_step(struct v4l2_h264_enc *enc,
+ struct vb2_v4l2_buffer *buffer);
+int v4l2_h264_enc_complete(struct v4l2_h264_enc *enc,
+ struct vb2_v4l2_buffer *buffer);
+
+#endif
--
2.53.0
^ permalink raw reply related
* [PATCH 03/14] media: h264: Add SPS video definitions
From: Paul Kocialkowski @ 2026-05-22 10:16 UTC (permalink / raw)
To: devicetree, imx, linux-arm-kernel, linux-kernel, linux-media
Cc: Rob Herring, Krzysztof Kozlowski, Conor Dooley, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Nicolas Dufresne,
Benjamin Gaignard, Philipp Zabel, Mauro Carvalho Chehab,
Hans Verkuil, Marco Felsch, Michael Tretter, Paul Kocialkowski
In-Reply-To: <20260522101653.2565125-1-paulk@sys-base.io>
These definitions are used to build the video part of the SPS,
including important fields such as frame cropping, timing info
(frame rate) and pixel coding.
Signed-off-by: Paul Kocialkowski <paulk@sys-base.io>
---
include/media/v4l2-h264.h | 115 ++++++++++++++++++++++++++++++++++++++
1 file changed, 115 insertions(+)
diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
index 1833c0556963..3b00a1b67fe5 100644
--- a/include/media/v4l2-h264.h
+++ b/include/media/v4l2-h264.h
@@ -12,6 +12,121 @@
#include <media/v4l2-ctrls.h>
+#define V4L2_H264_START_CODE_ANNEX_B 0x00000001
+
+#define V4L2_H264_NALU_TYPE_SLICE_NON_IDR 1
+#define V4L2_H264_NALU_TYPE_SLICE_IDR 5
+#define V4L2_H264_NALU_TYPE_SPS 7
+#define V4L2_H264_NALU_TYPE_PPS 8
+#define V4L2_H264_NALU_TYPE_AUD 9
+
+#define V4L2_H264_PRIMARY_PIC_TYPE_I 0
+#define V4L2_H264_PRIMARY_PIC_TYPE_IP 1
+#define V4L2_H264_PRIMARY_PIC_TYPE_IPB 2
+
+#define V4L2_H264_VUI_ASPECT_RATIO_IDC_EXTENDED 255
+
+#define V4L2_H264_VUI_VIDEO_COMPONENT 0
+#define V4L2_H264_VUI_VIDEO_PAL 1
+#define V4L2_H264_VUI_VIDEO_NTSC 2
+#define V4L2_H264_VUI_VIDEO_SECAM 3
+#define V4L2_H264_VUI_VIDEO_MAC 4
+#define V4L2_H264_VUI_VIDEO_UNSPECIFIED 5
+
+#define V4L2_H264_VUI_COLOUR_BT709 1
+#define V4L2_H264_VUI_COLOUR_UNSPECIFIED 2
+#define V4L2_H264_VUI_COLOUR_BT470_SYSTEM_M 4
+#define V4L2_H264_VUI_COLOUR_BT470_SYSTEM_BG 5
+#define V4L2_H264_VUI_COLOUR_SMPTE170M 6
+#define V4L2_H264_VUI_COLOUR_SMPTE240M 7
+#define V4L2_H264_VUI_COLOUR_BT2020 9
+
+#define V4L2_H264_VUI_TRANSFER_BT709 1
+#define V4L2_H264_VUI_TRANSFER_UNSPECIFIED 2
+#define V4L2_H264_VUI_TRANSFER_BT470_SYSTEM_M 4
+#define V4L2_H264_VUI_TRANSFER_BT470_SYSTEM_BG 5
+#define V4L2_H264_VUI_TRANSFER_SMPTE170M 6
+#define V4L2_H264_VUI_TRANSFER_SMPTE240M 7
+#define V4L2_H264_VUI_TRANSFER_LINEAR 8
+#define V4L2_H264_VUI_TRANSFER_SRGB 13
+
+#define V4L2_H264_VUI_MATRIX_IDENTITY 0
+#define V4L2_H264_VUI_MATRIX_BT709 1
+#define V4L2_H264_VUI_MATRIX_UNSPECIFIED 2
+#define V4L2_H264_VUI_MATRIX_BT470_SYSTEM_M 4
+#define V4L2_H264_VUI_MATRIX_BT470_SYSTEM_BG 5
+#define V4L2_H264_VUI_MATRIX_SMPTE170M 6
+#define V4L2_H264_VUI_MATRIX_SMPTE240M 7
+#define V4L2_H264_VUI_MATRIX_BT2020 9
+#define V4L2_H264_VUI_MATRIX_BT2020_CONST_LUM 10
+
+#define V4L2_H264_SPS_VIDEO_FLAG_FRAME_CROPPING BIT(0)
+#define V4L2_H264_SPS_VIDEO_FLAG_VUI_PARAMETERS_PRESENT BIT(1)
+#define V4L2_H264_SPS_VIDEO_FLAG_VUI_ASPECT_RATIO_INFO_PRESENT BIT(2)
+#define V4L2_H264_SPS_VIDEO_FLAG_VUI_OVERSCAN_INFO_PRESENT BIT(3)
+#define V4L2_H264_SPS_VIDEO_FLAG_VUI_OVERSCAN_APPROPRIATE BIT(4)
+#define V4L2_H264_SPS_VIDEO_FLAG_VUI_VIDEO_SIGNAL_TYPE_PRESENT BIT(5)
+#define V4L2_H264_SPS_VIDEO_FLAG_VUI_VIDEO_FULL_RANGE BIT(6)
+#define V4L2_H264_SPS_VIDEO_FLAG_VUI_COLOUR_DESCRIPTION_PRESENT BIT(7)
+#define V4L2_H264_SPS_VIDEO_FLAG_VUI_CHROMA_LOC_INFO_PRESENT BIT(8)
+#define V4L2_H264_SPS_VIDEO_FLAG_VUI_TIMING_INFO_PRESENT BIT(9)
+#define V4L2_H264_SPS_VIDEO_FLAG_VUI_FIXED_FRAME_RATE BIT(10)
+#define V4L2_H264_SPS_VIDEO_FLAG_VUI_NAL_HRD_PARAMETERS_PRESENT BIT(11)
+#define V4L2_H264_SPS_VIDEO_FLAG_VUI_VCL_HRD_PARAMETERS_PRESENT BIT(12)
+#define V4L2_H264_SPS_VIDEO_FLAG_VUI_LOW_DELAY_HRD BIT(13)
+#define V4L2_H264_SPS_VIDEO_FLAG_VUI_PIC_STRUCT_PRESENT BIT(14)
+#define V4L2_H264_SPS_VIDEO_FLAG_VUI_BITSTREAM_RESTRICTION BIT(15)
+#define V4L2_H264_SPS_VIDEO_FLAG_VUI_MOTION_VECTORS_OVER_PIC_BOUNDARIES BIT(16)
+
+struct v4l2_h264_sps_video_hrd {
+ u8 cpb_cnt_minus1;
+ u8 bit_rate_scale;
+ u8 cpb_size_scale;
+
+ u32 bit_rate_value_minus1[32];
+ u32 cpb_size_value_minus1[32];
+ u8 cbr_flag[32];
+
+ u8 initial_cpb_removal_delay_length_minus1;
+ u8 cpb_removal_delay_length_minus1;
+ u8 dpb_output_delay_length_minus1;
+ u8 time_offset_length;
+};
+
+struct v4l2_h264_sps_video {
+ u32 frame_crop_left_offset;
+ u32 frame_crop_right_offset;
+ u32 frame_crop_top_offset;
+ u32 frame_crop_bottom_offset;
+
+ u8 aspect_ratio_idc;
+ u16 sar_width;
+ u16 sar_height;
+
+ u8 video_format;
+ u8 colour_primaries;
+ u8 transfer_characteristics;
+ u8 matrix_coefficients;
+
+ u8 chroma_sample_loc_type_top_field;
+ u8 chroma_sample_loc_type_bottom_field;
+
+ u32 num_units_in_tick;
+ u32 time_scale;
+
+ struct v4l2_h264_sps_video_hrd nal_hrd;
+ struct v4l2_h264_sps_video_hrd vcl_hrd;
+
+ u32 max_bytes_per_pic_denom;
+ u32 max_bits_per_mb_denom;
+ u32 log2_max_mv_length_horizontal;
+ u32 log2_max_mv_length_vertical;
+ u32 max_num_reorder_frames;
+ u32 max_dec_frame_buffering;
+
+ u32 flags;
+};
+
/**
* struct v4l2_h264_reflist_builder - Reference list builder object
*
--
2.53.0
^ permalink raw reply related
* [PATCH 02/14] media: uapi: Add H.264 stateless encode support
From: Paul Kocialkowski @ 2026-05-22 10:16 UTC (permalink / raw)
To: devicetree, imx, linux-arm-kernel, linux-kernel, linux-media
Cc: Rob Herring, Krzysztof Kozlowski, Conor Dooley, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Nicolas Dufresne,
Benjamin Gaignard, Philipp Zabel, Mauro Carvalho Chehab,
Hans Verkuil, Marco Felsch, Michael Tretter, Paul Kocialkowski
In-Reply-To: <20260522101653.2565125-1-paulk@sys-base.io>
This introduces the stateless H.264 encode params control that allows
configuring an H.264 encode run.
Note that the current uAPI does not support explicit passing of
references and works following the sliding window decoded reference
picture marking process. This may change in the future.
Signed-off-by: Paul Kocialkowski <paulk@sys-base.io>
---
drivers/media/v4l2-core/v4l2-ctrls-core.c | 62 +++++++++++++++++++++++
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 4 ++
include/media/v4l2-ctrls.h | 2 +
include/uapi/linux/v4l2-controls.h | 33 ++++++++++++
include/uapi/linux/videodev2.h | 1 +
5 files changed, 102 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
index 85d07ef44f62..48217811b0f1 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -382,6 +382,9 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl)
case V4L2_CTRL_TYPE_H264_PRED_WEIGHTS:
pr_cont("H264_PRED_WEIGHTS");
break;
+ case V4L2_CTRL_TYPE_H264_ENCODE_PARAMS:
+ pr_cont("H264_ENCODE_PARAMS");
+ break;
case V4L2_CTRL_TYPE_FWHT_PARAMS:
pr_cont("FWHT_PARAMS");
break;
@@ -880,6 +883,7 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
struct v4l2_ctrl_h264_pred_weights *p_h264_pred_weights;
struct v4l2_ctrl_h264_slice_params *p_h264_slice_params;
struct v4l2_ctrl_h264_decode_params *p_h264_dec_params;
+ struct v4l2_ctrl_h264_encode_params *p_h264_enc_params;
struct v4l2_ctrl_hevc_sps *p_hevc_sps;
struct v4l2_ctrl_hevc_pps *p_hevc_pps;
struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering;
@@ -1102,6 +1106,61 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
zero_reserved(*p_h264_dec_params);
break;
+ case V4L2_CTRL_TYPE_H264_ENCODE_PARAMS:
+ p_h264_enc_params = p;
+
+ if (p_h264_enc_params->slice_type != V4L2_H264_SLICE_TYPE_B)
+ p_h264_enc_params->flags &=
+ ~V4L2_H264_ENCODE_FLAG_DIRECT_SPATIAL_MV_PRED;
+ if (!(p_h264_enc_params->flags & V4L2_H264_ENCODE_FLAG_IDR_PIC) ||
+ !p_h264_enc_params->nal_ref_idc)
+ p_h264_enc_params->flags &=
+ ~V4L2_H264_ENCODE_FLAG_LONG_TERM_REFERENCE;
+
+ if (p_h264_enc_params->flags & V4L2_H264_ENCODE_FLAG_IDR_PIC &&
+ p_h264_enc_params->slice_type != V4L2_H264_SLICE_TYPE_I)
+ return -EINVAL;
+ if (p_h264_enc_params->colour_plane_id > 2)
+ return -EINVAL;
+ if (p_h264_enc_params->cabac_init_idc > 2)
+ return -EINVAL;
+ if (p_h264_enc_params->disable_deblocking_filter_idc > 2)
+ return -EINVAL;
+ if (p_h264_enc_params->slice_alpha_c0_offset_div2 < -6 ||
+ p_h264_enc_params->slice_alpha_c0_offset_div2 > 6)
+ return -EINVAL;
+ if (p_h264_enc_params->slice_beta_offset_div2 < -6 ||
+ p_h264_enc_params->slice_beta_offset_div2 > 6)
+ return -EINVAL;
+
+ if (p_h264_enc_params->slice_type == V4L2_H264_SLICE_TYPE_I ||
+ p_h264_enc_params->slice_type == V4L2_H264_SLICE_TYPE_SI)
+ p_h264_enc_params->num_ref_idx_l0_active_minus1 = 0;
+ if (p_h264_enc_params->slice_type != V4L2_H264_SLICE_TYPE_B)
+ p_h264_enc_params->num_ref_idx_l1_active_minus1 = 0;
+
+ if (p_h264_enc_params->flags & V4L2_H264_ENCODE_FLAG_IDR_PIC) {
+ p_h264_enc_params->frame_num = 0;
+ p_h264_enc_params->pic_order_cnt_lsb = 0;
+ p_h264_enc_params->delta_pic_order_cnt_bottom = 0;
+ p_h264_enc_params->delta_pic_order_cnt0 = 0;
+ p_h264_enc_params->delta_pic_order_cnt1 = 0;
+ } else {
+ p_h264_enc_params->idr_pic_id = 0;
+ }
+
+ if (p_h264_enc_params->num_ref_idx_l0_active_minus1 >
+ (V4L2_H264_REF_LIST_LEN - 1))
+ return -EINVAL;
+ if (p_h264_enc_params->num_ref_idx_l1_active_minus1 >
+ (V4L2_H264_REF_LIST_LEN - 1))
+ return -EINVAL;
+ memset(&p_h264_enc_params->reserved0, 0,
+ sizeof(p_h264_enc_params->reserved0));
+ memset(&p_h264_enc_params->reserved1, 0,
+ sizeof(p_h264_enc_params->reserved1));
+ break;
+
case V4L2_CTRL_TYPE_VP8_FRAME:
p_vp8_frame = p;
@@ -1913,6 +1972,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
case V4L2_CTRL_TYPE_H264_PRED_WEIGHTS:
elem_size = sizeof(struct v4l2_ctrl_h264_pred_weights);
break;
+ case V4L2_CTRL_TYPE_H264_ENCODE_PARAMS:
+ elem_size = sizeof(struct v4l2_ctrl_h264_encode_params);
+ break;
case V4L2_CTRL_TYPE_VP8_FRAME:
elem_size = sizeof(struct v4l2_ctrl_vp8_frame);
break;
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index ad41f65374e2..4fa6dc7a3b00 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1216,6 +1216,7 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_STATELESS_H264_PPS: return "H264 Picture Parameter Set";
case V4L2_CID_STATELESS_H264_SCALING_MATRIX: return "H264 Scaling Matrix";
case V4L2_CID_STATELESS_H264_PRED_WEIGHTS: return "H264 Prediction Weight Table";
+ case V4L2_CID_STATELESS_H264_ENCODE_PARAMS: return "H264 Encode Parameters";
case V4L2_CID_STATELESS_H264_SLICE_PARAMS: return "H264 Slice Parameters";
case V4L2_CID_STATELESS_H264_DECODE_PARAMS: return "H264 Decode Parameters";
case V4L2_CID_STATELESS_FWHT_PARAMS: return "FWHT Stateless Parameters";
@@ -1555,6 +1556,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
case V4L2_CID_STATELESS_H264_PRED_WEIGHTS:
*type = V4L2_CTRL_TYPE_H264_PRED_WEIGHTS;
break;
+ case V4L2_CID_STATELESS_H264_ENCODE_PARAMS:
+ *type = V4L2_CTRL_TYPE_H264_ENCODE_PARAMS;
+ break;
case V4L2_CID_STATELESS_VP8_FRAME:
*type = V4L2_CTRL_TYPE_VP8_FRAME;
break;
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index 31fc1bee3797..54133cda7bc7 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -42,6 +42,7 @@ struct video_device;
* @p_h264_scaling_matrix: Pointer to a struct v4l2_ctrl_h264_scaling_matrix.
* @p_h264_slice_params: Pointer to a struct v4l2_ctrl_h264_slice_params.
* @p_h264_decode_params: Pointer to a struct v4l2_ctrl_h264_decode_params.
+ * @p_h264_encode_params: Pointer to a struct v4l2_ctrl_h264_encode_params.
* @p_h264_pred_weights: Pointer to a struct v4l2_ctrl_h264_pred_weights.
* @p_vp8_frame: Pointer to a VP8 frame params structure.
* @p_vp9_compressed_hdr_probs: Pointer to a VP9 frame compressed header probs structure.
@@ -76,6 +77,7 @@ union v4l2_ctrl_ptr {
struct v4l2_ctrl_h264_scaling_matrix *p_h264_scaling_matrix;
struct v4l2_ctrl_h264_slice_params *p_h264_slice_params;
struct v4l2_ctrl_h264_decode_params *p_h264_decode_params;
+ struct v4l2_ctrl_h264_encode_params *p_h264_encode_params;
struct v4l2_ctrl_h264_pred_weights *p_h264_pred_weights;
struct v4l2_ctrl_vp8_frame *p_vp8_frame;
struct v4l2_ctrl_hevc_sps *p_hevc_sps;
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 2d30107e047e..9add059ca02a 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -1568,6 +1568,7 @@ struct v4l2_h264_reference {
#define V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED 0x01
#define V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH 0x02
+#define V4L2_H264_SLICE_FLAG_NUM_REF_IDX_ACTIVE_OVERRIDE 0x04
#define V4L2_CID_STATELESS_H264_SLICE_PARAMS (V4L2_CID_CODEC_STATELESS_BASE + 6)
/**
@@ -1707,6 +1708,38 @@ struct v4l2_ctrl_h264_decode_params {
__u32 flags;
};
+#define V4L2_H264_ENCODE_FLAG_IDR_PIC 0x01
+#define V4L2_H264_ENCODE_FLAG_FIELD_PIC 0x02
+#define V4L2_H264_ENCODE_FLAG_BOTTOM_FIELD 0x04
+#define V4L2_H264_ENCODE_FLAG_DIRECT_SPATIAL_MV_PRED 0x08
+#define V4L2_H264_ENCODE_FLAG_NUM_REF_IDX_ACTIVE_OVERRIDE 0x10
+#define V4L2_H264_ENCODE_FLAG_NO_OUTPUT_OF_PRIOR_PICS 0x20
+#define V4L2_H264_ENCODE_FLAG_LONG_TERM_REFERENCE 0x40
+
+#define V4L2_CID_STATELESS_H264_ENCODE_PARAMS (V4L2_CID_CODEC_STATELESS_BASE + 8)
+struct v4l2_ctrl_h264_encode_params {
+ __u8 nal_ref_idc;
+ __u8 slice_type;
+ __u8 pic_parameter_set_id;
+ __u8 colour_plane_id;
+ __u16 frame_num;
+ __u16 idr_pic_id;
+ __u16 pic_order_cnt_lsb;
+ __u8 reserved0[2];
+ __s32 delta_pic_order_cnt_bottom;
+ __s32 delta_pic_order_cnt0;
+ __s32 delta_pic_order_cnt1;
+ __u8 num_ref_idx_l0_active_minus1;
+ __u8 num_ref_idx_l1_active_minus1;
+ __u8 cabac_init_idc;
+ __s8 slice_qp_delta;
+ __u8 disable_deblocking_filter_idc;
+ __s8 slice_alpha_c0_offset_div2;
+ __s8 slice_beta_offset_div2;
+ __u8 reserved1[6];
+ __u32 flags; /* V4L2_H264_ENCODE_FLAG_ */
+};
+
/* Stateless FWHT control, used by the vicodec driver */
/* Current FWHT version */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index becd08fdbddb..1d5e27b65544 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1964,6 +1964,7 @@ enum v4l2_ctrl_type {
V4L2_CTRL_TYPE_H264_SLICE_PARAMS = 0x0203,
V4L2_CTRL_TYPE_H264_DECODE_PARAMS = 0x0204,
V4L2_CTRL_TYPE_H264_PRED_WEIGHTS = 0x0205,
+ V4L2_CTRL_TYPE_H264_ENCODE_PARAMS = 0x0206,
V4L2_CTRL_TYPE_FWHT_PARAMS = 0x0220,
--
2.53.0
^ permalink raw reply related
* [PATCH 01/14] media: h264: Add a more generic reflist builder init
From: Paul Kocialkowski @ 2026-05-22 10:16 UTC (permalink / raw)
To: devicetree, imx, linux-arm-kernel, linux-kernel, linux-media
Cc: Rob Herring, Krzysztof Kozlowski, Conor Dooley, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Nicolas Dufresne,
Benjamin Gaignard, Philipp Zabel, Mauro Carvalho Chehab,
Hans Verkuil, Marco Felsch, Michael Tretter, Paul Kocialkowski
In-Reply-To: <20260522101653.2565125-1-paulk@sys-base.io>
This should become part of a larger reflist builder rework.
Signed-off-by: Paul Kocialkowski <paulk@sys-base.io>
---
drivers/media/v4l2-core/v4l2-h264.c | 69 +++++++++++++++++++++++++++++
include/media/v4l2-h264.h | 7 +++
2 files changed, 76 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
index c00197d095e7..4e5a177ca2c7 100644
--- a/drivers/media/v4l2-core/v4l2-h264.c
+++ b/drivers/media/v4l2-core/v4l2-h264.c
@@ -105,6 +105,75 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
}
EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
+void
+v4l2_h264_init_reflist_builder_gen(struct v4l2_h264_reflist_builder *b,
+ const struct v4l2_ctrl_h264_sps *sps,
+ const struct v4l2_h264_dpb_entry dpb[V4L2_H264_NUM_DPB_ENTRIES],
+ unsigned int pic_order_count, unsigned int frame_num, unsigned int fields)
+{
+ int cur_frame_num, max_frame_num;
+ unsigned int i;
+
+ max_frame_num = 1 << (sps->log2_max_frame_num_minus4 + 4);
+ cur_frame_num = frame_num;
+
+ memset(b, 0, sizeof(*b));
+ b->cur_pic_order_count = pic_order_count;
+ b->cur_pic_fields = fields;
+
+ for (i = 0; i < V4L2_H264_NUM_DPB_ENTRIES; i++) {
+ if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
+ continue;
+
+ if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)
+ b->refs[i].longterm = true;
+
+ /*
+ * Handle frame_num wraparound as described in section
+ * '8.2.4.1 Decoding process for picture numbers' of the spec.
+ * For long term references, frame_num is set to
+ * long_term_frame_idx which requires no wrapping.
+ */
+ if (!b->refs[i].longterm && dpb[i].frame_num > cur_frame_num)
+ b->refs[i].frame_num = (int)dpb[i].frame_num -
+ max_frame_num;
+ else
+ b->refs[i].frame_num = dpb[i].frame_num;
+
+ b->refs[i].top_field_order_cnt = dpb[i].top_field_order_cnt;
+ b->refs[i].bottom_field_order_cnt = dpb[i].bottom_field_order_cnt;
+
+ if (b->cur_pic_fields == V4L2_H264_FRAME_REF) {
+ u8 fields = V4L2_H264_FRAME_REF;
+
+ b->unordered_reflist[b->num_valid].index = i;
+ b->unordered_reflist[b->num_valid].fields = fields;
+ b->num_valid++;
+ continue;
+ }
+
+ if (dpb[i].fields & V4L2_H264_TOP_FIELD_REF) {
+ u8 fields = V4L2_H264_TOP_FIELD_REF;
+
+ b->unordered_reflist[b->num_valid].index = i;
+ b->unordered_reflist[b->num_valid].fields = fields;
+ b->num_valid++;
+ }
+
+ if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF) {
+ u8 fields = V4L2_H264_BOTTOM_FIELD_REF;
+
+ b->unordered_reflist[b->num_valid].index = i;
+ b->unordered_reflist[b->num_valid].fields = fields;
+ b->num_valid++;
+ }
+ }
+
+ for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
+ b->unordered_reflist[i].index = i;
+}
+EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder_gen);
+
static s32 v4l2_h264_get_poc(const struct v4l2_h264_reflist_builder *b,
const struct v4l2_h264_reference *ref)
{
diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
index 0d9eaa956123..1833c0556963 100644
--- a/include/media/v4l2-h264.h
+++ b/include/media/v4l2-h264.h
@@ -42,6 +42,7 @@ struct v4l2_h264_reflist_builder {
u8 cur_pic_fields;
struct v4l2_h264_reference unordered_reflist[V4L2_H264_REF_LIST_LEN];
+ /* FIXME: confusing with valid flag (active is checked). Proper terminology is "used" */
u8 num_valid;
};
@@ -51,6 +52,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
const struct v4l2_ctrl_h264_sps *sps,
const struct v4l2_h264_dpb_entry dpb[V4L2_H264_NUM_DPB_ENTRIES]);
+void
+v4l2_h264_init_reflist_builder_gen(struct v4l2_h264_reflist_builder *b,
+ const struct v4l2_ctrl_h264_sps *sps,
+ const struct v4l2_h264_dpb_entry dpb[V4L2_H264_NUM_DPB_ENTRIES],
+ unsigned int pic_order_count, unsigned int frame_num, unsigned int fields);
+
/**
* v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
*
--
2.53.0
^ permalink raw reply related
* [PATCH 00/14] media: Add V4L2 H.264 stateless encode and VC8000E support
From: Paul Kocialkowski @ 2026-05-22 10:16 UTC (permalink / raw)
To: devicetree, imx, linux-arm-kernel, linux-kernel, linux-media
Cc: Rob Herring, Krzysztof Kozlowski, Conor Dooley, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Nicolas Dufresne,
Benjamin Gaignard, Philipp Zabel, Mauro Carvalho Chehab,
Hans Verkuil, Marco Felsch, Michael Tretter, Paul Kocialkowski
This series introduces support for the V4L2 H.264 stateless encode uAPI,
core and support in the hantro driver for the Verisilicon VC8000E.
While this is a first version that will likely need some level of rework,
it is already usable for most common use-cases and supports constant
bitrate rate-control.
A GStreamer tree can be used to test the series at:
https://github.com/paulkocialkowski/gstreamer/tree/v4l2codecs/h264enc
And an example pipeline would look like:
gst-launch-1.0 videotestsrc pattern=smpte num-buffers=25 ! video/x-raw,width=640,height=480 ! v4l2slh264enc rate-control=cbr bitrate=8000000 qp-min=8 qp-max=42 ! h264parse ! matroskamux ! filesink location=encode.mkv
Note that documentation for the new uAPI is intentionally left out of
this series since it has not yet received approval.
Marco Felsch (2):
media: hantro: use hantro_decoded_buffer only for dst_vq
arm64: dts: imx8mp: add VC8000E encoder node
Paul Kocialkowski (12):
media: h264: Add a more generic reflist builder init
media: uapi: Add H.264 stateless encode support
media: h264: Add SPS video definitions
media: h264: Add stateless encode core
media: h264: Add stateless encode rbsp
media: h264: Add stateless encode reference management
media: h264: Add stateless encode rate control
media: verisilicon: Report default pixel coding for non-JPEG and fix
JPEG case
media: verisilicon: Cancel job with runtime pm put/clk disable on
failure
media: verisilicon: Add common encoder parm and frameintervals ioctls
media: verisilicon: Add support for the VC8000E H.264 encoder
media: verilisicon: imx8m: Add support for the VC8000E on i.MX8MP
arch/arm64/boot/dts/freescale/imx8mp.dtsi | 11 +
drivers/media/platform/verisilicon/Kconfig | 1 +
drivers/media/platform/verisilicon/Makefile | 2 +
drivers/media/platform/verisilicon/hantro.h | 17 +
.../media/platform/verisilicon/hantro_drv.c | 180 +-
.../media/platform/verisilicon/hantro_h264.c | 6 +-
.../media/platform/verisilicon/hantro_hw.h | 28 +
.../media/platform/verisilicon/hantro_v4l2.c | 123 +-
.../platform/verisilicon/hantro_vc8000e.c | 68 +
.../verisilicon/hantro_vc8000e_h264_enc.c | 883 +++++++
.../verisilicon/hantro_vc8000e_regs.h | 2129 +++++++++++++++++
.../media/platform/verisilicon/imx8m_vpu_hw.c | 113 +
drivers/media/v4l2-core/Kconfig | 4 +
drivers/media/v4l2-core/Makefile | 2 +
drivers/media/v4l2-core/v4l2-ctrls-core.c | 62 +
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 4 +
drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c | 1173 +++++++++
drivers/media/v4l2-core/v4l2-h264-enc-rc.c | 558 +++++
drivers/media/v4l2-core/v4l2-h264-enc.c | 1322 ++++++++++
drivers/media/v4l2-core/v4l2-h264.c | 69 +
include/media/v4l2-ctrls.h | 2 +
include/media/v4l2-h264-enc-rbsp.h | 72 +
include/media/v4l2-h264-enc-rc.h | 108 +
include/media/v4l2-h264-enc.h | 135 ++
include/media/v4l2-h264.h | 146 ++
include/uapi/linux/v4l2-controls.h | 33 +
include/uapi/linux/videodev2.h | 1 +
27 files changed, 7231 insertions(+), 21 deletions(-)
create mode 100644 drivers/media/platform/verisilicon/hantro_vc8000e.c
create mode 100644 drivers/media/platform/verisilicon/hantro_vc8000e_h264_enc.c
create mode 100644 drivers/media/platform/verisilicon/hantro_vc8000e_regs.h
create mode 100644 drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c
create mode 100644 drivers/media/v4l2-core/v4l2-h264-enc-rc.c
create mode 100644 drivers/media/v4l2-core/v4l2-h264-enc.c
create mode 100644 include/media/v4l2-h264-enc-rbsp.h
create mode 100644 include/media/v4l2-h264-enc-rc.h
create mode 100644 include/media/v4l2-h264-enc.h
--
2.53.0
^ permalink raw reply
* Re: [PATCH] arm64: tlb: Flush walk cache when unsharing PMD tables
From: Catalin Marinas @ 2026-05-22 10:13 UTC (permalink / raw)
To: Zeng Heng
Cc: yezhenyu2, zhurui3, will, akpm, npiggin, aneesh.kumar, peterz,
linux-kernel, wangkefeng.wang, linux-arm-kernel, linux-mm,
linux-arch, David Hildenbrand, zengheng4
In-Reply-To: <b7098272-a348-d918-774a-d5d2645fc7f4@huaweicloud.com>
On Fri, May 22, 2026 at 01:32:07PM +0800, Zeng Heng wrote:
> On 2026/5/21 23:15, Catalin Marinas wrote:
> > On Thu, May 21, 2026 at 04:05:07PM +0100, Catalin Marinas wrote:
> > > On Thu, May 21, 2026 at 03:30:11PM +0800, Zeng Heng wrote:
> > > > From: Zeng Heng <zengheng4@huawei.com>
> > > >
> > > > When huge_pmd_unshare() is called to unshare a PMD table, the
> > > > tlb_unshare_pmd_ptdesc() function sets tlb->unshared_tables=true
> > > > but the aarch64 tlb_flush() only checked tlb->freed_tables to
> > > > determine whether to use TLBF_NONE (vae1is, invalidates walk
> > > > cache) or TLBF_NOWALKCACHE (vale1is, leaf-only).
> > > >
> > > > This caused the stale PMD page table entry to remain in the walk cache
> > > > after unshare, potentially leading to incorrect page table walks.
> > > >
> > > > Fix by including unshared_tables in the check, so that when
> > > > unsharing tables, TLBF_NONE is used and the walk cache is properly
> > > > invalidated.
> > > >
> > > > Here is the detailed distinction between vae1is and vale1is:
> > > >
> > > > | Instruction Combination | Actual Invalidation Scope |
> > > > | ------------------------ | --------------------------------------------------|
> > > > | `VAE1IS` + TTL=`0` | All entries at all levels (full invalidation) |
> > > > | `VAE1IS` + TTL=`2` (L2) | Non-leaf at Level 0/1 + leaf at Level 2 |
> > > > | `VALE1IS` + TTL=`0` | Leaf entries at all levels (non-leaf not cleared) |
> > > > | `VALE1IS` + TTL=`2` (L2) | Leaf entry at Level 2 only |
> > > >
> > > > Signed-off-by: Zeng Heng <zengheng4@huawei.com>
> > > The fix looks fine but does it need:
> > >
> > > Fixes: 8ce720d5bd91 ("mm/hugetlb: fix excessive IPI broadcasts when unsharing PMD tables using mmu_gather")
> > > Cc: <stable@vger.kernel.org>
> > >
> > > > ---
> > > > arch/arm64/include/asm/tlb.h | 3 ++-
> > > > 1 file changed, 2 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h
> > > > index 10869d7731b8..751bd57bc3ba 100644
> > > > --- a/arch/arm64/include/asm/tlb.h
> > > > +++ b/arch/arm64/include/asm/tlb.h
> > > > @@ -53,7 +53,8 @@ static inline int tlb_get_level(struct mmu_gather *tlb)
> > > > static inline void tlb_flush(struct mmu_gather *tlb)
> > > > {
> > > > struct vm_area_struct vma = TLB_FLUSH_VMA(tlb->mm, 0);
> > > > - tlbf_t flags = tlb->freed_tables ? TLBF_NONE : TLBF_NOWALKCACHE;
> > > > + tlbf_t flags = (tlb->freed_tables || tlb->unshared_tables) ?
> > > > + TLBF_NONE : TLBF_NOWALKCACHE;
> > > > unsigned long stride = tlb_get_unmap_size(tlb);
> > > > int tlb_level = tlb_get_level(tlb);
> > Do we need this as well?
>
> The proposed fix has been validated against the issue scenarios and
> works as expected.
>
> Per the ARM Architecture Reference Manual, whether only the last-level
> page table entry is invalidated is determined by the instruction used
> (vale1is for leaf entry only, vae1is for walk cache including leaf entry and
> non-leaf entry), rather than the TTL field. The TTL field merely specifies
> which level the leaf entry belongs to.
Ah, yes, you are right. The TTL is still 2 in this case for a huge pmd,
we just want the walk cache leading to it to be invalidated. So no need
for the additional tlb_get_level().
Thanks.
--
Catalin
^ permalink raw reply
* [PATCH v5 5/5] arm64: dts: qcom: glymur: Add GPU cooling
From: Akhil P Oommen @ 2026-05-22 10:12 UTC (permalink / raw)
To: Bjorn Andersson, Konrad Dybcio, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Rob Clark, Sean Paul, Dmitry Baryshkov,
Abhinav Kumar, Jessica Zhang, Marijn Suijten, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Will Deacon, Robin Murphy, Joerg Roedel
Cc: linux-arm-msm, devicetree, linux-kernel, dri-devel, freedreno,
linux-arm-kernel, iommu, Akhil P Oommen,
Manaf Meethalavalappu Pallikunhi
In-Reply-To: <20260522-glymur-gpu-dt-v5-0-562c406b210c@oss.qualcomm.com>
From: Manaf Meethalavalappu Pallikunhi <manaf.pallikunhi@oss.qualcomm.com>
The GPU does not throttle its speed automatically when it
reaches high temperatures. Set up GPU cooling by throttling
the GPU speed when it reaches 95°C.
Signed-off-by: Manaf Meethalavalappu Pallikunhi <manaf.pallikunhi@oss.qualcomm.com>
Signed-off-by: Akhil P Oommen <akhilpo@oss.qualcomm.com>
---
arch/arm64/boot/dts/qcom/glymur.dtsi | 240 +++++++++++++++++++++++++++--------
1 file changed, 184 insertions(+), 56 deletions(-)
diff --git a/arch/arm64/boot/dts/qcom/glymur.dtsi b/arch/arm64/boot/dts/qcom/glymur.dtsi
index 01a2e32e503b..e109fb5b35a4 100644
--- a/arch/arm64/boot/dts/qcom/glymur.dtsi
+++ b/arch/arm64/boot/dts/qcom/glymur.dtsi
@@ -22,6 +22,7 @@
#include <dt-bindings/regulator/qcom,rpmh-regulator.h>
#include <dt-bindings/soc/qcom,rpmh-rsc.h>
#include <dt-bindings/spmi/spmi.h>
+#include <dt-bindings/thermal/thermal.h>
#include "glymur-ipcc.h"
@@ -7149,13 +7150,22 @@ aoss-7-critical {
};
thermal_gpu_0_0: gpu-0-0-thermal {
+ polling-delay-passive = <100>;
+
thermal-sensors = <&tsens7 1>;
+ cooling-maps {
+ map0 {
+ trip = <&gpu00_alert0>;
+ cooling-device = <&gpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+
trips {
- trip-point0 {
- temperature = <90000>;
- hysteresis = <5000>;
- type = "hot";
+ gpu00_alert0: trip-point0 {
+ temperature = <95000>;
+ hysteresis = <1000>;
+ type = "passive";
};
gpu-0-0-critical {
@@ -7164,16 +7174,26 @@ gpu-0-0-critical {
type = "critical";
};
};
+
};
thermal_gpu_0_1: gpu-0-1-thermal {
+ polling-delay-passive = <100>;
+
thermal-sensors = <&tsens7 2>;
+ cooling-maps {
+ map0 {
+ trip = <&gpu01_alert0>;
+ cooling-device = <&gpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+
trips {
- trip-point0 {
- temperature = <90000>;
- hysteresis = <5000>;
- type = "hot";
+ gpu01_alert0: trip-point0 {
+ temperature = <95000>;
+ hysteresis = <1000>;
+ type = "passive";
};
gpu-0-1-critical {
@@ -7185,13 +7205,22 @@ gpu-0-1-critical {
};
thermal_gpu_0_2: gpu-0-2-thermal {
+ polling-delay-passive = <100>;
+
thermal-sensors = <&tsens7 3>;
+ cooling-maps {
+ map0 {
+ trip = <&gpu02_alert0>;
+ cooling-device = <&gpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+
trips {
- trip-point0 {
- temperature = <90000>;
- hysteresis = <5000>;
- type = "hot";
+ gpu02_alert0: trip-point0 {
+ temperature = <95000>;
+ hysteresis = <1000>;
+ type = "passive";
};
gpu-0-2-critical {
@@ -7203,13 +7232,22 @@ gpu-0-2-critical {
};
thermal_gpu_1_0: gpu-1-0-thermal {
+ polling-delay-passive = <100>;
+
thermal-sensors = <&tsens7 4>;
+ cooling-maps {
+ map0 {
+ trip = <&gpu10_alert0>;
+ cooling-device = <&gpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+
trips {
- trip-point0 {
- temperature = <90000>;
- hysteresis = <5000>;
- type = "hot";
+ gpu10_alert0: trip-point0 {
+ temperature = <95000>;
+ hysteresis = <1000>;
+ type = "passive";
};
gpu-1-0-critical {
@@ -7221,13 +7259,22 @@ gpu-1-0-critical {
};
thermal_gpu_1_1: gpu-1-1-thermal {
+ polling-delay-passive = <100>;
+
thermal-sensors = <&tsens7 5>;
+ cooling-maps {
+ map0 {
+ trip = <&gpu11_alert0>;
+ cooling-device = <&gpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+
trips {
- trip-point0 {
- temperature = <90000>;
- hysteresis = <5000>;
- type = "hot";
+ gpu11_alert0: trip-point0 {
+ temperature = <95000>;
+ hysteresis = <1000>;
+ type = "passive";
};
gpu-1-1-critical {
@@ -7239,13 +7286,22 @@ gpu-1-1-critical {
};
thermal_gpu_1_2: gpu-1-2-thermal {
+ polling-delay-passive = <100>;
+
thermal-sensors = <&tsens7 6>;
+ cooling-maps {
+ map0 {
+ trip = <&gpu12_alert0>;
+ cooling-device = <&gpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+
trips {
- trip-point0 {
- temperature = <90000>;
- hysteresis = <5000>;
- type = "hot";
+ gpu12_alert0: trip-point0 {
+ temperature = <95000>;
+ hysteresis = <1000>;
+ type = "passive";
};
gpu-1-2-critical {
@@ -7257,13 +7313,22 @@ gpu-1-2-critical {
};
thermal_gpu_2_0: gpu-2-0-thermal {
+ polling-delay-passive = <100>;
+
thermal-sensors = <&tsens7 7>;
+ cooling-maps {
+ map0 {
+ trip = <&gpu20_alert0>;
+ cooling-device = <&gpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+
trips {
- trip-point0 {
- temperature = <90000>;
- hysteresis = <5000>;
- type = "hot";
+ gpu20_alert0: trip-point0 {
+ temperature = <95000>;
+ hysteresis = <1000>;
+ type = "passive";
};
gpu-2-0-critical {
@@ -7275,13 +7340,22 @@ gpu-2-0-critical {
};
thermal_gpu_2_1: gpu-2-1-thermal {
+ polling-delay-passive = <100>;
+
thermal-sensors = <&tsens7 8>;
+ cooling-maps {
+ map0 {
+ trip = <&gpu21_alert0>;
+ cooling-device = <&gpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+
trips {
- trip-point0 {
- temperature = <90000>;
- hysteresis = <5000>;
- type = "hot";
+ gpu21_alert0: trip-point0 {
+ temperature = <95000>;
+ hysteresis = <1000>;
+ type = "passive";
};
gpu-2-1-critical {
@@ -7293,13 +7367,22 @@ gpu-2-1-critical {
};
thermal_gpu_2_2: gpu-2-2-thermal {
+ polling-delay-passive = <100>;
+
thermal-sensors = <&tsens7 9>;
+ cooling-maps {
+ map0 {
+ trip = <&gpu22_alert0>;
+ cooling-device = <&gpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+
trips {
- trip-point0 {
- temperature = <90000>;
- hysteresis = <5000>;
- type = "hot";
+ gpu22_alert0: trip-point0 {
+ temperature = <95000>;
+ hysteresis = <1000>;
+ type = "passive";
};
gpu-2-2-critical {
@@ -7311,13 +7394,22 @@ gpu-2-2-critical {
};
thermal_gpu_3_0: gpu-3-0-thermal {
+ polling-delay-passive = <100>;
+
thermal-sensors = <&tsens7 10>;
+ cooling-maps {
+ map0 {
+ trip = <&gpu30_alert0>;
+ cooling-device = <&gpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+
trips {
- trip-point0 {
- temperature = <90000>;
- hysteresis = <5000>;
- type = "hot";
+ gpu30_alert0: trip-point0 {
+ temperature = <95000>;
+ hysteresis = <1000>;
+ type = "passive";
};
gpu-3-0-critical {
@@ -7329,13 +7421,22 @@ gpu-3-0-critical {
};
thermal_gpu_3_1: gpu-3-1-thermal {
+ polling-delay-passive = <100>;
+
thermal-sensors = <&tsens7 11>;
+ cooling-maps {
+ map0 {
+ trip = <&gpu31_alert0>;
+ cooling-device = <&gpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+
trips {
- trip-point0 {
- temperature = <90000>;
- hysteresis = <5000>;
- type = "hot";
+ gpu31_alert0: trip-point0 {
+ temperature = <95000>;
+ hysteresis = <1000>;
+ type = "passive";
};
gpu-3-1-critical {
@@ -7347,13 +7448,22 @@ gpu-3-1-critical {
};
thermal_gpu_3_2: gpu-3-2-thermal {
+ polling-delay-passive = <100>;
+
thermal-sensors = <&tsens7 12>;
+ cooling-maps {
+ map0 {
+ trip = <&gpu32_alert0>;
+ cooling-device = <&gpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+
trips {
- trip-point0 {
- temperature = <90000>;
- hysteresis = <5000>;
- type = "hot";
+ gpu32_alert0: trip-point0 {
+ temperature = <95000>;
+ hysteresis = <1000>;
+ type = "passive";
};
gpu-3-2-critical {
@@ -7365,13 +7475,22 @@ gpu-3-2-critical {
};
thermal_gpuss_0: gpuss-0-thermal {
+ polling-delay-passive = <100>;
+
thermal-sensors = <&tsens7 13>;
+ cooling-maps {
+ map0 {
+ trip = <&gpuss0_alert0>;
+ cooling-device = <&gpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+
trips {
- trip-point0 {
- temperature = <90000>;
- hysteresis = <5000>;
- type = "hot";
+ gpuss0_alert0: trip-point0 {
+ temperature = <95000>;
+ hysteresis = <1000>;
+ type = "passive";
};
gpuss-0-critical {
@@ -7383,13 +7502,22 @@ gpuss-0-critical {
};
thermal_gpuss_1: gpuss-1-thermal {
+ polling-delay-passive = <100>;
+
thermal-sensors = <&tsens7 14>;
+ cooling-maps {
+ map0 {
+ trip = <&gpuss1_alert0>;
+ cooling-device = <&gpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+ };
+ };
+
trips {
- trip-point0 {
- temperature = <90000>;
- hysteresis = <5000>;
- type = "hot";
+ gpuss1_alert0: trip-point0 {
+ temperature = <95000>;
+ hysteresis = <1000>;
+ type = "passive";
};
gpuss-1-critical {
--
2.51.0
^ permalink raw reply related
* [PATCH v5 4/5] arm64: dts: qcom: Add GPU support for Glymur
From: Akhil P Oommen @ 2026-05-22 10:12 UTC (permalink / raw)
To: Bjorn Andersson, Konrad Dybcio, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Rob Clark, Sean Paul, Dmitry Baryshkov,
Abhinav Kumar, Jessica Zhang, Marijn Suijten, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Will Deacon, Robin Murphy, Joerg Roedel
Cc: linux-arm-msm, devicetree, linux-kernel, dri-devel, freedreno,
linux-arm-kernel, iommu, Akhil P Oommen, Konrad Dybcio
In-Reply-To: <20260522-glymur-gpu-dt-v5-0-562c406b210c@oss.qualcomm.com>
The Adreno X2 series GPU present in Glymur SoC belongs to the A8x
family. It is a new HW IP with architectural improvements as well
as different set of hw configs like GMEM, num SPs, Caches sizes etc.
Add the GPU and GMU nodes to describe this hardware.
Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com>
Signed-off-by: Akhil P Oommen <akhilpo@oss.qualcomm.com>
---
arch/arm64/boot/dts/qcom/glymur.dtsi | 183 +++++++++++++++++++++++++++++++++++
1 file changed, 183 insertions(+)
diff --git a/arch/arm64/boot/dts/qcom/glymur.dtsi b/arch/arm64/boot/dts/qcom/glymur.dtsi
index 5e76a0d53f01..01a2e32e503b 100644
--- a/arch/arm64/boot/dts/qcom/glymur.dtsi
+++ b/arch/arm64/boot/dts/qcom/glymur.dtsi
@@ -3701,6 +3701,129 @@ hsc_noc: interconnect@2000000 {
#interconnect-cells = <2>;
};
+ gpu: gpu@3d00000 {
+ compatible = "qcom,adreno-44070001", "qcom,adreno";
+ reg = <0x0 0x03d00000 0x0 0x6c000>,
+ <0x0 0x03d9e000 0x0 0x2000>;
+ reg-names = "kgsl_3d0_reg_memory",
+ "cx_mem";
+
+ interrupts = <GIC_SPI 300 IRQ_TYPE_LEVEL_HIGH>;
+
+ iommus = <&adreno_smmu 0 0x0>,
+ <&adreno_smmu 1 0x0>;
+
+ operating-points-v2 = <&gpu_opp_table>;
+
+ qcom,gmu = <&gmu>;
+ #cooling-cells = <2>;
+
+ interconnects = <&hsc_noc MASTER_GFX3D QCOM_ICC_TAG_ALWAYS
+ &mc_virt SLAVE_EBI1 QCOM_ICC_TAG_ALWAYS>;
+ interconnect-names = "gfx-mem";
+
+ gpu_opp_table: opp-table {
+ compatible = "operating-points-v2-adreno",
+ "operating-points-v2";
+
+ opp-310000000 {
+ opp-hz = /bits/ 64 <310000000>;
+ opp-level = <RPMH_REGULATOR_LEVEL_LOW_SVS_D1>;
+ opp-peak-kBps = <2136719>;
+ opp-supported-hw = <0xf>;
+ /* ACD is disabled */
+ };
+
+ opp-410000000 {
+ opp-hz = /bits/ 64 <410000000>;
+ opp-level = <RPMH_REGULATOR_LEVEL_LOW_SVS>;
+ opp-peak-kBps = <6074219>;
+ opp-supported-hw = <0xf>;
+ /* ACD is disabled */
+ };
+
+ opp-572000000 {
+ opp-hz = /bits/ 64 <572000000>;
+ opp-level = <RPMH_REGULATOR_LEVEL_SVS>;
+ opp-peak-kBps = <12449219>;
+ opp-supported-hw = <0xf>;
+ qcom,opp-acd-level = <0xe02d5ffd>;
+ };
+
+ opp-760000000 {
+ opp-hz = /bits/ 64 <760000000>;
+ opp-level = <RPMH_REGULATOR_LEVEL_SVS_L1>;
+ opp-peak-kBps = <12449219>;
+ opp-supported-hw = <0xf>;
+ qcom,opp-acd-level = <0xc0285ffd>;
+ };
+
+ opp-820000000 {
+ opp-hz = /bits/ 64 <820000000>;
+ opp-level = <RPMH_REGULATOR_LEVEL_SVS_L2>;
+ opp-peak-kBps = <16500000>;
+ opp-supported-hw = <0xf>;
+ qcom,opp-acd-level = <0xa82e5ffd>;
+ };
+
+ opp-915000000 {
+ opp-hz = /bits/ 64 <915000000>;
+ opp-level = <RPMH_REGULATOR_LEVEL_NOM>;
+ opp-peak-kBps = <16500000>;
+ opp-supported-hw = <0xf>;
+ qcom,opp-acd-level = <0x882d5ffd>;
+ };
+
+ opp-1070000000 {
+ opp-hz = /bits/ 64 <1070000000>;
+ opp-level = <RPMH_REGULATOR_LEVEL_NOM_L1>;
+ opp-peak-kBps = <16500000>;
+ opp-supported-hw = <0xf>;
+ qcom,opp-acd-level = <0x882b5ffd>;
+ };
+
+ opp-1185000000 {
+ opp-hz = /bits/ 64 <1185000000>;
+ opp-level = <RPMH_REGULATOR_LEVEL_TURBO>;
+ opp-peak-kBps = <16500000>;
+ opp-supported-hw = <0xf>;
+ qcom,opp-acd-level = <0x882a5ffd>;
+ };
+
+ opp-1350000000 {
+ opp-hz = /bits/ 64 <1350000000>;
+ opp-level = <RPMH_REGULATOR_LEVEL_TURBO_L1>;
+ opp-peak-kBps = <18597657>;
+ opp-supported-hw = <0xf>;
+ qcom,opp-acd-level = <0x882a5ffd>;
+ };
+
+ opp-1550000000 {
+ opp-hz = /bits/ 64 <1550000000>;
+ opp-level = <RPMH_REGULATOR_LEVEL_TURBO_L3>;
+ opp-peak-kBps = <18597657>;
+ opp-supported-hw = <0x7>;
+ qcom,opp-acd-level = <0xa8295ffd>;
+ };
+
+ opp-1700000000 {
+ opp-hz = /bits/ 64 <1700000000>;
+ opp-level = <RPMH_REGULATOR_LEVEL_TURBO_L4>;
+ opp-peak-kBps = <18597657>;
+ opp-supported-hw = <0x7>;
+ qcom,opp-acd-level = <0x88295ffd>;
+ };
+
+ opp-1850000000 {
+ opp-hz = /bits/ 64 <1850000000>;
+ opp-level = <RPMH_REGULATOR_LEVEL_TURBO_L5>;
+ opp-peak-kBps = <18597657>;
+ opp-supported-hw = <0x3>;
+ qcom,opp-acd-level = <0x88285ffd>;
+ };
+ };
+ };
+
gxclkctl: clock-controller@3d64000 {
compatible = "qcom,glymur-gxclkctl";
reg = <0x0 0x03d64000 0x0 0x6000>;
@@ -3712,6 +3835,66 @@ gxclkctl: clock-controller@3d64000 {
#power-domain-cells = <1>;
};
+ gmu: gmu@3d6c000 {
+ compatible = "qcom,adreno-gmu-x285.1", "qcom,adreno-gmu";
+
+ reg = <0x0 0x03d6c000 0x0 0x32000>;
+ reg-names = "gmu";
+
+ interrupts = <GIC_SPI 304 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 305 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "hfi",
+ "gmu";
+
+ clocks = <&gpucc GPU_CC_AHB_CLK>,
+ <&gpucc GPU_CC_CX_GMU_CLK>,
+ <&gpucc GPU_CC_CXO_CLK>,
+ <&gcc GCC_GPU_GEMNOC_GFX_CLK>,
+ <&gpucc GPU_CC_HUB_CX_INT_CLK>,
+ <&gpucc GPU_CC_RSCC_HUB_AON_CLK>;
+ clock-names = "ahb",
+ "gmu",
+ "cxo",
+ "memnoc",
+ "hub",
+ "rscc";
+
+ power-domains = <&gpucc GPU_CC_CX_GDSC>,
+ <&gxclkctl GX_CLKCTL_GX_GDSC>;
+ power-domain-names = "cx",
+ "gx";
+
+ iommus = <&adreno_smmu 5 0x0>;
+
+ qcom,qmp = <&aoss_qmp>;
+
+ operating-points-v2 = <&gmu_opp_table>;
+
+ gmu_opp_table: opp-table {
+ compatible = "operating-points-v2";
+
+ opp-575000000 {
+ opp-hz = /bits/ 64 <575000000>;
+ opp-level = <RPMH_REGULATOR_LEVEL_LOW_SVS>;
+ };
+
+ opp-700000000 {
+ opp-hz = /bits/ 64 <700000000>;
+ opp-level = <RPMH_REGULATOR_LEVEL_SVS>;
+ };
+
+ opp-725000000 {
+ opp-hz = /bits/ 64 <725000000>;
+ opp-level = <RPMH_REGULATOR_LEVEL_SVS_L1>;
+ };
+
+ opp-750000000 {
+ opp-hz = /bits/ 64 <750000000>;
+ opp-level = <RPMH_REGULATOR_LEVEL_NOM>;
+ };
+ };
+ };
+
gpucc: clock-controller@3d90000 {
compatible = "qcom,glymur-gpucc";
reg = <0x0 0x03d90000 0x0 0x9800>;
--
2.51.0
^ permalink raw reply related
* [PATCH v5 3/5] arm64: dts: qcom: glymur: Add GPU smmu node
From: Akhil P Oommen @ 2026-05-22 10:11 UTC (permalink / raw)
To: Bjorn Andersson, Konrad Dybcio, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Rob Clark, Sean Paul, Dmitry Baryshkov,
Abhinav Kumar, Jessica Zhang, Marijn Suijten, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Will Deacon, Robin Murphy, Joerg Roedel
Cc: linux-arm-msm, devicetree, linux-kernel, dri-devel, freedreno,
linux-arm-kernel, iommu, Akhil P Oommen, Rajendra Nayak,
Konrad Dybcio, Dmitry Baryshkov
In-Reply-To: <20260522-glymur-gpu-dt-v5-0-562c406b210c@oss.qualcomm.com>
From: Rajendra Nayak <rajendra.nayak@oss.qualcomm.com>
Add the nodes to describe the GPU SMMU node.
Signed-off-by: Rajendra Nayak <rajendra.nayak@oss.qualcomm.com>
Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Signed-off-by: Akhil P Oommen <akhilpo@oss.qualcomm.com>
---
arch/arm64/boot/dts/qcom/glymur.dtsi | 38 ++++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)
diff --git a/arch/arm64/boot/dts/qcom/glymur.dtsi b/arch/arm64/boot/dts/qcom/glymur.dtsi
index ed9aac42fcbf..5e76a0d53f01 100644
--- a/arch/arm64/boot/dts/qcom/glymur.dtsi
+++ b/arch/arm64/boot/dts/qcom/glymur.dtsi
@@ -3729,6 +3729,44 @@ gpucc: clock-controller@3d90000 {
#power-domain-cells = <1>;
};
+ adreno_smmu: iommu@3da0000 {
+ compatible = "qcom,glymur-smmu-500", "qcom,adreno-smmu",
+ "qcom,smmu-500", "arm,mmu-500";
+ reg = <0x0 0x03da0000 0x0 0x40000>;
+ #iommu-cells = <2>;
+ #global-interrupts = <1>;
+ interrupts = <GIC_SPI 674 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 678 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 679 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 680 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 681 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 682 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 683 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 684 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 685 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 686 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 687 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 688 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 422 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 476 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 574 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 575 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 576 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 577 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 660 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 662 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 665 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 666 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 667 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 669 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 670 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 700 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&gpucc GPU_CC_GPU_SMMU_VOTE_CLK>;
+ clock-names = "hlos";
+ power-domains = <&gpucc GPU_CC_CX_GDSC>;
+ dma-coherent;
+ };
+
ipcc: mailbox@3e04000 {
compatible = "qcom,glymur-ipcc", "qcom,ipcc";
reg = <0x0 0x03e04000 0x0 0x1000>;
--
2.51.0
^ permalink raw reply related
* [PATCH v5 2/5] dt-bindings: display/msm: gpu: Document Adreno X2-185
From: Akhil P Oommen @ 2026-05-22 10:11 UTC (permalink / raw)
To: Bjorn Andersson, Konrad Dybcio, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Rob Clark, Sean Paul, Dmitry Baryshkov,
Abhinav Kumar, Jessica Zhang, Marijn Suijten, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Will Deacon, Robin Murphy, Joerg Roedel
Cc: linux-arm-msm, devicetree, linux-kernel, dri-devel, freedreno,
linux-arm-kernel, iommu, Akhil P Oommen
In-Reply-To: <20260522-glymur-gpu-dt-v5-0-562c406b210c@oss.qualcomm.com>
Adreno X2-185 GPU found in Glymur chipsets belongs to the A8x family.
It features a new slice architecture with 4 slices, significantly higher
bandwidth throughput compared to mobile counterparts, raytracing support,
and the highest GPU Fmax seen so far on an Adreno GPU (1850 Mhz), among
other improvements. Update the dt bindings documentation to describe this
GPU.
Signed-off-by: Akhil P Oommen <akhilpo@oss.qualcomm.com>
---
Documentation/devicetree/bindings/display/msm/gpu.yaml | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/Documentation/devicetree/bindings/display/msm/gpu.yaml b/Documentation/devicetree/bindings/display/msm/gpu.yaml
index 04b2328903ca..77caacd0fb3f 100644
--- a/Documentation/devicetree/bindings/display/msm/gpu.yaml
+++ b/Documentation/devicetree/bindings/display/msm/gpu.yaml
@@ -411,6 +411,21 @@ allOf:
- clocks
- clock-names
+ - if:
+ properties:
+ compatible:
+ contains:
+ const: qcom,adreno-44070001
+ then:
+ properties:
+ reg:
+ minItems: 2
+ maxItems: 2
+
+ reg-names:
+ minItems: 2
+ maxItems: 2
+
- if:
properties:
compatible:
@@ -434,6 +449,7 @@ allOf:
- qcom,adreno-43050a01
- qcom,adreno-43050c01
- qcom,adreno-43051401
+ - qcom,adreno-44070001
then: # Starting with A6xx, the clocks are usually defined in the GMU node
properties:
--
2.51.0
^ permalink raw reply related
* [PATCH v5 1/5] drm/msm/a8xx: Fix RSCC offset
From: Akhil P Oommen @ 2026-05-22 10:11 UTC (permalink / raw)
To: Bjorn Andersson, Konrad Dybcio, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Rob Clark, Sean Paul, Dmitry Baryshkov,
Abhinav Kumar, Jessica Zhang, Marijn Suijten, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Will Deacon, Robin Murphy, Joerg Roedel
Cc: linux-arm-msm, devicetree, linux-kernel, dri-devel, freedreno,
linux-arm-kernel, iommu, Akhil P Oommen
In-Reply-To: <20260522-glymur-gpu-dt-v5-0-562c406b210c@oss.qualcomm.com>
In A8xx, the RSCC block is part of GPU's register space. Update the
virtual base address of rscc to point to the correct address.
Fixes: 50e8a557d8d3 ("drm/msm/a8xx: Add support for A8x GMU")
Signed-off-by: Akhil P Oommen <akhilpo@oss.qualcomm.com>
---
drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
index 1b44b9e21ad8..cab4c46c6cf2 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
@@ -2357,7 +2357,12 @@ int a6xx_gmu_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node)
goto err_mmio;
}
} else if (adreno_is_a8xx(adreno_gpu)) {
- gmu->rscc = gmu->mmio + 0x19000;
+ /*
+ * On a8xx , RSCC lives at GPU base + 0x50000, which falls
+ * inside the GPU's kgsl_3d0_reg_memory range rather than the
+ * GMU's.
+ */
+ gmu->rscc = gpu->mmio + 0x50000;
} else {
gmu->rscc = gmu->mmio + 0x23000;
}
--
2.51.0
^ permalink raw reply related
* [PATCH v5 0/5] Devicetree support for Glymur GPU
From: Akhil P Oommen @ 2026-05-22 10:11 UTC (permalink / raw)
To: Bjorn Andersson, Konrad Dybcio, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Rob Clark, Sean Paul, Dmitry Baryshkov,
Abhinav Kumar, Jessica Zhang, Marijn Suijten, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Will Deacon, Robin Murphy, Joerg Roedel
Cc: linux-arm-msm, devicetree, linux-kernel, dri-devel, freedreno,
linux-arm-kernel, iommu, Akhil P Oommen, Rajendra Nayak,
Konrad Dybcio, Dmitry Baryshkov, Manaf Meethalavalappu Pallikunhi
This series adds the necessary Device Tree bits to enable GPU support
on the Glymur-based CRD devices. The Adreno X2-85 GPU present in Glymur
chipsets is based on the new Adreno A8x family of GPUs. It features a new
slice architecture with 4 slices, significantly higher bandwidth
throughput compared to mobile counterparts, raytracing support, and the
highest GPU Fmax seen so far on an Adreno GPU (1850 Mhz), among other
improvements.
This series includes patches that updates DT schema, add GPU SMMU &
GPU/GMU support. Keen-eyed readers may notice that the zap shader node
is missing. This is intentional: The Glymur-based laptop platforms
generally allow booting Linux at EL2 (yay!), which means the zap firmware
is not required here.
There is an update to the gxclkctl/drm drivers to properly support the IFPC
feature across all A8x GPUs. That series [1] is necessary to properly
support Glymur GPU:
[1] https://lore.kernel.org/lkml/20260427-gfx-clk-fixes-v2-0-797e54b3d464@oss.qualcomm.com/
Just FYI, on top of the linux-next, I had to pick below series [2] to boot
the device properly. But it is unrelated to GPU or this series:
[2] https://lore.kernel.org/all/20260331-qref_vote-v1-0-3fd7fbf87864@oss.qualcomm.com/
Signed-off-by: Akhil P Oommen <akhilpo@oss.qualcomm.com>
---
Changes in v5:
- Relax contraints for reg-names property (Krzysztof)
- Drop the smmu binding doc patch as it got picked up
- Link to v4: https://lore.kernel.org/r/20260513-glymur-gpu-dt-v4-0-f83832c3bc9a@oss.qualcomm.com
Changes in v4:
- Add a new patch for passive cooling support
- Link to v3: https://lore.kernel.org/r/20260512-glymur-gpu-dt-v3-0-84232dc21c03@oss.qualcomm.com
Changes in v3:
- Add a new patch to fix RSCC base vaddr in drm-msm
- Remove interconnect property from adreno smmu dt and the binding doc
- Add a contrait in GPU binding doc to limit the reg entries for Glymur
(Krzysztof)
- Link to v2: https://lore.kernel.org/r/20260501-glymur-gpu-dt-v2-0-2f128b5596bb@oss.qualcomm.com
Changes in v2:
- Keep GPU/GMU enabled by default and drop the enablement patch (Konrad)
- Drop zap shader node from DT
- A new patch to update GPU SMMU dt schema.
- Adjust reg range in dt nodes to avoid overlap.
- Removed cx_dbgc range as it is already stable across chipsets. This
region is now part of kgsl_3d0_reg_memory range.
- Link to v1: https://lore.kernel.org/r/20260405-glymur-gpu-dt-v1-0-2135eb11c562@oss.qualcomm.com
---
Akhil P Oommen (3):
drm/msm/a8xx: Fix RSCC offset
dt-bindings: display/msm: gpu: Document Adreno X2-185
arm64: dts: qcom: Add GPU support for Glymur
Manaf Meethalavalappu Pallikunhi (1):
arm64: dts: qcom: glymur: Add GPU cooling
Rajendra Nayak (1):
arm64: dts: qcom: glymur: Add GPU smmu node
.../devicetree/bindings/display/msm/gpu.yaml | 16 +
arch/arm64/boot/dts/qcom/glymur.dtsi | 461 ++++++++++++++++++---
drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 7 +-
3 files changed, 427 insertions(+), 57 deletions(-)
---
base-commit: c9bd03db3e792a99e9789fde20e91898e3a29e8a
change-id: 20260226-glymur-gpu-dt-339e5092606b
prerequisite-message-id: <20260410-glymur_mmcc_dt_config_v2-v3-0-acce9d106e72@oss.qualcomm.com>
prerequisite-patch-id: f7ab29f2f0241b6536d3b0c0593f0baa0e435221
prerequisite-patch-id: 56c830b7718129323b006e492aed9822d7c30079
Best regards,
--
Akhil P Oommen <akhilpo@oss.qualcomm.com>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox