From: Arun Iyengar <aiyengar1990@gmail.com>
To: Mauro Carvalho Chehab <mchehab@kernel.org>,
Paul Kocialkowski <paulk@sys-base.io>
Cc: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org,
Nicolas Dufresne <nicolas.dufresne@collabora.com>,
Hans Verkuil <hverkuil@kernel.org>,
Marco Felsch <m.felsch@pengutronix.de>,
Michael Tretter <m.tretter@pengutronix.de>,
Arun Iyengar <aiyengar1990@gmail.com>
Subject: [PATCH RFC 05/11] media: v4l2-core: Move the H.264 encode CBR rate control to a helper
Date: Tue, 10 Nov 2026 10:05:19 +0000 [thread overview]
Message-ID: <20261110-h264-encode-fixes-v1-5-7a64486df499@gmail.com> (raw)
In-Reply-To: <20261110-h264-encode-fixes-v1-0-7a64486df499@gmail.com>
The constant bitrate mode of the H.264 encode core does not depend on the
codec: it plans picture sizes over the GOP from the measured I/P size
ratio and IDR period, and moves one QP state by the QP error 6 *
log2(size / target), which holds for HEVC as well. Only the QP delta of
the I pictures differs between the codecs.
Move it to v4l2-enc-cbr, with the I picture QP delta as a parameter, so
that the HEVC encode core can use the same model. The H.264 core keeps
its mode callbacks and uses a QP delta of 2, as before. No functional
change.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
drivers/media/v4l2-core/Kconfig | 5 +
drivers/media/v4l2-core/Makefile | 1 +
drivers/media/v4l2-core/v4l2-enc-cbr.c | 300 +++++++++++++++++++++++++++++
drivers/media/v4l2-core/v4l2-h264-enc-rc.c | 246 ++---------------------
include/media/v4l2-enc-cbr.h | 64 ++++++
include/media/v4l2-h264-enc-rc.h | 35 +---
6 files changed, 387 insertions(+), 264 deletions(-)
diff --git a/drivers/media/v4l2-core/Kconfig b/drivers/media/v4l2-core/Kconfig
index a88eaf8af48c..b8b849a44abd 100644
--- a/drivers/media/v4l2-core/Kconfig
+++ b/drivers/media/v4l2-core/Kconfig
@@ -44,9 +44,14 @@ config V4L2_JPEG_HELPER
config V4L2_H264
tristate
+# Used by the stateless encode cores that need v4l2-enc-cbr.ko
+config V4L2_ENC_CBR
+ tristate
+
# Used by drivers that need v4l2-h264-enc.ko
config V4L2_H264_ENC
tristate
+ select V4L2_ENC_CBR
# Used by drivers that need v4l2-vp9.ko
config V4L2_VP9
diff --git a/drivers/media/v4l2-core/Makefile b/drivers/media/v4l2-core/Makefile
index 81c5582d6a34..3aa0855d7be7 100644
--- a/drivers/media/v4l2-core/Makefile
+++ b/drivers/media/v4l2-core/Makefile
@@ -26,6 +26,7 @@ videodev-$(CONFIG_VIDEO_V4L2_I2C) += v4l2-i2c.o
obj-$(CONFIG_V4L2_ASYNC) += v4l2-async.o
obj-$(CONFIG_V4L2_CCI) += v4l2-cci.o
+obj-$(CONFIG_V4L2_ENC_CBR) += v4l2-enc-cbr.o
obj-$(CONFIG_V4L2_FLASH_LED_CLASS) += v4l2-flash-led-class.o
obj-$(CONFIG_V4L2_FWNODE) += v4l2-fwnode.o
obj-$(CONFIG_V4L2_H264) += v4l2-h264.o
diff --git a/drivers/media/v4l2-core/v4l2-enc-cbr.c b/drivers/media/v4l2-core/v4l2-enc-cbr.c
new file mode 100644
index 000000000000..5bd10d494054
--- /dev/null
+++ b/drivers/media/v4l2-core/v4l2-enc-cbr.c
@@ -0,0 +1,300 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * V4L2 stateless encoder constant bitrate rate control
+ *
+ * One QP state is kept for the stream: it is the QP of the P (and B)
+ * pictures, and I pictures are coded a fixed number of steps below it, so
+ * that the quality of the reference pictures stays even across GOPs. Each
+ * picture gets a size target from a GOP plan, in which an I picture is
+ * "ratio" times the size of a P picture, with the ratio and the IDR period
+ * measured on the stream. The difference between the coded size so far and
+ * the size the plan expects at this point of the GOP is spread over the next
+ * pictures, so that the bitrate converges to the target.
+ *
+ * After each picture the QP state moves by a fraction of the QP error
+ * 6 * log2(size / target) (a QP step of 6 halves the size): slowly for small
+ * errors, so that the QP does not follow the size noise from picture to
+ * picture, and quickly for errors of several QP steps (a scene change, a
+ * poor start).
+ *
+ * The QP to size relation (6 QP steps per halving) is the same for H.264
+ * and HEVC, so the model does not depend on the codec. Only the QP delta of
+ * the I pictures is a parameter.
+ */
+
+#include <linux/export.h>
+#include <linux/int_log.h>
+#include <linux/math64.h>
+#include <linux/minmax.h>
+#include <linux/module.h>
+#include <linux/printk.h>
+#include <linux/string.h>
+#include <media/v4l2-enc-cbr.h>
+
+/* The accumulated size error is spread over this many pictures. */
+#define CBR_WINDOW 8
+/* Limit of the accumulated size error, in seconds of the bitrate */
+#define CBR_BUFFER_SECONDS 3
+/* I/P size ratio, 1/16 units: initial value (10x) and limits (1.5x .. 30x) */
+#define CBR_RATIO_INIT 160
+#define CBR_RATIO_MIN 24
+#define CBR_RATIO_MAX 480
+/* P pictures after an I picture when the I/P size ratio is measured */
+#define CBR_RATIO_PICS 4
+/* QP errors up to this are corrected by 1/8 per picture, 1/256 units */
+#define CBR_ERR_SLOW (3 * 256)
+/*
+ * Largest QP increase and decrease per picture, 1/256 units. Decreases are
+ * slower, so that pictures that are much smaller than their target, such as
+ * repeated pictures, don't pull the QP down far.
+ */
+#define CBR_STEP_UP_MAX (4 * 256)
+#define CBR_STEP_DOWN_MAX (2 * 256)
+
+/* log2(x) in 8.8 fixed point */
+static int cbr_log2_q8(u64 x)
+{
+ unsigned int shift = 0;
+
+ if (!x)
+ return 0;
+
+ while (x > U32_MAX) {
+ x >>= 1;
+ shift++;
+ }
+
+ return (int)(intlog2(x) >> 16) + shift * 256;
+}
+
+static unsigned int cbr_fps(const struct v4l2_fract *timeperframe)
+{
+ if (!timeperframe->numerator || !timeperframe->denominator)
+ return 30;
+
+ return max(DIV_ROUND_CLOSEST(timeperframe->denominator,
+ timeperframe->numerator), 1U);
+}
+
+static u64 cbr_nominal(u32 bitrate, const struct v4l2_fract *timeperframe)
+{
+ if (!timeperframe->numerator || !timeperframe->denominator)
+ return max(bitrate / 30, 1U);
+
+ return max_t(u64, div_u64((u64)bitrate * timeperframe->numerator,
+ timeperframe->denominator), 1);
+}
+
+/* Planned size of an I or P picture of the current GOP */
+static u64 cbr_plan(const struct v4l2_enc_cbr *cbr, unsigned int fps,
+ bool intra)
+{
+ unsigned int gop = cbr->gop;
+ u64 p;
+
+ /* Until the second I picture, assume at least two seconds. */
+ if (!gop)
+ gop = max(cbr->since_i + 1, 2 * fps);
+ if (gop <= 1)
+ return cbr->nominal;
+
+ p = div_u64(cbr->nominal * gop * 16, cbr->ratio + 16 * (gop - 1));
+ if (intra)
+ p = div_u64(p * cbr->ratio, 16);
+
+ return max_t(u64, p, 1);
+}
+
+/*
+ * Start QP of an I picture from its size in bits per luma sample:
+ * 20.3 - 6 * log2(bits per sample), clipped to 15..45. This is the relation
+ * the Rockchip MPP library uses for its first picture at 1080p, for H.264
+ * and HEVC alike.
+ */
+static int cbr_qp_initial(u64 bits, unsigned int samples)
+{
+ int qp_q8 = 20 * 256 + 77 +
+ 6 * (cbr_log2_q8(samples) - cbr_log2_q8(max_t(u64, bits, 1)));
+
+ return clamp(DIV_ROUND_CLOSEST(qp_q8, 256), 15, 45);
+}
+
+/**
+ * v4l2_enc_cbr_reset() - reset the rate control state
+ * @cbr: the rate control state
+ * @qp_delta_intra: QP of the I pictures below the QP of the P pictures
+ *
+ * The next picture starts the rate control afresh.
+ */
+void v4l2_enc_cbr_reset(struct v4l2_enc_cbr *cbr, int qp_delta_intra)
+{
+ memset(cbr, 0, sizeof(*cbr));
+ cbr->qp_delta_intra = qp_delta_intra;
+}
+EXPORT_SYMBOL_GPL(v4l2_enc_cbr_reset);
+
+/**
+ * v4l2_enc_cbr_begin() - set up the size target of the next picture
+ * @cbr: the rate control state
+ * @bitrate: target bitrate in bits per second
+ * @timeperframe: frame interval
+ * @samples: number of luma samples of a picture
+ * @intra: the picture is an I picture
+ * @qp_min: lowest QP
+ * @qp_max: highest QP
+ *
+ * Sets @cbr->target_bits. The QP of the picture is then given by
+ * v4l2_enc_cbr_qp(). A change of the bitrate keeps the QP state and the
+ * measured statistics.
+ *
+ * Return: 0, or -EINVAL if @bitrate is 0.
+ */
+int v4l2_enc_cbr_begin(struct v4l2_enc_cbr *cbr, u32 bitrate,
+ const struct v4l2_fract *timeperframe,
+ unsigned int samples, bool intra, int qp_min,
+ int qp_max)
+{
+ unsigned int fps = cbr_fps(timeperframe);
+ u64 nominal = cbr_nominal(bitrate, timeperframe);
+ s64 target, expected = 0;
+ int qp;
+
+ if (!bitrate)
+ return -EINVAL;
+
+ if (!cbr->bitrate) {
+ cbr->ratio = CBR_RATIO_INIT;
+ cbr->qp_q8 = -1;
+ }
+
+ if (cbr->bitrate != bitrate || cbr->nominal != nominal) {
+ cbr->bitrate = bitrate;
+ cbr->nominal = nominal;
+ cbr->buffer_bits = 0;
+ }
+
+ if (intra) {
+ /* An I picture closes the GOP: its length is the IDR period. */
+ if (cbr->seen_i)
+ cbr->gop = cbr->since_i + 1;
+ cbr->since_i = 0;
+ cbr->seen_i = true;
+ }
+
+ cbr->plan_bits = cbr_plan(cbr, fps, intra);
+
+ if (cbr->qp_q8 < 0) {
+ /* First picture: start QP from the bits per sample. */
+ qp = cbr_qp_initial(cbr_plan(cbr, fps, true), samples) +
+ cbr->qp_delta_intra;
+ cbr->qp_q8 = clamp_t(int, qp, qp_min, qp_max) * 256;
+ }
+
+ /*
+ * The buffer holds the coded minus the nominal sizes. Within a GOP the
+ * plan expects it to go up by the excess of the I picture and back
+ * down to zero by the savings of the P pictures until the next I
+ * picture. Spread the difference to that over the next pictures.
+ */
+ if (!intra && cbr->seen_i)
+ expected = (s64)cbr_plan(cbr, fps, true) - (s64)nominal +
+ (s64)cbr->since_i *
+ ((s64)cbr->plan_bits - (s64)nominal);
+
+ target = (s64)cbr->plan_bits -
+ div_s64(cbr->buffer_bits - max_t(s64, expected, 0),
+ CBR_WINDOW);
+ cbr->target_bits = clamp_t(s64, target,
+ max_t(u64, cbr->plan_bits / 4, 8),
+ cbr->plan_bits * 2);
+ cbr->intra = intra;
+
+ pr_debug("+ v4l2-enc-cbr: begin");
+ pr_debug(" nominal: %llu bits, plan: %llu bits, target: %llu bits",
+ cbr->nominal, cbr->plan_bits, cbr->target_bits);
+ pr_debug(" buffer: %lld bits, expected: %lld bits, gop: %u, ratio: %u",
+ cbr->buffer_bits, expected, cbr->gop, cbr->ratio);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_enc_cbr_begin);
+
+/**
+ * v4l2_enc_cbr_qp() - QP of the picture set up by v4l2_enc_cbr_begin()
+ * @cbr: the rate control state
+ *
+ * Return: the QP, not limited to the QP range of the codec.
+ */
+int v4l2_enc_cbr_qp(const struct v4l2_enc_cbr *cbr)
+{
+ int qp = DIV_ROUND_CLOSEST(cbr->qp_q8, 256);
+
+ if (cbr->intra)
+ qp -= cbr->qp_delta_intra;
+
+ return qp;
+}
+EXPORT_SYMBOL_GPL(v4l2_enc_cbr_qp);
+
+/**
+ * v4l2_enc_cbr_end() - account for a coded picture
+ * @cbr: the rate control state
+ * @bits: size of the coded picture in bits
+ * @timeperframe: frame interval
+ * @qp_min: lowest QP
+ * @qp_max: highest QP
+ */
+void v4l2_enc_cbr_end(struct v4l2_enc_cbr *cbr, u64 bits,
+ const struct v4l2_fract *timeperframe, int qp_min,
+ int qp_max)
+{
+ s64 limit = cbr->nominal * CBR_BUFFER_SECONDS * cbr_fps(timeperframe);
+ unsigned int ratio;
+ int err, step;
+
+ if (!cbr->bitrate || !cbr->target_bits)
+ return;
+
+ cbr->buffer_bits = clamp_t(s64, cbr->buffer_bits + (s64)bits -
+ (s64)cbr->nominal, -limit, limit);
+
+ /* QP error of the picture, 1/256 units */
+ err = 6 * (cbr_log2_q8(bits) - cbr_log2_q8(cbr->target_bits));
+
+ step = err / 8;
+ if (err > CBR_ERR_SLOW)
+ step += (err - CBR_ERR_SLOW) / 2;
+ else if (err < -CBR_ERR_SLOW)
+ step += (err + CBR_ERR_SLOW) / 2;
+ step = clamp(step, -CBR_STEP_DOWN_MAX, CBR_STEP_UP_MAX);
+
+ if (!cbr->intra) {
+ cbr->qp_q8 += step;
+ cbr->p_bits = cbr->p_bits ? (7 * cbr->p_bits + bits) / 8 : bits;
+ cbr->since_i++;
+
+ /* I/P size ratio, once a few P pictures were coded */
+ if (cbr->since_i == CBR_RATIO_PICS && cbr->i_bits) {
+ ratio = div64_u64(cbr->i_bits * 16, cbr->p_bits);
+ ratio = clamp(ratio, CBR_RATIO_MIN, CBR_RATIO_MAX);
+ cbr->ratio = (cbr->ratio + ratio + 1) / 2;
+ }
+ } else {
+ /*
+ * The size of an I picture mostly reflects the I/P ratio
+ * estimate: only all-intra streams steer the QP fully with it.
+ */
+ cbr->i_bits = bits;
+ cbr->qp_q8 += cbr->gop == 1 ? step : step / 4;
+ }
+
+ cbr->qp_q8 = clamp_t(int, cbr->qp_q8, qp_min * 256, qp_max * 256);
+
+ pr_debug("+ v4l2-enc-cbr: end");
+ pr_debug(" size: %llu bits, target: %llu bits, qp error: %d/256, qp: %d/256",
+ bits, cbr->target_bits, err, cbr->qp_q8);
+}
+EXPORT_SYMBOL_GPL(v4l2_enc_cbr_end);
+
+MODULE_DESCRIPTION("V4L2 stateless encoder constant bitrate rate control");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/v4l2-core/v4l2-h264-enc-rc.c b/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
index bbfe03febac8..c4f1718ead0e 100644
--- a/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
+++ b/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
@@ -5,7 +5,6 @@
* Copyright (C) 2025-2026 Paul Kocialkowski <paulk@sys-base.io>
*/
-#include <linux/int_log.h>
#include <linux/math64.h>
#include <linux/minmax.h>
#include <linux/module.h>
@@ -162,116 +161,15 @@ static const struct v4l2_h264_enc_rc_mode cq_mode = {
};
/*
- * Constant bitrate mode
- *
- * One QP state is kept for the stream: it is the QP of the P (and B)
- * pictures, and I pictures are coded a fixed number of steps below it, so
- * that the quality of the reference pictures stays even across GOPs. Each
- * picture gets a size target from a GOP plan, in which an I picture is
- * "ratio" times the size of a P picture, with the ratio and the IDR period
- * measured on the stream. The difference between the coded size so far and
- * the size the plan expects at this point of the GOP is spread over the next
- * pictures, so that the bitrate converges to the target.
- *
- * After each picture the QP state moves by a fraction of the QP error
- * 6 * log2(size / target) (a QP step of 6 halves the size): slowly for small
- * errors, so that the QP does not follow the size noise from picture to
- * picture, and quickly for errors of several QP steps (a scene change, a
- * poor start).
+ * Constant bitrate mode, with the codec independent rate control of
+ * v4l2-enc-cbr: I pictures 2 QP below the P pictures (the default of the
+ * Rockchip MPP library for H.264).
*/
-
-/* The accumulated size error is spread over this many pictures. */
-#define CBR_WINDOW 8
-/* Limit of the accumulated size error, in seconds of the bitrate */
-#define CBR_BUFFER_SECONDS 3
-/* I/P size ratio, 1/16 units: initial value (10x) and limits (1.5x .. 30x) */
-#define CBR_RATIO_INIT 160
-#define CBR_RATIO_MIN 24
-#define CBR_RATIO_MAX 480
-/* P pictures after an I picture when the I/P size ratio is measured */
-#define CBR_RATIO_PICS 4
-/* QP of the I pictures below the QP of the P pictures */
#define CBR_QP_DELTA_INTRA 2
-/* QP errors up to this are corrected by 1/8 per picture, 1/256 units */
-#define CBR_ERR_SLOW (3 * 256)
-/*
- * Largest QP increase and decrease per picture, 1/256 units. Decreases are
- * slower, so that pictures that are much smaller than their target, such as
- * repeated pictures, don't pull the QP down far.
- */
-#define CBR_STEP_UP_MAX (4 * 256)
-#define CBR_STEP_DOWN_MAX (2 * 256)
-
-/* log2(x) in 8.8 fixed point */
-static int cbr_log2_q8(u64 x)
-{
- unsigned int shift = 0;
-
- if (!x)
- return 0;
-
- while (x > U32_MAX) {
- x >>= 1;
- shift++;
- }
-
- return (int)(intlog2(x) >> 16) + shift * 256;
-}
-
-static unsigned int cbr_fps(const struct v4l2_fract *timeperframe)
-{
- if (!timeperframe->numerator || !timeperframe->denominator)
- return 30;
-
- return max(DIV_ROUND_CLOSEST(timeperframe->denominator,
- timeperframe->numerator), 1U);
-}
-
-static u64 cbr_nominal(u32 bitrate, const struct v4l2_fract *timeperframe)
-{
- if (!timeperframe->numerator || !timeperframe->denominator)
- return max(bitrate / 30, 1U);
-
- return max_t(u64, div_u64((u64)bitrate * timeperframe->numerator,
- timeperframe->denominator), 1);
-}
-
-/* Planned size of an I or P picture of the current GOP */
-static u64 cbr_plan(const struct v4l2_h264_enc_rc_cbr *cbr, unsigned int fps,
- bool intra)
-{
- unsigned int gop = cbr->gop;
- u64 p;
-
- /* Until the second I picture, assume at least two seconds. */
- if (!gop)
- gop = max(cbr->since_i + 1, 2 * fps);
- if (gop <= 1)
- return cbr->nominal;
-
- p = div_u64(cbr->nominal * gop * 16, cbr->ratio + 16 * (gop - 1));
- if (intra)
- p = div_u64(p * cbr->ratio, 16);
-
- return max_t(u64, p, 1);
-}
-
-/*
- * Start QP of an I picture from its size in bits per luma sample:
- * 20.3 - 6 * log2(bits per sample), clipped to 15..45. This is the relation
- * the Rockchip MPP library uses for its first picture at 1080p.
- */
-static int cbr_qp_initial(u64 bits, unsigned int samples)
-{
- int qp_q8 = 20 * 256 + 77 +
- 6 * (cbr_log2_q8(samples) - cbr_log2_q8(max_t(u64, bits, 1)));
-
- return clamp(DIV_ROUND_CLOSEST(qp_q8, 256), 15, 45);
-}
static int cbr_init(struct v4l2_h264_enc_rc *rc)
{
- memset(&rc->cbr, 0, sizeof(rc->cbr));
+ v4l2_enc_cbr_reset(&rc->cbr, CBR_QP_DELTA_INTRA);
return 0;
}
@@ -279,148 +177,34 @@ static int cbr_init(struct v4l2_h264_enc_rc *rc)
static int cbr_measure(struct v4l2_h264_enc_rc *rc, unsigned long *size)
{
struct v4l2_h264_enc_state *state = rc->state;
- struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
- struct v4l2_fract *timeperframe = &state->timeperframe;
bool intra = state->encode.slice_type == V4L2_H264_SLICE_TYPE_I;
- unsigned int fps = cbr_fps(timeperframe);
- u64 nominal = cbr_nominal(state->bitrate, timeperframe);
unsigned int samples = state->width_mbs * state->height_mbs * 256;
- s64 target, expected = 0;
- int qp;
-
- if (!state->bitrate)
- return -EINVAL;
-
- if (!cbr->bitrate) {
- cbr->ratio = CBR_RATIO_INIT;
- cbr->qp_q8 = -1;
- }
-
- /* A new bitrate keeps the QP state and the measured statistics. */
- if (cbr->bitrate != state->bitrate || cbr->nominal != nominal) {
- cbr->bitrate = state->bitrate;
- cbr->nominal = nominal;
- cbr->buffer_bits = 0;
- }
-
- if (intra) {
- /* An I picture closes the GOP: its length is the IDR period. */
- if (cbr->seen_i)
- cbr->gop = cbr->since_i + 1;
- cbr->since_i = 0;
- cbr->seen_i = true;
- }
-
- cbr->plan_bits = cbr_plan(cbr, fps, intra);
+ int ret;
- if (cbr->qp_q8 < 0) {
- /* First picture: start QP from the bits per sample. */
- qp = cbr_qp_initial(cbr_plan(cbr, fps, true), samples) +
- CBR_QP_DELTA_INTRA;
- cbr->qp_q8 = clamp_t(int, qp, state->qp_min, state->qp_max) *
- 256;
- }
+ ret = v4l2_enc_cbr_begin(&rc->cbr, state->bitrate,
+ &state->timeperframe, samples, intra,
+ state->qp_min, state->qp_max);
+ if (ret)
+ return ret;
- /*
- * The buffer holds the coded minus the nominal sizes. Within a GOP the
- * plan expects it to go up by the excess of the I picture and back
- * down to zero by the savings of the P pictures until the next I
- * picture. Spread the difference to that over the next pictures.
- */
- if (!intra && cbr->seen_i)
- expected = (s64)cbr_plan(cbr, fps, true) - (s64)nominal +
- (s64)cbr->since_i *
- ((s64)cbr->plan_bits - (s64)nominal);
-
- target = (s64)cbr->plan_bits -
- div_s64(cbr->buffer_bits - max_t(s64, expected, 0),
- CBR_WINDOW);
- cbr->target_bits = clamp_t(s64, target,
- max_t(u64, cbr->plan_bits / 4, 8),
- cbr->plan_bits * 2);
- cbr->intra = intra;
-
- pr_debug("+ v4l2-h264-enc-rc: cbr measure");
- pr_debug(" nominal: %llu bits, plan: %llu bits, target: %llu bits",
- cbr->nominal, cbr->plan_bits, cbr->target_bits);
- pr_debug(" buffer: %lld bits, expected: %lld bits, gop: %u, ratio: %u",
- cbr->buffer_bits, expected, cbr->gop, cbr->ratio);
-
- *size = max_t(unsigned long, cbr->target_bits / 8, 1);
+ *size = max_t(unsigned long, rc->cbr.target_bits / 8, 1);
return 0;
}
static int cbr_estimate(struct v4l2_h264_enc_rc *rc, unsigned int *qp)
{
- struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
- int value;
-
- value = DIV_ROUND_CLOSEST(cbr->qp_q8, 256);
- if (cbr->intra)
- value -= CBR_QP_DELTA_INTRA;
-
- *qp = clamp(value, 0, 51);
+ *qp = clamp(v4l2_enc_cbr_qp(&rc->cbr), 0, 51);
return 0;
}
-static void cbr_update(struct v4l2_h264_enc_rc *rc, u64 bits)
+static int cbr_complete(struct v4l2_h264_enc_rc *rc, unsigned long bytesused)
{
struct v4l2_h264_enc_state *state = rc->state;
- struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
- s64 limit = cbr->nominal * CBR_BUFFER_SECONDS *
- cbr_fps(&state->timeperframe);
- unsigned int ratio;
- int err, step;
-
- if (!cbr->bitrate || !cbr->target_bits)
- return;
-
- cbr->buffer_bits = clamp_t(s64, cbr->buffer_bits + (s64)bits -
- (s64)cbr->nominal, -limit, limit);
-
- /* QP error of the picture, 1/256 units */
- err = 6 * (cbr_log2_q8(bits) - cbr_log2_q8(cbr->target_bits));
-
- step = err / 8;
- if (err > CBR_ERR_SLOW)
- step += (err - CBR_ERR_SLOW) / 2;
- else if (err < -CBR_ERR_SLOW)
- step += (err + CBR_ERR_SLOW) / 2;
- step = clamp(step, -CBR_STEP_DOWN_MAX, CBR_STEP_UP_MAX);
-
- if (!cbr->intra) {
- cbr->qp_q8 += step;
- cbr->p_bits = cbr->p_bits ? (7 * cbr->p_bits + bits) / 8 : bits;
- cbr->since_i++;
-
- /* I/P size ratio, once a few P pictures were coded */
- if (cbr->since_i == CBR_RATIO_PICS && cbr->i_bits) {
- ratio = div64_u64(cbr->i_bits * 16, cbr->p_bits);
- ratio = clamp(ratio, CBR_RATIO_MIN, CBR_RATIO_MAX);
- cbr->ratio = (cbr->ratio + ratio + 1) / 2;
- }
- } else {
- /*
- * The size of an I picture mostly reflects the I/P ratio
- * estimate: only all-intra streams steer the QP fully with it.
- */
- cbr->i_bits = bits;
- cbr->qp_q8 += cbr->gop == 1 ? step : step / 4;
- }
- cbr->qp_q8 = clamp_t(int, cbr->qp_q8, state->qp_min * 256,
- state->qp_max * 256);
-
- pr_debug("+ v4l2-h264-enc-rc: cbr update");
- pr_debug(" size: %llu bits, target: %llu bits, qp error: %d/256, qp: %d/256",
- bits, cbr->target_bits, err, cbr->qp_q8);
-}
-
-static int cbr_complete(struct v4l2_h264_enc_rc *rc, unsigned long bytesused)
-{
- cbr_update(rc, (u64)bytesused * 8);
+ v4l2_enc_cbr_end(&rc->cbr, (u64)bytesused * 8, &state->timeperframe,
+ state->qp_min, state->qp_max);
return v4l2_h264_enc_rc_stats_collect(rc, bytesused);
}
diff --git a/include/media/v4l2-enc-cbr.h b/include/media/v4l2-enc-cbr.h
new file mode 100644
index 000000000000..6b66d121b0cc
--- /dev/null
+++ b/include/media/v4l2-enc-cbr.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * V4L2 stateless encoder constant bitrate rate control
+ *
+ * A codec independent frame-level rate control for the constant bitrate
+ * mode of the stateless encode cores.
+ */
+
+#ifndef _MEDIA_V4L2_ENC_CBR_H
+#define _MEDIA_V4L2_ENC_CBR_H
+
+#include <linux/types.h>
+#include <linux/videodev2.h>
+
+/**
+ * struct v4l2_enc_cbr - constant bitrate rate control state
+ * @qp_delta_intra: QP of the I pictures below the QP of the P pictures
+ * @bitrate: bitrate the state was set up for, 0 before the first picture
+ * @nominal: nominal picture size in bits (bitrate times frame interval)
+ * @qp_q8: QP of the P pictures in 1/256 units, -1 before the first picture
+ * @ratio: I/P picture size ratio in 1/16 units
+ * @gop: measured IDR period in pictures, 0 until two I pictures were seen
+ * @since_i: pictures coded since the last I picture
+ * @seen_i: an I picture was coded
+ * @intra: the current picture is an I picture
+ * @p_bits: running mean of the P picture sizes
+ * @i_bits: size of the last I picture
+ * @buffer_bits: accumulated difference between the coded and nominal sizes
+ * @plan_bits: planned size of the current picture
+ * @target_bits: size target of the current picture
+ *
+ * Only @qp_delta_intra is set by the user, after v4l2_enc_cbr_reset().
+ * @target_bits is valid from v4l2_enc_cbr_begin() of a picture until its
+ * v4l2_enc_cbr_end().
+ */
+struct v4l2_enc_cbr {
+ int qp_delta_intra;
+
+ u32 bitrate;
+ u64 nominal;
+ int qp_q8;
+ unsigned int ratio;
+ unsigned int gop;
+ unsigned int since_i;
+ bool seen_i;
+ bool intra;
+ u64 p_bits;
+ u64 i_bits;
+ s64 buffer_bits;
+ u64 plan_bits;
+ u64 target_bits;
+};
+
+void v4l2_enc_cbr_reset(struct v4l2_enc_cbr *cbr, int qp_delta_intra);
+int v4l2_enc_cbr_begin(struct v4l2_enc_cbr *cbr, u32 bitrate,
+ const struct v4l2_fract *timeperframe,
+ unsigned int samples, bool intra, int qp_min,
+ int qp_max);
+int v4l2_enc_cbr_qp(const struct v4l2_enc_cbr *cbr);
+void v4l2_enc_cbr_end(struct v4l2_enc_cbr *cbr, u64 bits,
+ const struct v4l2_fract *timeperframe, int qp_min,
+ int qp_max);
+
+#endif
diff --git a/include/media/v4l2-h264-enc-rc.h b/include/media/v4l2-h264-enc-rc.h
index 72d78a594e40..76781fe2d844 100644
--- a/include/media/v4l2-h264-enc-rc.h
+++ b/include/media/v4l2-h264-enc-rc.h
@@ -10,6 +10,7 @@
#include <linux/types.h>
#include <linux/v4l2-controls.h>
+#include <media/v4l2-enc-cbr.h>
#include <media/v4l2-h264.h>
#define V4L2_H264_ENC_RC_STATS_TYPE_COUNT 10
@@ -83,38 +84,6 @@ struct v4l2_h264_enc_rc_stats {
unsigned int index;
};
-/**
- * struct v4l2_h264_enc_rc_cbr - constant bitrate mode state
- * @bitrate: bitrate the state was set up for, 0 before the first picture
- * @nominal: nominal picture size in bits (bitrate times frame interval)
- * @qp_q8: QP of the P pictures in 1/256 units, -1 before the first picture
- * @ratio: I/P picture size ratio in 1/16 units
- * @gop: measured IDR period in pictures, 0 until two I pictures were seen
- * @since_i: pictures coded since the last I picture
- * @seen_i: an I picture was coded
- * @intra: the current picture is an I picture
- * @p_bits: running mean of the P picture sizes
- * @i_bits: size of the last I picture
- * @buffer_bits: accumulated difference between the coded and nominal sizes
- * @plan_bits: planned size of the current picture
- * @target_bits: size target of the current picture
- */
-struct v4l2_h264_enc_rc_cbr {
- u32 bitrate;
- u64 nominal;
- int qp_q8;
- unsigned int ratio;
- unsigned int gop;
- unsigned int since_i;
- bool seen_i;
- bool intra;
- u64 p_bits;
- u64 i_bits;
- s64 buffer_bits;
- u64 plan_bits;
- u64 target_bits;
-};
-
struct v4l2_h264_enc_rc {
const struct v4l2_h264_enc_rc_ops *ops;
void *private_data;
@@ -124,7 +93,7 @@ struct v4l2_h264_enc_rc {
void *mode_data;
struct v4l2_h264_enc_rc_stats stats;
- struct v4l2_h264_enc_rc_cbr cbr;
+ struct v4l2_enc_cbr cbr;
unsigned long size;
unsigned int qp;
};
--
2.55.0
WARNING: multiple messages have this Message-ID (diff)
From: Arun Iyengar via B4 Relay <devnull+aiyengar1990.gmail.com@kernel.org>
To: Mauro Carvalho Chehab <mchehab@kernel.org>,
Paul Kocialkowski <paulk@sys-base.io>
Cc: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org,
Nicolas Dufresne <nicolas.dufresne@collabora.com>,
Hans Verkuil <hverkuil@kernel.org>,
Marco Felsch <m.felsch@pengutronix.de>,
Michael Tretter <m.tretter@pengutronix.de>,
Arun Iyengar <aiyengar1990@gmail.com>
Subject: [PATCH RFC 05/11] media: v4l2-core: Move the H.264 encode CBR rate control to a helper
Date: Tue, 10 Nov 2026 10:05:19 +0000 [thread overview]
Message-ID: <20261110-h264-encode-fixes-v1-5-7a64486df499@gmail.com> (raw)
In-Reply-To: <20261110-h264-encode-fixes-v1-0-7a64486df499@gmail.com>
From: Arun Iyengar <aiyengar1990@gmail.com>
The constant bitrate mode of the H.264 encode core does not depend on the
codec: it plans picture sizes over the GOP from the measured I/P size
ratio and IDR period, and moves one QP state by the QP error 6 *
log2(size / target), which holds for HEVC as well. Only the QP delta of
the I pictures differs between the codecs.
Move it to v4l2-enc-cbr, with the I picture QP delta as a parameter, so
that the HEVC encode core can use the same model. The H.264 core keeps
its mode callbacks and uses a QP delta of 2, as before. No functional
change.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
drivers/media/v4l2-core/Kconfig | 5 +
drivers/media/v4l2-core/Makefile | 1 +
drivers/media/v4l2-core/v4l2-enc-cbr.c | 300 +++++++++++++++++++++++++++++
drivers/media/v4l2-core/v4l2-h264-enc-rc.c | 246 ++---------------------
include/media/v4l2-enc-cbr.h | 64 ++++++
include/media/v4l2-h264-enc-rc.h | 35 +---
6 files changed, 387 insertions(+), 264 deletions(-)
diff --git a/drivers/media/v4l2-core/Kconfig b/drivers/media/v4l2-core/Kconfig
index a88eaf8af48c..b8b849a44abd 100644
--- a/drivers/media/v4l2-core/Kconfig
+++ b/drivers/media/v4l2-core/Kconfig
@@ -44,9 +44,14 @@ config V4L2_JPEG_HELPER
config V4L2_H264
tristate
+# Used by the stateless encode cores that need v4l2-enc-cbr.ko
+config V4L2_ENC_CBR
+ tristate
+
# Used by drivers that need v4l2-h264-enc.ko
config V4L2_H264_ENC
tristate
+ select V4L2_ENC_CBR
# Used by drivers that need v4l2-vp9.ko
config V4L2_VP9
diff --git a/drivers/media/v4l2-core/Makefile b/drivers/media/v4l2-core/Makefile
index 81c5582d6a34..3aa0855d7be7 100644
--- a/drivers/media/v4l2-core/Makefile
+++ b/drivers/media/v4l2-core/Makefile
@@ -26,6 +26,7 @@ videodev-$(CONFIG_VIDEO_V4L2_I2C) += v4l2-i2c.o
obj-$(CONFIG_V4L2_ASYNC) += v4l2-async.o
obj-$(CONFIG_V4L2_CCI) += v4l2-cci.o
+obj-$(CONFIG_V4L2_ENC_CBR) += v4l2-enc-cbr.o
obj-$(CONFIG_V4L2_FLASH_LED_CLASS) += v4l2-flash-led-class.o
obj-$(CONFIG_V4L2_FWNODE) += v4l2-fwnode.o
obj-$(CONFIG_V4L2_H264) += v4l2-h264.o
diff --git a/drivers/media/v4l2-core/v4l2-enc-cbr.c b/drivers/media/v4l2-core/v4l2-enc-cbr.c
new file mode 100644
index 000000000000..5bd10d494054
--- /dev/null
+++ b/drivers/media/v4l2-core/v4l2-enc-cbr.c
@@ -0,0 +1,300 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * V4L2 stateless encoder constant bitrate rate control
+ *
+ * One QP state is kept for the stream: it is the QP of the P (and B)
+ * pictures, and I pictures are coded a fixed number of steps below it, so
+ * that the quality of the reference pictures stays even across GOPs. Each
+ * picture gets a size target from a GOP plan, in which an I picture is
+ * "ratio" times the size of a P picture, with the ratio and the IDR period
+ * measured on the stream. The difference between the coded size so far and
+ * the size the plan expects at this point of the GOP is spread over the next
+ * pictures, so that the bitrate converges to the target.
+ *
+ * After each picture the QP state moves by a fraction of the QP error
+ * 6 * log2(size / target) (a QP step of 6 halves the size): slowly for small
+ * errors, so that the QP does not follow the size noise from picture to
+ * picture, and quickly for errors of several QP steps (a scene change, a
+ * poor start).
+ *
+ * The QP to size relation (6 QP steps per halving) is the same for H.264
+ * and HEVC, so the model does not depend on the codec. Only the QP delta of
+ * the I pictures is a parameter.
+ */
+
+#include <linux/export.h>
+#include <linux/int_log.h>
+#include <linux/math64.h>
+#include <linux/minmax.h>
+#include <linux/module.h>
+#include <linux/printk.h>
+#include <linux/string.h>
+#include <media/v4l2-enc-cbr.h>
+
+/* The accumulated size error is spread over this many pictures. */
+#define CBR_WINDOW 8
+/* Limit of the accumulated size error, in seconds of the bitrate */
+#define CBR_BUFFER_SECONDS 3
+/* I/P size ratio, 1/16 units: initial value (10x) and limits (1.5x .. 30x) */
+#define CBR_RATIO_INIT 160
+#define CBR_RATIO_MIN 24
+#define CBR_RATIO_MAX 480
+/* P pictures after an I picture when the I/P size ratio is measured */
+#define CBR_RATIO_PICS 4
+/* QP errors up to this are corrected by 1/8 per picture, 1/256 units */
+#define CBR_ERR_SLOW (3 * 256)
+/*
+ * Largest QP increase and decrease per picture, 1/256 units. Decreases are
+ * slower, so that pictures that are much smaller than their target, such as
+ * repeated pictures, don't pull the QP down far.
+ */
+#define CBR_STEP_UP_MAX (4 * 256)
+#define CBR_STEP_DOWN_MAX (2 * 256)
+
+/* log2(x) in 8.8 fixed point */
+static int cbr_log2_q8(u64 x)
+{
+ unsigned int shift = 0;
+
+ if (!x)
+ return 0;
+
+ while (x > U32_MAX) {
+ x >>= 1;
+ shift++;
+ }
+
+ return (int)(intlog2(x) >> 16) + shift * 256;
+}
+
+static unsigned int cbr_fps(const struct v4l2_fract *timeperframe)
+{
+ if (!timeperframe->numerator || !timeperframe->denominator)
+ return 30;
+
+ return max(DIV_ROUND_CLOSEST(timeperframe->denominator,
+ timeperframe->numerator), 1U);
+}
+
+static u64 cbr_nominal(u32 bitrate, const struct v4l2_fract *timeperframe)
+{
+ if (!timeperframe->numerator || !timeperframe->denominator)
+ return max(bitrate / 30, 1U);
+
+ return max_t(u64, div_u64((u64)bitrate * timeperframe->numerator,
+ timeperframe->denominator), 1);
+}
+
+/* Planned size of an I or P picture of the current GOP */
+static u64 cbr_plan(const struct v4l2_enc_cbr *cbr, unsigned int fps,
+ bool intra)
+{
+ unsigned int gop = cbr->gop;
+ u64 p;
+
+ /* Until the second I picture, assume at least two seconds. */
+ if (!gop)
+ gop = max(cbr->since_i + 1, 2 * fps);
+ if (gop <= 1)
+ return cbr->nominal;
+
+ p = div_u64(cbr->nominal * gop * 16, cbr->ratio + 16 * (gop - 1));
+ if (intra)
+ p = div_u64(p * cbr->ratio, 16);
+
+ return max_t(u64, p, 1);
+}
+
+/*
+ * Start QP of an I picture from its size in bits per luma sample:
+ * 20.3 - 6 * log2(bits per sample), clipped to 15..45. This is the relation
+ * the Rockchip MPP library uses for its first picture at 1080p, for H.264
+ * and HEVC alike.
+ */
+static int cbr_qp_initial(u64 bits, unsigned int samples)
+{
+ int qp_q8 = 20 * 256 + 77 +
+ 6 * (cbr_log2_q8(samples) - cbr_log2_q8(max_t(u64, bits, 1)));
+
+ return clamp(DIV_ROUND_CLOSEST(qp_q8, 256), 15, 45);
+}
+
+/**
+ * v4l2_enc_cbr_reset() - reset the rate control state
+ * @cbr: the rate control state
+ * @qp_delta_intra: QP of the I pictures below the QP of the P pictures
+ *
+ * The next picture starts the rate control afresh.
+ */
+void v4l2_enc_cbr_reset(struct v4l2_enc_cbr *cbr, int qp_delta_intra)
+{
+ memset(cbr, 0, sizeof(*cbr));
+ cbr->qp_delta_intra = qp_delta_intra;
+}
+EXPORT_SYMBOL_GPL(v4l2_enc_cbr_reset);
+
+/**
+ * v4l2_enc_cbr_begin() - set up the size target of the next picture
+ * @cbr: the rate control state
+ * @bitrate: target bitrate in bits per second
+ * @timeperframe: frame interval
+ * @samples: number of luma samples of a picture
+ * @intra: the picture is an I picture
+ * @qp_min: lowest QP
+ * @qp_max: highest QP
+ *
+ * Sets @cbr->target_bits. The QP of the picture is then given by
+ * v4l2_enc_cbr_qp(). A change of the bitrate keeps the QP state and the
+ * measured statistics.
+ *
+ * Return: 0, or -EINVAL if @bitrate is 0.
+ */
+int v4l2_enc_cbr_begin(struct v4l2_enc_cbr *cbr, u32 bitrate,
+ const struct v4l2_fract *timeperframe,
+ unsigned int samples, bool intra, int qp_min,
+ int qp_max)
+{
+ unsigned int fps = cbr_fps(timeperframe);
+ u64 nominal = cbr_nominal(bitrate, timeperframe);
+ s64 target, expected = 0;
+ int qp;
+
+ if (!bitrate)
+ return -EINVAL;
+
+ if (!cbr->bitrate) {
+ cbr->ratio = CBR_RATIO_INIT;
+ cbr->qp_q8 = -1;
+ }
+
+ if (cbr->bitrate != bitrate || cbr->nominal != nominal) {
+ cbr->bitrate = bitrate;
+ cbr->nominal = nominal;
+ cbr->buffer_bits = 0;
+ }
+
+ if (intra) {
+ /* An I picture closes the GOP: its length is the IDR period. */
+ if (cbr->seen_i)
+ cbr->gop = cbr->since_i + 1;
+ cbr->since_i = 0;
+ cbr->seen_i = true;
+ }
+
+ cbr->plan_bits = cbr_plan(cbr, fps, intra);
+
+ if (cbr->qp_q8 < 0) {
+ /* First picture: start QP from the bits per sample. */
+ qp = cbr_qp_initial(cbr_plan(cbr, fps, true), samples) +
+ cbr->qp_delta_intra;
+ cbr->qp_q8 = clamp_t(int, qp, qp_min, qp_max) * 256;
+ }
+
+ /*
+ * The buffer holds the coded minus the nominal sizes. Within a GOP the
+ * plan expects it to go up by the excess of the I picture and back
+ * down to zero by the savings of the P pictures until the next I
+ * picture. Spread the difference to that over the next pictures.
+ */
+ if (!intra && cbr->seen_i)
+ expected = (s64)cbr_plan(cbr, fps, true) - (s64)nominal +
+ (s64)cbr->since_i *
+ ((s64)cbr->plan_bits - (s64)nominal);
+
+ target = (s64)cbr->plan_bits -
+ div_s64(cbr->buffer_bits - max_t(s64, expected, 0),
+ CBR_WINDOW);
+ cbr->target_bits = clamp_t(s64, target,
+ max_t(u64, cbr->plan_bits / 4, 8),
+ cbr->plan_bits * 2);
+ cbr->intra = intra;
+
+ pr_debug("+ v4l2-enc-cbr: begin");
+ pr_debug(" nominal: %llu bits, plan: %llu bits, target: %llu bits",
+ cbr->nominal, cbr->plan_bits, cbr->target_bits);
+ pr_debug(" buffer: %lld bits, expected: %lld bits, gop: %u, ratio: %u",
+ cbr->buffer_bits, expected, cbr->gop, cbr->ratio);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_enc_cbr_begin);
+
+/**
+ * v4l2_enc_cbr_qp() - QP of the picture set up by v4l2_enc_cbr_begin()
+ * @cbr: the rate control state
+ *
+ * Return: the QP, not limited to the QP range of the codec.
+ */
+int v4l2_enc_cbr_qp(const struct v4l2_enc_cbr *cbr)
+{
+ int qp = DIV_ROUND_CLOSEST(cbr->qp_q8, 256);
+
+ if (cbr->intra)
+ qp -= cbr->qp_delta_intra;
+
+ return qp;
+}
+EXPORT_SYMBOL_GPL(v4l2_enc_cbr_qp);
+
+/**
+ * v4l2_enc_cbr_end() - account for a coded picture
+ * @cbr: the rate control state
+ * @bits: size of the coded picture in bits
+ * @timeperframe: frame interval
+ * @qp_min: lowest QP
+ * @qp_max: highest QP
+ */
+void v4l2_enc_cbr_end(struct v4l2_enc_cbr *cbr, u64 bits,
+ const struct v4l2_fract *timeperframe, int qp_min,
+ int qp_max)
+{
+ s64 limit = cbr->nominal * CBR_BUFFER_SECONDS * cbr_fps(timeperframe);
+ unsigned int ratio;
+ int err, step;
+
+ if (!cbr->bitrate || !cbr->target_bits)
+ return;
+
+ cbr->buffer_bits = clamp_t(s64, cbr->buffer_bits + (s64)bits -
+ (s64)cbr->nominal, -limit, limit);
+
+ /* QP error of the picture, 1/256 units */
+ err = 6 * (cbr_log2_q8(bits) - cbr_log2_q8(cbr->target_bits));
+
+ step = err / 8;
+ if (err > CBR_ERR_SLOW)
+ step += (err - CBR_ERR_SLOW) / 2;
+ else if (err < -CBR_ERR_SLOW)
+ step += (err + CBR_ERR_SLOW) / 2;
+ step = clamp(step, -CBR_STEP_DOWN_MAX, CBR_STEP_UP_MAX);
+
+ if (!cbr->intra) {
+ cbr->qp_q8 += step;
+ cbr->p_bits = cbr->p_bits ? (7 * cbr->p_bits + bits) / 8 : bits;
+ cbr->since_i++;
+
+ /* I/P size ratio, once a few P pictures were coded */
+ if (cbr->since_i == CBR_RATIO_PICS && cbr->i_bits) {
+ ratio = div64_u64(cbr->i_bits * 16, cbr->p_bits);
+ ratio = clamp(ratio, CBR_RATIO_MIN, CBR_RATIO_MAX);
+ cbr->ratio = (cbr->ratio + ratio + 1) / 2;
+ }
+ } else {
+ /*
+ * The size of an I picture mostly reflects the I/P ratio
+ * estimate: only all-intra streams steer the QP fully with it.
+ */
+ cbr->i_bits = bits;
+ cbr->qp_q8 += cbr->gop == 1 ? step : step / 4;
+ }
+
+ cbr->qp_q8 = clamp_t(int, cbr->qp_q8, qp_min * 256, qp_max * 256);
+
+ pr_debug("+ v4l2-enc-cbr: end");
+ pr_debug(" size: %llu bits, target: %llu bits, qp error: %d/256, qp: %d/256",
+ bits, cbr->target_bits, err, cbr->qp_q8);
+}
+EXPORT_SYMBOL_GPL(v4l2_enc_cbr_end);
+
+MODULE_DESCRIPTION("V4L2 stateless encoder constant bitrate rate control");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/v4l2-core/v4l2-h264-enc-rc.c b/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
index bbfe03febac8..c4f1718ead0e 100644
--- a/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
+++ b/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
@@ -5,7 +5,6 @@
* Copyright (C) 2025-2026 Paul Kocialkowski <paulk@sys-base.io>
*/
-#include <linux/int_log.h>
#include <linux/math64.h>
#include <linux/minmax.h>
#include <linux/module.h>
@@ -162,116 +161,15 @@ static const struct v4l2_h264_enc_rc_mode cq_mode = {
};
/*
- * Constant bitrate mode
- *
- * One QP state is kept for the stream: it is the QP of the P (and B)
- * pictures, and I pictures are coded a fixed number of steps below it, so
- * that the quality of the reference pictures stays even across GOPs. Each
- * picture gets a size target from a GOP plan, in which an I picture is
- * "ratio" times the size of a P picture, with the ratio and the IDR period
- * measured on the stream. The difference between the coded size so far and
- * the size the plan expects at this point of the GOP is spread over the next
- * pictures, so that the bitrate converges to the target.
- *
- * After each picture the QP state moves by a fraction of the QP error
- * 6 * log2(size / target) (a QP step of 6 halves the size): slowly for small
- * errors, so that the QP does not follow the size noise from picture to
- * picture, and quickly for errors of several QP steps (a scene change, a
- * poor start).
+ * Constant bitrate mode, with the codec independent rate control of
+ * v4l2-enc-cbr: I pictures 2 QP below the P pictures (the default of the
+ * Rockchip MPP library for H.264).
*/
-
-/* The accumulated size error is spread over this many pictures. */
-#define CBR_WINDOW 8
-/* Limit of the accumulated size error, in seconds of the bitrate */
-#define CBR_BUFFER_SECONDS 3
-/* I/P size ratio, 1/16 units: initial value (10x) and limits (1.5x .. 30x) */
-#define CBR_RATIO_INIT 160
-#define CBR_RATIO_MIN 24
-#define CBR_RATIO_MAX 480
-/* P pictures after an I picture when the I/P size ratio is measured */
-#define CBR_RATIO_PICS 4
-/* QP of the I pictures below the QP of the P pictures */
#define CBR_QP_DELTA_INTRA 2
-/* QP errors up to this are corrected by 1/8 per picture, 1/256 units */
-#define CBR_ERR_SLOW (3 * 256)
-/*
- * Largest QP increase and decrease per picture, 1/256 units. Decreases are
- * slower, so that pictures that are much smaller than their target, such as
- * repeated pictures, don't pull the QP down far.
- */
-#define CBR_STEP_UP_MAX (4 * 256)
-#define CBR_STEP_DOWN_MAX (2 * 256)
-
-/* log2(x) in 8.8 fixed point */
-static int cbr_log2_q8(u64 x)
-{
- unsigned int shift = 0;
-
- if (!x)
- return 0;
-
- while (x > U32_MAX) {
- x >>= 1;
- shift++;
- }
-
- return (int)(intlog2(x) >> 16) + shift * 256;
-}
-
-static unsigned int cbr_fps(const struct v4l2_fract *timeperframe)
-{
- if (!timeperframe->numerator || !timeperframe->denominator)
- return 30;
-
- return max(DIV_ROUND_CLOSEST(timeperframe->denominator,
- timeperframe->numerator), 1U);
-}
-
-static u64 cbr_nominal(u32 bitrate, const struct v4l2_fract *timeperframe)
-{
- if (!timeperframe->numerator || !timeperframe->denominator)
- return max(bitrate / 30, 1U);
-
- return max_t(u64, div_u64((u64)bitrate * timeperframe->numerator,
- timeperframe->denominator), 1);
-}
-
-/* Planned size of an I or P picture of the current GOP */
-static u64 cbr_plan(const struct v4l2_h264_enc_rc_cbr *cbr, unsigned int fps,
- bool intra)
-{
- unsigned int gop = cbr->gop;
- u64 p;
-
- /* Until the second I picture, assume at least two seconds. */
- if (!gop)
- gop = max(cbr->since_i + 1, 2 * fps);
- if (gop <= 1)
- return cbr->nominal;
-
- p = div_u64(cbr->nominal * gop * 16, cbr->ratio + 16 * (gop - 1));
- if (intra)
- p = div_u64(p * cbr->ratio, 16);
-
- return max_t(u64, p, 1);
-}
-
-/*
- * Start QP of an I picture from its size in bits per luma sample:
- * 20.3 - 6 * log2(bits per sample), clipped to 15..45. This is the relation
- * the Rockchip MPP library uses for its first picture at 1080p.
- */
-static int cbr_qp_initial(u64 bits, unsigned int samples)
-{
- int qp_q8 = 20 * 256 + 77 +
- 6 * (cbr_log2_q8(samples) - cbr_log2_q8(max_t(u64, bits, 1)));
-
- return clamp(DIV_ROUND_CLOSEST(qp_q8, 256), 15, 45);
-}
static int cbr_init(struct v4l2_h264_enc_rc *rc)
{
- memset(&rc->cbr, 0, sizeof(rc->cbr));
+ v4l2_enc_cbr_reset(&rc->cbr, CBR_QP_DELTA_INTRA);
return 0;
}
@@ -279,148 +177,34 @@ static int cbr_init(struct v4l2_h264_enc_rc *rc)
static int cbr_measure(struct v4l2_h264_enc_rc *rc, unsigned long *size)
{
struct v4l2_h264_enc_state *state = rc->state;
- struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
- struct v4l2_fract *timeperframe = &state->timeperframe;
bool intra = state->encode.slice_type == V4L2_H264_SLICE_TYPE_I;
- unsigned int fps = cbr_fps(timeperframe);
- u64 nominal = cbr_nominal(state->bitrate, timeperframe);
unsigned int samples = state->width_mbs * state->height_mbs * 256;
- s64 target, expected = 0;
- int qp;
-
- if (!state->bitrate)
- return -EINVAL;
-
- if (!cbr->bitrate) {
- cbr->ratio = CBR_RATIO_INIT;
- cbr->qp_q8 = -1;
- }
-
- /* A new bitrate keeps the QP state and the measured statistics. */
- if (cbr->bitrate != state->bitrate || cbr->nominal != nominal) {
- cbr->bitrate = state->bitrate;
- cbr->nominal = nominal;
- cbr->buffer_bits = 0;
- }
-
- if (intra) {
- /* An I picture closes the GOP: its length is the IDR period. */
- if (cbr->seen_i)
- cbr->gop = cbr->since_i + 1;
- cbr->since_i = 0;
- cbr->seen_i = true;
- }
-
- cbr->plan_bits = cbr_plan(cbr, fps, intra);
+ int ret;
- if (cbr->qp_q8 < 0) {
- /* First picture: start QP from the bits per sample. */
- qp = cbr_qp_initial(cbr_plan(cbr, fps, true), samples) +
- CBR_QP_DELTA_INTRA;
- cbr->qp_q8 = clamp_t(int, qp, state->qp_min, state->qp_max) *
- 256;
- }
+ ret = v4l2_enc_cbr_begin(&rc->cbr, state->bitrate,
+ &state->timeperframe, samples, intra,
+ state->qp_min, state->qp_max);
+ if (ret)
+ return ret;
- /*
- * The buffer holds the coded minus the nominal sizes. Within a GOP the
- * plan expects it to go up by the excess of the I picture and back
- * down to zero by the savings of the P pictures until the next I
- * picture. Spread the difference to that over the next pictures.
- */
- if (!intra && cbr->seen_i)
- expected = (s64)cbr_plan(cbr, fps, true) - (s64)nominal +
- (s64)cbr->since_i *
- ((s64)cbr->plan_bits - (s64)nominal);
-
- target = (s64)cbr->plan_bits -
- div_s64(cbr->buffer_bits - max_t(s64, expected, 0),
- CBR_WINDOW);
- cbr->target_bits = clamp_t(s64, target,
- max_t(u64, cbr->plan_bits / 4, 8),
- cbr->plan_bits * 2);
- cbr->intra = intra;
-
- pr_debug("+ v4l2-h264-enc-rc: cbr measure");
- pr_debug(" nominal: %llu bits, plan: %llu bits, target: %llu bits",
- cbr->nominal, cbr->plan_bits, cbr->target_bits);
- pr_debug(" buffer: %lld bits, expected: %lld bits, gop: %u, ratio: %u",
- cbr->buffer_bits, expected, cbr->gop, cbr->ratio);
-
- *size = max_t(unsigned long, cbr->target_bits / 8, 1);
+ *size = max_t(unsigned long, rc->cbr.target_bits / 8, 1);
return 0;
}
static int cbr_estimate(struct v4l2_h264_enc_rc *rc, unsigned int *qp)
{
- struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
- int value;
-
- value = DIV_ROUND_CLOSEST(cbr->qp_q8, 256);
- if (cbr->intra)
- value -= CBR_QP_DELTA_INTRA;
-
- *qp = clamp(value, 0, 51);
+ *qp = clamp(v4l2_enc_cbr_qp(&rc->cbr), 0, 51);
return 0;
}
-static void cbr_update(struct v4l2_h264_enc_rc *rc, u64 bits)
+static int cbr_complete(struct v4l2_h264_enc_rc *rc, unsigned long bytesused)
{
struct v4l2_h264_enc_state *state = rc->state;
- struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
- s64 limit = cbr->nominal * CBR_BUFFER_SECONDS *
- cbr_fps(&state->timeperframe);
- unsigned int ratio;
- int err, step;
-
- if (!cbr->bitrate || !cbr->target_bits)
- return;
-
- cbr->buffer_bits = clamp_t(s64, cbr->buffer_bits + (s64)bits -
- (s64)cbr->nominal, -limit, limit);
-
- /* QP error of the picture, 1/256 units */
- err = 6 * (cbr_log2_q8(bits) - cbr_log2_q8(cbr->target_bits));
-
- step = err / 8;
- if (err > CBR_ERR_SLOW)
- step += (err - CBR_ERR_SLOW) / 2;
- else if (err < -CBR_ERR_SLOW)
- step += (err + CBR_ERR_SLOW) / 2;
- step = clamp(step, -CBR_STEP_DOWN_MAX, CBR_STEP_UP_MAX);
-
- if (!cbr->intra) {
- cbr->qp_q8 += step;
- cbr->p_bits = cbr->p_bits ? (7 * cbr->p_bits + bits) / 8 : bits;
- cbr->since_i++;
-
- /* I/P size ratio, once a few P pictures were coded */
- if (cbr->since_i == CBR_RATIO_PICS && cbr->i_bits) {
- ratio = div64_u64(cbr->i_bits * 16, cbr->p_bits);
- ratio = clamp(ratio, CBR_RATIO_MIN, CBR_RATIO_MAX);
- cbr->ratio = (cbr->ratio + ratio + 1) / 2;
- }
- } else {
- /*
- * The size of an I picture mostly reflects the I/P ratio
- * estimate: only all-intra streams steer the QP fully with it.
- */
- cbr->i_bits = bits;
- cbr->qp_q8 += cbr->gop == 1 ? step : step / 4;
- }
- cbr->qp_q8 = clamp_t(int, cbr->qp_q8, state->qp_min * 256,
- state->qp_max * 256);
-
- pr_debug("+ v4l2-h264-enc-rc: cbr update");
- pr_debug(" size: %llu bits, target: %llu bits, qp error: %d/256, qp: %d/256",
- bits, cbr->target_bits, err, cbr->qp_q8);
-}
-
-static int cbr_complete(struct v4l2_h264_enc_rc *rc, unsigned long bytesused)
-{
- cbr_update(rc, (u64)bytesused * 8);
+ v4l2_enc_cbr_end(&rc->cbr, (u64)bytesused * 8, &state->timeperframe,
+ state->qp_min, state->qp_max);
return v4l2_h264_enc_rc_stats_collect(rc, bytesused);
}
diff --git a/include/media/v4l2-enc-cbr.h b/include/media/v4l2-enc-cbr.h
new file mode 100644
index 000000000000..6b66d121b0cc
--- /dev/null
+++ b/include/media/v4l2-enc-cbr.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * V4L2 stateless encoder constant bitrate rate control
+ *
+ * A codec independent frame-level rate control for the constant bitrate
+ * mode of the stateless encode cores.
+ */
+
+#ifndef _MEDIA_V4L2_ENC_CBR_H
+#define _MEDIA_V4L2_ENC_CBR_H
+
+#include <linux/types.h>
+#include <linux/videodev2.h>
+
+/**
+ * struct v4l2_enc_cbr - constant bitrate rate control state
+ * @qp_delta_intra: QP of the I pictures below the QP of the P pictures
+ * @bitrate: bitrate the state was set up for, 0 before the first picture
+ * @nominal: nominal picture size in bits (bitrate times frame interval)
+ * @qp_q8: QP of the P pictures in 1/256 units, -1 before the first picture
+ * @ratio: I/P picture size ratio in 1/16 units
+ * @gop: measured IDR period in pictures, 0 until two I pictures were seen
+ * @since_i: pictures coded since the last I picture
+ * @seen_i: an I picture was coded
+ * @intra: the current picture is an I picture
+ * @p_bits: running mean of the P picture sizes
+ * @i_bits: size of the last I picture
+ * @buffer_bits: accumulated difference between the coded and nominal sizes
+ * @plan_bits: planned size of the current picture
+ * @target_bits: size target of the current picture
+ *
+ * Only @qp_delta_intra is set by the user, after v4l2_enc_cbr_reset().
+ * @target_bits is valid from v4l2_enc_cbr_begin() of a picture until its
+ * v4l2_enc_cbr_end().
+ */
+struct v4l2_enc_cbr {
+ int qp_delta_intra;
+
+ u32 bitrate;
+ u64 nominal;
+ int qp_q8;
+ unsigned int ratio;
+ unsigned int gop;
+ unsigned int since_i;
+ bool seen_i;
+ bool intra;
+ u64 p_bits;
+ u64 i_bits;
+ s64 buffer_bits;
+ u64 plan_bits;
+ u64 target_bits;
+};
+
+void v4l2_enc_cbr_reset(struct v4l2_enc_cbr *cbr, int qp_delta_intra);
+int v4l2_enc_cbr_begin(struct v4l2_enc_cbr *cbr, u32 bitrate,
+ const struct v4l2_fract *timeperframe,
+ unsigned int samples, bool intra, int qp_min,
+ int qp_max);
+int v4l2_enc_cbr_qp(const struct v4l2_enc_cbr *cbr);
+void v4l2_enc_cbr_end(struct v4l2_enc_cbr *cbr, u64 bits,
+ const struct v4l2_fract *timeperframe, int qp_min,
+ int qp_max);
+
+#endif
diff --git a/include/media/v4l2-h264-enc-rc.h b/include/media/v4l2-h264-enc-rc.h
index 72d78a594e40..76781fe2d844 100644
--- a/include/media/v4l2-h264-enc-rc.h
+++ b/include/media/v4l2-h264-enc-rc.h
@@ -10,6 +10,7 @@
#include <linux/types.h>
#include <linux/v4l2-controls.h>
+#include <media/v4l2-enc-cbr.h>
#include <media/v4l2-h264.h>
#define V4L2_H264_ENC_RC_STATS_TYPE_COUNT 10
@@ -83,38 +84,6 @@ struct v4l2_h264_enc_rc_stats {
unsigned int index;
};
-/**
- * struct v4l2_h264_enc_rc_cbr - constant bitrate mode state
- * @bitrate: bitrate the state was set up for, 0 before the first picture
- * @nominal: nominal picture size in bits (bitrate times frame interval)
- * @qp_q8: QP of the P pictures in 1/256 units, -1 before the first picture
- * @ratio: I/P picture size ratio in 1/16 units
- * @gop: measured IDR period in pictures, 0 until two I pictures were seen
- * @since_i: pictures coded since the last I picture
- * @seen_i: an I picture was coded
- * @intra: the current picture is an I picture
- * @p_bits: running mean of the P picture sizes
- * @i_bits: size of the last I picture
- * @buffer_bits: accumulated difference between the coded and nominal sizes
- * @plan_bits: planned size of the current picture
- * @target_bits: size target of the current picture
- */
-struct v4l2_h264_enc_rc_cbr {
- u32 bitrate;
- u64 nominal;
- int qp_q8;
- unsigned int ratio;
- unsigned int gop;
- unsigned int since_i;
- bool seen_i;
- bool intra;
- u64 p_bits;
- u64 i_bits;
- s64 buffer_bits;
- u64 plan_bits;
- u64 target_bits;
-};
-
struct v4l2_h264_enc_rc {
const struct v4l2_h264_enc_rc_ops *ops;
void *private_data;
@@ -124,7 +93,7 @@ struct v4l2_h264_enc_rc {
void *mode_data;
struct v4l2_h264_enc_rc_stats stats;
- struct v4l2_h264_enc_rc_cbr cbr;
+ struct v4l2_enc_cbr cbr;
unsigned long size;
unsigned int qp;
};
--
2.55.0
next prev parent reply other threads:[~2026-11-10 10:05 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-10 10:05 [PATCH RFC 00/11] media: h264: Stateless encode core follow-ups: parameter sets, POC type 2, CBR and VBR Arun Iyengar
2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 01/11] media: h264: Only write the SPS and PPS of a stateless encode when they change Arun Iyengar
2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 02/11] media: h264: Support picture order count type 2 in stateless encode Arun Iyengar
2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 03/11] media: h264: Write the stateless encode SPS and PPS with nal_ref_idc 3 Arun Iyengar
2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 04/11] media: v4l2-h264-enc-rc: Rework the constant bitrate mode Arun Iyengar
2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` Arun Iyengar [this message]
2026-11-10 10:05 ` [PATCH RFC 05/11] media: v4l2-core: Move the H.264 encode CBR rate control to a helper Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 06/11] media: v4l2-core: Bound the I picture size in the encode CBR rate control Arun Iyengar
2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 07/11] media: v4l2-core: Halve the I picture QP increase of the encode CBR Arun Iyengar
2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 08/11] media: v4l2-core: Re-encode oversized pictures in " Arun Iyengar
2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 09/11] media: v4l2-core: Fix all-intra streams in the encode CBR rate control Arun Iyengar
2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 10/11] media: v4l2-core: Add a peak bitrate to the encode " Arun Iyengar
2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 11/11] media: h264: Add a VBR mode to the stateless " Arun Iyengar
2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-11 1:20 ` [PATCH RFC 00/11] media: h264: Stateless encode core follow-ups: parameter sets, POC type 2, CBR and VBR Nicolas Dufresne
2026-11-12 16:20 ` Paul Kocialkowski
2026-11-13 8:47 ` Arun Iyengar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261110-h264-encode-fixes-v1-5-7a64486df499@gmail.com \
--to=aiyengar1990@gmail.com \
--cc=hverkuil@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=m.felsch@pengutronix.de \
--cc=m.tretter@pengutronix.de \
--cc=mchehab@kernel.org \
--cc=nicolas.dufresne@collabora.com \
--cc=paulk@sys-base.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.