All of lore.kernel.org
 help / color / mirror / Atom feed
From: Arun Iyengar <aiyengar1990@gmail.com>
To: Mauro Carvalho Chehab <mchehab@kernel.org>,
	 Paul Kocialkowski <paulk@sys-base.io>
Cc: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org,
	 Nicolas Dufresne <nicolas.dufresne@collabora.com>,
	 Hans Verkuil <hverkuil@kernel.org>,
	Marco Felsch <m.felsch@pengutronix.de>,
	 Michael Tretter <m.tretter@pengutronix.de>,
	 Arun Iyengar <aiyengar1990@gmail.com>
Subject: [PATCH RFC 04/11] media: v4l2-h264-enc-rc: Rework the constant bitrate mode
Date: Tue, 10 Nov 2026 10:05:18 +0000	[thread overview]
Message-ID: <20261110-h264-encode-fixes-v1-4-7a64486df499@gmail.com> (raw)
In-Reply-To: <20261110-h264-encode-fixes-v1-0-7a64486df499@gmail.com>

The CBR mode of the H.264 encode core misses its bitrate and has a large
size spike after each IDR picture:

- the QP only moves when a picture misses its size target by more than
  25% (the ratio table has a dead band from 0.8 to 1.25), so a steady
  error below that is never corrected and the QP stays near its start
  value;
- P pictures get 0.8 times the average size left in a 50 picture budget,
  which settles at about 0.89 times the nominal picture size: the
  bitrate stays 5-9% low;
- I and P pictures keep separate QP histories, so an IDR picture is coded
  with the I QP of the previous GOP, seconds old, and the first P
  picture after it uses the P history against a reference of a very
  different quality, and overshoots its target several times;
- the start QP comes from a fixed bits per macroblock table and a P
  target 6 times below the I table.

Replace it with a model like the frame-level rate control of the Rockchip
MPP library: one QP state for the stream (the QP of the P pictures), I
pictures 2 QP below it, a start QP from the bits per luma sample of the
first I picture, picture size targets from a GOP plan with the I/P size
ratio and the IDR period measured on the stream, and the deviation of
the coded size from that plan spread over the next 8 pictures. After
each picture the QP moves by 1/8 of its QP error, 6 * log2(size /
target), and by half of the part beyond 3 QP steps.

The constant QP and constant quality modes are not changed.

Board results (RK3588S Hantro H1, 1080p60 clips of 300 pictures, GOP 120,
2-16 Mb/s, GStreamer v4l2slh264enc): the bitrate error goes from
-8.3..+2.3% to +0.3..+2.3% (-0.0..+0.2% over whole GOPs), PSNR-Y
improves by 0.2-0.6 dB, the first P picture after an IDR goes from up to
20 times to at most 1.4 times the median P size, and the encode speed is
unchanged.

Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
 drivers/media/v4l2-core/v4l2-h264-enc-rc.c | 426 +++++++++++++++--------------
 include/media/v4l2-h264-enc-rc.h           |  34 +++
 2 files changed, 248 insertions(+), 212 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-h264-enc-rc.c b/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
index 8f9622395eb6..bbfe03febac8 100644
--- a/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
+++ b/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
@@ -5,6 +5,9 @@
  * Copyright (C) 2025-2026 Paul Kocialkowski <paulk@sys-base.io>
  */
 
+#include <linux/int_log.h>
+#include <linux/math64.h>
+#include <linux/minmax.h>
 #include <linux/module.h>
 #include <media/v4l2-h264-enc.h>
 
@@ -158,275 +161,274 @@ static const struct v4l2_h264_enc_rc_mode cq_mode = {
 	.estimate	= cq_estimate,
 };
 
-static int cbr_measure(struct v4l2_h264_enc_rc *rc, unsigned long *size)
-{
-	struct v4l2_h264_enc_state *state = rc->state;
-	struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
-	struct v4l2_fract *timeperframe = &state->timeperframe;
-	struct v4l2_h264_enc_rc_stats *stats = &rc->stats;
-	unsigned long long budget;
-	unsigned long target;
-	unsigned long target_nominal;
-
-	pr_debug("+ v4l2-h264-enc-rc: cbr measure");
-
-	/* Nominal average size target for exact bitrate. */
-	target_nominal = state->bitrate * timeperframe->numerator /
-			 timeperframe->denominator / 8;
-
-	/* Total size budget for twice the size window. */
-	budget = 2 * V4L2_H264_ENC_RC_STATS_SIZE_COUNT * state->bitrate *
-		 timeperframe->numerator / timeperframe->denominator / 8;
-
-	if (budget < stats->size_total) {
-		target = target_nominal;
-		goto complete;
-	}
+/*
+ * Constant bitrate mode
+ *
+ * One QP state is kept for the stream: it is the QP of the P (and B)
+ * pictures, and I pictures are coded a fixed number of steps below it, so
+ * that the quality of the reference pictures stays even across GOPs. Each
+ * picture gets a size target from a GOP plan, in which an I picture is
+ * "ratio" times the size of a P picture, with the ratio and the IDR period
+ * measured on the stream. The difference between the coded size so far and
+ * the size the plan expects at this point of the GOP is spread over the next
+ * pictures, so that the bitrate converges to the target.
+ *
+ * After each picture the QP state moves by a fraction of the QP error
+ * 6 * log2(size / target) (a QP step of 6 halves the size): slowly for small
+ * errors, so that the QP does not follow the size noise from picture to
+ * picture, and quickly for errors of several QP steps (a scene change, a
+ * poor start).
+ */
 
-	/* Remaining size budget after deducing total size in window. */
-	budget -= stats->size_total;
-	/* Average size target from remaining size budget. */
-	target = budget / (2 * V4L2_H264_ENC_RC_STATS_SIZE_COUNT - stats->count);
+/* The accumulated size error is spread over this many pictures. */
+#define CBR_WINDOW		8
+/* Limit of the accumulated size error, in seconds of the bitrate */
+#define CBR_BUFFER_SECONDS	3
+/* I/P size ratio, 1/16 units: initial value (10x) and limits (1.5x .. 30x) */
+#define CBR_RATIO_INIT		160
+#define CBR_RATIO_MIN		24
+#define CBR_RATIO_MAX		480
+/* P pictures after an I picture when the I/P size ratio is measured */
+#define CBR_RATIO_PICS		4
+/* QP of the I pictures below the QP of the P pictures */
+#define CBR_QP_DELTA_INTRA	2
+/* QP errors up to this are corrected by 1/8 per picture, 1/256 units */
+#define CBR_ERR_SLOW		(3 * 256)
+/*
+ * Largest QP increase and decrease per picture, 1/256 units. Decreases are
+ * slower, so that pictures that are much smaller than their target, such as
+ * repeated pictures, don't pull the QP down far.
+ */
+#define CBR_STEP_UP_MAX		(4 * 256)
+#define CBR_STEP_DOWN_MAX	(2 * 256)
 
-complete:
-	pr_debug("  nominal target: %lu bytes", target_nominal);
-	pr_debug("  uniform target: %lu bytes", target);
+/* log2(x) in 8.8 fixed point */
+static int cbr_log2_q8(u64 x)
+{
+	unsigned int shift = 0;
 
-	/* Bump size for intra frames and reduce for pred frames.
-	 * We are better off with good quality initial references.
-	 * Maintain average assuming 1 intra frame for 24 inter frames.
-	 */
-	if (encode->slice_type == V4L2_H264_SLICE_TYPE_I)
-		target = target * 6;
-	else
-		target = target * 80 / 100;
+	if (!x)
+		return 0;
 
-	/* Reduce target size on bitrate overshoot to calm things down. */
-	if (9 * stats->size_total > 10 * stats->count * target_nominal) {
-		pr_debug("  overshoot, reducing target size");
-		target = 80 * target / 100;
+	while (x > U32_MAX) {
+		x >>= 1;
+		shift++;
 	}
 
-	pr_debug("  final target: %lu bytes", target);
-
-	*size = target;
-
-	return 0;
+	return (int)(intlog2(x) >> 16) + shift * 256;
 }
 
-/* Intra bit size per macroblock estimation for QP starting at 10. */
-static unsigned int cbr_qp_initial_bits_mb[] = {
-	437, 411, 376, 355, 318, 303, 291, 271, 252, 238, 218, 204, 188, 171,
-	158, 149, 133, 124, 115, 105, 98, 92, 83, 77, 70, 63, 57, 52, 45, 39,
-	34, 31, 30, 29, 26, 23, 22, 20, 18, 17, 16, 15
-};
-
-static int cbr_qp_initial(struct v4l2_h264_enc_rc *rc, unsigned int *qp)
+static unsigned int cbr_fps(const struct v4l2_fract *timeperframe)
 {
-	struct v4l2_h264_enc_state *state = rc->state;
-	struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
-	unsigned long size_mb;
-	unsigned int qp_initial;
-	unsigned int threshold;
-	unsigned int i;
+	if (!timeperframe->numerator || !timeperframe->denominator)
+		return 30;
 
-	size_mb = 8 * rc->size / (state->width_mbs * state->height_mbs);
-
-	pr_debug("  initial estimate from %lu bits/mb", size_mb);
+	return max(DIV_ROUND_CLOSEST(timeperframe->denominator,
+				     timeperframe->numerator), 1U);
+}
 
-	for (i = 0; i < ARRAY_SIZE(cbr_qp_initial_bits_mb); i++) {
-		threshold = cbr_qp_initial_bits_mb[i];
+static u64 cbr_nominal(u32 bitrate, const struct v4l2_fract *timeperframe)
+{
+	if (!timeperframe->numerator || !timeperframe->denominator)
+		return max(bitrate / 30, 1U);
 
-		/* Expect inter frames to be 6 times smaller. */
-		if (encode->slice_type != V4L2_H264_SLICE_TYPE_I)
-			threshold = cbr_qp_initial_bits_mb[i] / 6;
+	return max_t(u64, div_u64((u64)bitrate * timeperframe->numerator,
+				  timeperframe->denominator), 1);
+}
 
-		if (size_mb >= threshold) {
-			qp_initial = 10 + i;
-			break;
-		}
-	}
+/* Planned size of an I or P picture of the current GOP */
+static u64 cbr_plan(const struct v4l2_h264_enc_rc_cbr *cbr, unsigned int fps,
+		    bool intra)
+{
+	unsigned int gop = cbr->gop;
+	u64 p;
 
-	/* Fallback to the lowest QP (last in the list). */
-	if (i == ARRAY_SIZE(cbr_qp_initial_bits_mb))
-		qp_initial = 10 + i - 1;
+	/* Until the second I picture, assume at least two seconds. */
+	if (!gop)
+		gop = max(cbr->since_i + 1, 2 * fps);
+	if (gop <= 1)
+		return cbr->nominal;
 
-	*qp = qp_initial;
+	p = div_u64(cbr->nominal * gop * 16, cbr->ratio + 16 * (gop - 1));
+	if (intra)
+		p = div_u64(p * cbr->ratio, 16);
 
-	return 0;
+	return max_t(u64, p, 1);
 }
 
 /*
- * This table gives the QP decrease/increase (coded as index, with QP = 0 in
- * the middle) for each size ratio (in base 1000).
- *
- * Intra is a bit agressive since we generally have few intra slices to adapt
- * to a scene change. The QP diff range goes from -4 to +4.
- */
-static unsigned long cbr_ratio_qp_diff_steps_intra[] = {
-	3000, 2000, 1500, 1250, 800, 666, 500, 333
-};
-
-/*
- * Inter is less agressive since we generally have more inter slices to adapt
- * to a scene change and want to give more stability to QP. The QP diff range
- * goes from -3 to +3.
+ * Start QP of an I picture from its size in bits per luma sample:
+ * 20.3 - 6 * log2(bits per sample), clipped to 15..45. This is the relation
+ * the Rockchip MPP library uses for its first picture at 1080p.
  */
-static unsigned long cbr_ratio_qp_diff_steps_inter[] = {
-	2000, 1500, 1250, 800, 666, 500
-};
-
-static int cbr_ratio_qp_diff(unsigned long ratio, unsigned char slice_type)
+static int cbr_qp_initial(u64 bits, unsigned int samples)
 {
-	unsigned long *steps;
-	unsigned int count;
-	unsigned int i;
-
-	if (slice_type == V4L2_H264_SLICE_TYPE_I) {
-		steps = cbr_ratio_qp_diff_steps_intra;
-		count = ARRAY_SIZE(cbr_ratio_qp_diff_steps_intra);
-	} else {
-		steps = cbr_ratio_qp_diff_steps_inter;
-		count = ARRAY_SIZE(cbr_ratio_qp_diff_steps_inter);
-	}
+	int qp_q8 = 20 * 256 + 77 +
+		    6 * (cbr_log2_q8(samples) - cbr_log2_q8(max_t(u64, bits, 1)));
 
-	WARN_ON(count % 2);
+	return clamp(DIV_ROUND_CLOSEST(qp_q8, 256), 15, 45);
+}
 
-	for (i = 0; i < count; i++)
-		if (ratio > steps[i])
-			return -1 * (int)count / 2 + i;
+static int cbr_init(struct v4l2_h264_enc_rc *rc)
+{
+	memset(&rc->cbr, 0, sizeof(rc->cbr));
 
-	return count / 2;
+	return 0;
 }
 
-static int cbr_estimate(struct v4l2_h264_enc_rc *rc, unsigned int *qp)
+static int cbr_measure(struct v4l2_h264_enc_rc *rc, unsigned long *size)
 {
 	struct v4l2_h264_enc_state *state = rc->state;
-	struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
-	struct v4l2_h264_enc_rc_stats *stats = &rc->stats;
-	struct v4l2_h264_enc_rc_stats_type *type;
-	unsigned int type_index[3];
-	unsigned int ratio_qp[3];
-	unsigned int penalty[3];
-	unsigned int penalty_total;
-	unsigned int weight[3];
-	unsigned int weight_total;
-	int ratio_qp_diff;
-	unsigned long diff;
-	unsigned long diff_closest;
-	unsigned int slot_closest;
-	unsigned int index;
-	unsigned int count;
-	unsigned int total;
-	unsigned long ratio;
-	unsigned int i;
-
-	if (!stats->count)
-		return cbr_qp_initial(rc, qp);
-
-	if (encode->slice_type == V4L2_H264_SLICE_TYPE_I)
-		type = &stats->intra;
-	else if (encode->slice_type == V4L2_H264_SLICE_TYPE_P)
-		type = &stats->pred;
-	else if (encode->slice_type == V4L2_H264_SLICE_TYPE_B)
-		type = &stats->bipred;
-	else
+	struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
+	struct v4l2_fract *timeperframe = &state->timeperframe;
+	bool intra = state->encode.slice_type == V4L2_H264_SLICE_TYPE_I;
+	unsigned int fps = cbr_fps(timeperframe);
+	u64 nominal = cbr_nominal(state->bitrate, timeperframe);
+	unsigned int samples = state->width_mbs * state->height_mbs * 256;
+	s64 target, expected = 0;
+	int qp;
+
+	if (!state->bitrate)
 		return -EINVAL;
 
-	pr_debug("+ v4l2-h264-enc-rc: cbr estimate");
-
-	if (!type->count)
-		return cbr_qp_initial(rc, qp);
-
-	for (i = 0; i < 3; i++) {
-		if (i == type->count)
-			break;
-
-		index = type->index;
-		if (index <= i)
-			index = type->count + index - (i + 1);
-		else
-			index -= i + 1;
-
-		/* Set initial penalty based on frame age. */
-		penalty[i] = 4 * i;
-		type_index[i] = index;
-
-		/* Track frame with size closest to target. */
-		diff = abs((long)rc->size - type->size[index]);
-
-		if (!i || diff < diff_closest) {
-			diff_closest = diff;
-			slot_closest = i;
-		}
+	if (!cbr->bitrate) {
+		cbr->ratio = CBR_RATIO_INIT;
+		cbr->qp_q8 = -1;
 	}
 
-	count = i;
-	penalty_total = 0;
-
-	for (i = 0; i < count; i++) {
-		/* Add penalty for frames with larger size difference. */
-		if (i != slot_closest)
-			penalty[i] += 10;
+	/* A new bitrate keeps the QP state and the measured statistics. */
+	if (cbr->bitrate != state->bitrate || cbr->nominal != nominal) {
+		cbr->bitrate = state->bitrate;
+		cbr->nominal = nominal;
+		cbr->buffer_bits = 0;
+	}
 
-		penalty_total += penalty[i];
+	if (intra) {
+		/* An I picture closes the GOP: its length is the IDR period. */
+		if (cbr->seen_i)
+			cbr->gop = cbr->since_i + 1;
+		cbr->since_i = 0;
+		cbr->seen_i = true;
+	}
 
-		index = type_index[i];
+	cbr->plan_bits = cbr_plan(cbr, fps, intra);
 
-		/* Calculate QP from target to observed size ratio. */
-		ratio = 1000UL * rc->size / type->size[index];
+	if (cbr->qp_q8 < 0) {
+		/* First picture: start QP from the bits per sample. */
+		qp = cbr_qp_initial(cbr_plan(cbr, fps, true), samples) +
+		     CBR_QP_DELTA_INTRA;
+		cbr->qp_q8 = clamp_t(int, qp, state->qp_min, state->qp_max) *
+			     256;
+	}
 
-		ratio_qp_diff = cbr_ratio_qp_diff(ratio, encode->slice_type);
+	/*
+	 * The buffer holds the coded minus the nominal sizes. Within a GOP the
+	 * plan expects it to go up by the excess of the I picture and back
+	 * down to zero by the savings of the P pictures until the next I
+	 * picture. Spread the difference to that over the next pictures.
+	 */
+	if (!intra && cbr->seen_i)
+		expected = (s64)cbr_plan(cbr, fps, true) - (s64)nominal +
+			   (s64)cbr->since_i *
+			   ((s64)cbr->plan_bits - (s64)nominal);
+
+	target = (s64)cbr->plan_bits -
+		 div_s64(cbr->buffer_bits - max_t(s64, expected, 0),
+			 CBR_WINDOW);
+	cbr->target_bits = clamp_t(s64, target,
+				   max_t(u64, cbr->plan_bits / 4, 8),
+				   cbr->plan_bits * 2);
+	cbr->intra = intra;
 
-		if ((int)type->qp[index] + ratio_qp_diff < 0)
-			ratio_qp[i] = 0;
-		else if ((int)type->qp[index] + ratio_qp_diff > 51)
-			ratio_qp[i] = 51;
-		else
-			ratio_qp[i] = type->qp[index] + ratio_qp_diff;
+	pr_debug("+ v4l2-h264-enc-rc: cbr measure");
+	pr_debug("  nominal: %llu bits, plan: %llu bits, target: %llu bits",
+		 cbr->nominal, cbr->plan_bits, cbr->target_bits);
+	pr_debug("  buffer: %lld bits, expected: %lld bits, gop: %u, ratio: %u",
+		 cbr->buffer_bits, expected, cbr->gop, cbr->ratio);
 
-		diff = abs((long)rc->size - type->size[index]);
+	*size = max_t(unsigned long, cbr->target_bits / 8, 1);
 
-		pr_debug("  - backlog %d size: %lu (diff: %lu), qp: %u, penalty %u, ratio qp: %u",
-			 -(i + 1), type->size[index], diff, type->qp[index],
-			 penalty[i], ratio_qp[i]);
-	}
+	return 0;
+}
 
-	total = 0;
-	weight_total = 0;
+static int cbr_estimate(struct v4l2_h264_enc_rc *rc, unsigned int *qp)
+{
+	struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
+	int value;
 
-	for (i = 0; i < count; i++) {
-		/* Weight each calculated QP using the penalty ratio. */
-		if (penalty_total)
-			weight[i] = 1000 * (penalty_total - penalty[i]) /
-				    penalty_total;
-		else
-			weight[i] = 1000;
+	value = DIV_ROUND_CLOSEST(cbr->qp_q8, 256);
+	if (cbr->intra)
+		value -= CBR_QP_DELTA_INTRA;
 
-		/* Give non-zero low weight to full penalty cases. */
-		if (weight[i] < 200)
-			weight[i] = 200;
+	*qp = clamp(value, 0, 51);
 
-		total += ratio_qp[i] * weight[i];
-		weight_total += weight[i];
+	return 0;
+}
 
-		pr_debug("  - backlog %d weight: %u", -(i + 1),
-			 weight[i]);
+static void cbr_update(struct v4l2_h264_enc_rc *rc, u64 bits)
+{
+	struct v4l2_h264_enc_state *state = rc->state;
+	struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
+	s64 limit = cbr->nominal * CBR_BUFFER_SECONDS *
+		    cbr_fps(&state->timeperframe);
+	unsigned int ratio;
+	int err, step;
+
+	if (!cbr->bitrate || !cbr->target_bits)
+		return;
+
+	cbr->buffer_bits = clamp_t(s64, cbr->buffer_bits + (s64)bits -
+				   (s64)cbr->nominal, -limit, limit);
+
+	/* QP error of the picture, 1/256 units */
+	err = 6 * (cbr_log2_q8(bits) - cbr_log2_q8(cbr->target_bits));
+
+	step = err / 8;
+	if (err > CBR_ERR_SLOW)
+		step += (err - CBR_ERR_SLOW) / 2;
+	else if (err < -CBR_ERR_SLOW)
+		step += (err + CBR_ERR_SLOW) / 2;
+	step = clamp(step, -CBR_STEP_DOWN_MAX, CBR_STEP_UP_MAX);
+
+	if (!cbr->intra) {
+		cbr->qp_q8 += step;
+		cbr->p_bits = cbr->p_bits ? (7 * cbr->p_bits + bits) / 8 : bits;
+		cbr->since_i++;
+
+		/* I/P size ratio, once a few P pictures were coded */
+		if (cbr->since_i == CBR_RATIO_PICS && cbr->i_bits) {
+			ratio = div64_u64(cbr->i_bits * 16, cbr->p_bits);
+			ratio = clamp(ratio, CBR_RATIO_MIN, CBR_RATIO_MAX);
+			cbr->ratio = (cbr->ratio + ratio + 1) / 2;
+		}
+	} else {
+		/*
+		 * The size of an I picture mostly reflects the I/P ratio
+		 * estimate: only all-intra streams steer the QP fully with it.
+		 */
+		cbr->i_bits = bits;
+		cbr->qp_q8 += cbr->gop == 1 ? step : step / 4;
 	}
 
-	*qp = total / weight_total;
+	cbr->qp_q8 = clamp_t(int, cbr->qp_q8, state->qp_min * 256,
+			     state->qp_max * 256);
 
-	return 0;
+	pr_debug("+ v4l2-h264-enc-rc: cbr update");
+	pr_debug("  size: %llu bits, target: %llu bits, qp error: %d/256, qp: %d/256",
+		 bits, cbr->target_bits, err, cbr->qp_q8);
 }
 
 static int cbr_complete(struct v4l2_h264_enc_rc *rc, unsigned long bytesused)
 {
+	cbr_update(rc, (u64)bytesused * 8);
+
 	return v4l2_h264_enc_rc_stats_collect(rc, bytesused);
 }
 
 static const struct v4l2_h264_enc_rc_mode cbr_mode = {
 	.name		= "constant bitrate",
 	.type		= V4L2_H264_ENC_RC_TYPE_CBR,
+	.init		= cbr_init,
 	.measure	= cbr_measure,
 	.estimate	= cbr_estimate,
 	.complete	= cbr_complete,
diff --git a/include/media/v4l2-h264-enc-rc.h b/include/media/v4l2-h264-enc-rc.h
index 8b453c3a212e..72d78a594e40 100644
--- a/include/media/v4l2-h264-enc-rc.h
+++ b/include/media/v4l2-h264-enc-rc.h
@@ -8,6 +8,7 @@
 #ifndef _MEDIA_V4L2_H264_ENC_RC_H
 #define _MEDIA_V4L2_H264_ENC_RC_H
 
+#include <linux/types.h>
 #include <linux/v4l2-controls.h>
 #include <media/v4l2-h264.h>
 
@@ -82,6 +83,38 @@ struct v4l2_h264_enc_rc_stats {
 	unsigned int index;
 };
 
+/**
+ * struct v4l2_h264_enc_rc_cbr - constant bitrate mode state
+ * @bitrate: bitrate the state was set up for, 0 before the first picture
+ * @nominal: nominal picture size in bits (bitrate times frame interval)
+ * @qp_q8: QP of the P pictures in 1/256 units, -1 before the first picture
+ * @ratio: I/P picture size ratio in 1/16 units
+ * @gop: measured IDR period in pictures, 0 until two I pictures were seen
+ * @since_i: pictures coded since the last I picture
+ * @seen_i: an I picture was coded
+ * @intra: the current picture is an I picture
+ * @p_bits: running mean of the P picture sizes
+ * @i_bits: size of the last I picture
+ * @buffer_bits: accumulated difference between the coded and nominal sizes
+ * @plan_bits: planned size of the current picture
+ * @target_bits: size target of the current picture
+ */
+struct v4l2_h264_enc_rc_cbr {
+	u32 bitrate;
+	u64 nominal;
+	int qp_q8;
+	unsigned int ratio;
+	unsigned int gop;
+	unsigned int since_i;
+	bool seen_i;
+	bool intra;
+	u64 p_bits;
+	u64 i_bits;
+	s64 buffer_bits;
+	u64 plan_bits;
+	u64 target_bits;
+};
+
 struct v4l2_h264_enc_rc {
 	const struct v4l2_h264_enc_rc_ops *ops;
 	void *private_data;
@@ -91,6 +124,7 @@ struct v4l2_h264_enc_rc {
 	void *mode_data;
 
 	struct v4l2_h264_enc_rc_stats stats;
+	struct v4l2_h264_enc_rc_cbr cbr;
 	unsigned long size;
 	unsigned int qp;
 };

-- 
2.55.0


WARNING: multiple messages have this Message-ID (diff)
From: Arun Iyengar via B4 Relay <devnull+aiyengar1990.gmail.com@kernel.org>
To: Mauro Carvalho Chehab <mchehab@kernel.org>,
	 Paul Kocialkowski <paulk@sys-base.io>
Cc: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org,
	 Nicolas Dufresne <nicolas.dufresne@collabora.com>,
	 Hans Verkuil <hverkuil@kernel.org>,
	Marco Felsch <m.felsch@pengutronix.de>,
	 Michael Tretter <m.tretter@pengutronix.de>,
	 Arun Iyengar <aiyengar1990@gmail.com>
Subject: [PATCH RFC 04/11] media: v4l2-h264-enc-rc: Rework the constant bitrate mode
Date: Tue, 10 Nov 2026 10:05:18 +0000	[thread overview]
Message-ID: <20261110-h264-encode-fixes-v1-4-7a64486df499@gmail.com> (raw)
In-Reply-To: <20261110-h264-encode-fixes-v1-0-7a64486df499@gmail.com>

From: Arun Iyengar <aiyengar1990@gmail.com>

The CBR mode of the H.264 encode core misses its bitrate and has a large
size spike after each IDR picture:

- the QP only moves when a picture misses its size target by more than
  25% (the ratio table has a dead band from 0.8 to 1.25), so a steady
  error below that is never corrected and the QP stays near its start
  value;
- P pictures get 0.8 times the average size left in a 50 picture budget,
  which settles at about 0.89 times the nominal picture size: the
  bitrate stays 5-9% low;
- I and P pictures keep separate QP histories, so an IDR picture is coded
  with the I QP of the previous GOP, seconds old, and the first P
  picture after it uses the P history against a reference of a very
  different quality, and overshoots its target several times;
- the start QP comes from a fixed bits per macroblock table and a P
  target 6 times below the I table.

Replace it with a model like the frame-level rate control of the Rockchip
MPP library: one QP state for the stream (the QP of the P pictures), I
pictures 2 QP below it, a start QP from the bits per luma sample of the
first I picture, picture size targets from a GOP plan with the I/P size
ratio and the IDR period measured on the stream, and the deviation of
the coded size from that plan spread over the next 8 pictures. After
each picture the QP moves by 1/8 of its QP error, 6 * log2(size /
target), and by half of the part beyond 3 QP steps.

The constant QP and constant quality modes are not changed.

Board results (RK3588S Hantro H1, 1080p60 clips of 300 pictures, GOP 120,
2-16 Mb/s, GStreamer v4l2slh264enc): the bitrate error goes from
-8.3..+2.3% to +0.3..+2.3% (-0.0..+0.2% over whole GOPs), PSNR-Y
improves by 0.2-0.6 dB, the first P picture after an IDR goes from up to
20 times to at most 1.4 times the median P size, and the encode speed is
unchanged.

Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
 drivers/media/v4l2-core/v4l2-h264-enc-rc.c | 426 +++++++++++++++--------------
 include/media/v4l2-h264-enc-rc.h           |  34 +++
 2 files changed, 248 insertions(+), 212 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-h264-enc-rc.c b/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
index 8f9622395eb6..bbfe03febac8 100644
--- a/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
+++ b/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
@@ -5,6 +5,9 @@
  * Copyright (C) 2025-2026 Paul Kocialkowski <paulk@sys-base.io>
  */
 
+#include <linux/int_log.h>
+#include <linux/math64.h>
+#include <linux/minmax.h>
 #include <linux/module.h>
 #include <media/v4l2-h264-enc.h>
 
@@ -158,275 +161,274 @@ static const struct v4l2_h264_enc_rc_mode cq_mode = {
 	.estimate	= cq_estimate,
 };
 
-static int cbr_measure(struct v4l2_h264_enc_rc *rc, unsigned long *size)
-{
-	struct v4l2_h264_enc_state *state = rc->state;
-	struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
-	struct v4l2_fract *timeperframe = &state->timeperframe;
-	struct v4l2_h264_enc_rc_stats *stats = &rc->stats;
-	unsigned long long budget;
-	unsigned long target;
-	unsigned long target_nominal;
-
-	pr_debug("+ v4l2-h264-enc-rc: cbr measure");
-
-	/* Nominal average size target for exact bitrate. */
-	target_nominal = state->bitrate * timeperframe->numerator /
-			 timeperframe->denominator / 8;
-
-	/* Total size budget for twice the size window. */
-	budget = 2 * V4L2_H264_ENC_RC_STATS_SIZE_COUNT * state->bitrate *
-		 timeperframe->numerator / timeperframe->denominator / 8;
-
-	if (budget < stats->size_total) {
-		target = target_nominal;
-		goto complete;
-	}
+/*
+ * Constant bitrate mode
+ *
+ * One QP state is kept for the stream: it is the QP of the P (and B)
+ * pictures, and I pictures are coded a fixed number of steps below it, so
+ * that the quality of the reference pictures stays even across GOPs. Each
+ * picture gets a size target from a GOP plan, in which an I picture is
+ * "ratio" times the size of a P picture, with the ratio and the IDR period
+ * measured on the stream. The difference between the coded size so far and
+ * the size the plan expects at this point of the GOP is spread over the next
+ * pictures, so that the bitrate converges to the target.
+ *
+ * After each picture the QP state moves by a fraction of the QP error
+ * 6 * log2(size / target) (a QP step of 6 halves the size): slowly for small
+ * errors, so that the QP does not follow the size noise from picture to
+ * picture, and quickly for errors of several QP steps (a scene change, a
+ * poor start).
+ */
 
-	/* Remaining size budget after deducing total size in window. */
-	budget -= stats->size_total;
-	/* Average size target from remaining size budget. */
-	target = budget / (2 * V4L2_H264_ENC_RC_STATS_SIZE_COUNT - stats->count);
+/* The accumulated size error is spread over this many pictures. */
+#define CBR_WINDOW		8
+/* Limit of the accumulated size error, in seconds of the bitrate */
+#define CBR_BUFFER_SECONDS	3
+/* I/P size ratio, 1/16 units: initial value (10x) and limits (1.5x .. 30x) */
+#define CBR_RATIO_INIT		160
+#define CBR_RATIO_MIN		24
+#define CBR_RATIO_MAX		480
+/* P pictures after an I picture when the I/P size ratio is measured */
+#define CBR_RATIO_PICS		4
+/* QP of the I pictures below the QP of the P pictures */
+#define CBR_QP_DELTA_INTRA	2
+/* QP errors up to this are corrected by 1/8 per picture, 1/256 units */
+#define CBR_ERR_SLOW		(3 * 256)
+/*
+ * Largest QP increase and decrease per picture, 1/256 units. Decreases are
+ * slower, so that pictures that are much smaller than their target, such as
+ * repeated pictures, don't pull the QP down far.
+ */
+#define CBR_STEP_UP_MAX		(4 * 256)
+#define CBR_STEP_DOWN_MAX	(2 * 256)
 
-complete:
-	pr_debug("  nominal target: %lu bytes", target_nominal);
-	pr_debug("  uniform target: %lu bytes", target);
+/* log2(x) in 8.8 fixed point */
+static int cbr_log2_q8(u64 x)
+{
+	unsigned int shift = 0;
 
-	/* Bump size for intra frames and reduce for pred frames.
-	 * We are better off with good quality initial references.
-	 * Maintain average assuming 1 intra frame for 24 inter frames.
-	 */
-	if (encode->slice_type == V4L2_H264_SLICE_TYPE_I)
-		target = target * 6;
-	else
-		target = target * 80 / 100;
+	if (!x)
+		return 0;
 
-	/* Reduce target size on bitrate overshoot to calm things down. */
-	if (9 * stats->size_total > 10 * stats->count * target_nominal) {
-		pr_debug("  overshoot, reducing target size");
-		target = 80 * target / 100;
+	while (x > U32_MAX) {
+		x >>= 1;
+		shift++;
 	}
 
-	pr_debug("  final target: %lu bytes", target);
-
-	*size = target;
-
-	return 0;
+	return (int)(intlog2(x) >> 16) + shift * 256;
 }
 
-/* Intra bit size per macroblock estimation for QP starting at 10. */
-static unsigned int cbr_qp_initial_bits_mb[] = {
-	437, 411, 376, 355, 318, 303, 291, 271, 252, 238, 218, 204, 188, 171,
-	158, 149, 133, 124, 115, 105, 98, 92, 83, 77, 70, 63, 57, 52, 45, 39,
-	34, 31, 30, 29, 26, 23, 22, 20, 18, 17, 16, 15
-};
-
-static int cbr_qp_initial(struct v4l2_h264_enc_rc *rc, unsigned int *qp)
+static unsigned int cbr_fps(const struct v4l2_fract *timeperframe)
 {
-	struct v4l2_h264_enc_state *state = rc->state;
-	struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
-	unsigned long size_mb;
-	unsigned int qp_initial;
-	unsigned int threshold;
-	unsigned int i;
+	if (!timeperframe->numerator || !timeperframe->denominator)
+		return 30;
 
-	size_mb = 8 * rc->size / (state->width_mbs * state->height_mbs);
-
-	pr_debug("  initial estimate from %lu bits/mb", size_mb);
+	return max(DIV_ROUND_CLOSEST(timeperframe->denominator,
+				     timeperframe->numerator), 1U);
+}
 
-	for (i = 0; i < ARRAY_SIZE(cbr_qp_initial_bits_mb); i++) {
-		threshold = cbr_qp_initial_bits_mb[i];
+static u64 cbr_nominal(u32 bitrate, const struct v4l2_fract *timeperframe)
+{
+	if (!timeperframe->numerator || !timeperframe->denominator)
+		return max(bitrate / 30, 1U);
 
-		/* Expect inter frames to be 6 times smaller. */
-		if (encode->slice_type != V4L2_H264_SLICE_TYPE_I)
-			threshold = cbr_qp_initial_bits_mb[i] / 6;
+	return max_t(u64, div_u64((u64)bitrate * timeperframe->numerator,
+				  timeperframe->denominator), 1);
+}
 
-		if (size_mb >= threshold) {
-			qp_initial = 10 + i;
-			break;
-		}
-	}
+/* Planned size of an I or P picture of the current GOP */
+static u64 cbr_plan(const struct v4l2_h264_enc_rc_cbr *cbr, unsigned int fps,
+		    bool intra)
+{
+	unsigned int gop = cbr->gop;
+	u64 p;
 
-	/* Fallback to the lowest QP (last in the list). */
-	if (i == ARRAY_SIZE(cbr_qp_initial_bits_mb))
-		qp_initial = 10 + i - 1;
+	/* Until the second I picture, assume at least two seconds. */
+	if (!gop)
+		gop = max(cbr->since_i + 1, 2 * fps);
+	if (gop <= 1)
+		return cbr->nominal;
 
-	*qp = qp_initial;
+	p = div_u64(cbr->nominal * gop * 16, cbr->ratio + 16 * (gop - 1));
+	if (intra)
+		p = div_u64(p * cbr->ratio, 16);
 
-	return 0;
+	return max_t(u64, p, 1);
 }
 
 /*
- * This table gives the QP decrease/increase (coded as index, with QP = 0 in
- * the middle) for each size ratio (in base 1000).
- *
- * Intra is a bit agressive since we generally have few intra slices to adapt
- * to a scene change. The QP diff range goes from -4 to +4.
- */
-static unsigned long cbr_ratio_qp_diff_steps_intra[] = {
-	3000, 2000, 1500, 1250, 800, 666, 500, 333
-};
-
-/*
- * Inter is less agressive since we generally have more inter slices to adapt
- * to a scene change and want to give more stability to QP. The QP diff range
- * goes from -3 to +3.
+ * Start QP of an I picture from its size in bits per luma sample:
+ * 20.3 - 6 * log2(bits per sample), clipped to 15..45. This is the relation
+ * the Rockchip MPP library uses for its first picture at 1080p.
  */
-static unsigned long cbr_ratio_qp_diff_steps_inter[] = {
-	2000, 1500, 1250, 800, 666, 500
-};
-
-static int cbr_ratio_qp_diff(unsigned long ratio, unsigned char slice_type)
+static int cbr_qp_initial(u64 bits, unsigned int samples)
 {
-	unsigned long *steps;
-	unsigned int count;
-	unsigned int i;
-
-	if (slice_type == V4L2_H264_SLICE_TYPE_I) {
-		steps = cbr_ratio_qp_diff_steps_intra;
-		count = ARRAY_SIZE(cbr_ratio_qp_diff_steps_intra);
-	} else {
-		steps = cbr_ratio_qp_diff_steps_inter;
-		count = ARRAY_SIZE(cbr_ratio_qp_diff_steps_inter);
-	}
+	int qp_q8 = 20 * 256 + 77 +
+		    6 * (cbr_log2_q8(samples) - cbr_log2_q8(max_t(u64, bits, 1)));
 
-	WARN_ON(count % 2);
+	return clamp(DIV_ROUND_CLOSEST(qp_q8, 256), 15, 45);
+}
 
-	for (i = 0; i < count; i++)
-		if (ratio > steps[i])
-			return -1 * (int)count / 2 + i;
+static int cbr_init(struct v4l2_h264_enc_rc *rc)
+{
+	memset(&rc->cbr, 0, sizeof(rc->cbr));
 
-	return count / 2;
+	return 0;
 }
 
-static int cbr_estimate(struct v4l2_h264_enc_rc *rc, unsigned int *qp)
+static int cbr_measure(struct v4l2_h264_enc_rc *rc, unsigned long *size)
 {
 	struct v4l2_h264_enc_state *state = rc->state;
-	struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
-	struct v4l2_h264_enc_rc_stats *stats = &rc->stats;
-	struct v4l2_h264_enc_rc_stats_type *type;
-	unsigned int type_index[3];
-	unsigned int ratio_qp[3];
-	unsigned int penalty[3];
-	unsigned int penalty_total;
-	unsigned int weight[3];
-	unsigned int weight_total;
-	int ratio_qp_diff;
-	unsigned long diff;
-	unsigned long diff_closest;
-	unsigned int slot_closest;
-	unsigned int index;
-	unsigned int count;
-	unsigned int total;
-	unsigned long ratio;
-	unsigned int i;
-
-	if (!stats->count)
-		return cbr_qp_initial(rc, qp);
-
-	if (encode->slice_type == V4L2_H264_SLICE_TYPE_I)
-		type = &stats->intra;
-	else if (encode->slice_type == V4L2_H264_SLICE_TYPE_P)
-		type = &stats->pred;
-	else if (encode->slice_type == V4L2_H264_SLICE_TYPE_B)
-		type = &stats->bipred;
-	else
+	struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
+	struct v4l2_fract *timeperframe = &state->timeperframe;
+	bool intra = state->encode.slice_type == V4L2_H264_SLICE_TYPE_I;
+	unsigned int fps = cbr_fps(timeperframe);
+	u64 nominal = cbr_nominal(state->bitrate, timeperframe);
+	unsigned int samples = state->width_mbs * state->height_mbs * 256;
+	s64 target, expected = 0;
+	int qp;
+
+	if (!state->bitrate)
 		return -EINVAL;
 
-	pr_debug("+ v4l2-h264-enc-rc: cbr estimate");
-
-	if (!type->count)
-		return cbr_qp_initial(rc, qp);
-
-	for (i = 0; i < 3; i++) {
-		if (i == type->count)
-			break;
-
-		index = type->index;
-		if (index <= i)
-			index = type->count + index - (i + 1);
-		else
-			index -= i + 1;
-
-		/* Set initial penalty based on frame age. */
-		penalty[i] = 4 * i;
-		type_index[i] = index;
-
-		/* Track frame with size closest to target. */
-		diff = abs((long)rc->size - type->size[index]);
-
-		if (!i || diff < diff_closest) {
-			diff_closest = diff;
-			slot_closest = i;
-		}
+	if (!cbr->bitrate) {
+		cbr->ratio = CBR_RATIO_INIT;
+		cbr->qp_q8 = -1;
 	}
 
-	count = i;
-	penalty_total = 0;
-
-	for (i = 0; i < count; i++) {
-		/* Add penalty for frames with larger size difference. */
-		if (i != slot_closest)
-			penalty[i] += 10;
+	/* A new bitrate keeps the QP state and the measured statistics. */
+	if (cbr->bitrate != state->bitrate || cbr->nominal != nominal) {
+		cbr->bitrate = state->bitrate;
+		cbr->nominal = nominal;
+		cbr->buffer_bits = 0;
+	}
 
-		penalty_total += penalty[i];
+	if (intra) {
+		/* An I picture closes the GOP: its length is the IDR period. */
+		if (cbr->seen_i)
+			cbr->gop = cbr->since_i + 1;
+		cbr->since_i = 0;
+		cbr->seen_i = true;
+	}
 
-		index = type_index[i];
+	cbr->plan_bits = cbr_plan(cbr, fps, intra);
 
-		/* Calculate QP from target to observed size ratio. */
-		ratio = 1000UL * rc->size / type->size[index];
+	if (cbr->qp_q8 < 0) {
+		/* First picture: start QP from the bits per sample. */
+		qp = cbr_qp_initial(cbr_plan(cbr, fps, true), samples) +
+		     CBR_QP_DELTA_INTRA;
+		cbr->qp_q8 = clamp_t(int, qp, state->qp_min, state->qp_max) *
+			     256;
+	}
 
-		ratio_qp_diff = cbr_ratio_qp_diff(ratio, encode->slice_type);
+	/*
+	 * The buffer holds the coded minus the nominal sizes. Within a GOP the
+	 * plan expects it to go up by the excess of the I picture and back
+	 * down to zero by the savings of the P pictures until the next I
+	 * picture. Spread the difference to that over the next pictures.
+	 */
+	if (!intra && cbr->seen_i)
+		expected = (s64)cbr_plan(cbr, fps, true) - (s64)nominal +
+			   (s64)cbr->since_i *
+			   ((s64)cbr->plan_bits - (s64)nominal);
+
+	target = (s64)cbr->plan_bits -
+		 div_s64(cbr->buffer_bits - max_t(s64, expected, 0),
+			 CBR_WINDOW);
+	cbr->target_bits = clamp_t(s64, target,
+				   max_t(u64, cbr->plan_bits / 4, 8),
+				   cbr->plan_bits * 2);
+	cbr->intra = intra;
 
-		if ((int)type->qp[index] + ratio_qp_diff < 0)
-			ratio_qp[i] = 0;
-		else if ((int)type->qp[index] + ratio_qp_diff > 51)
-			ratio_qp[i] = 51;
-		else
-			ratio_qp[i] = type->qp[index] + ratio_qp_diff;
+	pr_debug("+ v4l2-h264-enc-rc: cbr measure");
+	pr_debug("  nominal: %llu bits, plan: %llu bits, target: %llu bits",
+		 cbr->nominal, cbr->plan_bits, cbr->target_bits);
+	pr_debug("  buffer: %lld bits, expected: %lld bits, gop: %u, ratio: %u",
+		 cbr->buffer_bits, expected, cbr->gop, cbr->ratio);
 
-		diff = abs((long)rc->size - type->size[index]);
+	*size = max_t(unsigned long, cbr->target_bits / 8, 1);
 
-		pr_debug("  - backlog %d size: %lu (diff: %lu), qp: %u, penalty %u, ratio qp: %u",
-			 -(i + 1), type->size[index], diff, type->qp[index],
-			 penalty[i], ratio_qp[i]);
-	}
+	return 0;
+}
 
-	total = 0;
-	weight_total = 0;
+static int cbr_estimate(struct v4l2_h264_enc_rc *rc, unsigned int *qp)
+{
+	struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
+	int value;
 
-	for (i = 0; i < count; i++) {
-		/* Weight each calculated QP using the penalty ratio. */
-		if (penalty_total)
-			weight[i] = 1000 * (penalty_total - penalty[i]) /
-				    penalty_total;
-		else
-			weight[i] = 1000;
+	value = DIV_ROUND_CLOSEST(cbr->qp_q8, 256);
+	if (cbr->intra)
+		value -= CBR_QP_DELTA_INTRA;
 
-		/* Give non-zero low weight to full penalty cases. */
-		if (weight[i] < 200)
-			weight[i] = 200;
+	*qp = clamp(value, 0, 51);
 
-		total += ratio_qp[i] * weight[i];
-		weight_total += weight[i];
+	return 0;
+}
 
-		pr_debug("  - backlog %d weight: %u", -(i + 1),
-			 weight[i]);
+static void cbr_update(struct v4l2_h264_enc_rc *rc, u64 bits)
+{
+	struct v4l2_h264_enc_state *state = rc->state;
+	struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
+	s64 limit = cbr->nominal * CBR_BUFFER_SECONDS *
+		    cbr_fps(&state->timeperframe);
+	unsigned int ratio;
+	int err, step;
+
+	if (!cbr->bitrate || !cbr->target_bits)
+		return;
+
+	cbr->buffer_bits = clamp_t(s64, cbr->buffer_bits + (s64)bits -
+				   (s64)cbr->nominal, -limit, limit);
+
+	/* QP error of the picture, 1/256 units */
+	err = 6 * (cbr_log2_q8(bits) - cbr_log2_q8(cbr->target_bits));
+
+	step = err / 8;
+	if (err > CBR_ERR_SLOW)
+		step += (err - CBR_ERR_SLOW) / 2;
+	else if (err < -CBR_ERR_SLOW)
+		step += (err + CBR_ERR_SLOW) / 2;
+	step = clamp(step, -CBR_STEP_DOWN_MAX, CBR_STEP_UP_MAX);
+
+	if (!cbr->intra) {
+		cbr->qp_q8 += step;
+		cbr->p_bits = cbr->p_bits ? (7 * cbr->p_bits + bits) / 8 : bits;
+		cbr->since_i++;
+
+		/* I/P size ratio, once a few P pictures were coded */
+		if (cbr->since_i == CBR_RATIO_PICS && cbr->i_bits) {
+			ratio = div64_u64(cbr->i_bits * 16, cbr->p_bits);
+			ratio = clamp(ratio, CBR_RATIO_MIN, CBR_RATIO_MAX);
+			cbr->ratio = (cbr->ratio + ratio + 1) / 2;
+		}
+	} else {
+		/*
+		 * The size of an I picture mostly reflects the I/P ratio
+		 * estimate: only all-intra streams steer the QP fully with it.
+		 */
+		cbr->i_bits = bits;
+		cbr->qp_q8 += cbr->gop == 1 ? step : step / 4;
 	}
 
-	*qp = total / weight_total;
+	cbr->qp_q8 = clamp_t(int, cbr->qp_q8, state->qp_min * 256,
+			     state->qp_max * 256);
 
-	return 0;
+	pr_debug("+ v4l2-h264-enc-rc: cbr update");
+	pr_debug("  size: %llu bits, target: %llu bits, qp error: %d/256, qp: %d/256",
+		 bits, cbr->target_bits, err, cbr->qp_q8);
 }
 
 static int cbr_complete(struct v4l2_h264_enc_rc *rc, unsigned long bytesused)
 {
+	cbr_update(rc, (u64)bytesused * 8);
+
 	return v4l2_h264_enc_rc_stats_collect(rc, bytesused);
 }
 
 static const struct v4l2_h264_enc_rc_mode cbr_mode = {
 	.name		= "constant bitrate",
 	.type		= V4L2_H264_ENC_RC_TYPE_CBR,
+	.init		= cbr_init,
 	.measure	= cbr_measure,
 	.estimate	= cbr_estimate,
 	.complete	= cbr_complete,
diff --git a/include/media/v4l2-h264-enc-rc.h b/include/media/v4l2-h264-enc-rc.h
index 8b453c3a212e..72d78a594e40 100644
--- a/include/media/v4l2-h264-enc-rc.h
+++ b/include/media/v4l2-h264-enc-rc.h
@@ -8,6 +8,7 @@
 #ifndef _MEDIA_V4L2_H264_ENC_RC_H
 #define _MEDIA_V4L2_H264_ENC_RC_H
 
+#include <linux/types.h>
 #include <linux/v4l2-controls.h>
 #include <media/v4l2-h264.h>
 
@@ -82,6 +83,38 @@ struct v4l2_h264_enc_rc_stats {
 	unsigned int index;
 };
 
+/**
+ * struct v4l2_h264_enc_rc_cbr - constant bitrate mode state
+ * @bitrate: bitrate the state was set up for, 0 before the first picture
+ * @nominal: nominal picture size in bits (bitrate times frame interval)
+ * @qp_q8: QP of the P pictures in 1/256 units, -1 before the first picture
+ * @ratio: I/P picture size ratio in 1/16 units
+ * @gop: measured IDR period in pictures, 0 until two I pictures were seen
+ * @since_i: pictures coded since the last I picture
+ * @seen_i: an I picture was coded
+ * @intra: the current picture is an I picture
+ * @p_bits: running mean of the P picture sizes
+ * @i_bits: size of the last I picture
+ * @buffer_bits: accumulated difference between the coded and nominal sizes
+ * @plan_bits: planned size of the current picture
+ * @target_bits: size target of the current picture
+ */
+struct v4l2_h264_enc_rc_cbr {
+	u32 bitrate;
+	u64 nominal;
+	int qp_q8;
+	unsigned int ratio;
+	unsigned int gop;
+	unsigned int since_i;
+	bool seen_i;
+	bool intra;
+	u64 p_bits;
+	u64 i_bits;
+	s64 buffer_bits;
+	u64 plan_bits;
+	u64 target_bits;
+};
+
 struct v4l2_h264_enc_rc {
 	const struct v4l2_h264_enc_rc_ops *ops;
 	void *private_data;
@@ -91,6 +124,7 @@ struct v4l2_h264_enc_rc {
 	void *mode_data;
 
 	struct v4l2_h264_enc_rc_stats stats;
+	struct v4l2_h264_enc_rc_cbr cbr;
 	unsigned long size;
 	unsigned int qp;
 };

-- 
2.55.0



  parent reply	other threads:[~2026-11-10 10:05 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-10 10:05 [PATCH RFC 00/11] media: h264: Stateless encode core follow-ups: parameter sets, POC type 2, CBR and VBR Arun Iyengar
2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 01/11] media: h264: Only write the SPS and PPS of a stateless encode when they change Arun Iyengar
2026-11-10 10:05   ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 02/11] media: h264: Support picture order count type 2 in stateless encode Arun Iyengar
2026-11-10 10:05   ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 03/11] media: h264: Write the stateless encode SPS and PPS with nal_ref_idc 3 Arun Iyengar
2026-11-10 10:05   ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` Arun Iyengar [this message]
2026-11-10 10:05   ` [PATCH RFC 04/11] media: v4l2-h264-enc-rc: Rework the constant bitrate mode Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 05/11] media: v4l2-core: Move the H.264 encode CBR rate control to a helper Arun Iyengar
2026-11-10 10:05   ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 06/11] media: v4l2-core: Bound the I picture size in the encode CBR rate control Arun Iyengar
2026-11-10 10:05   ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 07/11] media: v4l2-core: Halve the I picture QP increase of the encode CBR Arun Iyengar
2026-11-10 10:05   ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 08/11] media: v4l2-core: Re-encode oversized pictures in " Arun Iyengar
2026-11-10 10:05   ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 09/11] media: v4l2-core: Fix all-intra streams in the encode CBR rate control Arun Iyengar
2026-11-10 10:05   ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 10/11] media: v4l2-core: Add a peak bitrate to the encode " Arun Iyengar
2026-11-10 10:05   ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 11/11] media: h264: Add a VBR mode to the stateless " Arun Iyengar
2026-11-10 10:05   ` Arun Iyengar via B4 Relay
2026-11-11  1:20 ` [PATCH RFC 00/11] media: h264: Stateless encode core follow-ups: parameter sets, POC type 2, CBR and VBR Nicolas Dufresne
2026-11-12 16:20   ` Paul Kocialkowski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261110-h264-encode-fixes-v1-4-7a64486df499@gmail.com \
    --to=aiyengar1990@gmail.com \
    --cc=hverkuil@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-media@vger.kernel.org \
    --cc=m.felsch@pengutronix.de \
    --cc=m.tretter@pengutronix.de \
    --cc=mchehab@kernel.org \
    --cc=nicolas.dufresne@collabora.com \
    --cc=paulk@sys-base.io \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.