Devicetree
 help / color / mirror / Atom feed
From: sashiko-bot@kernel.org
To: "Sofus Forstreuter" <sofus.c@icloud.com>
Cc: devicetree@vger.kernel.org, robh@kernel.org, conor+dt@kernel.org,
	media-ci@linuxtv.org
Subject: Re: [PATCH v3 09/17] media: apple: avd: add hevc support
Date: Tue, 20 Oct 2026 11:40:30 +0000	[thread overview]
Message-ID: <sashiko-outbox-178901@kernel.org> (raw)
In-Reply-To: <20261020-avd-v3-9-85cbaa2125d8@icloud.com>

Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
- [Critical] media: apple: avd: OOB read of PPS tile arrays in compute_tiles_non_uniform()
- [Critical] media: apple: avd: tile size sum underflow in compute_tiles_non_uniform()
- [Critical] media: apple: avd: entry_point_offsets sum overflow in avd_hevc_run_preamble()
- [High] media: apple: avd: wrong plane length field in stream_slice_mv()
- [High] media: apple: avd: OOB stack read of row_bd in submit_slice_segment()
- [High] media: apple: avd: missing SPS block size and dimension validation
--

commit 9e1098e858acf03c63d37b3cb57614f4681a182e
Author: Sofus Forstreuter <sofus.c@icloud.com>

media: apple: avd: add hevc support

This patch adds stateless V4L2 HEVC decoding support to the Apple Video
Decoder (AVD) driver.

> diff --git a/drivers/media/platform/apple/avd/avd-hevc.c b/drivers/media/platform/apple/avd/avd-hevc.c
> new file mode 100644
> index 0000000000000..36ae24c640a10
> --- /dev/null
> +++ b/drivers/media/platform/apple/avd/avd-hevc.c

[ ... ]

> +#define HEVC_SCL_DIMS	0x127ffff
> +
> +/* this is for level 7.1 */
> +#define HEVC_MAX_TILE_COLS	40
> +#define HEVC_MAX_TILE_ROWS	44

[ ... ]

> +static void stream_slice_mv(struct avd_ctx *ctx, struct avd_hevc_run *run,
> +			    const struct v4l2_ctrl_hevc_slice_params *sl,
> +			    bool is_first)
> +{

[ ... ]

> +	if (ref_valid) {
> +		dma_addr_t mv_color_addr =
> +			vb2_dma_contig_plane_dma_addr(&ref->base.vb.vb2_buf,
> +						      0) +
> +			(ref->base.vb.planes[0].length -
> +			 mv_color_size(fmt_width(ctx), fmt_height(ctx)));
> +		pusha(mv_color_addr, "mv_colo", 0);
> +	}
> +}

[Severity: High]
Should stream_slice_mv() use ref->base.vb.vb2_buf.planes[0].length instead
of ref->base.vb.planes[0].length here?

In avd_hevc_run_preamble(), the destination buffer's mv_color address is
computed from vb2_buf.planes[0].length:

	dst_len = run->base.bufs.dst->vb2_buf.planes[0].length;
	mv_color_len = mv_color_size(fmt_width(ctx), fmt_height(ctx));
	run->addresses.mv_color = run->base.y_out + (dst_len - mv_color_len);

When capture buffers are queued using VB2_MEMORY_DMABUF with
v4l2_plane.length set to 0, __prepare_dmabuf() in videobuf2-core.c sets
vb2_buf.planes[0].length = dbuf->size, while leaving
vb2_v4l2_buffer.planes[0].length as 0.

When decoding a P or B slice with
V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_TEMPORAL_MVP_ENABLED and a non-intra
collocated reference frame, subtracting mv_color_size() from
ref->base.vb.planes[0].length (0) underflows the 32-bit unsigned value and
programs the hardware with an out-of-bounds DMA address.

Also, in set_slice(), coded_in and size are computed from sl->bit_size,
sl->data_byte_offset, and entry_point_offsets without bounds checking
against the source buffer plane size.

> +
> +static void set_slice(struct avd_ctx *ctx, struct avd_hevc_run *run,
> +		      const struct v4l2_ctrl_hevc_slice_params *sl, u32 size,
> +		      u32 offset, u32 flags)
> +{
> +	dma_addr_t coded_in =
> +		run->base.coded_in + offset + sl->data_byte_offset;
> +	push(AVD_OP_CODED_DATA | flags | AVD_OP_CODED_IN_HI(coded_in),
> +	     "cm3_cmd_set_coded_slice");
> +	push(AVD_OP_CODED_IN_LO(coded_in), "slc_bd8_slice_addr");
> +	push(size, "slc_bdc_slice_size");
> +}
> +
> +static int submit_slice_segment(struct avd_ctx *ctx, struct avd_hevc_run *run,
> +				const struct v4l2_ctrl_hevc_slice_params *sl,
> +				int row, int col, u32 col_bd[23],
> +				u32 row_bd[23], u32 pic_in_cts_width,
> +				u32 pic_in_cts_height, bool first_slice,
> +				bool hflip, bool vflip, u32 coded_flags,
> +				u32 last_tile_block)
> +{

[ ... ]

> +	} else {
> +		tile_boundary = AVD_OP_SL_LOC_Y(row_bd[row]) |
> +				AVD_OP_SL_LOC_X(col_bd[col]);
> +	}
> +
> +	if (coded_flags & NEW_TILE_ID) {
> +		push(AVD_OP_SL_DIM_START |
> +			     (coded_flags & NEW_SLICE ? tile_block :
> +							tile_boundary),
> +		     "cm3_cmd_set_ctb_xy");
> +
> +		/* tile boundary end */
> +		if (pps->flags & V4L2_HEVC_PPS_FLAG_TILES_ENABLED)
> +			push(AVD_SL_DIM_END_ROW((hflip ? 4 : 0) |
> +						(vflip ? 8 : 0)) |
> +				     AVD_SL_DIM_END_COL(col) |
> +				     AVD_SL_DIM_END_Y(row_bd[row + 1] - 1) |
> +				     AVD_SL_DIM_END_X(col_bd[col + 1] - 1),
> +			     "cm3_set_ctb_xy");

[Severity: High]
Can submit_slice_segment() read past the end of row_bd[] on the stack?

In stream_slices(), pos is incremented for every segment where
(slice_flag & NEW_TILE_ID) is set, and row is computed as:

	row = pos / num_cols;
	col = pos % num_cols;

Across up to 1800 slices and 1760 entry point offsets, pos can reach up to
3560 while num_cols (pps->num_tile_columns_minus1 + 1) can be 1. Because
row is not checked against pps->num_tile_rows_minus1 or HEVC_MAX_TILE_ROWS,
submit_slice_segment() can index row_bd[row] and row_bd[row + 1] past the
end of run.tile_info.row_bd[HEVC_MAX_TILE_ROWS + 1].

Note also that submit_slice_segment() declares u32 col_bd[23] and
u32 row_bd[23], which do not match HEVC_MAX_TILE_COLS + 1 and
HEVC_MAX_TILE_ROWS + 1.

[ ... ]

> +static void compute_tiles_non_uniform(struct avd_hevc_run *run,
> +				      u16 log2_min_cb_size, u16 width,
> +				      u16 height, u32 pic_in_cts_width,
> +				      u32 pic_in_cts_height, u16 *column_width,
> +				      u16 *row_height)
> +{
> +	const struct v4l2_ctrl_hevc_pps *pps = run->pps;
> +	u32 sum = 0;
> +	int i;
> +
> +	for (i = 0; i < pps->num_tile_columns_minus1; i++) {
> +		column_width[i] = MIN(pps->column_width_minus1[i] + 1,
> +				      pic_in_cts_width);
> +		sum += column_width[i];
> +	}
> +	column_width[i] = pic_in_cts_width - sum;
> +
> +	sum = 0;
> +	for (i = 0; i < pps->num_tile_rows_minus1; i++) {
> +		row_height[i] = MIN(pps->row_height_minus1[i] + 1,
> +				    pic_in_cts_height);
> +		sum += row_height[i];
> +	}
> +	row_height[i] = pic_in_cts_height - sum;
> +}

[Severity: Critical]
Can compute_tiles_non_uniform() read out of bounds from
pps->column_width_minus1[] and pps->row_height_minus1[]?

In include/uapi/linux/v4l2-controls.h, struct v4l2_ctrl_hevc_pps defines:

	__u8 column_width_minus1[20];
	__u8 row_height_minus1[22];

However, HEVC_MAX_TILE_COLS is 40 and HEVC_MAX_TILE_ROWS is 44, so
avd_hevc_validate_pps() allows pps->num_tile_columns_minus1 up to 39 and
pps->num_tile_rows_minus1 up to 43. When V4L2_HEVC_PPS_FLAG_TILES_ENABLED
is set without V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING, these loops read
pps->column_width_minus1[i] up to index 38 and pps->row_height_minus1[i]
up to index 42, reading past both arrays and the end of struct
v4l2_ctrl_hevc_pps.

[Severity: Critical]
In addition, what happens in compute_tiles_non_uniform() if the cumulative
sum of column widths or row heights exceeds pic_in_cts_width or
pic_in_cts_height?

Clamping each individual column or row with MIN() does not prevent sum from
exceeding pic_in_cts_width or pic_in_cts_height across multiple columns or
rows. When sum > pic_in_cts_width (or sum > pic_in_cts_height), the final
assignment:

	column_width[i] = pic_in_cts_width - sum;
	...
	row_height[i] = pic_in_cts_height - sum;

underflows in u16 to a large value (up to 65535). compute_bd() then
computes col_bd and row_bd boundaries exceeding pic_in_ctbs_width and
pic_in_ctbs_height, causing compute_tile_ids() to read
ctb_addr_rs_to_ts[y * pic_in_ctbs_width + x] and write tile_ids[...] out of
bounds on the heap:

> +static void compute_tile_ids(struct avd_hevc_run *run, u32 pic_in_ctbs_width,
> +			     u32 *col_bd, u32 *row_bd, u32 *ctb_addr_rs_to_ts,
> +			     u32 *tile_ids)
> +{
> +	const struct v4l2_ctrl_hevc_pps *pps = run->pps;
> +	int j, i, y, x;
> +	u32 tile_idx;
> +
> +	for (j = 0, tile_idx = 0; j <= pps->num_tile_rows_minus1; j++)
> +		for (i = 0; i <= pps->num_tile_columns_minus1; i++, tile_idx++)
> +			for (y = row_bd[j]; y < row_bd[j + 1]; y++)
> +				for (x = col_bd[i]; x < col_bd[i + 1]; x++)
> +					tile_ids[ctb_addr_rs_to_ts
> +							 [y * pic_in_ctbs_width +
> +							  x]] = tile_idx;
> +}

[ ... ]

> +static int avd_hevc_validate_sps(struct avd_ctx *ctx,
> +				 const struct v4l2_ctrl_hevc_sps *sps)
> +{
> +	if (sps->pic_width_in_luma_samples > ctx->coded_fmt.fmt.pix_mp.width ||
> +	    sps->pic_height_in_luma_samples > ctx->coded_fmt.fmt.pix_mp.height)
> +		return -EINVAL;
> +
> +	if (sps->bit_depth_chroma_minus8 != sps->bit_depth_luma_minus8)
> +		return -EINVAL;
> +
> +	if (sps->bit_depth_luma_minus8 > 2)
> +		/* not implemented */
> +		return -EINVAL;
> +
> +	return 0;
> +}

[Severity: High]
Does avd_hevc_validate_sps() also need to validate that picture dimensions
are non-zero and bound log2_min_luma_coding_block_size_minus3 and
log2_diff_max_min_luma_coding_block_size?

In avd_hevc_run(), avd_hevc_compute_tiles() runs before
avd_hevc_alloc_scratch() (where the !max_cu_width check is located) and
executes:

	log2_min_cb_size = sps->log2_min_luma_coding_block_size_minus3 + 3;

	max_cu_width = 1 << (sps->log2_diff_max_min_luma_coding_block_size +
			     log2_min_cb_size);
	pic_in_ctbs_width = (width + max_cu_width - 1) / max_cu_width;
	pic_in_ctbs_height = (height + max_cu_width - 1) / max_cu_width;
	pic_in_ctbs_size = pic_in_ctbs_height * pic_in_ctbs_width;

If the shift exponent is >= 31, 1 << ... triggers signed 32-bit shift
undefined behavior; if it is >= 32 (or wraps to 0), dividing by max_cu_width
divides by zero, and if it is 31, max_cu_width is 0x80000000, making
pic_in_ctbs_width = 0 and causing division by zero in compute_rs_to_ts() or
submit_slice_segment(), as well as 32-bit overflow in
((max_col * max_cu_width) / 4) * bit_depth in avd_hevc_alloc_scratch().

Similarly, if width or height is 0 (for example, if streaming starts before
setting V4L2_CID_STATELESS_HEVC_SPS), pic_in_ctbs_size is 0 and
kzalloc_objs() returns ZERO_SIZE_PTR, which is then dereferenced. Also,
because avd_hevc_validate_sps() is only called from avd_hevc_try_ctrl(),
decreasing the coded format dimensions via VIDIOC_S_FMT after setting the
SPS control leaves stale SPS dimensions larger than the format.

> +
> +static int avd_hevc_validate_pps(struct avd_ctx *ctx,
> +				 const struct v4l2_ctrl_hevc_pps *pps)
> +{
> +	if (pps->num_tile_columns_minus1 + 1 > HEVC_MAX_TILE_COLS ||
> +	    pps->num_tile_rows_minus1 + 1 > HEVC_MAX_TILE_ROWS)
> +		return -EINVAL;
> +
> +	return 0;
> +}

[ ... ]

> +static int avd_hevc_run_preamble(struct avd_ctx *ctx, struct avd_hevc_run *run)
> +{

[ ... ]

> +	for (i = 0; i < run->num_slices; i++)
> +		sum += (*run->sl)[i].num_entry_point_offsets;
> +
> +	if (sum && sum != run->num_entry_point_offsets)
> +		return -EINVAL;

[Severity: Critical]
Can sum overflow when accumulating (*run->sl)[i].num_entry_point_offsets in
avd_hevc_run_preamble()?

(*run->sl)[i].num_entry_point_offsets is an unvalidated __u32 accumulated
into a 32-bit signed int sum. If two or more slices have
num_entry_point_offsets values whose 32-bit sum wraps to 0 or to
run->num_entry_point_offsets, this check passes while avd_init_job() only
allocates run.num_slices + run.num_entry_point_offsets + 1 segments.

In stream_slices():

	to = tiles_enabled ? sl->num_entry_point_offsets + 1 : 1;

	for (i = 0; i < to; i++) {
		...
		if (tiles_enabled && to > 1) {
			if (i < sl->num_entry_point_offsets) {
				size = (*run->entry_point_offsets)
					       [entry_point_idx++];

When tiles_enabled is true, this loop can then read
(*run->entry_point_offsets)[entry_point_idx++] out of bounds on the heap
and write past the end of ctx->job.segments via push() and
avd_end_segment(). Moreover, if sl->num_entry_point_offsets >= 0x7fffffff,
int to = sl->num_entry_point_offsets + 1 overflows to <= 0, skipping the
inner loop and its sl->slice_segment_addr >= pic_in_ctbs_size check for
slice s, so slice s + 1 indexes
tile_info->ctb_addr_rs_to_ts[(*run->sl)[s].slice_segment_addr] out of
bounds.

> +
> +	dst_len = run->base.bufs.dst->vb2_buf.planes[0].length;
> +
> +	mv_color_len = mv_color_size(fmt_width(ctx), fmt_height(ctx));
> +
> +	run->addresses.mv_color = run->base.y_out + (dst_len - mv_color_len);
> +	return 0;
> +}

[ ... ]

> +static int avd_hevc_run(struct avd_ctx *ctx)
> +{
> +	struct avd_hevc_run run = {};
> +	struct avd_decoded_buffer *dst;
> +	int ret;
> +
> +	ret = avd_hevc_run_preamble(ctx, &run);
> +	if (ret)
> +		goto postamble;
> +
> +	dst = vb2_to_avd_decoded_buf(&run.base.bufs.dst->vb2_buf);
> +	update_dec_buf_info(dst, &(*run.sl)[0]);
> +
> +	ret = avd_hevc_compute_tiles(ctx, &run);
> +	if (ret)
> +		goto done;
> +
> +	ret = avd_hevc_alloc_scratch(ctx, &run);
> +	if (ret)
> +		goto done;
> +
> +	ret = avd_init_job(ctx, AVD_CODEC_HEVC,
> +			   run.num_slices + run.num_entry_point_offsets + 1);
> +	if (ret)
> +		goto done;

[ ... ]

> +	return ret;
> +}

[ ... ]

> diff --git a/drivers/media/platform/apple/avd/avd-v4l2.c b/drivers/media/platform/apple/avd/avd-v4l2.c
> index 0f83a17f16fae..34db42e008332 100644
> --- a/drivers/media/platform/apple/avd/avd-v4l2.c
> +++ b/drivers/media/platform/apple/avd/avd-v4l2.c

[ ... ]

> +	{
> +		.cfg.id = V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS,
> +		.cfg.flags = V4L2_CTRL_FLAG_DYNAMIC_ARRAY,
> +		.cfg.dims = { 1760 },
> +		.cfg.max = 0xffffffff,
> +		.cfg.step = 1,
> +	},

[ ... ]

> diff --git a/drivers/media/platform/apple/avd/avd.h b/drivers/media/platform/apple/avd/avd.h
> index bd6177ab1eb25..828d25f85902e 100644
> --- a/drivers/media/platform/apple/avd/avd.h
> +++ b/drivers/media/platform/apple/avd/avd.h

[ ... ]

> +struct avd_hevc_decoded_buffer_info {
> +	bool is_intra;
> +};

[ ... ]

>  extern const struct v4l2_ctrl_ops avd_ctrl_ops;

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261020-avd-v3-0-85cbaa2125d8@icloud.com?part=9

  reply	other threads:[~2026-10-20 11:40 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-20  9:37 [PATCH v3 00/17] media: apple: add avd driver Sofus Forstreuter
2026-10-20  9:37 ` [PATCH v3 01/17] dt-bindings: media: add apple,avd Sofus Forstreuter
2026-10-20  9:37 ` [PATCH v3 02/17] media: v4l2: Add P210 pixel format Sofus Forstreuter
2026-10-20  9:37 ` [PATCH v3 03/17] media: v4l2: Add Apple interchange pixel formats Sofus Forstreuter
2026-10-20  9:56   ` sashiko-bot
2026-10-20  9:37 ` [PATCH v3 04/17] media: v4l2-ctrls: validate av1 tile info Sofus Forstreuter
2026-10-20 10:10   ` sashiko-bot
2026-10-20  9:37 ` [PATCH v3 05/17] media: v4l2-ctrls: validate vp9 tile_rows_log2 Sofus Forstreuter
2026-10-20  9:37 ` [PATCH v3 06/17] media: apple: add avd driver Sofus Forstreuter
2026-10-20 10:34   ` sashiko-bot
2026-10-20  9:37 ` [PATCH v3 07/17] media: apple: avd: add h264 support Sofus Forstreuter
2026-10-20 10:55   ` sashiko-bot
2026-10-20  9:37 ` [PATCH v3 08/17] media: apple: avd: add vp9 support Sofus Forstreuter
2026-10-20 11:22   ` sashiko-bot
2026-10-20  9:37 ` [PATCH v3 09/17] media: apple: avd: add hevc support Sofus Forstreuter
2026-10-20 11:40   ` sashiko-bot [this message]
2026-10-20  9:37 ` [PATCH v3 10/17] media: apple: avd: add av1 support Sofus Forstreuter
2026-10-20 12:50   ` sashiko-bot
2026-10-20  9:37 ` [PATCH v3 11/17] arm64: dts: apple: t8103: add avd nodes Sofus Forstreuter
2026-10-20 13:27   ` sashiko-bot
2026-10-20  9:37 ` [PATCH v3 12/17] arm64: dts: apple: t8112: " Sofus Forstreuter
2026-10-20  9:37 ` [PATCH v3 13/17] arm64: dts: apple: t8122: " Sofus Forstreuter
2026-10-20 14:02   ` sashiko-bot
2026-10-20  9:37 ` [PATCH v3 14/17] arm64: dts: apple: t600x: " Sofus Forstreuter
2026-10-20  9:37 ` [PATCH v3 15/17] arm64: dts: apple: t602x: " Sofus Forstreuter
2026-10-20  9:37 ` [PATCH v3 16/17] arm64: dts: apple: t6030: " Sofus Forstreuter
2026-10-20 14:40   ` sashiko-bot
2026-10-20  9:37 ` [PATCH v3 17/17] arm64: dts: apple: t6031: " Sofus Forstreuter
2026-10-20 15:02   ` sashiko-bot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=sashiko-outbox-178901@kernel.org \
    --to=sashiko-bot@kernel.org \
    --cc=conor+dt@kernel.org \
    --cc=devicetree@vger.kernel.org \
    --cc=media-ci@linuxtv.org \
    --cc=robh@kernel.org \
    --cc=sashiko-reviews@lists.linux.dev \
    --cc=sofus.c@icloud.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox