From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5BD4F3A75B6 for ; Wed, 13 May 2026 18:19:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.181 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778696375; cv=none; b=recoGtpFvEL9q//aTwDvplehoE53Z0wPyrCNg5FYSnB2CusRBNKVP4+XMFP5QWHjCvQiu3WHhiaC8aR84K/zPQVTbZNihMtZ1Of2y+ID2uk8L4HzjtJ0oNvMl1oeFXZAy+CrFnww9QxS6x/fM47NxiArEqWOT4U6mSIckAz6PpE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778696375; c=relaxed/simple; bh=kjNupAK/J8F0Ik6qBviODgZBc/q9fZtKcmtllAC9dz0=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=Hql9meSYqBC/POxG9I7cHXRYO99qqylYehGf5hKmKP1DWvUSHiGbDhJ4Co+I2svqFRlScHwINVS+E6xBuVv1LAhNzbsf/+w/EcdnBwSseg7li9gx9kmOCcyXJqT3bZeTTQTvl4cmZehZgtFTo042lK6yP7MzhIACECWkBdC2QpU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Nb/ls1vM; arc=none smtp.client-ip=209.85.222.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Nb/ls1vM" Received: by mail-qk1-f181.google.com with SMTP id af79cd13be357-90b2fcf90a0so456258485a.1 for ; Wed, 13 May 2026 11:19:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778696372; x=1779301172; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=KPjNzhHK50cDvgdayyNCGaEkmYz/Q2VYJa8N/oM2vHs=; b=Nb/ls1vM3sTdg6EXEvU9aDMTlSiZSW5BtCW6Viy7/zcdTZ+ZJpQk+6IQroCavW06hS CbhTJhWORgvy6ktqWenkD/RcaBFFa/VGyHja/gT5d4yQxgh+VcmWS9cIO8vpsPPvoZl4 ++3qY6cNa43w8J+3SrvwdF1KPkWV/+/9m9qz/TZZdol1LHu4PTh5Um7sVqna7lFgPCMF wMm+yHHn8rMC4PIlIgWP7DLP3GV6hnCVdewA06N9oy0pylsFAp4WeRoqSdNVPAusU9od SGwxL8LI5UxxkwbQkZzGOPZx5xSO51LTguxiXqTY3r1vLKixNV3VvMExXtEsXUorbYaH GrFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778696372; x=1779301172; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=KPjNzhHK50cDvgdayyNCGaEkmYz/Q2VYJa8N/oM2vHs=; b=FaU0UX2subx9r2LcXFAC9f6viRZ5VP4HMmNfdvh94PM5vWbxNk/qwGWMs7sYvzqE/E NbHp29m2O//V/yzVFbUFHElmYUmsVJo4y8mQstnoaBYqdWOqIpR4xisAvyVJY99lbR9o 149jt14giQc+uWsRUJwswn8Vtcl6j5fEDYQP50ND/37saaoPRhQtGJ8lRcJzOQ7cXqOD 5UYE5yLn00thjqZiE43o+3nPCAuCdyMPAB1F5IAyOH9/5TpYjORm/e+twee+kI+nwKVm rduBcD/+A3z8EJhnPFl4INhbQUGqs/r/5dcT9qlJoYmkmOurzq8IhHjSK7pJSTM0/2oX Ac6A== X-Forwarded-Encrypted: i=1; AFNElJ/3vkkzdfhsXg9iiHEs/+2w87JO9fDfgi9eMC3NTYjiBUc6gS730k6ZM2u+VysMuhPT8cuHBtNFc9h5oFU=@vger.kernel.org X-Gm-Message-State: AOJu0Yy41vOi42X/8CArPwJJbOUigkaa/7WIGDagwsrom42svJNROz4B J8/wkKgU62I/Yqmj6tEmn26GcmFYJlcxE43doz4rEepfmZoeCbrzYI7d X-Gm-Gg: Acq92OEEqtWJTkSWFPuqzRV6fDQ0gRqannUGrCsg5GGkdDGK4w/pzU76I9dB3vg6CdL lRg9Mhz2qlwa5bt7NKcwanDvu8MlY7rHW8VWBvdCzRj6ogyEKwdSE9hWr0ZhkMdu4EA36dj/L9w P0ZxdaTIxRU6hfWzTCvkK50uBS9X1lrPFP59rclwuRp2Y/bmPq+06Hu0LekeWe0mbZl8fvnvnBx wnG8nd5VeLw6JehcSBur45mQgGkFt4efFpuy1W7yCuX7gKj1OofHqSjBp+UHXvDFzcR+SVdhe0v wWOQqHkiFgNYrOteI9J/C000qdpfnznGMX5ENcyS4qG2f8bWJL5cvT+a1bx9zbgiKg30Dv+tTLy Dx6DZISZXZrjg5NX7KsaJaG95G/VPoGdFZaYOlb3TY78d0Vlt2yqC7KC8Z25fBpXNpMpjG3nVZy SDB4KpdaaTry2QA/9wYKBHqKsgErzjcr+ZC3zMWZsWJouoyR6Qq98IFBQVMGRG/jzEP4HxYT+Mi FaJBOcZQjPB54NpM3p5PH1Vely6HGrSProNw9A/bmxJik7QpNgjjw== X-Received: by 2002:a05:620a:8886:b0:910:1c85:4adb with SMTP id af79cd13be357-910b110235bmr62071185a.37.1778696372150; Wed, 13 May 2026 11:19:32 -0700 (PDT) Received: from server0.tail6e7dd.ts.net (c-68-48-65-54.hsd1.mi.comcast.net. [68.48.65.54]) by smtp.gmail.com with ESMTPSA id af79cd13be357-910bd02f12dsm27871485a.40.2026.05.13.11.19.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 May 2026 11:19:31 -0700 (PDT) From: Michael Bommarito To: Detlev Casanova , Ezequiel Garcia , Mauro Carvalho Chehab , Heiko Stuebner , linux-media@vger.kernel.org Cc: linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH] media: rkvdec: hevc: cap EXT SPS RPS control counts before descriptor assembly Date: Wed, 13 May 2026 14:19:22 -0400 Message-ID: <20260513181922.2075438-1-michael.bommarito@gmail.com> X-Mailer: git-send-email 2.53.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS and V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS are registered as dynamic-size controls with a per-control element cap of 65. The V4L2 control core enforces only the payload-element cap. It does not bound the spec-derived count fields that the rkvdec HEVC helper later uses to walk fixed hardware descriptor tables and temporary helper arrays: - struct rkvdec_rps::refs[32] - struct rkvdec_rps::short_term_ref_sets[64] - struct calculated_rps_st_set::delta_poc_s0[16] / delta_poc_s1[16] A userspace V4L2 client that can open the Rockchip RKVDEC m2m decoder node may submit SPS/RPS controls whose counts exceed those capacities or whose prediction reference index underflows. rkvdec_hevc_assemble_hw_rps() then walks past the descriptor table or temporary-array bounds. KASAN under a small KUnit harness wrapping the real helper reports slab-out-of-bounds in all of: - num_short_term_ref_pic_sets = 65 (write past short_term_ref_sets[64]) - num_long_term_ref_pics_sps = 33 (write past refs[32], intra-struct) - ext_sps_st_rps[i].num_negative_pics or num_positive_pics > 16 (write past delta_poc_s0[16] inside calculated_rps_st_set) - INTER_REF_PIC_SET_PRED with delta_idx_minus1 + 1 > i (u8 ref_rps_idx underflow then OOB read on calculated_rps_st_sets) Validate the SPS/RPS counts before calling the assembly helpers. The cap values match both the HEVC spec ranges (num_short_term_ref_pic_sets <= 64, num_long_term_ref_pics_sps <= 32) and the fixed driver descriptor and helper-array capacities. Reject controls whose counts exceed those, and reject prediction entries whose reference index would underflow. Fixes: c9a59dc2acc7 ("media: rkvdec: Add HEVC support for the VDPU381 variant") Signed-off-by: Michael Bommarito Assisted-by: Claude:claude-opus-4-7 --- I don't have a RK3588 / RK3576 board to confirm this through a real /dev/videoN request path yet, but was convinced enough by: 1. Static reach: registration of EXT_SPS_ST_RPS / EXT_SPS_LT_RPS with .dims = { 65 } at drivers/media/platform/rockchip/rkvdec/rkvdec.c :239-287, the SPS-count-driven loops in rkvdec-hevc-common.c :213-225 and :228-251, and v4l2-ctrls-core.c :1213-1277, which validates only EXT RPS flags and not the spec-derived count fields. 2. A KUnit harness (separate, not in this patch) that allocates one struct rkvdec_rps + a single calculated_rps_st_sets element via kunit_kzalloc / kzalloc and calls the real rkvdec_hevc_assemble_hw_rps() helper. Under UML + KASAN_GENERIC with the kasan_multi_shot boot param, on a stock tree it produces these reports: BUG: KASAN: slab-out-of-bounds in rkvdec_hevc_assemble_hw_rps+0xb0c/0x1080 (num_short_term_ref_pic_sets = 65, write of size 36 0 bytes past the rps allocation) BUG: KASAN: slab-out-of-bounds in rkvdec_hevc_assemble_hw_rps (num_negative_pics = 64, write past the single-element kzalloc'd calculated_rps_st_sets buffer) BUG: KASAN: slab-use-after-free / slab-out-of-bounds reads via u8 ref_rps_idx underflow at calculated_rps_st_sets[255] (INTER_REF_PIC_SET_PRED with delta_idx_minus1 = 0, idx = 0) The num_long_term_ref_pics_sps = 33 case is invisible to KASAN (the OOB write lands inside struct rkvdec_rps) but corrupts short_term_ref_sets[0]; the harness asserts that case explicitly. 3. Same harness on the patched tree: all five cases (four adversarial plus a legitimate-limit regression with ST=64, LT=32, num_neg=1) pass clean, no KASAN reports. If hardware-side validation actually does reject these counts before rkvdec_hevc_assemble_hw_rps() runs and this patch is unnecessary, please say so and I will withdraw it. If it is reachable, I will follow up with a runtime hardware splat once the Orange Pi board I bought arrives. Let me know if you want a patch set with the KUnit harnesses too. checkpatch.pl: 0 errors / 0 warnings. .../rockchip/rkvdec/rkvdec-hevc-common.c | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c index 3119f3bc9f98..895fb16bc572 100644 --- a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c +++ b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c @@ -408,9 +408,58 @@ static void rkvdec_hevc_prepare_hw_st_rps(struct rkvdec_hevc_run *run, struct rk memcpy(cache, run->ext_sps_st_rps, sizeof(struct v4l2_ctrl_hevc_ext_sps_st_rps)); } +/* + * V4L2 caps the EXT_SPS RPS payload length but not the SPS-derived counts + * that the helpers walk. Caps match the HEVC spec ranges. + */ +#define RKVDEC_HEVC_MAX_SHORT_TERM_REF_PIC_SETS 64 +#define RKVDEC_HEVC_MAX_LONG_TERM_REF_PICS_SPS 32 +#define RKVDEC_HEVC_MAX_RPS_NEG_POS_PICS 16 + +static int rkvdec_hevc_validate_rps_ctrls(struct rkvdec_hevc_run *run) +{ + const struct v4l2_ctrl_hevc_sps *sps = run->sps; + + if (run->ext_sps_lt_rps && + sps->num_long_term_ref_pics_sps > + RKVDEC_HEVC_MAX_LONG_TERM_REF_PICS_SPS) + return -EINVAL; + + if (run->ext_sps_st_rps) { + unsigned int i; + + if (sps->num_short_term_ref_pic_sets > + RKVDEC_HEVC_MAX_SHORT_TERM_REF_PIC_SETS) + return -EINVAL; + + for (i = 0; i < sps->num_short_term_ref_pic_sets; i++) { + const struct v4l2_ctrl_hevc_ext_sps_st_rps *r = + &run->ext_sps_st_rps[i]; + + if (r->num_negative_pics > + RKVDEC_HEVC_MAX_RPS_NEG_POS_PICS || + r->num_positive_pics > + RKVDEC_HEVC_MAX_RPS_NEG_POS_PICS) + return -EINVAL; + + if ((r->flags & + V4L2_HEVC_EXT_SPS_ST_RPS_FLAG_INTER_REF_PIC_SET_PRED) && + (unsigned int)r->delta_idx_minus1 + 1 > i) + return -EINVAL; + } + } + + return 0; +} + void rkvdec_hevc_assemble_hw_rps(struct rkvdec_hevc_run *run, struct rkvdec_rps *rps, struct v4l2_ctrl_hevc_ext_sps_st_rps *st_cache) { + if (rkvdec_hevc_validate_rps_ctrls(run)) { + pr_err_ratelimited("rkvdec: rejecting HEVC SPS/RPS controls with out-of-range counts\n"); + return; + } + rkvdec_hevc_prepare_hw_st_rps(run, rps, st_cache); rkvdec_hevc_assemble_hw_lt_rps(run, rps); } -- 2.53.0