From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0EC16C27C79 for ; Thu, 20 Jun 2024 15:04:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version: Content-Transfer-Encoding:Content-Type:References:In-Reply-To:Date:Cc:To:From :Subject:Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Q2uUCNx0/9k8tOiqpzcUKTRsiQd90Xg5p6u2EUxQ2N4=; b=AmVj1jwxOXasHIuuD32q4hsSDv O9wHLdwCrY0CrWoEgk9ZZsr4nGHojBf9bkMJk6okh+ps/zgxxmHPUPo5qtNAfUDkqXEy94Afr5g2s 9EOdebFqYcZcMqMWVEynTEMFnZmAVTi4lFFU8pJGE3JHQlVQJzI7SVSpBped1pLADEd+EuQZd0/gE 0Cs2buJSqq+e9ZZrvTt2pmXDEyB9fyUW7s7cmadmyGiL39Y9hTkA5+muHWoc186tX0g7rqJ0NG+uO uUlCI1zHvtaYr4Lp02dITAFc/rSyLFWNcABd1KMe7n7MNbGWc6pqPaWnomNEks6xkzlxkoTX00LWu w4imq4Sg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sKJKc-00000005ZCT-1718; Thu, 20 Jun 2024 15:04:10 +0000 Received: from madrid.collaboradmins.com ([2a00:1098:ed:100::25]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sKJKY-00000005ZAp-2KkW; Thu, 20 Jun 2024 15:04:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1718895841; bh=x/vZAHZVKlbt1TM2vckPahTO9LL1ZOh2lyhpbR2qC0g=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=M4AFZ3nCd77ATlvL9HOelsIQ0B1xH9kKkCDwtoVUzNbsW6pWjogrjNAlrvj5FbPWS cubpgiijA0MrP6coFE67bTImg4BDZTo3Ii+Yiup5/lLwVFn9H3QJm7N0m9b+61xLCC ZRDATubUG9LqaK8eWKxacVtTOdXPgw9f0hBDcrG+R/0HFBmkG7ShsH2/RlD7qZXek4 KysfT8FpG0RGWyz1CdNsEPsrAdnU8UAC0Fs7V7GsFI2GvJoo19mDP+7feajl03+n/b 7G3H9YuZQm84Rln+px3aSrCoho92e2/fNtGDbGFwAcOZJ6bJOXkAntIWArNXIeEOp/ 55BivXZ/uMlDg== Received: from nicolas-tpx395.localdomain (cola.collaboradmins.com [195.201.22.229]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: nicolas) by madrid.collaboradmins.com (Postfix) with ESMTPSA id E58E73780C1F; Thu, 20 Jun 2024 15:03:57 +0000 (UTC) Message-ID: <2349746d488f4edf9c7c40df5e15ff70d3ec67b7.camel@collabora.com> Subject: Re: [PATCH v2 2/4] media: rockchip: Introduce the rkvdec2 driver From: Nicolas Dufresne To: Jianfeng Liu , detlev.casanova@collabora.com Cc: andy.yan@rock-chips.com, benjamin.gaignard@collabora.com, boris.brezillon@collabora.com, conor+dt@kernel.org, daniel.almeida@collabora.com, devicetree@vger.kernel.org, didi.debian@cknow.org, dsimic@manjaro.org, ezequiel@vanguardiasur.com.ar, gregkh@linuxfoundation.org, heiko@sntech.de, hverkuil-cisco@xs4all.nl, jonas@kwiboo.se, knaerzche@gmail.com, krzk+dt@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, mchehab@kernel.org, paul.kocialkowski@bootlin.com, robh@kernel.org, sebastian.reichel@collabora.com Date: Thu, 20 Jun 2024 11:03:54 -0400 In-Reply-To: <20240619174623.270706-1-liujianfeng1994@gmail.com> References: <20240619150029.59730-3-detlev.casanova@collabora.com> <20240619174623.270706-1-liujianfeng1994@gmail.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.52.2 (3.52.2-1.fc40) MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240620_080406_880651_B9C82BCD X-CRM114-Status: GOOD ( 25.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Jianfeng, Le jeudi 20 juin 2024 =C3=A0 01:46 +0800, Jianfeng Liu a =C3=A9crit=C2=A0: > Hi Detlev, >=20 > On Wed, 19 Jun 2024 10:57:19 -0400, Detlev Casanova wrote: > > + if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY)) > > + height *=3D 2; > > + > > + if (width > ctx->coded_fmt.fmt.pix_mp.width || > > + height > ctx->coded_fmt.fmt.pix_mp.height) > > + return -EINVAL; >=20 > I did further invesatigation on chromium. I find that before real video > is decoded, chromium will call VIDIOC_STREAMON twice with value of > sps->flags 0: >=20 > At the first time width and height are 16; ctx->coded_fmt.fmt.pix_mp.widt= h > and coded_fmt.fmt.pix_mp.height are 16, which are the min size of decoder= ; This falls short of a specification to support the uninitialized usage by Chromium. That being said, we do make an effort to try and have a valid def= ault SPS control and OUTPUT format combination. So my suggestion would be to set V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY in the default compound control init. Thi= s way, 0x0 get translate to 16x16 instead of 16x32, thus will work with more drivers. Chromium these days is not being tested on anything else then MTK, which ha= s a 64x64 minimum size, this is why they never get into this issue. This driver validation is entirely correct. Removing in some cases is unsafe, it would = need to be replaced with STREAMON only validation of the CAPTURE sizes (which currently is validate by implied propagation of valid SPS/OUTPUT). **not even compiled tested, just to illustrate** diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2= - core/v4l2-ctrls-core.c index c4d995f32191..a55e165ea9c3 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c @@ -111,6 +111,7 @@ static void std_init_compound(const struct v4l2_ctrl *c= trl, u32 idx, struct v4l2_ctrl_vp9_frame *p_vp9_frame; struct v4l2_ctrl_fwht_params *p_fwht_params; struct v4l2_ctrl_h264_scaling_matrix *p_h264_scaling_matrix; + struct v4l2_ctrl_h264_sps *p_h264_sps; struct v4l2_ctrl_av1_sequence *p_av1_sequence; void *p =3D ptr.p + idx * ctrl->elem_size; =20 @@ -179,6 +180,18 @@ static void std_init_compound(const struct v4l2_ctrl *= ctrl, u32 idx, */ memset(p_h264_scaling_matrix, 16, sizeof(*p_h264_scaling_matrix)); break; + case V4L2_CTRL_TYPE_H264_SPS: + p_h264_sps =3D p; + /* + * Without V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY, + * frame_mbs_only_flag set to 0 will translate to a miniumu= m + * height of 32 (see H.264 specification 7-8). Some driver = may + * have a minimum size lower then 32, which would fail + * validation with the SPS value. Set this flag, so that th= ere + * is now doubling in the height, allowing a valid default. + */ + p_h264_sps->flags =3D V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY; + break; } } Nicolas > At the second time width and height are still 16; while > coded_fmt.fmt.pix_mp.width is 1920 and coded_fmt.fmt.pix_mp.height is > 1088, which are the real size of video. >=20 > So VIDIOC_STREAMON will fall at the first time call because sps->flags is > 0 so V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set, and then height is > doubled to 32 which is larger than 16. >=20 > What do you think if we skip doubling height if sps->flags is 0 and at th= e > same time V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set? The following hac= k > did fix my chromium: >=20 > --- a/drivers/staging/media/rkvdec2/rkvdec2-h264.c > +++ b/drivers/staging/media/rkvdec2/rkvdec2-h264.c > @@ -767,7 +767,7 @@ static int rkvdec2_h264_validate_sps(struct rkvdec2_c= tx *ctx, > * which is half the final height (see (7-18) in the > * specification) > */ > - if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY)) > + if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY) && sps->fla= gs) > height *=3D 2; > =20 > if (width > ctx->coded_fmt.fmt.pix_mp.width || >=20 > Best regards > Jianfeng