Re: [PATCH v2 2/4] media: rockchip: Introduce the rkvdec2 driver

From: Detlev Casanova <detlev.casanova@collabora.com>
To: Jianfeng Liu <liujianfeng1994@gmail.com>
Cc: andy.yan@rock-chips.com, benjamin.gaignard@collabora.com,
	boris.brezillon@collabora.com, conor+dt@kernel.org,
	daniel.almeida@collabora.com, devicetree@vger.kernel.org,
	didi.debian@cknow.org, dsimic@manjaro.org,
	ezequiel@vanguardiasur.com.ar, gregkh@linuxfoundation.org,
	heiko@sntech.de, hverkuil-cisco@xs4all.nl, jonas@kwiboo.se,
	knaerzche@gmail.com, krzk+dt@kernel.org,
	linux-arm-kernel@lists.infradead.org,
	linux-kernel@vger.kernel.org, linux-media@vger.kernel.org,
	linux-rockchip@lists.infradead.org,
	linux-staging@lists.linux.dev, mchehab@kernel.org,
	nicolas.dufresne@collabora.com, paul.kocialkowski@bootlin.com,
	robh@kernel.org, sebastian.reichel@collabora.com
Subject: Re: [PATCH v2 2/4] media: rockchip: Introduce the rkvdec2 driver
Date: Thu, 20 Jun 2024 10:07:41 -0400	[thread overview]
Message-ID: <2187039.irdbgypaU6@arisu> (raw)
In-Reply-To: <20240619174623.270706-1-liujianfeng1994@gmail.com>

[-- Attachment #1.1: Type: text/plain, Size: 2164 bytes --]

Hi Jianfeng,

On Wednesday, June 19, 2024 1:46:23 P.M. EDT Jianfeng Liu wrote:
> Hi Detlev,
> 
> On Wed, 19 Jun 2024 10:57:19 -0400, Detlev Casanova wrote:
> >+	if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
> >+		height *= 2;
> >+
> >+	if (width > ctx->coded_fmt.fmt.pix_mp.width ||
> >+	    height > ctx->coded_fmt.fmt.pix_mp.height)
> >+		return -EINVAL;
> 
> I did further invesatigation on chromium. I find that before real video
> is decoded, chromium will call VIDIOC_STREAMON twice with value of
> sps->flags 0:
> 
> At the first time width and height are 16; ctx->coded_fmt.fmt.pix_mp.width
> and coded_fmt.fmt.pix_mp.height are 16, which are the min size of decoder;
> At the second time width and height are still 16; while
> coded_fmt.fmt.pix_mp.width is 1920 and coded_fmt.fmt.pix_mp.height is
> 1088, which are the real size of video.
> 
> So VIDIOC_STREAMON will fall at the first time call because sps->flags is
> 0 so V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set, and then height is
> doubled to 32 which is larger than 16.
> 
> What do you think if we skip doubling height if sps->flags is 0 and at the
> same time V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set? The following hack
> did fix my chromium:
> 
> --- a/drivers/staging/media/rkvdec2/rkvdec2-h264.c
> +++ b/drivers/staging/media/rkvdec2/rkvdec2-h264.c
> @@ -767,7 +767,7 @@ static int rkvdec2_h264_validate_sps(struct rkvdec2_ctx
> *ctx, * which is half the final height (see (7-18) in the
>          * specification)
>          */
> -       if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
> +       if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY) && sps->flags)
> height *= 2;
> 
>         if (width > ctx->coded_fmt.fmt.pix_mp.width ||

I'm not sure what Chromium is trying to do here. But the spec is clear that 
height should be multiplied by 2 (That's actually 7-8, not 7-18):

  FrameHeightInMbs = ( 2 – frame_mbs_only_flag ) * PicHeightInMapUnits

This feels like hacking the driver to please a specific userspace application, 
so I'd like to understand better what chromium is doing.

Regards,
Detlev.

[-- Attachment #1.2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip