Re: [yocto-patches] [meta-rockchip PATCH v2] enable HW VPU decoding for SoCs that have stateless VPUs

[Trevor Woerner <twoerner@gmail.com>]

On Wed 2024-08-28 @ 06:23:59 PM, Quentin Schulz via lists.yoctoproject.org wrote:
> From: Quentin Schulz <quentin.schulz@cherry.de>
> 
> v4l2codecs is the gstreamer plugin for V4L2 stateless video hardware
> decoding. The Rockchip SoCs that have a VPU all seems to be based on
> Hantro, RKVDEC or RKVDECv2, all stateless encoding/decoding VPUs.
> 
> Therefore, let's enable VPU decoding in Gstreamer whenever possible,
> when the SoC supports it.
> 
> PX30, RK3066, RK3188, RK3288, RK3328, RK3399, RK356x and RK3588(s) all
> have at least one Hantro VPU.
> 
> RK3328, RK3399, RK356x and RK3588(s) all have at least one
> RKVDEC/RKVDECv2 VPU (though not necessarily supported in the upstream
> kernel just yet).
> 
> === PX30
> Tested on PX30 Ringneck with with Haikou+Haikou Video Demo adapter:
> 
> $ gst-launch-1.0 filesrc location=$FILE ! parsebin ! v4l2slh264dec ! waylandsink
> 
> with FILE storing the path to any h264 file, e.g.
> https://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_h264.mov
> https://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_1080p_h264.mov
> 
> Needed packages are:
> - weston
> - gstreamer1.0-plugins-bad (for waylandsink and v4l2slh264dec)
> - gstreamer1.0-plugins-base (for parsebin)
> 
> A few frames are dropped every other second for 1080p but otherwise
> smooth.
> 
> === RK3399
> Tested on RK3399 Puma with Haikou:
> 
> $ gst-launch-1.0 filesrc location=$FILE ! parsebin ! v4l2slh264dec ! waylandsink
> 
> with FILE storing the path to any h264 file, e.g.
> https://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_1080p_h264.mov
> https://download.blender.org/demo/movies/BBB/bbb_sunflower_2160p_30fps_normal.mp4.zip
> 
> Needed packages are:
> - weston
> - gstreamer1.0-plugins-bad (for waylandsink and v4l2codecs)
> - gstreamer1.0-plugins-base (for parsebin)
> 
> === RK3588
> 
> Tested on a RK3588 Tiger with Haikou+Haikou Video Demo adapter - on a
> downstream v6.6 (upstream-based, not Rockchip BSP-based) with DSI
> patches - :
> 
> $ gst-launch-1.0 filesrc location=$FILE ! parsebin ! v4l2slav1dec ! fakesink
> 
> with FILE storing the path to any AV1 file, e.g.
> http://download.opencontent.netflix.com.s3.amazonaws.com/AV1/cmaf/spark-8b-59.94fps/spark_606kbps_432p.mp4
> https://woolyss.com/f/av1-nosound-chimera.mp4
> https://woolyss.com/f/av1-opus-sita.webm
> 
> Needed packages are:
> - gstreamer1.0-plugins-bad (for fakesink and v4l2slav1dec)
> - gstreamer1.0-plugins-base (for parsebin)
> 
> For some reason though, waylandsink is very choppy. Combining
> fpsdisplaysink with fakesink shows a ~60fps when decoding the 432p file,
> ~24fps for the two others.
> Note that 10b-depth isn't supported (at least in my setup).
> 
> Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de>
> ---
> (Most) Rockchip SoCs have a VPU that can be used with v4l2codecs plugin
> from gstreamer. In the kernel this VPU is supported by the Hantro,
> RKVDEC or RKVDECv2 drivers which do both stateless HW video encoding and
> decoding, however gstreamer only supports decoding for now, see [1] for
> encoding support.
> 
> All Rockchip SoCs supported in this layer to the exception of the RK3308
> have at least one stateless VPU according to the compatible list in the
> Linux kernel, so all SoC includes (except the one for the RK3308) now
> set ENABLE_STATELESS_VPU_GST value to enable VPU decoding via gstreamer.
> 
> Note that RK3588 upstream Linux kernel only supports AV1 decoding right
> now. Also, there's currently no video output support, so this was tested
> on a private downstream branch supporting DSI on RK3588 Tiger with
> Haikou + Haikou Video Demo adapter).
> 
> Note that Gstreamer v4l2codecs only added support for av1 decoding in
> 1.23.1, so the RK3588 part of this series doesn't necessarily make sense
> to be backported right now to other branches (e.g. scarthgap or
> kirkstone), at the very least until more than AV1 decoding is supported
> (which will probably land this year or the next).
> 
> [1] https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5676
> ---
> Changes in v2:
> - rename HAS_HANTRO to ENABLE_STATELESS_VPU_GST
> - add documentation in README
> - squash all commits
> - mention rkvdec/rkvdecv2 as well
> - Link to v1: https://lore.kernel.org/r/20240820-gst-hantro-v1-0-335c4eaf8e8b@cherry.de
> ---
>  README                                             | 22 ++++++++++++++++++++++
>  conf/machine/include/px30.inc                      |  2 ++
>  conf/machine/include/rk3066.inc                    |  2 ++
>  conf/machine/include/rk3188.inc                    |  2 ++
>  conf/machine/include/rk3288.inc                    |  2 ++
>  conf/machine/include/rk3328.inc                    |  2 ++
>  conf/machine/include/rk3399.inc                    |  2 ++
>  conf/machine/include/rk3566.inc                    |  2 ++
>  conf/machine/include/rk3568.inc                    |  2 ++
>  conf/machine/include/rk3588s.inc                   |  2 ++
>  .../gstreamer/gstreamer1.0-plugins-bad_%.bbappend  |  1 +
>  11 files changed, 41 insertions(+)

Applied to meta-rockchip, master branch. Thanks!