Re: how to use stateless-mpeg2-vp8-h264-v4 on imx8m with vpu?

[Martin Kepplinger <martin.kepplinger@puri.sm>]

On 30.06.20 14:59, Ezequiel Garcia wrote:
> Hi Martin,
> 
> On Tue, 2020-06-30 at 13:16 +0200, Martin Kepplinger wrote:
>> Hi Ezequiel,
>>
>> On the Librem 5 (imx8mq) I try to decode a (h264) full hd video file
>> using the VPU.
>>
>> I'm running the following tree that doesn't change much, but adds the
>> VPU dts description:
>> https://source.puri.sm/martin.kepplinger/linux-next/-/commits/5.8-rc3/librem5___vpu
>> (output of "v4l-ctl --all" at the bottom).
>>
> 
> OK, this means the VPU is enabled with the Hantro driver, so far so good.
> 
>>
>> I now run your version of ffmpeg:
>> https://gitlab.collabora.com/ezequiel/ffmpeg/-/commits/stateless-mpeg2-vp8-h264-v4
>> (with minor build fixes) but simply doing "ffplay <file>" seems like not
>> using /dev/video0
>>
>> Is this supposed to work? If so, do I need to know anything about ffmpeg
>> to be able to test this? How do you verify that the v4l2 mem2mem device
>> (vpu) is being used?
>>
> 
> Not sure what state the branch is on, and also IIRC you should use that together
> with mpv for proper zero-copy operation.
> 
> I think the libre elec community can help you with that, as they maintain
> proper ffmpeg/mpv branches.

thanks for the hint.

> 
> Do you need this to work with ffmpeg, or is GStreamer also OK?

gstreamer would be equally fine (or easier as I've used that before) for
me. Last time I checked I think I'd have to build many more libraries
from source in order to test though. That's the only reason why I've
tried ffmpeg.

> 
> Thanks,
> Ezequiel
> 
>> thanks a lot,
>>                                 martin
>>
>>
>>
>> purism@pureos:~/ffmpeg_sources/ffmpeg$ v4l2-ctl --all
>> Driver Info:
>> 	Driver name      : hantro-vpu
>> 	Card type        : nxp,imx8mq-vpu-dec
>> 	Bus info         : platform: hantro-vpu
>> 	Driver version   : 5.8.0
>> 	Capabilities     : 0x84204000
>> 		Video Memory-to-Memory Multiplanar
>> 		Streaming
>> 		Extended Pix Format
>> 		Device Capabilities
>> 	Device Caps      : 0x04204000
>> 		Video Memory-to-Memory Multiplanar
>> 		Streaming
>> 		Extended Pix Format
>> Media Driver Info:
>> 	Driver name      : hantro-vpu
>> 	Model            : hantro-vpu
>> 	Serial           :
>> 	Bus info         : platform: hantro-vpu
>> 	Media version    : 5.8.0
>> 	Hardware revision: 0x00000000 (0)
>> 	Driver version   : 5.8.0
>> Interface Info:
>> 	ID               : 0x0300000c
>> 	Type             : V4L Video
>> Entity Info:
>> 	ID               : 0x00000001 (1)
>> 	Name             : nxp,imx8mq-vpu-dec-source
>> 	Function         : V4L2 I/O
>> 	Pad 0x01000002   : 0: Source
>> 	  Link 0x02000008: to remote pad 0x1000004 of entity
>> 'nxp,imx8mq-vpu-dec-proc': Data, Enabled, Immutable
>> Priority: 2
>> Format Video Capture Multiplanar:
>> 	Width/Height      : 48/48
>> 	Pixel Format      : 'NV12' (Y/CbCr 4:2:0)
>> 	Field             : None
>> 	Number of planes  : 1
>> 	Flags             :
>> 	Colorspace        : JPEG
>> 	Transfer Function : Default
>> 	YCbCr/HSV Encoding: Default
>> 	Quantization      : Default
>> 	Plane 0           :
>> 	   Bytes per Line : 48
>> 	   Size Image     : 3456
>> Format Video Output Multiplanar:
>> 	Width/Height      : 48/48
>> 	Pixel Format      : 'MG2S' (MPEG-2 Parsed Slice Data)
>> 	Field             : None
>> 	Number of planes  : 1
>> 	Flags             :
>> 	Colorspace        : JPEG
>> 	Transfer Function : Default
>> 	YCbCr/HSV Encoding: Default
>> 	Quantization      : Default
>> 	Plane 0           :
>> 	   Bytes per Line : 0
>> 	   Size Image     : 4608
>>
>> Codec Controls
>>
>>         mpeg_2_slice_parameters 0x009909fa (unknown): type=103
>> flags=has-payload
>>    mpeg_2_quantization_matrices 0x009909fb (unknown): type=104
>> flags=has-payload
>>                    h264_profile 0x00990a6b (menu)   : min=0 max=4
>> default=2 value=2
>>     h264_sequence_parameter_set 0x00990ce8 (unknown): type=110
>> flags=has-payload
>>      h264_picture_parameter_set 0x00990ce9 (unknown): type=111
>> flags=has-payload
>>             h264_scaling_matrix 0x00990cea (unknown): type=112
>> flags=has-payload
>>           h264_slice_parameters 0x00990ceb (unknown): type=113
>> flags=has-payload
>>          h264_decode_parameters 0x00990cec (unknown): type=114
>> flags=has-payload
>>                h264_decode_mode 0x00990ced (menu)   : min=1 max=1
>> default=1 value=1
>>                 h264_start_code 0x00990cee (menu)   : min=1 max=1
>> default=1 value=1
>>                vp8_frame_header 0x009910d0 (unknown): type=301
>> flags=has-payload
> 
>