Re: [PATCH 1/3] media: rockchip: Introduce the rkvdec2 driver

Re: [PATCH 1/3] media: rockchip: Introduce the rkvdec2 driver

[Piotr Oniszczuk]

> Wiadomość napisana przez Detlev Casanova <detlev.casanova@collabora.com> w dniu 15 cze 2024, o godz. 21:44:
>> 
>> 
>> 

> Yes, the vdpu34x decoder on rk356x socs should be supported by this driver but 
> I don't have boards to test that unfortunately.
> 

Detlev,

Just FYI:

I done some tests of rkvdec2 on 6.12.11 on 3588, 3568 and 3566

For enabling rkvdec2 on 356x i:
-add 356x compatible in rkvdec2.c
-add dtsi nodes like this: https://github.com/warpme/minimyth2/blob/master/script/kernel/linux-6.12/files/1078-arm64-dtsi-rockchip-rk356x-add-rkvdec2-video-decoder-nodes.patch

With this i can say:
-on rk3588 i have some hevc 4k decoding perfectly but some others are failing
-on rk3566/3568 only subset of 3588’s samples is decoded well. (but is works then works perfectly fine)
-when not failing on 3588 sample fails on 356x - is see errors like:

[   95.666669] iova: 0x00000000f2e80000 already mapped to 0x0000000037378000 cannot remap to phys: 0x000000002f8c0000 prot: 0x3
[   95.745082] iova: 0x00000000f2f46000 already mapped to 0x00000000372b6000 cannot remap to phys: 0x000000003a403000 prot: 0x3
[   95.822012] iova: 0x00000000f2ee6000 already mapped to 0x0000000037126000 cannot remap to phys: 0x000000003a803000 prot: 0x3
[   95.896802] iova: 0x00000000f2ec6000 already mapped to 0x0000000029fe6000 cannot remap to phys: 0x000000003a403000 prot: 0x3
 turning-off iommu makes above errors disappear - but sample still fails.

If anybody hints me for way/tool to analyse of playing/failing samples to catch: what encoding specifics makes given sample failing to decode  on rkvdec2 - i'll be more that happy to provide details…     
(doing simple mediainfo <file> shows no differences for me…)

Re: [PATCH 1/3] media: rockchip: Introduce the rkvdec2 driver

[Nicolas Dufresne]

Hi Piotr,

Le mercredi 29 janvier 2025 à 15:48 +0100, Piotr Oniszczuk a écrit :
> 
> > Wiadomość napisana przez Detlev Casanova <detlev.casanova@collabora.com> w dniu 15 cze 2024, o godz. 21:44:
> > > 
> > > 
> > > 
> 
> > Yes, the vdpu34x decoder on rk356x socs should be supported by this driver but 
> > I don't have boards to test that unfortunately.
> > 
> 
> Detlev,
> 
> Just FYI:
> 
> I done some tests of rkvdec2 on 6.12.11 on 3588, 3568 and 3566
> 
> For enabling rkvdec2 on 356x i:
> -add 356x compatible in rkvdec2.c
> -add dtsi nodes like this: https://github.com/warpme/minimyth2/blob/master/script/kernel/linux-6.12/files/1078-arm64-dtsi-rockchip-rk356x-add-rkvdec2-video-decoder-nodes.patch
> 
> With this i can say:
> -on rk3588 i have some hevc 4k decoding perfectly but some others are failing
> -on rk3566/3568 only subset of 3588’s samples is decoded well. (but is works then works perfectly fine)
> -when not failing on 3588 sample fails on 356x - is see errors like:
> 
> [   95.666669] iova: 0x00000000f2e80000 already mapped to 0x0000000037378000 cannot remap to phys: 0x000000002f8c0000 prot: 0x3
> [   95.745082] iova: 0x00000000f2f46000 already mapped to 0x00000000372b6000 cannot remap to phys: 0x000000003a403000 prot: 0x3
> [   95.822012] iova: 0x00000000f2ee6000 already mapped to 0x0000000037126000 cannot remap to phys: 0x000000003a803000 prot: 0x3
> [   95.896802] iova: 0x00000000f2ec6000 already mapped to 0x0000000029fe6000 cannot remap to phys: 0x000000003a403000 prot: 0x3
>  turning-off iommu makes above errors disappear - but sample still fails.
> 
> If anybody hints me for way/tool to analyse of playing/failing samples to catch: what encoding specifics makes given sample failing to decode  on rkvdec2 - i'll be more that happy to provide details…     
> (doing simple mediainfo <file> shows no differences for me…)

Not sure where it is saved, but we have a mainline kernel with the MPP services
driver up-levelled (you can probably do that yourself, its not that hard). You
have to have the MPP library and probably the gstreamer plugins from rockchip
installed.

The general approach is to dump the register prior to every codec trigger, and
compare. Appart from memory addresses, pretty much all register should be
identical for decoders. If you happen to have a stream that fails on MPP, simply
report that to them, they are generally fast on fixing their own stuff. And we
can then used that knowledge to fix our implementation.

As for most codec, when working with larger group of developpers, its better to
start with getting the ITU conformance tests passing. Once we reach an excellant
score then we can start looking at specific streams. The benefit is that ITU
conformance can be run with fluster which removes a lot of possible human
errors.

Nicolas

p.s. Jonas Karlman has been working on IOMMU fixes and resets that helps recover
from faults that are sticky in the dedicated IOMMUs despite the HW self reset
mechanism.

Re: [PATCH 1/3] media: rockchip: Introduce the rkvdec2 driver

[Detlev Casanova]

Hi Piotr,

On Wednesday, 29 January 2025 09:48:51 EST Piotr Oniszczuk wrote:
> > Wiadomość napisana przez Detlev Casanova <detlev.casanova@collabora.com> w
> > dniu 15 cze 2024, o godz. 21:44:
> > 
> > 
> > 
> > 
> > Yes, the vdpu34x decoder on rk356x socs should be supported by this driver
> > but I don't have boards to test that unfortunately.
> 
> Detlev,
> 
> Just FYI:
> 
> I done some tests of rkvdec2 on 6.12.11 on 3588, 3568 and 3566
> 
> For enabling rkvdec2 on 356x i:
> -add 356x compatible in rkvdec2.c
> -add dtsi nodes like this:
> https://github.com/warpme/minimyth2/blob/master/script/kernel/linux-6.12/fi
> les/1078-arm64-dtsi-rockchip-rk356x-add-rkvdec2-video-decoder-nodes.patch
> 
> With this i can say:
> -on rk3588 i have some hevc 4k decoding perfectly but some others are
> failing -on rk3566/3568 only subset of 3588’s samples is decoded well. (but
> is works then works perfectly fine) -when not failing on 3588 sample fails
> on 356x - is see errors like:
> 
> [   95.666669] iova: 0x00000000f2e80000 already mapped to 0x0000000037378000
> cannot remap to phys: 0x000000002f8c0000 prot: 0x3 [   95.745082] iova:
> 0x00000000f2f46000 already mapped to 0x00000000372b6000 cannot remap to
> phys: 0x000000003a403000 prot: 0x3 [   95.822012] iova: 0x00000000f2ee6000
> already mapped to 0x0000000037126000 cannot remap to phys:
> 0x000000003a803000 prot: 0x3 [   95.896802] iova: 0x00000000f2ec6000
> already mapped to 0x0000000029fe6000 cannot remap to phys:
> 0x000000003a403000 prot: 0x3 turning-off iommu makes above errors disappear
> - but sample still fails.

I suppose you tested with my hevc branch, which is not really ready yet (Some 
ref frames will work but usually, it won't) Can you confirm which branch/commit 
you based your tests on ?

For the iommu, do you see those errors like that only on 356x or also on 3588 
? The hevc branch should have the iommu patches to fix that kind of things. 
(but note that hevc support is really new, so it may have bugs with buffer 
allocations)

> If anybody hints me for way/tool to analyse of playing/failing samples to
> catch: what encoding specifics makes given sample failing to decode  on
> rkvdec2 - i'll be more that happy to provide details… (doing simple
> mediainfo <file> shows no differences for me…)

Few features are supported for HEVC as of now:
 - No scanlist support (only default 16x16 blocks will work)
 - Long term reference frames are also not configured yet.
 - hevc 10 bits is also not supported yet

These are specific to the encoding and mediainfo won't really give you 
information on that, except maybe on the 10 bits format.

You can also checkout YUView (https://github.com/IENT/YUView) to get 
information on media files structure, but I have had issues with HEVC support 
lately.

Regards,

Detlev.

Re: [PATCH 1/3] media: rockchip: Introduce the rkvdec2 driver

[Piotr Oniszczuk]

> Wiadomość napisana przez Nicolas Dufresne <nicolas@ndufresne.ca> w dniu 29 sty 2025, o godz. 17:20:
> 
> Hi Piotr,
> 
> Le mercredi 29 janvier 2025 à 15:48 +0100, Piotr Oniszczuk a écrit :
>> 
> 
> As for most codec, when working with larger group of developpers, its better to
> start with getting the ITU conformance tests passing. Once we reach an excellant
> score then we can start looking at specific streams. The benefit is that ITU
> conformance can be run with fluster which removes a lot of possible human
> errors.
> 

Ah indeed.
ITU conformance approach is for sure best one.

By asking for way/tool to analyse of playing/failing samples I was just curious: is there any pattern here (e.g. specific encoding mode; etc).
I assumed - it there is - this might be good pointer in narrowing decoder code which needs fixing. 

Re: [PATCH 1/3] media: rockchip: Introduce the rkvdec2 driver

[Piotr Oniszczuk]

> Wiadomość napisana przez Detlev Casanova <detlev.casanova@collabora.com> w dniu 29 sty 2025, o godz. 17:50:
> 
> Hi Piotr,
> 
> On Wednesday, 29 January 2025 09:48:51 EST Piotr Oniszczuk wrote:
> 
> I suppose you tested with my hevc branch, which is not really ready yet (Some 
> ref frames will work but usually, it won't) Can you confirm which branch/commit 
> you based your tests on ?

Indeed  - i’m using your rkvdec2 hevc code from: https://gitlab.collabora.com/detlev/linux/-/commits/add-rkvdec2-driver-hevc with iommu from https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux/-/commit/bc47c445bfd9586115e9bcf5f231c5a5c5f0f828
and https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux/-/commit/9a9fd791513bc0d02c2242c88f23b41bd47de30a
 

> 
> For the iommu, do you see those errors like that only on 356x or also on 3588 
> ?

3588 has no any iommu errors. 
I see iommu errors only on 356x
 

> The hevc branch should have the iommu patches to fix that kind of things.

I’m using:
https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux/-/commit/bc47c445bfd9586115e9bcf5f231c5a5c5f0f828
https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux/-/commit/9a9fd791513bc0d02c2242c88f23b41bd47de30a

I see there is: https://gitlab.collabora.com/detlev/linux/-/commits/add-rkvdec2-driver-iommu
Let me update iommu code to above branch and report back here how it goes in 3588 and 356x


> 
> (but note that hevc support is really new, so it may have bugs with buffer 
> allocations)

It is already great work.
I’m awaiting for hevc in 35xx during last 2+ years :-)  

> 
>> If anybody hints me for way/tool to analyse of playing/failing samples to
>> catch: what encoding specifics makes given sample failing to decode  on
>> rkvdec2 - i'll be more that happy to provide details… (doing simple
>> mediainfo <file> shows no differences for me…)
> 
> Few features are supported for HEVC as of now:
> - No scanlist support (only default 16x16 blocks will work)
> - Long term reference frames are also not configured yet.
> - hevc 10 bits is also not supported yet
> 
> These are specific to the encoding and mediainfo won't really give you 
> information on that, except maybe on the 10 bits format.
> 
> You can also checkout YUView (https://github.com/IENT/YUView) to get 
> information on media files structure, but I have had issues with HEVC support 
> lately.

oh great. will do and report!

Re: [PATCH 1/3] media: rockchip: Introduce the rkvdec2 driver

[Piotr Oniszczuk]

Update to my last email: 

> Wiadomość napisana przez Piotr Oniszczuk <piotr.oniszczuk@gmail.com> w dniu 30 sty 2025, o godz. 11:46:
> 
> 
> I’m using:
> https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux/-/commit/bc47c445bfd9586115e9bcf5f231c5a5c5f0f828
> https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux/-/commit/9a9fd791513bc0d02c2242c88f23b41bd47de30a
> 
> I see there is: https://gitlab.collabora.com/detlev/linux/-/commits/add-rkvdec2-driver-iommu
> Let me update iommu code to above branch and report back here how it goes in 3588 and 356x

I updated to https://gitlab.collabora.com/detlev/linux/-/commits/add-rkvdec2-driver-iommu and basically i see no change (the same iommu errors on 356x; 3588 all clean)   

> 
> 
>> 
>> You can also checkout YUView (https://github.com/IENT/YUView) to get 
>> information on media files structure, but I have had issues with HEVC support 
>> lately.
> 
> oh great. will do and report!
> 

Huh - here in need to switch to Linux as on maxOS i have issue with ffmpeg ver. mismatch in YUView
Will report next days….