Re: [RFC PATCH 0/2] Fixes on CM3 helper for plane shaper LUT

[Melissa Wen <mwen@igalia.com>]

On 09/12/2025 12:12, Harry Wentland wrote:
>
> On 2025-12-09 09:44, Melissa Wen wrote:
>>
>> On 09/12/2025 11:31, Melissa Wen wrote:
>>>
>>> On 08/12/2025 22:34, Matthew Schwartz wrote:
>>>>> On Dec 8, 2025, at 3:48 PM, Melissa Wen <mwen@igalia.com> wrote:
>>>>>
>>>>> There are some unexpected banding and shimmer effects when using
>>>>> steamOS/gamescope color pipeline for HDR on DCN32 or newer families.
>>>>> Those problems are not present in Steam Deck (DCN301). It happens on
>>>>> DCN32 because plane shaper LUT uses DCN30 CM3 helper to translate curves
>>>>> instead of DCN10 CM helper. This series identifies the necessary changes
>>>>> on CM3 helper to reduce differences on color transformation made by
>>>>> those two helpers.
>>>>>
>>>>> Patch 1 aims to solve the shimmer/colorful points that looks like a
>>>>> wrong map of black values on red/green/blue colors. Patch 2 extends the
>>>>> delta clamping fix made in commit 27fc10d1095f ("drm/amd/display: Fix
>>>>> the delta clamping for shaper LUT") to solve some banding effects.
>>>>>
>>>>> Banding is not fully solved by any helper and needs further
>>>>> investigation.
>>>>>
>>>>> One easy way to check the current and expected behavior is moving the
>>>>> cursor (doing composition) to get the expected result from GFX. When the
>>>>> cursor disappears, those color transformations are back to be done by
>>>>> the display hw.
>>>> Hi Melissa,
>>>>
>>>> Could you share how you’re testing the gamescope color pipeline with HDR on DCN32, i.e display and connection type? Are any extra gamescope or kernel patches required?
>>>>
>>>> At least on my own DCN32 setup (AMD 7900XTX) + my primary monitor (an LG 45gx950a-b) via DisplayPort or my DCN35 setup + integrated HDR OLED screen (Legion Go 2), gamescope always composites when HDR is enabled. I applied your patches on top of kernel 6.18, and my kernel is built with CONFIG_DRM_AMD_COLOR_STEAMDECK=y (the downstream name of AMD_PRIVATE_COLOR for SteamOS), so that shouldn't be an issue. I tried everything from 1280x720p -> 5120x2160p, and it does not work on any resolution.
>>> Hi Matt,
>>>
>>> You need to hack the DPP color caps to enabled SHAPER/3D and BLEND LUTs as below:
>>>
>>> diff --git i/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c w/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c
>>> index b276fec3e479..96b4f3239fb1 100644
>>> --- i/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c
>>> +++ w/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c
>>> @@ -2256,8 +2256,8 @@ static bool dcn32_resource_construct(
>>>          dc->caps.color.dpp.gamma_corr = 1;
>>>          dc->caps.color.dpp.dgam_rom_for_yuv = 0;
>>>
>>> -       dc->caps.color.dpp.hw_3d_lut = 0;
>>> -       dc->caps.color.dpp.ogam_ram = 0;  // no OGAM in DPP since DCN1
>>> +       dc->caps.color.dpp.hw_3d_lut = 1;
>>> +       dc->caps.color.dpp.ogam_ram = 1;  // no OGAM in DPP since DCN1
>>>          // no OGAM ROM on DCN2 and later ASICs
>>>          dc->caps.color.dpp.ogam_rom_caps.srgb = 0;
>>>          dc->caps.color.dpp.ogam_rom_caps.bt2020 = 0;
>>>
>>> In short, you need to change `caps.color.dpp.hw_3d_lut` and `caps.color.dpp.ogam_ram` to 1 in the dcnX_resource.c file to say there is a "plane" color caps.
>>> The thing is that, in DCN32+, these color caps are not part of DPP anymore, they are MPC capabilities in MCM that can be moved before or after blending.
>>> But the current kernel implementation checks DPP color caps to expose plane color proprerties.
>>> Checking MPC and where the MCM is positioned would be more complex, but not impossible. Something to improve in the future yes.
>> Just found this: dpp_color_caps.hw_3d_lut || dm->dc->caps.color.mpc.preblend (https://gitlab.freedesktop.org/agd5f/linux/-/blob/amd-staging-drm-next/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c#L1636)
>>
>> Should be enough for new kernel versions. So you might need only the blend LUT hack.
>>
>>> You need to confirm that your `drm_info` shows all AMD plane color properties, but gamescope basically checks CTM and BLEND_TF as you can see here:
>>> https://github.com/ValveSoftware/gamescope/blob/master/src/Backends/DRMBackend.cpp#L3347
>>>
> Are you testing this with AMD_PRIVATE_COLOR, or with the newly merged color pipeline API? If it's the former, then the kernel needs to be built with an explicit -DAMD_PRIVATE_COLOR for this to work.

I'm testing with cflags, but AFAIU Matthew is using a downstream kernel 
version where there is an extra commit that enables AMD_PRIVATE_COLOR 
via config option ("CONFIG_DRM_AMD_COLOR_STEAMDECK=y").
Depends on this kernel version, the hack for 3D LUT and BLEND LUT are 
both necessary.

>
> Harry
>
>>> Let me know if it works for you.
>>>
>>> BR,
>>>
>>> Melissa
>>>
>>>> Thanks,
>>>> Matt
>>>>
>>>>> Lemme know your thoughts!
>>>>>
>>>>> Melissa
>>>>>
>>>>> Melissa Wen (2):
>>>>>    drm/amd/display: fix wrong color value mapping on DCN32 shaper LUT
>>>>>    drm/amd/display: extend delta clamping logic to CM3 LUT helper
>>>>>
>>>>> .../amd/display/dc/dcn30/dcn30_cm_common.c    | 32 +++++++++++++++----
>>>>> .../display/dc/dwb/dcn30/dcn30_cm_common.h    |  2 +-
>>>>> .../amd/display/dc/hwss/dcn30/dcn30_hwseq.c   |  9 +++---
>>>>> .../amd/display/dc/hwss/dcn32/dcn32_hwseq.c   | 17 ++++++----
>>>>> .../amd/display/dc/hwss/dcn401/dcn401_hwseq.c | 16 ++++++----
>>>>> 5 files changed, 50 insertions(+), 26 deletions(-)
>>>>>
>>>>> -- 
>>>>> 2.51.0
>>>>>