[PATCH] drm/amd/display: drop RGBA8888 from plane format lists

[PATCH] drm/amd/display: drop RGBA8888 from plane format lists

[MuXian Chen]

DC's enum surface_pixel_format has no RGBA8888 (memory order A,B,G,R):
fill_dc_plane_info_and_addr() has no case for it and returns -EINVAL.
Yet rgb_formats[] and overlay_formats[] advertise DRM_FORMAT_RGBA8888, so
compositors (KWin on Wayland) pick it for layers with alpha and every
atomic commit fails once per frame, flooding dmesg with

  amdgpu 0000:06:00.0: [drm] *ERROR* Unsupported screen format RA24
  little-endian (0x34324152)

and forcing a fallback to composition on every frame.

Remove the two entries.  No working configuration is lost: a plane that
advertises RGBA8888 can never convert it in fill_dc_plane_info_and_addr(),
on any display generation.

Tested on Raphael 610M (DCN 3.1.5): before, ~240 atomic commits per second
failed with -EINVAL; after, the plane format list no longer contains RA24
and dmesg is clean.

Link: https://gitlab.freedesktop.org/drm/amd/-/work_items/5339
Signed-off-by: MuXian Chen <a18165943161@gmail.com>
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
index 9139823..6d5454f 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
@@ -49,11 +49,13 @@
  * For future use cases we should either initialize them dynamically based on
  * plane capabilities, or initialize this array to all formats, so internal drm
  * check will succeed, and let DC implement proper check
+ *
+ * DC has no surface format for DRM_FORMAT_RGBA8888, so it must not be
+ * advertised here or in overlay_formats.
  */
 static const uint32_t rgb_formats[] = {
 	DRM_FORMAT_XRGB8888,
 	DRM_FORMAT_ARGB8888,
-	DRM_FORMAT_RGBA8888,
 	DRM_FORMAT_XRGB2101010,
 	DRM_FORMAT_XBGR2101010,
 	DRM_FORMAT_ARGB2101010,
@@ -70,7 +72,6 @@ static const uint32_t rgb_formats[] = {
 static const uint32_t overlay_formats[] = {
 	DRM_FORMAT_XRGB8888,
 	DRM_FORMAT_ARGB8888,
-	DRM_FORMAT_RGBA8888,
 	DRM_FORMAT_XBGR8888,
 	DRM_FORMAT_ABGR8888,
 	DRM_FORMAT_RGB565,
-- 
2.56.0

Re: [PATCH] drm/amd/display: drop RGBA8888 from plane format lists

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] drm/amd/display: broken KUnit test dm_test_get_plane_formats()
--

commit f2e839f51941e28c7b37c695841346e690a5a04b
Author: MuXian Chen <a18165943161@gmail.com>

drm/amd/display: drop RGBA8888 from plane format lists

This commit removes DRM_FORMAT_RGBA8888 from rgb_formats[] and
overlay_formats[] because DC lacks a corresponding surface pixel format,
causing fill_dc_plane_info_and_addr() to fail with -EINVAL when a
compositor selects it.

Link: https://gitlab.freedesktop.org/drm/amd/-/work_items/5339

> diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
> index 06d9ff2c32c52..931c4ad677c7e 100644
> --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
> +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
> @@ -48,11 +48,13 @@
>   * For future use cases we should either initialize them dynamically based on
>   * plane capabilities, or initialize this array to all formats, so internal drm
>   * check will succeed, and let DC implement proper check
> + *
> + * DC has no surface format for DRM_FORMAT_RGBA8888, so it must not be
> + * advertised here or in overlay_formats.
>   */
>  static const uint32_t rgb_formats[] = {
>  	DRM_FORMAT_XRGB8888,
>  	DRM_FORMAT_ARGB8888,
> -	DRM_FORMAT_RGBA8888,
>  	DRM_FORMAT_XRGB2101010,
>  	DRM_FORMAT_XBGR2101010,
>  	DRM_FORMAT_ARGB2101010,
> @@ -69,7 +71,6 @@ static const uint32_t rgb_formats[] = {
>  static const uint32_t overlay_formats[] = {
>  	DRM_FORMAT_XRGB8888,
>  	DRM_FORMAT_ARGB8888,
> -	DRM_FORMAT_RGBA8888,

[Severity: Medium]
As reported by Sashiko on v1 [1], removing DRM_FORMAT_RGBA8888 reduces
ARRAY_SIZE(rgb_formats) from 14 to 13 and ARRAY_SIZE(overlay_formats) from
9 to 8.

In drivers/gpu/drm/amd/display/amdgpu_dm/tests/amdgpu_dm_plane_test.c,
dm_test_get_plane_formats() calls amdgpu_dm_plane_get_plane_formats() and
checks the returned format counts:

	plane->type = DRM_PLANE_TYPE_PRIMARY;
	KUNIT_EXPECT_EQ(test, amdgpu_dm_plane_get_plane_formats(plane, NULL, formats, 32), 14);

	cap->pixel_format_support.nv12 = true;
	cap->pixel_format_support.p010 = true;
	cap->pixel_format_support.fp16 = true;
	KUNIT_EXPECT_EQ(test, amdgpu_dm_plane_get_plane_formats(plane, cap, formats, 32), 20);

	plane->type = DRM_PLANE_TYPE_OVERLAY;
	KUNIT_EXPECT_EQ(test, amdgpu_dm_plane_get_plane_formats(plane, NULL, formats, 32), 9);

	universal_cap->type = DC_PLANE_TYPE_DCN_UNIVERSAL;
	KUNIT_EXPECT_EQ(test,
			amdgpu_dm_plane_get_plane_formats(plane, universal_cap, formats, 32),
			14);

Should these four KUNIT_EXPECT_EQ assertions in dm_test_get_plane_formats()
be updated to expect 13, 19, 8, and 13 so the amdgpu_dm_plane KUnit test
suite does not fail?

[1] https://lore.kernel.org/r/20261030171816.11610-1-a18165943161@gmail.com

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261030164327.8461-1-a18165943161@gmail.com?part=1