[PATCH RESEND] drm/amd/display: drop RGBA8888 from plane format lists

[PATCH RESEND] drm/amd/display: drop RGBA8888 from plane format lists

[MuXian Chen]

DC's enum surface_pixel_format has no RGBA8888 (memory order A,B,G,R):
fill_dc_plane_info_and_addr() has no case for it and returns -EINVAL.
Yet rgb_formats[] and overlay_formats[] advertise DRM_FORMAT_RGBA8888, so
compositors (KWin on Wayland) pick it for layers with alpha and every
atomic commit fails once per frame, flooding dmesg with

  amdgpu 0000:06:00.0: [drm] *ERROR* Unsupported screen format RA24
  little-endian (0x34324152)

and forcing a fallback to composition on every frame.

Remove the two entries.  No working configuration is lost: a plane that
advertises RGBA8888 can never convert it in fill_dc_plane_info_and_addr(),
on any display generation.

Tested on Raphael 610M (DCN 3.1.5): before, ~240 atomic commits per second
failed with -EINVAL; after, the plane format list no longer contains RA24
and dmesg is clean.

Link: https://gitlab.freedesktop.org/drm/amd/-/work_items/5339
Signed-off-by: MuXian Chen <a18165943161@gmail.com>
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
index 9139823..6d5454f 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
@@ -49,11 +49,13 @@
  * For future use cases we should either initialize them dynamically based on
  * plane capabilities, or initialize this array to all formats, so internal drm
  * check will succeed, and let DC implement proper check
+ *
+ * DC has no surface format for DRM_FORMAT_RGBA8888, so it must not be
+ * advertised here or in overlay_formats.
  */
 static const uint32_t rgb_formats[] = {
 	DRM_FORMAT_XRGB8888,
 	DRM_FORMAT_ARGB8888,
-	DRM_FORMAT_RGBA8888,
 	DRM_FORMAT_XRGB2101010,
 	DRM_FORMAT_XBGR2101010,
 	DRM_FORMAT_ARGB2101010,
@@ -70,7 +72,6 @@ static const uint32_t rgb_formats[] = {
 static const uint32_t overlay_formats[] = {
 	DRM_FORMAT_XRGB8888,
 	DRM_FORMAT_ARGB8888,
-	DRM_FORMAT_RGBA8888,
 	DRM_FORMAT_XBGR8888,
 	DRM_FORMAT_ABGR8888,
 	DRM_FORMAT_RGB565,
-- 
2.56.0

Re: [PATCH RESEND] drm/amd/display: drop RGBA8888 from plane format lists

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] drm/amd/display: broken KUnit test dm_test_get_plane_formats()
--

commit 7e1267e34b886f02722f18cb9144a71e1afc6858
Author: MuXian Chen <a18165943161@gmail.com>

drm/amd/display: drop RGBA8888 from plane format lists

This commit removes DRM_FORMAT_RGBA8888 from rgb_formats[] and
overlay_formats[] because DC has no surface pixel format for RGBA8888 and
fails atomic commits in fill_dc_plane_info_and_addr() when compositors select
it.

Link: https://gitlab.freedesktop.org/drm/amd/-/work_items/5339

> diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
> index 91398230d4083..6d5454fb5e379 100644
> --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
> +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
> @@ -49,11 +49,13 @@
>   * For future use cases we should either initialize them dynamically based on
>   * plane capabilities, or initialize this array to all formats, so internal drm
>   * check will succeed, and let DC implement proper check
> + *
> + * DC has no surface format for DRM_FORMAT_RGBA8888, so it must not be
> + * advertised here or in overlay_formats.
>   */
>  static const uint32_t rgb_formats[] = {
>  	DRM_FORMAT_XRGB8888,
>  	DRM_FORMAT_ARGB8888,
> -	DRM_FORMAT_RGBA8888,
>  	DRM_FORMAT_XRGB2101010,
>  	DRM_FORMAT_XBGR2101010,
>  	DRM_FORMAT_ARGB2101010,
> @@ -70,7 +72,6 @@ static const uint32_t rgb_formats[] = {
>  static const uint32_t overlay_formats[] = {
>  	DRM_FORMAT_XRGB8888,
>  	DRM_FORMAT_ARGB8888,
> -	DRM_FORMAT_RGBA8888,

[Severity: Medium]
Does removing DRM_FORMAT_RGBA8888 from rgb_formats[] and overlay_formats[] in
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c break the KUnit test
dm_test_get_plane_formats() in
drivers/gpu/drm/amd/display/amdgpu_dm/tests/amdgpu_dm_plane_test.c?

Because amdgpu_dm_plane_get_plane_formats() populates the format list by
iterating over ARRAY_SIZE(rgb_formats) (reduced from 14 to 13) for primary and
universal planes and ARRAY_SIZE(overlay_formats) (reduced from 9 to 8) for
overlay planes, running the amdgpu_dm_plane KUnit suite triggers failures in
dm_test_get_plane_formats():

	plane->type = DRM_PLANE_TYPE_PRIMARY;
	KUNIT_EXPECT_EQ(test, amdgpu_dm_plane_get_plane_formats(plane, NULL, formats, 32), 14);

	cap->pixel_format_support.nv12 = true;
	cap->pixel_format_support.p010 = true;
	cap->pixel_format_support.fp16 = true;
	KUNIT_EXPECT_EQ(test, amdgpu_dm_plane_get_plane_formats(plane, cap, formats, 32), 20);

	plane->type = DRM_PLANE_TYPE_OVERLAY;
	KUNIT_EXPECT_EQ(test, amdgpu_dm_plane_get_plane_formats(plane, NULL, formats, 32), 9);

	universal_cap->type = DC_PLANE_TYPE_DCN_UNIVERSAL;
	KUNIT_EXPECT_EQ(test,
			amdgpu_dm_plane_get_plane_formats(plane, universal_cap, formats, 32),
			14);

Should these expected counts in dm_test_get_plane_formats() be updated to 13,
19, 8, and 13 to match the new array sizes?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261030171816.11610-1-a18165943161@gmail.com?part=1

[PATCH v2] drm/amd/display: drop RGBA8888 from plane format lists

[MuXian Chen]

DC's enum surface_pixel_format has no RGBA8888 (memory order A,B,G,R):
fill_dc_plane_info_and_addr() has no case for it and returns -EINVAL.
Yet rgb_formats[] and overlay_formats[] advertise DRM_FORMAT_RGBA8888, so
compositors (KWin on Wayland) pick it for layers with alpha and every
atomic commit fails once per frame, flooding dmesg with

  amdgpu 0000:06:00.0: [drm] *ERROR* Unsupported screen format RA24
  little-endian (0x34324152)

and forcing a fallback to composition on every frame.

Remove the two entries.  No working configuration is lost: a plane that
advertises RGBA8888 can never convert it in fill_dc_plane_info_and_addr(),
on any display generation.

Update the KUnit expectations in dm_test_get_plane_formats() accordingly:
14/20/9/14 -> 13/19/8/13.

Tested on Raphael 610M (DCN 3.1.5): before, ~240 atomic commits per second
failed with -EINVAL; after, the plane format list no longer contains RA24
and dmesg is clean.

Link: https://gitlab.freedesktop.org/drm/amd/-/work_items/5339
Signed-off-by: MuXian Chen <a18165943161@gmail.com>
---
v2:
 - update dm_test_get_plane_formats() expectations after dropping RGBA8888
   (reported by Sashiko AI review).
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c   | 5 +++--
 .../amd/display/amdgpu_dm/tests/amdgpu_dm_plane_test.c    | 8 ++++----
 2 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
index 9139823..6d5454f 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
@@ -49,11 +49,13 @@
  * For future use cases we should either initialize them dynamically based on
  * plane capabilities, or initialize this array to all formats, so internal drm
  * check will succeed, and let DC implement proper check
+ *
+ * DC has no surface format for DRM_FORMAT_RGBA8888, so it must not be
+ * advertised here or in overlay_formats.
  */
 static const uint32_t rgb_formats[] = {
 	DRM_FORMAT_XRGB8888,
 	DRM_FORMAT_ARGB8888,
-	DRM_FORMAT_RGBA8888,
 	DRM_FORMAT_XRGB2101010,
 	DRM_FORMAT_XBGR2101010,
 	DRM_FORMAT_ARGB2101010,
@@ -70,7 +72,6 @@ static const uint32_t rgb_formats[] = {
 static const uint32_t overlay_formats[] = {
 	DRM_FORMAT_XRGB8888,
 	DRM_FORMAT_ARGB8888,
-	DRM_FORMAT_RGBA8888,
 	DRM_FORMAT_XBGR8888,
 	DRM_FORMAT_ABGR8888,
 	DRM_FORMAT_RGB565,
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/tests/amdgpu_dm_plane_test.c b/drivers/gpu/drm/amd/display/amdgpu_dm/tests/amdgpu_dm_plane_test.c
index 7422e8b..3e7ef6d 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/tests/amdgpu_dm_plane_test.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/tests/amdgpu_dm_plane_test.c
@@ -277,20 +277,20 @@ static void dm_test_get_plane_formats(struct kunit *test)
 	KUNIT_ASSERT_NOT_NULL(test, universal_cap);
 
 	plane->type = DRM_PLANE_TYPE_PRIMARY;
-	KUNIT_EXPECT_EQ(test, amdgpu_dm_plane_get_plane_formats(plane, NULL, formats, 32), 14);
+	KUNIT_EXPECT_EQ(test, amdgpu_dm_plane_get_plane_formats(plane, NULL, formats, 32), 13);
 
 	cap->pixel_format_support.nv12 = true;
 	cap->pixel_format_support.p010 = true;
 	cap->pixel_format_support.fp16 = true;
-	KUNIT_EXPECT_EQ(test, amdgpu_dm_plane_get_plane_formats(plane, cap, formats, 32), 20);
+	KUNIT_EXPECT_EQ(test, amdgpu_dm_plane_get_plane_formats(plane, cap, formats, 32), 19);
 
 	plane->type = DRM_PLANE_TYPE_OVERLAY;
-	KUNIT_EXPECT_EQ(test, amdgpu_dm_plane_get_plane_formats(plane, NULL, formats, 32), 9);
+	KUNIT_EXPECT_EQ(test, amdgpu_dm_plane_get_plane_formats(plane, NULL, formats, 32), 8);
 
 	universal_cap->type = DC_PLANE_TYPE_DCN_UNIVERSAL;
 	KUNIT_EXPECT_EQ(test,
 			amdgpu_dm_plane_get_plane_formats(plane, universal_cap, formats, 32),
-			14);
+			13);
 
 	plane->type = DRM_PLANE_TYPE_CURSOR;
 	KUNIT_EXPECT_EQ(test, amdgpu_dm_plane_get_plane_formats(plane, NULL, formats, 32), 1);
-- 
2.56.0