Re: [PATCH] drm/ast: Fix big-endian support

[Thomas Zimmermann <tzimmermann@suse.de>]

Hi

Am 12.12.25 um 21:15 schrieb René Rebe:
> Hi,
>
> On Thu, 11 Dec 2025 15:31:01 +0100 (CET), René Rebe <rene@exactco.de> wrote:
>
>>>   	/* write checksum + signature */
>>> +	writel(swab32(csum), dst);
>>> +	writel(swab32(width), dst + AST_HWC_SIGNATURE_SizeX);
>>> +	writel(swab32(height), dst + AST_HWC_SIGNATURE_SizeY);
>>> +	writel(swab32(0), dst + AST_HWC_SIGNATURE_HOTSPOTX);
>>> +	writel(swab32(0), dst + AST_HWC_SIGNATURE_HOTSPOTY);
>>> +#else
>>> +	memcpy_toio(dst, src, AST_HWC_SIZE);
>>>   	dst += AST_HWC_SIZE;
>>> +
>>> +	/* write checksum + signature */
>>>   	writel(csum, dst);
>>>   	writel(width, dst + AST_HWC_SIGNATURE_SizeX);
>>>   	writel(height, dst + AST_HWC_SIGNATURE_SizeY);
>>>   	writel(0, dst + AST_HWC_SIGNATURE_HOTSPOTX);
>>>   	writel(0, dst + AST_HWC_SIGNATURE_HOTSPOTY);
>>> +#endif
>> I'm pretty sure this will break the cursor, as the position was
>> working correctly and I only had to swap the cursor image data. The
>> csum will also not be identical anyway, as the checksum function
>> computes it in native byte order. Theoretically that would have to be
>> changed. However, I do not see where it is really used, maybe only
>> some special remote desktop vendor protocol that I'm not using. Maybe
>> the exact checksum does not even matter and is only used as
>> optimization to not resend an unchanged cursor image.
>>
>> I'll send a final version after validating it w/ HW later.
> I just sent a more minimally tested V4 removing the superflous unused
> fmt_cnv_state to ast_set_cursor_image you somehow had added.

That is a leftover form earlier drm_fb_swab() code.

>
> As the additional writen cursor RDP service writes are untested, I
> left them out.
>
> There is still something suspect or buggy: the 2-bit X11 cursor is
> filled with transparency, while the ARGB RGB channel work now. Modern
> ARGB Xcursor theme also look strange. Not sure if that is just due to
> loosing 4-bit precision and thus half of the dynamic range with all
> the shadows. To me at least the 2-bit transparent X cursor looks like
> a fmt conversion bug in some layer that we would need to continue
> debugging another day. At least the framebuffer / installer text would
> be more readble upstream now, too ;-)

The kernel's ast driver doesn't support 2-bit cursors. It's likely a 
problem somewhere in Xorg or the rendering libraries.

Best regards
Thomas

>
> Thanks,
> 	René
>

-- 
--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com
GF: Jochen Jaser, Andrew McDonald, Werner Knoblich, (HRB 36809, AG Nürnberg)