[PATCH] [efifb] Fix 16 color palette entry calculation

[PATCH] [efifb] Fix 16 color palette entry calculation

[Max Staudt]

When using efifb with a 16-bit (5:6:5) visual, fbcon's text is rendered
in the wrong colors - e.g. text gray (#aaaaaa) is rendered as green
(#50bc50) and neighboring pixels have slightly different values
(such as #50bc78).

The reason is that fbcon loads its 16 color palette through
efifb_setcolreg(), which in turn calculates a 32-bit value to write
into memory for each palette index.
Until now, this code could only handle 8-bit visuals and didn't mask
overlapping values when ORing them.

With this patch, fbcon displays the correct colors when a qemu VM is
booted in 16-bit mode (in GRUB: "set gfxpayload=800x600x16").

Signed-off-by: Max Staudt <mstaudt@suse.de>
---
 drivers/video/fbdev/efifb.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/video/fbdev/efifb.c b/drivers/video/fbdev/efifb.c
index 924bad4..37a37c4 100644
--- a/drivers/video/fbdev/efifb.c
+++ b/drivers/video/fbdev/efifb.c
@@ -50,9 +50,9 @@ static int efifb_setcolreg(unsigned regno, unsigned red, unsigned green,
 		return 1;
 
 	if (regno < 16) {
-		red   >>= 8;
-		green >>= 8;
-		blue  >>= 8;
+		red   >>= 16 - info->var.red.length;
+		green >>= 16 - info->var.green.length;
+		blue  >>= 16 - info->var.blue.length;
 		((u32 *)(info->pseudo_palette))[regno] =
 			(red   << info->var.red.offset)   |
 			(green << info->var.green.offset) |
-- 
2.6.6

Re: [PATCH] [efifb] Fix 16 color palette entry calculation

[Greg KH]

On Tue, Jun 07, 2016 at 03:45:43PM +0200, Max Staudt wrote:
> When using efifb with a 16-bit (5:6:5) visual, fbcon's text is rendered
> in the wrong colors - e.g. text gray (#aaaaaa) is rendered as green
> (#50bc50) and neighboring pixels have slightly different values
> (such as #50bc78).
> 
> The reason is that fbcon loads its 16 color palette through
> efifb_setcolreg(), which in turn calculates a 32-bit value to write
> into memory for each palette index.
> Until now, this code could only handle 8-bit visuals and didn't mask
> overlapping values when ORing them.
> 
> With this patch, fbcon displays the correct colors when a qemu VM is
> booted in 16-bit mode (in GRUB: "set gfxpayload=800x600x16").
> 
> Signed-off-by: Max Staudt <mstaudt@suse.de>
> ---
>  drivers/video/fbdev/efifb.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 

<formletter>

This is not the correct way to submit patches for inclusion in the
stable kernel tree.  Please read Documentation/stable_kernel_rules.txt
for how to do this properly.

</formletter>

[PATCH RESEND] fbdev/efifb: Fix 16 color palette entry calculation

[Max Staudt]

When using efifb with a 16-bit (5:6:5) visual, fbcon's text is rendered
in the wrong colors - e.g. text gray (#aaaaaa) is rendered as green
(#50bc50) and neighboring pixels have slightly different values
(such as #50bc78).

The reason is that fbcon loads its 16 color palette through
efifb_setcolreg(), which in turn calculates a 32-bit value to write
into memory for each palette index.
Until now, this code could only handle 8-bit visuals and didn't mask
overlapping values when ORing them.

With this patch, fbcon displays the correct colors when a qemu VM is
booted in 16-bit mode (in GRUB: "set gfxpayload=800x600x16").

Fixes: 7c83172b98e5 ("x86_64 EFI boot support: EFI frame buffer driver")  # v2.6.24+
Signed-off-by: Max Staudt <mstaudt@suse.de>
Acked-By: Peter Jones <pjones@redhat.com>
Cc: stable@vger.kernel.org
---
 drivers/video/fbdev/efifb.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/video/fbdev/efifb.c b/drivers/video/fbdev/efifb.c
index 924bad4..37a37c4 100644
--- a/drivers/video/fbdev/efifb.c
+++ b/drivers/video/fbdev/efifb.c
@@ -50,9 +50,9 @@ static int efifb_setcolreg(unsigned regno, unsigned red, unsigned green,
 		return 1;
 
 	if (regno < 16) {
-		red   >>= 8;
-		green >>= 8;
-		blue  >>= 8;
+		red   >>= 16 - info->var.red.length;
+		green >>= 16 - info->var.green.length;
+		blue  >>= 16 - info->var.blue.length;
 		((u32 *)(info->pseudo_palette))[regno] =
 			(red   << info->var.red.offset)   |
 			(green << info->var.green.offset) |
-- 
2.6.6

Re: [PATCH RESEND] fbdev/efifb: Fix 16 color palette entry calculation

[Tomi Valkeinen]

[-- Attachment #1.1: Type: text/plain, Size: 1063 bytes --]



On 13/06/16 20:15, Max Staudt wrote:
> When using efifb with a 16-bit (5:6:5) visual, fbcon's text is rendered
> in the wrong colors - e.g. text gray (#aaaaaa) is rendered as green
> (#50bc50) and neighboring pixels have slightly different values
> (such as #50bc78).
> 
> The reason is that fbcon loads its 16 color palette through
> efifb_setcolreg(), which in turn calculates a 32-bit value to write
> into memory for each palette index.
> Until now, this code could only handle 8-bit visuals and didn't mask
> overlapping values when ORing them.
> 
> With this patch, fbcon displays the correct colors when a qemu VM is
> booted in 16-bit mode (in GRUB: "set gfxpayload=800x600x16").
> 
> Fixes: 7c83172b98e5 ("x86_64 EFI boot support: EFI frame buffer driver")  # v2.6.24+
> Signed-off-by: Max Staudt <mstaudt@suse.de>
> Acked-By: Peter Jones <pjones@redhat.com>
> Cc: stable@vger.kernel.org
> ---
>  drivers/video/fbdev/efifb.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)

Thanks, queued for v4.9.

 Tomi



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]