[PATCH][BUG] Fix the graphic corruption issue on IA64 machines

[PATCH][BUG] Fix the graphic corruption issue on IA64 machines

[izumi]

[-- Attachment #1: Type: text/plain, Size: 1761 bytes --]

Hi,

I encountered the graphic corruption issue on IA64 machines by the
following operations.

 0. enable VGA console(default setting). disable serial console setting.
 1. boot the system at run level 3 and login via text-mode console
(/dev/tty1)
    as the root user.
 2. disable console blanking.
    # setterm -blank 0
 3. start X.
 4. shutdown X.
 5. start X again.
 6. open the gnome-terminal and write someting to /dev/console.
    # ls -l  > /dev/console

The cause of this problem may be VGA console driver's misunderstanding
mode(text/graphic).

I confirmed this problem is fixed by the attached patch, but I don't
know this is the correct fix.

--------------------------------------------------------------------

VGA console driver can misunderstand the current mode(Text/Graphic) under
"disable console blanking" setting.

- start X
  -->  vt_ioctl()    drivers/char/vt_ioctl.c
        do_blank_screen(1)     drivers/char/vt.c
           sw->con_blank(vc_cons[currcons].d, 1, 0)  drivers/char/vt.c
       vgacon_blank()     drivers/video/console/vgacon.c
　        vga_is_gfx = 1 /* enter Graphic mode */

- shutdown X
  -->   vt_ioctl()
          do_unblank_screen(1)    drivers/char/vt.c
            sw->con_blank(vc_cons[currcons].d, 0, leaving_gfx)
        vgacon_blank()
　         vga_is_gfx = 0 /* leave Graphic mode */

When "disable console blank" is set (=> blankinterval=0),
"do_unblank_screen()" function
returns without changing "blank_state", and when "blank_state" is
"blank_off",
"do_blank_screen() function returns without invoking sw->con_blank()
function.
That's why VGA console driver can misunderstand the current mode.

-----------------------------------------------------------------------

Regards,
Taku Izumi


[-- Attachment #2: vt-blank.patch --]
[-- Type: text/plain, Size: 1295 bytes --]

Fix the graphic corruption issue on IA64 machines.
VGA console driver can misunderstand the current mode(Text/Graphic) under 
"disable console blanking" setting. When "disable console blank" is set (blankinterval=0), "do_unblank_screen()" function returns without changing "blank_state", and when "blank_state" is "blank_off", "do_blank_screen() function returns without invoking sw->con_blank() function. That's why VGA console driver can misunderstand the current mode.

Signed-off-by: Nobuhiro Tachino <ntachino@redhat.com>
Signed-off-by: Taku Izumi <izumi2005@soft.fujitsu.com>
---
 drivers/char/vt.c |    2 +-
 1 files changed, 1 insertion(+), 1 deletion(-)


Index: linux-2.6.21.5/drivers/char/vt.c
========================================================================
--- linux-2.6.21.5.org/drivers/char/vt.c	2007-06-13 12:03:11.000000000 +0900
+++ linux-2.6.21.5/drivers/char/vt.c	2007-06-13 12:07:48.000000000 +0900
@@ -3419,12 +3419,12 @@ void do_unblank_screen(int leaving_gfx)
 		return;
 	}
 	vc = vc_cons[fg_console].d;
+	blank_state = blank_normal_wait;
 	if (vc->vc_mode != KD_TEXT)
 		return; /* but leave console_blanked != 0 */
 
 	if (blankinterval) {
 		mod_timer(&console_timer, jiffies + blankinterval);
-		blank_state = blank_normal_wait;
 	}
 
 	console_blanked = 0;

Re: [PATCH][BUG] Fix the graphic corruption issue on IA64 machines

[izumi]

[-- Attachment #1: Type: text/plain, Size: 525 bytes --]

HI,

As a result of the discussion with Pete Zaitcev(zaitcev@redhat.com),
I re-create a patch. This attached patch is revised version.

He pointed out that the former patch may violate the assumptions and
was not safe. Concretely speaking, he concerned that an unexpected
problem may arise somewhere if "blank_state", which is intended to
reflect the state of timer, was shuffled arround.

This revised patch reflects his pointed out. I confirmed this also fixed
the problem.

Regards,
Taku Izumi <izumi.taku@jp.fujitsu.com>

[-- Attachment #2: vt-blank-rev2.patch --]
[-- Type: text/plain, Size: 1649 bytes --]

Fix the graphic corruption issue on IA64 machines.
VGA console driver can misunderstand the current mode(Text/Graphic) under 
"disable console blanking" setting. When "disable console blank" is set (blankinterval=0), 
"do_unblank_screen()" function returns without changing "blank_state", and when 
"blank_state" is "blank_off", "do_blank_screen() function returns without 
invoking sw->con_blank() function. That's why VGA console driver can misunderstand 
the current mode.

Signed-off-by: Nobuhiro Tachino <tachino@jp.fujitsu.com>
Signed-off-by: Pete Zaitcev <zaitcev@redhat.com>
Signed-off-by: Taku Izumi <izumi.taku@jp.fujitsu.com>
---
 drivers/char/vt.c |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)


Index: linux-2.6.22/drivers/char/vt.c
=============================================================================
--- linux-2.6.22.org/drivers/char/vt.c	2007-06-27 11:40:03.000000000 -0400
+++ linux-2.6.22/drivers/char/vt.c	2007-06-27 11:24:32.000000000 -0400
@@ -3491,9 +3491,6 @@ void do_blank_screen(int entering_gfx)
 		}
 		return;
 	}
-	if (blank_state != blank_normal_wait)
-		return;
-	blank_state = blank_off;
 
 	/* entering graphics mode? */
 	if (entering_gfx) {
@@ -3501,10 +3498,15 @@ void do_blank_screen(int entering_gfx)
 		save_screen(vc);
 		vc->vc_sw->con_blank(vc, -1, 1);
 		console_blanked = fg_console + 1;
+		blank_state = blank_off;
 		set_origin(vc);
 		return;
 	}
 
+	if (blank_state != blank_normal_wait)
+		return;
+	blank_state = blank_off;
+
 	/* don't blank graphics */
 	if (vc->vc_mode != KD_TEXT) {
 		console_blanked = fg_console + 1;