[PATCH resend] vt: fix potential race in VT_WAITACTIVE handler

[PATCH resend] vt: fix potential race in VT_WAITACTIVE handler

[Michal Januszewski]

From: Michal Januszewski <spock@gentoo.org>

On a multiprocessor machine the VT_WAITACTIVE ioctl call may return 0
if fg_console has already been updated in redraw_screen() but the
console switch itself hasn't been completed. Fix this by checking
fg_console in vt_waitactive() with the console sem held.

Signed-off-by: Michal Januszewski <spock@gentoo.org>
---
This is the 2nd version of this patch. It incorporates Andrew's
suggestions, ie. calls set_current_state() after down() and adds
a comment explaining why acquiring the console sem is necessary.

 drivers/char/vt_ioctl.c |   14 ++++++++++++--
 1 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/drivers/char/vt_ioctl.c b/drivers/char/vt_ioctl.c
index 1fa2da8..0508293 100644
--- a/drivers/char/vt_ioctl.c
+++ b/drivers/char/vt_ioctl.c
@@ -1039,10 +1039,20 @@ int vt_waitactive(int vt)
 
 	add_wait_queue(&vt_activate_queue, &wait);
 	for (;;) {
-		set_current_state(TASK_INTERRUPTIBLE);
 		retval = 0;
-		if (vt == fg_console)
+
+		/* Synchronize with redraw_screen(). By acquiring the console
+		 * semaphore we make sure that the console switch is completed
+		 * before we return. If we didn't wait for the semaphore, we
+		 * could return at a point where fg_console has already been
+		 * updated, but the console switch hasn't been completed. */
+		acquire_console_sem();
+		set_current_state(TASK_INTERRUPTIBLE);
+		if (vt == fg_console) {
+			release_console_sem();
 			break;
+		}
+		release_console_sem();
 		retval = -EINTR;
 		if (signal_pending(current))
 			break;


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: [PATCH resend] vt: fix potential race in VT_WAITACTIVE handler

[Antonino A. Daplas]

On Sun, 2007-04-01 at 14:43 +0200, Michal Januszewski wrote:
> From: Michal Januszewski <spock@gentoo.org>
> 
> On a multiprocessor machine the VT_WAITACTIVE ioctl call may return 0
> if fg_console has already been updated in redraw_screen() but the
> console switch itself hasn't been completed. Fix this by checking
> fg_console in vt_waitactive() with the console sem held.
> 
> Signed-off-by: Michal Januszewski <spock@gentoo.org>
Acked-by: Antonino Daplas <adaplas@pol.net>

Andrew,

Drop the old version I sent to you.

Tony



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV