[PATCH] tty_set_ldisc receive_room fix

[PATCH] tty_set_ldisc receive_room fix

[Paul Fulghum]

Fix tty_set_ldisc in tty_io.c so that tty->receive_room is
only cleared if actually changing line disciplines.

Without this fix a problem occurs when requesting the line
discipline to change to the same line discipline. In this case
tty->receive_room is cleared but ldisc->open() is not called
to set tty->receive_room back to a sane value. The result
is that tty->receive_room is stuck at 0 preventing the tty
flip buffer from passing receive data to the line discipline.

For example: a switch from N_TTY to N_TTY followed by
a select() call for read input results in data never being
received because tty->receive_room is stuck at zero.

A switch from N_TTY to N_TTY followed by
a read() call works because the read() call itself
sets tty->receive_room correctly (but select does not).

Previously (< 2.6.18) this was not a problem because
the tty flip buffer pushed data to the line discipline
without regard for tty->receive room.

Signed-off-by: Paul Fulghum <paulkf@microgate.com>


--- a/drivers/char/tty_io.c	2007-04-25 22:08:32.000000000 -0500
+++ b/drivers/char/tty_io.c	2007-05-04 14:25:08.000000000 -0500
@@ -936,13 +936,6 @@ restart:
 		return -EINVAL;
 
 	/*
-	 *	No more input please, we are switching. The new ldisc
-	 *	will update this value in the ldisc open function
-	 */
-
-	tty->receive_room = 0;
-
-	/*
 	 *	Problem: What do we do if this blocks ?
 	 */
 
@@ -953,6 +946,13 @@ restart:
 		return 0;
 	}
 
+	/*
+	 *	No more input please, we are switching. The new ldisc
+	 *	will update this value in the ldisc open function
+	 */
+
+	tty->receive_room = 0;
+
 	o_ldisc = tty->ldisc;
 	o_tty = tty->link;
 

Re: [PATCH] tty_set_ldisc receive_room fix

[Alan Cox]

> Previously (< 2.6.18) this was not a problem because
> the tty flip buffer pushed data to the line discipline
> without regard for tty->receive room.

 
> Signed-off-by: Paul Fulghum <paulkf@microgate.com>

Acked-by: Alan Cox <alan@redhat.com>