[PATCH -next 04/27] tty: Clarify re-open behavior of master ptys

Peter Hurley
2014-10-16T20:28:16Z

Re-opening master ptys is not allowed. Once opened and for the remaining
lifetime of the master pty, its tty count is 1. If its tty count has
dropped to 0, then the master pty was closed and TTY_CLOSING was set,
and destruction may begin imminently.

Besides the normal case of a legacy BSD pty master being re-opened
(which always returns -EIO), this code is only reachable in 2 degenerate
cases:
1. The pty master is the controlling terminal (this is possible through
the TIOCSCTTY ioctl). pty masters are not designed to be controlling
terminals and it's an oversight that tiocsctty() ever let that happen.
The attempted open of /dev/tty will always fail. No known program does
this.
2. The legacy BSD pty slave was opened first. The slave open will fail
in pty_open() and tty_release() will commence. But before tty_release()
claims the tty_mutex, there is a very small window where a parallel
master open might succeed. In a test of racing legacy BSD slave and
master parallel opens, where:
slave open attempts: 10000 success:4527 failure:5473
master open attempts: 11728 success:5789 failure:5939
only 8 master open attempts would have succeeded reaching this code and
successfully opened the master pty. This case is not possible with
SysV ptys.

Always return -EIO if a master pty is re-opened or the slave is opened
first and the master opened in parallel (for legacy BSD ptys).

Furthermore, now that changing the slave's count is not required,
the tty_lock is sufficient for preventing concurrent changes to the
tty being re-opened (or failing re-opening).

Signed-off-by: Peter Hurley <peter@hurleysoftware.com>
---
drivers/tty/tty_io.c | 15 ++++-----------
1 file changed, 4 insertions(+), 11 deletions(-)

diff --git a/drivers/tty/tty_io.c b/drivers/tty/tty_io.c
index fd06512..5b6253f 100644
--- a/drivers/tty/tty_io.c
+++ b/drivers/tty/tty_io.c
@@ -1444,9 +1444,9 @@ void tty_driver_remove_tty(struct tty_driver *driver, struct tty_struct *tty)
* @tty - the tty to open
*
* Return 0 on success, -errno on error.
+ * Re-opens on master ptys are not allowed and return -EIO.
*
- * Locking: tty_mutex must be held from the time the tty was found
- * till this open completes.
+ * Locking: Caller must hold tty_lock
*/
static int tty_reopen(struct tty_struct *tty)
{
@@ -1456,16 +1456,9 @@ static int tty_reopen(struct tty_struct *tty)
return -EIO;

if (driver->type == TTY_DRIVER_TYPE_PTY &&
- driver->subtype == PTY_TYPE_MASTER) {
- /*
- * special case for PTY masters: only one open permitted,
- * and the slave side open count is incremented as well.
- */
- if (tty->count)
- return -EIO;
+ driver->subtype == PTY_TYPE_MASTER)
+ return -EIO;

- tty->link->count++;
- }
tty->count++;

WARN_ON(!tty->ldisc);
--
2.1.1