Re: regression, bisected: openpty fails from 3.7 onwards without devpts

[Jiri Slaby <jslaby@suse.cz>]

[-- Attachment #1: Type: text/plain, Size: 305 bytes --]

On 01/10/2013 11:51 PM, Jiri Slaby wrote:
> On 01/10/2013 11:45 PM, Alan Cox wrote:
>> So we should just fix TIOCGPTN on a pty with no suitable name answer to
>> return -EINVAL
> 
> Yes, I agree as I'm expressed in my second mail. Sorry for the confusion.

Does the attached patch help?

-- 
js
suse labs

[-- Attachment #2: 0001-pty-return-EINVAL-for-TIOCGPTN-for-BSD-ptys.patch --]
[-- Type: text/x-patch, Size: 1307 bytes --]

>From e7f1242ca1d8d6a17c87a85091cc977d2040ef15 Mon Sep 17 00:00:00 2001
From: Jiri Slaby <jslaby@suse.cz>
Date: Fri, 11 Jan 2013 12:06:27 +0100
Subject: [PATCH] pty: return EINVAL for TIOCGPTN for BSD ptys

Commit bbb63c514a3464342967237a51a21ea8f61ab951 (drivers:tty:fix up
ENOIOCTLCMD error handling) changed the default return value from tty
ioctl to be ENOTTY and not EINVAL. This is appropriate.

But in case of TIOCGPTN for the old BSD ptys glibc started failing
because it expects EINVAL to be returned. Only then it continues to
obtain the pts name the other way around.

So fix this case by explicit return of EINVAL in this case.

Signed-off-by: Jiri Slaby <jslaby@suse.cz>
Reported-by: Florian Westphal <fw@strlen.de>
Cc: Alan Cox <alan@linux.intel.com>
---
 drivers/tty/pty.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/tty/pty.c b/drivers/tty/pty.c
index d04c39d..d2fd6f0 100644
--- a/drivers/tty/pty.c
+++ b/drivers/tty/pty.c
@@ -441,6 +441,8 @@ static int pty_bsd_ioctl(struct tty_struct *tty,
 		return pty_get_pktmode(tty, (int __user *)arg);
 	case TIOCSIG:    /* Send signal to other side of pty */
 		return pty_signal(tty, (int) arg);
+	case TIOCGPTN: /* TTY returns ENOTTY, but glibc expects EINVAL here */
+		return -EINVAL;
 	}
 	return -ENOIOCTLCMD;
 }
-- 
1.8.1