[PATCH] connect() return value.

[PATCH] connect() return value.

[Geoffrey Lee]

[-- Attachment #1: Type: text/plain, Size: 1106 bytes --]

Hi,


I posted this on linux-kernel but didn't get a reply.

Anyway here's a patch for it.



Problem:

(1)

While you try to connect() to port 0 on ipv4 / ipv6, it happily does it.
This doesn't seem to be correct, as I remember port 0 is reserved.
For reference FreeBSD / Solaris / OSF return -EADDRNOTAVAIL for this.

This is what my patch fixes, for ipv4 and ipv6 for both tcp and udp,
I don't know what should be done for the other protocols that is 
supported.


(2)

connect() doesn't return -EINTR on signal, rather it exits the signal
handler after processing it and returns an error if an error occurred
for the connection or does the connect to completion. The Linux man page 
implies that we don't ever return an -EINTR on Linux, but I don't know what 
is the correct behavior.  For reference, FreeBSD / Solaris / OSF return
-EINTR for an interrupted connect().

The patch also addresses this. However, it is noted on OSF that even
though connect() returns with -EINTR if the connection succeeds you can
still send and receive through that socket.


(Please CC:, not on list. Thanks).

	-- G.
	

[-- Attachment #2: network-patch --]
[-- Type: text/plain, Size: 2385 bytes --]

diff -ruNp linux-2.4.19/net/ipv4/tcp_ipv4.c linux-2.4.19-glee/net/ipv4/tcp_ipv4.c
--- linux-2.4.19/net/ipv4/tcp_ipv4.c	2002-08-03 19:26:04.000000000 +1000
+++ linux-2.4.19-glee/net/ipv4/tcp_ipv4.c	2002-08-10 19:34:24.000000000 +1000
@@ -763,6 +763,9 @@ int tcp_v4_connect(struct sock *sk, stru
 	if (usin->sin_family != AF_INET)
 		return(-EAFNOSUPPORT);
 
+	if (usin->sin_port == 0)
+		return(-EADDRNOTAVAIL);
+
 	nexthop = daddr = usin->sin_addr.s_addr;
 	if (sk->protinfo.af_inet.opt && sk->protinfo.af_inet.opt->srr) {
 		if (daddr == 0)
diff -ruNp linux-2.4.19/net/ipv4/udp.c linux-2.4.19-glee/net/ipv4/udp.c
--- linux-2.4.19/net/ipv4/udp.c	2002-08-03 19:26:04.000000000 +1000
+++ linux-2.4.19-glee/net/ipv4/udp.c	2002-08-10 19:34:52.000000000 +1000
@@ -723,6 +723,9 @@ int udp_connect(struct sock *sk, struct 
 	if (usin->sin_family != AF_INET) 
 	  	return -EAFNOSUPPORT;
 
+	if (usin->sin_port == 0)
+		return -EADDRNOTAVAIL;
+
 	sk_dst_reset(sk);
 
 	oif = sk->bound_dev_if;
diff -ruNp linux-2.4.19/net/ipv6/tcp_ipv6.c linux-2.4.19-glee/net/ipv6/tcp_ipv6.c
--- linux-2.4.19/net/ipv6/tcp_ipv6.c	2002-08-03 19:26:04.000000000 +1000
+++ linux-2.4.19-glee/net/ipv6/tcp_ipv6.c	2002-08-10 19:35:57.000000000 +1000
@@ -539,6 +539,9 @@ static int tcp_v6_connect(struct sock *s
 	if (usin->sin6_family != AF_INET6) 
 		return(-EAFNOSUPPORT);
 
+	if (usin->sin6_port == 0)
+		return -EADDRNOTAVAIL;
+
 	fl.fl6_flowlabel = 0;
 	if (np->sndflow) {
 		fl.fl6_flowlabel = usin->sin6_flowinfo&IPV6_FLOWINFO_MASK;
diff -ruNp linux-2.4.19/net/ipv6/udp.c linux-2.4.19-glee/net/ipv6/udp.c
--- linux-2.4.19/net/ipv6/udp.c	2002-08-03 19:26:04.000000000 +1000
+++ linux-2.4.19-glee/net/ipv6/udp.c	2002-08-10 19:36:22.000000000 +1000
@@ -231,6 +231,9 @@ int udpv6_connect(struct sock *sk, struc
 	if (usin->sin6_family != AF_INET6) 
 	  	return -EAFNOSUPPORT;
 
+	if (usin->sin6_port == 0)
+		return -EADDRNOTAVAIL;
+
 	fl.fl6_flowlabel = 0;
 	if (np->sndflow) {
 		fl.fl6_flowlabel = usin->sin6_flowinfo&IPV6_FLOWINFO_MASK;
diff -ruNp linux-2.4.19/net/socket.c linux-2.4.19-glee/net/socket.c
--- linux-2.4.19/net/socket.c	2002-08-03 19:26:04.000000000 +1000
+++ linux-2.4.19-glee/net/socket.c	2002-08-10 19:37:23.000000000 +1000
@@ -1118,6 +1118,9 @@ asmlinkage long sys_connect(int fd, stru
 out_put:
 	sockfd_put(sock);
 out:
+	if (signal_pending(current))
+		return -EINTR;
+
 	return err;
 }
 

Re: [PATCH] connect() return value.

[kuznet]

Hello!

> connect() doesn't return -EINTR on signal,

It does, when you ask not to restart after signal.


> The patch also addresses this.

It would be better if it did not. :-)

Alexey

Re: [PATCH] connect() return value.

[Geoffrey Lee]

Yo,

On Mon, Aug 12, 2002 at 12:46:51AM +0400, kuznet@ms2.inr.ac.ru wrote:
> Hello!
> 
> > connect() doesn't return -EINTR on signal,
> 
> It does, when you ask not to restart after signal.
> 

True. What I was getting at is the default behavior of an OS with 
interruptibility for connect(), i.e. using signal() instead of
specifying / unspecifying the SA_RESTART flag with sigaction(), though
I will acknowledge that if you're using signal() you may be in for 
some unpleasant surprises with signal interruptibility anyway.

I've done a bit more testing. It seems that OSF1 (Digital UNIX) will not 
return -EINTR by default with signal(), contrary to what I have previously
stated. I must have been on drugs. However, Solaris will do this 
(SunOS 5.6) by default with signal(), and I wanted to know what 
is the "correct" behavior for connect() by default with signal().



So, current standings are with connect() interruptibility by default
(i.e. with signal()) are:

FreeBSD 4.6-STABLE: -EINTR
Digital UNIX (OSF1 V4.0): To completion
SunOS 5.6: -EINTR
Linux (2.4): To completion


> 
> > The patch also addresses this.
> 
> It would be better if it did not. :-)
> 



I only patched it from what I observed with other Unices. :-)

I patched it largely due to observing what FreeBSD does with their
sockets connect() call.

Because my patched addresses 2 issues, if you are going to apply the
patch for the -EADDRNOTAVAIL issue, and you want a re-worked patch
without the connect() -EINTR issue, please let me know. But
it should be trivial to hand edit my prevoius patch posted.



	-- G.
	

Re: [PATCH] connect() return value.

[kuznet]

Hello!

>					     I wanted to know what 
> is the "correct" behavior for connect() by default with signal().

This simply does not matter, that's answer.

Actually, it is not so easy to implement restartable connect().
I think this is the only reason why it not restartable in some OSes.

BTW could you make the following experiments on the same OSes:
connect() on nonblocking socket, then repeat the connect()
until it returns EISCONN. I guess the behaviour also will be different.


> it should be trivial to hand edit my prevoius patch posted.

Yes, of course. Though the first part of it also from the class
"does not matter", it is worth to do this just for sanity.

Alexey

Re: [PATCH] connect() return value.

[Geoffrey Lee]

On Mon, Aug 12, 2002 at 05:28:38AM +0400, kuznet@ms2.inr.ac.ru wrote:
> Hello!
> 
> >					     I wanted to know what 
> > is the "correct" behavior for connect() by default with signal().
> 
> This simply does not matter, that's answer.
> 
> Actually, it is not so easy to implement restartable connect().
> I think this is the only reason why it not restartable in some OSes.
> 


True, probably.

Solaris will not restart a connect() even with the SA_RESTART set
in sa_flags.


> BTW could you make the following experiments on the same OSes:
> connect() on nonblocking socket, then repeat the connect()
> until it returns EISCONN. I guess the behaviour also will be different.
> 
> 



In brief, it is what we expect. After we do a connect() on a
non-blocking socket, it returns -EINPROGRESS. On subsequent
attempts it returns -EALREADY, then after it is connected
it returns -EISCONN. This behavior is consistent in all operating
systems which I have tested, which include the following:

Digital UNIX (OSF1 4.0)
Solaris (SunOS 5.6)
Linux (2.4.18)



On a LAN, when we run the program on first try, it is usually possible
to see -EINPROGRESS, -EALREADY, then -EISCONN. On subsequent
runs it normally returns -EINPROGRESS, then -EISCONN. This 
behavior can be explained by arp interaction and is consistent
with what we expect. 

On a WAN link, we see it return -EINPROGRESS, -EALREADY, then
-EISCONN all of the time.

Not tested on BSD yet, but I expect the results to be consistent
with my current findings. I can post the results later if you 
want them.

This is all fine, but what is interesting is what follows aftewards.
It was noted in Richard Steven's UNIX Network Programming that there are
several ways which you could find out to see if the connection has
succeeded in the background. One way mentioned is to call a read()
with a length of 0. With this method, it implies that read() must return
an error for an in-progress socket, otherwise there would be no way to
distinguish between a in-progress socket or a completed socket. To test
this, we write a program to connect to a open TCP port on a remote machine
on the WAN (to maximize the delay).

We set the socket to non-blocking then we issue a TCP connect(). 
connect() will return immediately while the connection is attempted
in the background asynchroniously. After the return from connect() 
we immediately issue a read() with a length of 0, and we note the
return value from read, the errno (if applicable). Finally, we issue
a second connect() and we note the return value, and errno (if 
applicable).

This is what happens on Digital UNIX (OSF1 4.0):

read() returns 0 with an errno of 36. 36 on Digital UNIX corresponds to
-EINPROGRESS. The errno isn't from the read but from our first
connect(). On second connect(), it returns -1, with an errno of
-EALREADY.

On Solaris (SunOS 5.6):

read() returns -1 with an errno of ENOTCONN. On second connect() it
returns -1 with an errno of EALREADY.

On Linux (v 2.4.18):

read() returns 0 with an errno of 115. On Linux 115 corresponds to 
-EINPROGRESS. So as with Digital UNIX, the errno isn't from read but
from our first conect(). On second connect(), it returns -1 with
an errno of -EALREADY.


Is using write() instead of read() a better way? 

To do this, we change our previous program, instead of reading 0
bytes this time we issue a write() of 0 bytes.

On Solaris (SunOS 5.6):

write() returns -1 with an errno of -ENOTCONN.

On Digital UNIX (OSF1 4.0):

write() returns -1 with an errno of -ENOTCONN. We note that previously
a read() returned 0. This behavior is inconsistent. There is a bug in 
Digital UNIX.

On Linux (v 2.4.18):

write() returns 0.


We note that Solaris and Linux are consistent with their read() and write()
for a in-progress socket.

I guess the moral story is don't try to use read() or write() to test 
if a socket is connected or not. :-) It looks to me that issuing a 
second connect() and see if it returns -EISCONN is probably a more
portable way. Of course, following up on the previous discussion, one
must handle a -EALREADY from connect() as well.



> > it should be trivial to hand edit my prevoius patch posted.
> 
> Yes, of course. Though the first part of it also from the class
> "does not matter", it is worth to do this just for sanity.
> 

Yep. Though I would really like it if it conforms to what the rest of 
the Unices does with that behavior.

So, we can agree that the first part should be applied on the next
networking merge? :-)


	-- G.

Re: [PATCH] connect() return value.

[kuznet]

Hello!

> it returns -EISCONN. This behavior is consistent in all operating
> systems which I have tested, which include the following:

This is very strange. Linux should return success once.

Probably, I should explain why I ask this. This is a necessary
element of implementation of restartable connect(). If connect()
is restarted after signal, it should return success on connected
socket sometimes rather than -EISCONN.


> On Linux (v 2.4.18):
> 
> read() returns 0 with an errno of 115. On Linux 115 corresponds to 
> -EINPROGRESS.

I am sorry, this machine is not running Linux.

Zero length read() always returns zero, not depending of socket state.



> Yep. Though I would really like it if it conforms to what the rest of 
> the Unices does with that behavior.

Non-sense. We are not going to emulate pathological cases.


> So, we can agree

Did we argue about this? :-)

Alexey

Re: [PATCH] connect() return value.

[Geoffrey Lee]

On Tue, Aug 13, 2002 at 08:44:21AM +0400, kuznet@ms2.inr.ac.ru wrote:
> Hello!
> 
> > it returns -EISCONN. This behavior is consistent in all operating
> > systems which I have tested, which include the following:
> 
> This is very strange. Linux should return success once.
> 
> Probably, I should explain why I ask this. This is a necessary
> element of implementation of restartable connect(). If connect()
> is restarted after signal, it should return success on connected
> socket sometimes rather than -EISCONN.
> 



I see.

Possibly, I should be a bit clearer, to make sure that I am not
not confusing with what are you are asking for.

The summaries so far, for Linux:


(1) no signals (so it runs the sighandler but doesn't return from the
sys call until it completes), non-blocking socket:


-EINPROGRESS, -EALREADY, -EISCONN, in that order. Never returns 0.


(2) SIGALRM, set for 1 second, (and reschedules another for 1 second),
non-blocking


-EINPROGRESS, -EALREADY, -EISCONN, in that order. We note that in 
-EALREADY a signal is processed. That is the SIGALRM handler.


(3) SIGALRM, blocking:

will run sighandler but connect() returns 0 once, running connect()
again gives EISCONN.



Is that what you expected?


> 
> > On Linux (v 2.4.18):
> > 
> > read() returns 0 with an errno of 115. On Linux 115 corresponds to 
> > -EINPROGRESS.
> 
> I am sorry, this machine is not running Linux.
> 
> Zero length read() always returns zero, not depending of socket state.
> 
> 



It doesn't seem to be standardized what one should return.

So read()  / write()  is not a portable way to test if a socket has been
connected with non-blocking connect.



	-- G.
	

Re: [PATCH] connect() return value.

[kuznet]

Hello!

> (1) no signals (so it runs the sighandler but doesn't return from the
> sys call until it completes), non-blocking socket:
> 
> 
> -EINPROGRESS, -EALREADY, -EISCONN, in that order. Never returns 0.


socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 3
fcntl64(0x3, 0x4, 0x800, 0x400151cc)    = 0
connect(3, {sin_family=AF_INET, sin_port=htons(9), sin_addr=inet_addr("193.233.7.75")}}, 16) = -1 EINPROGRESS (Operation now in progress)
connect(3, {sin_family=AF_INET, sin_port=htons(9), sin_addr=inet_addr("193.233.7.75")}}, 16) = -1 EALREADY (Operation already in progress)
connect(3, {sin_family=AF_INET, sin_port=htons(9), sin_addr=inet_addr("193.233.7.75")}}, 16) = -1 EALREADY (Operation already in progress)
connect(3, {sin_family=AF_INET, sin_port=htons(9), sin_addr=inet_addr("193.233.7.75")}}, 16) = -1 EALREADY (Operation already in progress)
connect(3, {sin_family=AF_INET, sin_port=htons(9), sin_addr=inet_addr("193.233.7.75")}}, 16) = -1 EALREADY (Operation already in progress)
connect(3, {sin_family=AF_INET, sin_port=htons(9), sin_addr=inet_addr("193.233.7.75")}}, 16) = 0
connect(3, {sin_family=AF_INET, sin_port=htons(9), sin_addr=inet_addr("193.233.7.75")}}, 16) = -1 EISCONN (Transport endpoint is already connected)

> Is that what you expected?

Well, it proves that one of us makes tests not on linux.

Alexey

Re: [PATCH] connect() return value.

[Geoffrey Lee]

> > Is that what you expected?
> 
> Well, it proves that one of us makes tests not on linux.
> 


Ah, yes you're right. I was feeling stupid and I forgot to clear the
errno because re-trying the connect().

It looks my brain has gone mouldy. Sorry. :-)



	-- G.

Re: [PATCH] connect() return value.

[kuznet]

Hello!

> Ah, yes you're right. I was feeling stupid and I forgot to clear the
> errno because re-trying the connect().

Good. :-)

Could you repeat corrected test with repreated nonblocking connect()
on those OSes?

Alexey

Re: [PATCH] connect() return value.

[Geoffrey Lee]

[-- Attachment #1: Type: text/plain, Size: 2129 bytes --]

On Tue, Aug 13, 2002 at 05:47:14PM +0400, kuznet@ms2.inr.ac.ru wrote:
> Hello!
> 
> > Ah, yes you're right. I was feeling stupid and I forgot to clear the
> > errno because re-trying the connect().
> 
> Good. :-)
> 
> Could you repeat corrected test with repreated nonblocking connect()
> on those OSes?
> 

Alright, it looks like I've been smoking too much pot. I'm going to post
the source code [ugly] here as well so what you see is what you get (no more
oops-I-forgot-to-clear-errno oddities again :-)


Let's redo the experiment from the start:

This is the following OS I will test against:

SunOS 5.6
OSF1 4.0
Linux v2.4.18

Source files:
connect.c
connect3.c

connect.c tests for interruptibility with a particular OS.
connect3.c tests for what connect returns if the socket is set 
into non-blocking mode.


First off, we note the behavior of the with regard to interruptibility 
of connect. I use SIGALRM, sigaction with SA_RESTART, and connect to a 
valid IP where the host is down / no host is present. Default blocking
behavior.

SunOS 5.6:

sighandler called, connect fails with -EINTR


OSF1 4.0

sighandler called, connect fails with -EINTR

(I have no idea why I "corrected" myself before that it's restartable,
possibly some cruft I left behind with a read() following connect. So
a return of -EINTR is my Final Answer. Sorry for that confusion.)

Linux 2.4.18:

sighandler is called, but we see that the program does not exit, this
seems to imply that the connect is restarted.


Now, we try with no signals, connect to a valid IP where host is up with
a valid port listening. We use fcntl() to set the socket to
non-blocking mode.



SunOS 5.6

-EINPROGRESS, -EALREADY, -EISCONN, in that order.


OSF1 4.0


-EINPROGRESS, -EALREADY, -EISCONN in that order.



Linux 2.4.18

-EINPROGRESS, -EALREADY, 0 (Success), -EISCONN, in that order.


So Linux does indeed return 0, and this is the magic value that
we are looking for. People writing portably will be required to
handle 0 in Linux, as well as possibly an error of -EISCONN (this
error is ok and seems to be expected in OSF1 4.0 and SunOS 5.6).




	-- G.

[-- Attachment #2: connect.c --]
[-- Type: text/x-csrc, Size: 1700 bytes --]

#include <sys/socket.h>

#include <netinet/in.h>

#include <errno.h>
#include <signal.h>
#include <stdio.h>
#include <string.h>

#include <unistd.h>

#define PORT 22
#define HOST "129.94.44.44"

void handler(int);

int main(int argc, char **argv)
{


	struct sockaddr_in saddr;
	int ret;
	int c;
	int fd;
	char buf[BUFSIZ];
	int val;
	size_t len;
	struct sigaction act, oact;
	char *host = HOST;
	unsigned short port = PORT;

	val = 0;
	len = sizeof(val);

	while ((c = getopt(argc, argv, "h:p:")) != -1) {
                switch (c) {
                        case 'h':
                                host = strdup(optarg);
                                break;
                        case 'p':
                                port = atoi(optarg);
                                break;
                        default:
                                break;
                }
        }      

	bzero(&act, sizeof(struct sigaction));
	bzero(&oact, sizeof(struct sigaction));

	act.sa_flags |= SA_RESTART;
	act.sa_handler = handler;

	ret = sigaction(SIGALRM, &act, &oact);


	if (ret < 0) {

		printf("sigaction\n");
		exit(1);
	}

	memset(&saddr, 0, sizeof(struct sockaddr_in));

	saddr.sin_family = AF_INET;
	saddr.sin_port   = htons(port);
	saddr.sin_addr.s_addr = inet_addr(host);

	fd = socket(AF_INET, SOCK_STREAM, 0);

	if (fd < 0) {
		printf("socket\n");
		exit(1);
	}
	


	alarm(1);

	errno = 0;

	ret = connect(fd, (struct sockaddr *)&saddr, 
		sizeof(struct sockaddr));

	printf("returned from connect\n");

	if (ret < 0) {
		printf("connect\n");
		printf("%s\n", strerror(errno));

	}

	close(fd);

	return(0);
}

void handler(int signo)
{

	printf("in signal hander\n");
	return ;

}

[-- Attachment #3: connect3.c --]
[-- Type: text/x-csrc, Size: 1473 bytes --]

#include <sys/socket.h>

#include <netinet/in.h>

#include <errno.h>
#include <fcntl.h>
#include <signal.h>
#include <stdio.h>
#include <string.h>

#include <unistd.h>

#define HOST "192.168.0.1"
#define PORT 22

void handler(int);

int main(int argc, char **argv)
{
	struct sockaddr_in saddr;
	int ret;
	int fd;
	int i;
	char buf[BUFSIZ];
	int val;
	int c;
	unsigned short port = PORT;
	size_t len;
	char *host = HOST;

	val = 0;
	len = sizeof(val);

	while ((c = getopt(argc, argv, "h:p:")) != -1) {
		switch (c) {
			case 'h':
				host = strdup(optarg);
				break;
			case 'p':
				port = atoi(optarg);
				break;
			default:
				break;
		}
	}
	
	memset(&saddr, 0, sizeof(struct sockaddr_in));

	saddr.sin_family = AF_INET;
	saddr.sin_port   = htons(port);
	saddr.sin_addr.s_addr = inet_addr(host);

	fd = socket(AF_INET, SOCK_STREAM, 0);

	if (fd < 0) {
		printf("socket\n");
		exit(1);
	}

	val = fcntl(fd, F_GETFL);

	if (val < 0) {
		printf("fcntl get\n");
		exit(1);
	}

	ret = fcntl(fd, F_SETFL, val | O_NONBLOCK);


	if (ret < 0) {
		printf("fcntl set\n");	
		exit(1);
	}



	for (i = 1 ;; i++) {

		errno = 0;

		printf("entering %d connect\n", i);
		ret = connect(fd, (struct sockaddr *)&saddr, 
			sizeof(struct sockaddr_in));

		printf("leaving %d connect, error %d msg = %s\n",
			i, errno, strerror(errno));

		if (errno == EISCONN)
			break;
	
	}

	close(fd);

	exit(0);
}

void handler(int signo)
{

	alarm(1);
	printf("in signal hander\n");
	return ;

}

Re: [PATCH] connect() return value.

[kuznet]

Hello!

> First off,
et al.

Great, I think we will change this too.

Could you repeat that test with read() after incomplete connect?
That your test was apparently invalid because of the same errno mistake,
so results may be different. Only, please, use read() with non-zero length.

Alexey

Re: [PATCH] connect() return value.

[Geoffrey Lee]

[-- Attachment #1: Type: text/plain, Size: 3621 bytes --]

On Tue, Aug 13, 2002 at 07:28:41PM +0400, kuznet@ms2.inr.ac.ru wrote:
> Hello!
> 
> > First off,
> et al.
> 
> Great, I think we will change this too.
> 


What are you intending to change? :-) connect behaivor or read behavior?


> Could you repeat that test with read() after incomplete connect?
> That your test was apparently invalid because of the same errno mistake,
> so results may be different. Only, please, use read() with non-zero length.
> 


Sure.
(though I don't think it should matter since with connect() the test
was bogus because I forgot to reset errno everytime I went through
the loop).

I will test for write as well, because read and write should be
consistent with each other.

Again, I will be attaching the programs that I used for this test.

In this mail:

connect-read.c (test for 0 length read)
connect-write.c (test for 0 length write)

Both are modified versions of the original connect.c.


Tested operating systems:


SunOS 5.6
OSF1 4.0
Linux 2.4.18


In this test we will be using the programs to connect to a open TCP
port on a host. We connect to a host on the WAN in an attempt to
maximize the connect process.

First we create a socket and we set it to non-blocking. We connect once,
and test for an errno of EINPROGRESS (expected). We set the errno value
to 0, to guarantee that should read / write return an error it is from
the read / write call. After that, we then we issue a 0 length
read / write, and test for an error. Finally, we issue a second connect,
to make sure that we get an EALREADY for errno, to make sure that the 3 way
handshake is not complete and we indeed tried to write to / read from
a in-progress socket.



SunOS 5.6

We find the errno after the first connect is indeed EINPROGRESS. A 
read with 0 length returns -1, with an errno of 134. That corresponds
to ENOTCONN on Solaris. The second connect returns EALREADY, and we
are sure that the 3 way handshake is not complete.

Similarly for write, when we issue a zero-length write, we find write
returns -1 with an errno of 134 (ENOTCONN).



OSF1 4.0


We find that we get a expected EINPROGRESS after the first connect.
For read, it returns 0. On the second connect, it returns EALREADY.

It is interesting to note that on OSF1 4.0, write returns an errno
of 57 (ENOTCONN). Behavior is not consistent with their socket read
and write. We have found a bug in Digital UNIX.


Linux 2.4.18

We find that we indeed get a EINPROGRESS after the first connected.
After a read, we find that read returns 0. On second connect, it
returns an error with errno set to EALREADY.

Similarly for write, when we issue a zero-length write, we find
write returns 0.


So to summarize:

To write portably to test for a connected socket, you can:

* use connect, but be prepared to handle an errno of 0 from Linux;
* handle the differences between read / write. In particular Digital
  UNIX has the extremely unsocial behaivor of returning something
  totally different with read and write.
* Use getpeername. We expect ENOTCONN if it is not completed, or
  some error occurred. Another implementation detail: OSF1 4.0
  and Solaris require getpeername's second argument to be valid, or
  -EBADF will be returned. Linux does not require this and returns
  -ENOTCONN even if the second argument is NULL. SunOS 5.6 and OSF1 4.0
  both return -ENOTCONN if you run getpeername with a non-NULL second
  argument.


  i.e.
  getpeername(fd, NULL, NULL) --> ok on linux, not ok on OSF1/SunOS
  getpeername(fd, &peeraddr, NULL) --> ok on all



Non-blocking sockets is as non-portable as it can be. :-)


  
	-- G.
  



	-- G.
	

[-- Attachment #2: connect-read.c --]
[-- Type: text/x-csrc, Size: 1771 bytes --]

#include <errno.h>
#include <fcntl.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

#include <sys/socket.h>
#include <netinet/in.h>


extern char *optarg;

#define HOST "192.168.0.1"
#define PORT 0

void handler(int);

int main(int argc, char **argv)
{
	struct sockaddr_in saddr;
	int ret;
	int fd;
	char buf[BUFSIZ];
	int val;
	int c;
	unsigned short port = PORT;
	size_t len;
	char *host = HOST;

	val = 0;
	len = sizeof(val);

	while ((c = getopt(argc, argv, "h:p:")) != -1) {
		switch (c) {
			case 'h':
				host = (char *)strdup(optarg);
				break;
			case 'p':
				port = atoi(optarg);
				break;
			default:
				break;
		}
	}
	
	memset(&saddr, 0, sizeof(struct sockaddr_in));

	saddr.sin_family = AF_INET;
	saddr.sin_port   = htons(port);
	saddr.sin_addr.s_addr = inet_addr(host);

	fd = socket(AF_INET, SOCK_STREAM, 0);

	if (fd < 0) {
		printf("socket\n");
		exit(1);
	}

	val = fcntl(fd, F_GETFL);

	if (val < 0) {
		printf("fcntl get\n");
		exit(1);
	}

	ret = fcntl(fd, F_SETFL, val | O_NONBLOCK);

	if (ret < 0) {
		printf("fcntl set\n");	
		exit(1);
	}



	ret = connect(fd, (struct sockaddr *)&saddr, 
			sizeof(struct sockaddr_in));

	if (ret < 0) {
		if (errno == EINPROGRESS) {
			printf("connect: EINPROGRESS (expected)\n");
		} else {
			printf("unexpected errno from connect, errno = %d"
			    " msg = %s\n", errno, strerror(errno));
		}
	}

	/* make sure errno is 0 */
	errno = 0;

	val = read(fd, buf, 0);
		printf("read() returns %d, errno = %d, msg = %s\n", 
			val, errno, (val < 0) ? strerror(errno) : "");

	errno = 0;
	ret = connect(fd, (struct sockaddr *)&saddr,
		sizeof(struct sockaddr_in));

	printf("connect returns %d, err = %s\n", ret, strerror(errno));

	close(fd);

	exit(0);
}

[-- Attachment #3: connect-write.c --]
[-- Type: text/x-csrc, Size: 1900 bytes --]

#include <errno.h>
#include <fcntl.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

#include <sys/socket.h>
#include <netinet/in.h>


extern char *optarg;

#define HOST "192.168.0.1"
#define PORT 0

void handler(int);

int main(int argc, char **argv)
{
	struct sockaddr_in saddr;
	int ret;
	int fd;
	char buf[BUFSIZ];
	int val;
	int c;
	unsigned short port = PORT;
	size_t len;
	char *host = HOST;

	val = 0;
	len = sizeof(val);

	while ((c = getopt(argc, argv, "h:p:")) != -1) {
		switch (c) {
			case 'h':
				host = (char *)strdup(optarg);
				break;
			case 'p':
				port = atoi(optarg);
				break;
			default:
				break;
		}
	}
	
	memset(&saddr, 0, sizeof(struct sockaddr_in));

	saddr.sin_family = AF_INET;
	saddr.sin_port   = htons(port);
	saddr.sin_addr.s_addr = inet_addr(host);

	fd = socket(AF_INET, SOCK_STREAM, 0);

	if (fd < 0) {
		printf("socket\n");
		exit(1);
	}

	val = fcntl(fd, F_GETFL);

	if (val < 0) {
		printf("fcntl get\n");
		exit(1);
	}

	ret = fcntl(fd, F_SETFL, val | O_NONBLOCK);

	if (ret < 0) {
		printf("fcntl set\n");	
		exit(1);
	}


	ret = connect(fd, (struct sockaddr *)&saddr, 
			sizeof(struct sockaddr_in));


	if (ret < 0) {
                if (errno == EINPROGRESS) {
                        printf("connect: EINPROGRESS (expected)\n");
                } else {
                        printf("unexpected errno from connect, errno = %d"
                            " msg = %s\n", errno, strerror(errno));
                }
        }

        /* make sure errno is 0 */
        errno = 0;

	val = write(fd, "1", 0);
		printf("write() returns %d, errno = %d, msg = %s\n", 
			val, errno, (val < 0) ? strerror(errno) : "");

	errno = 0;

	ret = connect(fd, (struct sockaddr *)&saddr,
		sizeof(struct sockaddr_in));

	printf("connect returns %d, err = %s\n", ret, strerror(errno));

	close(fd);

	exit(0);
}

Re: [PATCH] connect() return value.

[kuznet]

Hello!

> What are you intending to change? :-)

connect.

> > Could you repeat that test with read() after incomplete connect?
> > That your test was apparently invalid because of the same errno mistake,
> > so results may be different. Only, please, use read() with non-zero length.
				 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


> the read / write call. After that, we then we issue a 0 length
> read / write,

I asked you not to do this. This is pathological case, and return
value from this presents only academic interest.


> We find the errno after the first connect is indeed EINPROGRESS. A 
> read with 0 length returns -1, with an errno of 134. That corresponds
> to ENOTCONN on Solaris.

F.e. if Solaris behaves in this way on normal not-nil read/write,
it is fatally buggy. I do not understand why it works after this,
just by plain luck.

read/write on not-yet-connected socket can only block (when blocking)
or return EAGAIN otherwise. Well, or complete sucessfully.

Alexey

Re: [PATCH] connect() return value.

[Geoffrey Lee]

> 
> I asked you not to do this. This is pathological case, and return
> value from this presents only academic interest.
> 
> 


Burk.  Sorry for that error.


> > We find the errno after the first connect is indeed EINPROGRESS. A 
> > read with 0 length returns -1, with an errno of 134. That corresponds
> > to ENOTCONN on Solaris.
> 
> F.e. if Solaris behaves in this way on normal not-nil read/write,
> it is fatally buggy. I do not understand why it works after this,
> just by plain luck.
> 
> read/write on not-yet-connected socket can only block (when blocking)
> or return EAGAIN otherwise. Well, or complete sucessfully.
>


Ok, let's try again.

Same OS, same conditions same program used, but this time we read and
write with a count of 1.


OSF1 4.0:

It seems even with a non-nil write it is very anti-social and returns
different error codes.

For read it returns -EWOULDBLOCK, while for write it returns -ENOTCONN.
On OSF1, the errno EWOULDBLOCK is the same as EAGAIN.

It looks like for socket reads and writes, OSF1 does the sanity
checks for read and write in a different order. This is a bug.


SunOS 5.6:

-ENOTCONN for both a read and write of 1 byte count.



Linux 2.4.18:

-EAGAIN in both read and write of one byte.



As you said that the fact that Solaris works at all with those semantics
is pure luck, so we try to understand it a bit better. We modify the program
further, and after the connection succeeds, we read some data and we print it
out.

For the write case, we use a simple `echo' server and write some data to
the server and read back from it, and see that it is ok. We sleep for a 
amount of time between the second connect call and the read / write call
to give it ample time for the 3 way handshake to complete. As a control
the same tests is done across all 3 operating systems.


We first start the server and telnet to it and make sure that it works
alright. We verify that this is true. We also start the server on the 
same computer to rule out endianness problems.


SunOS 5.6:

read case is ok.
write case is ok.

OSF1 4.0:

read case is ok.
write case is ok.

Linux 2.4.18:

read case is ok.
write case is ok.


So SunOS with -ENOTCONN semantics work. So does OSF1, but we note write
returns -ENOTCONN.

So if you say that it is indeed a bug, then both SunOS and OSF1 are
lucky.

But now that we have determined that is is not portable to use a read or 
a write to test if a non-blocking socket is connected or not, I'd like 
to hear your reasons why a read or a write returning -ENOTCONN is a
buggy behavior.



	-- G.
	

Re: [PATCH] connect() return value.

[kuznet]

Hello!

> to hear your reasons why a read or a write returning -ENOTCONN is a

Because any error but EAGAIN/EINTR is failure. As you convinced
yourself there is no good way to detect connection completion,
so in fact writing correct program is next to impossible.

Actually, never in my life I have seen a pattern of program
with waits for EISCONN after EINTR on connect() (but lots of them
fail when not seeing 0), so I have no idea why they work under these OSes.
Apparently, they fail randomly.

Alexey

Re: [PATCH] connect() return value.

[Geoffrey Lee]

> 
> Because any error but EAGAIN/EINTR is failure. As you convinced


Yep. I thought about this as well.

If you used read with a byte count of 0 / 1, and it returned -1 to
the program with errno set to ENOTCONN, you cannot be confident whether
it is trying to complete a 3 way handshake or the connection failed.


> yourself there is no good way to detect connection completion,
> so in fact writing correct program is next to impossible.
>


I am a bit jaded about non-blocking connects on Unix, because is even
more non-portable than I ever imagined.


> Actually, never in my life I have seen a pattern of program
> with waits for EISCONN after EINTR on connect() (but lots of them
> fail when not seeing 0), so I have no idea why they work under these OSes.
> Apparently, they fail randomly.
> 


Yep.

On Solaris / Digital UNIX the 3 way handshake completes asynchronously
on EINTR. Checking for a EISCONN is one non-perfect solution to check
for a successful connection establishment.

So, in the near future, we can expect Linux to not return 0 for
non-blocking connects (i.e. the change you mentioned)?


	-- G.

Re: [PATCH] connect() return value.

[kuznet]

Hello!

> So, in the near future, we can expect Linux to not return 0 for
> non-blocking connects (i.e. the change you mentioned)?

I wanted, but I changed my opinion about this. It is pretty strange to mimic
behavior of solaris/tru64 and Co, which is really stinking.

So, despite of our current behavior is not perfect and even not
quite self-consistent, it is the best one. Well, and it does not
contradict to sus/posix.

So, connect() remains restartable, and it will return 0 on success
instead of EISCONN crap.

Alexey

Re: [PATCH] connect() return value.

[David S. Miller]

   From: glee@gnupilgrims.org
   Date: Thu, 15 Aug 2002 11:25:43 +0800
   
   I research how sockets are done on various Unix as a hobby.

How extensive are your notes on poll() behavior on TCP
sockets? :-)

Re: [PATCH] connect() return value.

[glee]

On Wed, Aug 14, 2002 at 09:25:39PM +0400, kuznet@ms2.inr.ac.ru wrote:
> Hello!
> 
> > So, in the near future, we can expect Linux to not return 0 for
> > non-blocking connects (i.e. the change you mentioned)?
> 
> I wanted, but I changed my opinion about this. It is pretty strange to mimic
> behavior of solaris/tru64 and Co, which is really stinking.
> 

It is true it is bad to go for a technically inferior solution that
EINTR must be handled by the userspace code.

> So, despite of our current behavior is not perfect and even not
> quite self-consistent, it is the best one. Well, and it does not
> contradict to sus/posix.
>


Agreed for both points, as far as I recall it is not standardized,
not even what "may" or "should" be done.


> So, connect() remains restartable, and it will return 0 on success
> instead of EISCONN crap.


Yep.

I research how sockets are done on various Unix as a hobby. So, 
one of the motives of my original letter was that I wanted to know
whether there is any special reasons (well now I know after this 
discussion) why Linux's connect is restartable. 

I will make a special note of that in my notes.

Thanks for this fruitful discussion. :-)



	-- G.
	

Re: [PATCH] connect() return value.

[Geoffrey Lee]

On Wed, Aug 14, 2002 at 08:15:14PM -0700, David S. Miller wrote:
>    From: glee@gnupilgrims.org
>    Date: Thu, 15 Aug 2002 11:25:43 +0800
>    
>    I research how sockets are done on various Unix as a hobby.
> 
> How extensive are your notes on poll() behavior on TCP
> sockets? :-)


Hmm, in which regard?

Is there something specific that you are searching for? :-)


	-- G.

Re: [PATCH] connect() return value.

[David S. Miller]

   From: Geoffrey Lee <glee@gnupilgrims.org>
   Date: Thu, 15 Aug 2002 21:02:10 +1000

   On Wed, Aug 14, 2002 at 08:15:14PM -0700, David S. Miller wrote:
   > How extensive are your notes on poll() behavior on TCP
   > sockets? :-)

   Hmm, in which regard?
   
   Is there something specific that you are searching for? :-)

POLLHUP in particular.  I remember we verified that our behavior
matched Solaris but no checks were performed against others.

Re: [PATCH] connect() return value.

[Geoffrey Lee]

[-- Attachment #1: Type: text/plain, Size: 3231 bytes --]

On Thu, Aug 15, 2002 at 02:23:37PM -0700, David S. Miller wrote:
>    From: Geoffrey Lee <glee@gnupilgrims.org>
>    Date: Thu, 15 Aug 2002 21:02:10 +1000
> 
>    On Wed, Aug 14, 2002 at 08:15:14PM -0700, David S. Miller wrote:
>    > How extensive are your notes on poll() behavior on TCP
>    > sockets? :-)
> 
>    Hmm, in which regard?
>    
>    Is there something specific that you are searching for? :-)
> 
> POLLHUP in particular.  I remember we verified that our behavior
> matched Solaris but no checks were performed against others.


In all honesty, none for that. But it would be a worthwhile experiment
I think. Let us see what happens.


Today, we are going to see what happens with POLLHUP on different Unix
implementations. Source code is provided.

poll.c (server that uses poll)
linger.c (our client to invoke the POLLHUP condition)

Note in poll.c I used size_t as the 3rd argument to accept instead
of socklen_t, as only Linux had socklen_t.

We also see that we get a compile warning when compiling the client as
on SunOS 5.6 the 4th argument to setsockopt is a char *. But we can
ignore that warning.


The following operating systems will be tested against:

OSF1 4.0 (Digital UNIX)
SunOS 5.6 (Solaris)
Linux 2.4.18


We will start the server on a machine with some specified port. We 
call listen for the specified socket, and accept. Next, we block
in the call to poll with fd set to the socket descriptor returned
by accept, and events and revents zeroed. On the call to poll, we 
specify -1 as a senitel that we want to block forever until something
interesting arrives (the error condition).

On the client, we connect to the server to the specified port
as normal. We specify the SO_LINGER option, with l_onoff set to 1 and
linger time set to 0. After we successfully connect to the server, we
call the close call immediately, to issue a RST to the server.


This is what happens on Linux 2.4.18:

$ ./poll -p 8888 
accept ok.
poll: POLLERR set
poll: POLLHUP set
sever terminating
$ 

On the client side:

$ ./linger -h [ip] -p 8888
connecting ..
closing ...
closed.
$ 


So, sending a RST will trigger POLLHUP and POLLERR on Linux.


On OSF1 4.0, we get this:

On the server:
$ ./poll -p 8888
accept ok.

On the client side:
$ ./linger -h [ip] -p 8888
connecting ..
closing ...
closed.
$ 


So on Digital UNIX, it blocks forever in the call to poll.

That's strange. Let's see what happens on SunOS 5.6:

./poll -p 8888
accept ok.


On the client side:

./linger -h [ip] -p 8888
connecting ..
closing ...
closed.
$ 


So on SunOS 5.6, it also blocks forever.


Do you know which Solaris was tested the last time it was found that
Linux behaves the same way as Solaris?

By the way, while we are on the subject of poll, what should one include
to get POLLRDNORM and INFTIM?

I found on OSF1 4.0 one can have both by including <poll.h>,
on SunOS 5.6, one can get POLLRDNORM by including <poll.h>, for INFTIM,
one must include <sys/stropts.h> as well.

On Linux, one will not be able to get them by including both <poll.h>
or <sys/stropts.h>. <sys/poll.h> will not work either. by including 
<linux/poll.h>, then one can get POLLRDNORM defined. But I could not
find INFTIM anywhere on Linux.


	-- G.



[-- Attachment #2: linger.c --]
[-- Type: text/x-csrc, Size: 1251 bytes --]

#include <sys/socket.h>

#include <netinet/in.h>

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h> /* getopt */

#define HOST "192.168.0.1"
#define PORT (22)

int main(argc, argv)
	int argc;
	char *argv[];
{
	int sockfd;
	int c;
	struct linger ling;
	struct sockaddr_in servaddr;
	char *host = HOST;
	unsigned short port = PORT;
	


	while ((c = getopt(argc, argv, "h:p:")) != -1) {
		switch (c) {
			case 'h':
				host = (char *)strdup(optarg);
				break;
			case 'p':
				port = atoi(optarg);
				break;
			default:
				break;
		}
	}

	sockfd = socket(AF_INET, SOCK_STREAM, 0);

	if (sockfd < 0) {
		printf("socket\n");
		exit(1);
	}

	bzero(&servaddr, sizeof(struct sockaddr_in));

	servaddr.sin_family = AF_INET;
	servaddr.sin_port   = htons(port);
	servaddr.sin_addr.s_addr   = inet_addr(host);

	ling.l_onoff = 1;
	ling.l_linger = 0;

	if (setsockopt(sockfd, SOL_SOCKET, SO_LINGER, &ling, sizeof(ling)) < 0) {
		printf("setsockopt\n");
		exit(1);
	}

	printf("connecting ..\n");
	if (connect(sockfd, (struct sockaddr *)&servaddr, sizeof(struct sockaddr_in)) < 0) {
		printf("connect: %s\n", strerror(errno));
		exit(1);
	}

	printf("closing ...\n");
	close(sockfd);	/* send RST */
	printf("closed.\n");

	exit(0);
}

[-- Attachment #3: poll.c --]
[-- Type: text/x-csrc, Size: 1702 bytes --]

#include <poll.h>
#include <sys/socket.h>

#include <netinet/in.h>

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h> /* strdup */
#include <unistd.h> /* getopt */

#define PORT (22)

#define BACKLOG 20

#ifndef INFTIM
#define INFTIM (-1)
#endif


extern char *optarg;

int main(argc, argv)
	int argc;
	char *argv[];
{
	struct pollfd nfd;
	struct sockaddr_in saddr, caddr;
	int fd;
	int c;
	int clientfd;
	size_t len;
	unsigned short port = PORT;

	
	bzero(&saddr, sizeof(struct sockaddr_in));
	bzero(&caddr, sizeof(struct sockaddr_in));

	while ((c = getopt(argc, argv, "h:p:")) != -1) {
		switch (c) {
			case 'p':
				port = atoi(optarg);
				break;
			default:
				break;
		}
	}

	saddr.sin_family = AF_INET;
	saddr.sin_port   = htons(port);
	saddr.sin_addr.s_addr   = INADDR_ANY;

	fd = socket(AF_INET, SOCK_STREAM, 0);

	if (fd < 0) {
		printf("socket\n");
		exit(1);
	}

	if (bind(fd, (struct sockaddr *)&saddr, sizeof(struct sockaddr_in)) < 0) {
		printf("bind\n");
		exit(1);
	}

	if (listen(fd, BACKLOG) < 0) {
		printf("listen\n");
		exit(1);
	}

	len = sizeof(struct sockaddr_in);

	if ((clientfd = accept(fd, (struct sockaddr *)&caddr, &len)) < 0) {
		printf("accept: %s\n", strerror(errno)), exit(1);
	}

	printf("accept ok.\n");

	/* reset */	
	nfd.fd = clientfd;
	nfd.events = 0;
	nfd.revents = 0;

	if (poll(&nfd, 1, INFTIM) < 0) 
		printf("poll: %s\n", strerror(errno)), exit(1);

	if (nfd.revents & POLLERR) 
		printf("poll: POLLERR set\n");

	if (nfd.revents & POLLHUP)
		printf("poll: POLLHUP set\n");

	if (nfd.revents & POLLNVAL)
		printf("poll: POLLNVAL set\n"); /* should not happen */

	close(clientfd);
	printf("sever terminating\n");
	exit(0);
}