Lack of Documentation about SA_RESTART...

Lack of Documentation about SA_RESTART...

[Paolo Ornati]

The documentation (man pages & info libc) doesn't cover well interaction
between various syscalls and SA_RESTART flag of sigaction()... I wonder
why!

	> MAN SIGACTION <

"SA_RESTART
Provide behaviour compatible with BSD signal semantics by
making certain system calls restartable across signals."

	certain?!? :-O!


	> INFO LIBC <

"   One important exception is `EINTR' (*note Interrupted Primitives::).
Many stream I/O implementations will treat it as an ordinary error,
which can be quite inconvenient.  You can avoid this hassle by
installing all signals with the `SA_RESTART' flag."


" -- Macro: int SA_RESTART
     This flag controls what happens when a signal is delivered during
     certain primitives (such as `open', `read' or `write'), and the
     signal handler returns normally.  There are two alternatives: the
     library function can resume, or it can return failure with error
     code `EINTR'."

	Ok, "info libc" is a bit better.


But what I'm looking for is a list of syscalls that are automatically
restarted when SA_RESTART is set, and especially in what conditions.

For example: read(), write(), open() are obviously restarted, but even
on non-blocking fd?
And what about connect() and select() for example?

There are a lot of syscalls that can fail with "EINTR"! What's the
advantage of using SA_RESTART if one doesn't know what syscalls are
restarted?

One should always check for "EINTR" or use "TEMP_FAILURE_RETRY()" macro
as suggested in "info libc" !


Looking at the source I can easly see that a syscall is retarted when it
returns "-ERESTARTSYS" and SA_RESTART flag is set. Should I look at the
code for every syscall / particular condition?


Example of behavior: according to source code it seems that "connect()"
(the "net/ipv4/af_inet.c : inet_stream_connect()" implementation)
returns -ERESTARTSYS if interrupted, but if the socket is in
non-blocking mode it returns -EINTR.


SUMMARY:

1) there is a reason for this lack of documentation?

2) what can I safely assume about syscalls restart when using SA_RESTART
flag?


Bye,

-- 
	Paolo Ornati
	Linux 2.6.12.2 on x86_64

Re: Lack of Documentation about SA_RESTART...

[Theodore Ts'o]

On Mon, Jul 11, 2005 at 12:32:37PM +0200, Paolo Ornati wrote:
> But what I'm looking for is a list of syscalls that are automatically
> restarted when SA_RESTART is set, and especially in what conditions.
> 
> For example: read(), write(), open() are obviously restarted, but even
> on non-blocking fd?
> And what about connect() and select() for example?
> 
> There are a lot of syscalls that can fail with "EINTR"! What's the
> advantage of using SA_RESTART if one doesn't know what syscalls are
> restarted?

According to the Single Unix Specification V3, all functions that
return EINTR are supposed to restart if a process receives a signal
where signal handler has been installed with the SA_RESTART flag.  

There may be a few places in the kernel where this isn't happenning,
that's what the specification states. 

My preference is to always check for EINTR out of sheer paranoia and
for portability, since SA_RESTART is an optional part (XSI) of the
specification which is not necessarily implemented by all Unix
systems, and contrary to popular belief, there _are_ Linux-like
systems beyond just Linux that an application writer might want their
programs to be portable to.  You know, quaint legacy systems like
Solaris, HP-UX, AIX, etc.  :-)

> Example of behavior: according to source code it seems that "connect()"
> (the "net/ipv4/af_inet.c : inet_stream_connect()" implementation)
> returns -ERESTARTSYS if interrupted, but if the socket is in
> non-blocking mode it returns -EINTR.

If the socket is non-blocking mode, and there isn't a connection ready
to be received, then the socket is going to return with some kind of
errno set; either EINPROGRESS (Operation now in progress) or EALREADY
(Operation already in progress), or if a signal came in during the
system call (which would happen pretty rarely, the window for the race
condition is pretty small) it will return EINTR.  I _think_ that a
close reading of the specification would state that the system call
should be restarted, and since a connection isn't ready, then at that
point EINPROGRESS or EALREADY would be returned.  

But this is a pretty small corner case, and in practice happens quite
rarely.  There might be some existing code that expects EINTR in this
case, which might explain this, or perhaps it was thought that it was
useful for the application to know that a system call was handled in
the middle of a non-blocking system call (but what's the point, since
no notification would be given if the signal was processed right
before or right after the system call).  

Still, I'd suggest that the old internet dictum of being conservative
in what you send and liberal in what you accept applies here.  If you
want your program to be robust and portable, check for EINTR.  

						- Ted

Re: Lack of Documentation about SA_RESTART...

[Philippe Troin]

"Theodore Ts'o" <tytso@mit.edu> writes:

> On Mon, Jul 11, 2005 at 12:32:37PM +0200, Paolo Ornati wrote:
> > But what I'm looking for is a list of syscalls that are automatically
> > restarted when SA_RESTART is set, and especially in what conditions.
> > 
> > For example: read(), write(), open() are obviously restarted, but even
> > on non-blocking fd?
> > And what about connect() and select() for example?
> > 
> > There are a lot of syscalls that can fail with "EINTR"! What's the
> > advantage of using SA_RESTART if one doesn't know what syscalls are
> > restarted?
> 
> According to the Single Unix Specification V3, all functions that
> return EINTR are supposed to restart if a process receives a signal
> where signal handler has been installed with the SA_RESTART flag.  

Except for select() and poll(), which should always return EINTR even
when interrupted with a SA_RESTART signal.

Phil.

Re: Lack of Documentation about SA_RESTART...

[Paolo Ornati]

On 11 Jul 2005 20:30:20 -0700
Philippe Troin <phil@fifi.org> wrote:

> Except for select() and poll(), which should always return EINTR even
> when interrupted with a SA_RESTART signal.

SUSV3 says this for select():
"If SA_RESTART has been set for the interrupting signal, it is
implementation-defined whether the function restarts or returns with
[EINTR]"

and says nothing for poll()...

But it says nothing also for "pause()", for example... and I doubt that
it's supposed to be automatically restarted :)

-- 
	Paolo Ornati
	Linux 2.6.12.2 on x86_64

Re: Lack of Documentation about SA_RESTART...

[Paolo Ornati]

On Mon, 11 Jul 2005 10:34:27 -0400
Theodore Ts'o <tytso@mit.edu> wrote:

> According to the Single Unix Specification V3, all functions that
> return EINTR are supposed to restart if a process receives a signal
> where signal handler has been installed with the SA_RESTART flag.  

Thanks to have cited SUS... I'm reading it and is much cleaner than
manpages :-)

"SA_RESTART
    This flag affects the behavior of interruptible
functions; that is, those specified to fail with errno set to [EINTR].
If set, and a function specified as interruptible is interrupted by this
signal, the function shall restart and shall not fail with [EINTR]
unless otherwise specified. If the flag is not set, interruptible
functions interrupted by this signal shall fail with errno set to
[EINTR]"

Note the "unless otherwise specified".

> 
> > Example of behavior: according to source code it seems that
> > "connect()" (the "net/ipv4/af_inet.c : inet_stream_connect()"
> > implementation) returns -ERESTARTSYS if interrupted, but if the
> > socket is in non-blocking mode it returns -EINTR.
> 
> If the socket is non-blocking mode, and there isn't a connection ready
> to be received, then the socket is going to return with some kind of
> errno set; either EINPROGRESS (Operation now in progress) or EALREADY
> (Operation already in progress), or if a signal came in during the
> system call (which would happen pretty rarely, the window for the race
> condition is pretty small) it will return EINTR.  I _think_ that a
> close reading of the specification would state that the system call
> should be restarted, and since a connection isn't ready, then at that
> point EINPROGRESS or EALREADY would be returned.  
> 

Reading SUSV3 it seems that doing a connection with non-blocking socket
CANNOT return EINTR... and this make sense.

The particular case you analized (blocking connect interrupted by a
SA_RESTART signal) is interesting... and since SUSV3 says
	"but the connection request shall not be aborted, and the connection
	shall be established asynchronously" (with select() or poll()...)
both for EINPROGRESS and EINTR, I think it's quite stupit to
automatically restart it and then return EALREADY.

The logically correct behaviur with blocking connect interrupted and
then restarted should be to continue the blocking wait... IHMO.

-- 
	Paolo Ornati
	Linux 2.6.12.2 on x86_64

Re: Lack of Documentation about SA_RESTART...

[Paolo Ornati]

[-- Attachment #1: Type: text/plain, Size: 1259 bytes --]

On Tue, 12 Jul 2005 10:38:11 +0200
Paolo Ornati <ornati@fastwebnet.it> wrote:

> The particular case you analized (blocking connect interrupted by a
> SA_RESTART signal) is interesting... and since SUSV3 says
> 	"but the connection request shall not be aborted, and the
> 	connection shall be established asynchronously" (with select()
> 	or poll()...)
> both for EINPROGRESS and EINTR, I think it's quite stupit to
> automatically restart it and then return EALREADY.
> 
> The logically correct behaviur with blocking connect interrupted and
> then restarted should be to continue the blocking wait... IHMO.

it seems that Linux is doing the Right Thing... see the attached
program...

$ make
gcc -O2 -Wall -o conntest connect_test.c


FROM ANOTHER CONSOLE: this is needed to block connect()...
# iptables -A OUTPUT -p tcp --dport 3500 -m state --state NEW -j DROP


$ ./conntest WITHOUT_SA_RESTART
connect(): errno = 4		# EINTR, as expected
Cannot setup client!


$ ./conntest	# connect is restarted after SIGALRM, and then it blocks again


FROM ANOTHER CONSOLE:
# iptables -D OUTPUT -p tcp --dport 3500 -m state --state NEW -j DROP


and then "conntest" (thanks to TCP protocol retries) will terminate.



:-)

-- 
	Paolo Ornati
	Linux 2.6.12.2 on x86_64

[-- Attachment #2: connect_test.c --]
[-- Type: text/x-csrc, Size: 1692 bytes --]

#include <stdio.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <netinet/tcp.h>
#include <signal.h>
#include <unistd.h> 

void sighandler(int sig)
{
	/* nothing :) */
}

int setup_alarm_handler(int flags)
{
	struct sigaction sa = {
		.sa_handler = &sighandler,
		.sa_flags = flags
	};
	return sigaction(SIGALRM, &sa, NULL);
}

int setup_server(int port)
{
	int sock;
	struct sockaddr_in sa = {
		.sin_family = AF_INET,
		.sin_port = htons(port),
		.sin_addr.s_addr = htonl(INADDR_ANY)
	};

	if ((sock=socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0)
		goto error;
	if (bind(sock, (struct sockaddr*)&sa, sizeof(sa)) < 0)
		goto error_clean;
	if (listen(sock, 16) < 0)
		goto error_clean;
	return sock;

 error_clean:
	close(sock);
 error:
	return -1;
}

int setup_client(const char *server, int port)
{
	int sock;
	struct sockaddr_in sa = {
		.sin_family = AF_INET,
		.sin_port = htons(port),
		.sin_addr.s_addr = inet_addr(server)
	};

	if ((sock=socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0)
		goto error;
	if (connect(sock, (struct sockaddr*)&sa, sizeof(sa)) < 0) {
		printf("connect(): errno = %d\n", errno);
		goto error_clean;
	}
	return sock;

 error_clean:
	close(sock);
 error:
	return -1;
}

int main(int argc, char *argv[])
{
	int server, client;
	int flags = SA_RESTART;

	if (argc > 1)
		flags = 0;

	if (setup_alarm_handler(flags))
		return 1;
	
	server = setup_server(3500);
	if (server < 0) {
		printf("Cannot setup server!\n");
		return 1;
	}
	alarm(1);
	client = setup_client("127.0.0.1", 3500);
	if (client < 0) {
		printf("Cannot setup client!\n");
		return 1;
	}
	printf("Ok!\n");
	return 0;
}

[-- Attachment #3: Makefile --]
[-- Type: application/octet-stream, Size: 49 bytes --]

conntest: connect_test.c
	gcc -O2 -Wall -o $@ $<

Re: Lack of Documentation about SA_RESTART...

[Theodore Ts'o]

On Tue, Jul 12, 2005 at 10:38:11AM +0200, Paolo Ornati wrote:
> The particular case you analized (blocking connect interrupted by a
> SA_RESTART signal) is interesting... and since SUSV3 says
> 	"but the connection request shall not be aborted, and the connection
> 	shall be established asynchronously" (with select() or poll()...)
> both for EINPROGRESS and EINTR, I think it's quite stupit to
> automatically restart it and then return EALREADY.
> 
> The logically correct behaviur with blocking connect interrupted and
> then restarted should be to continue the blocking wait... IHMO.

I was looking at what happened with a *non-blocking* connect
interrupted by an SA_RESTART signal.  Since it is non-blocking, it
will never continue with the wait.  The only question is whether it
should return with an EINTR (which is what it currently does) or
return with whatever error code it would have returned if the signal
had not been delievered in the first place.  We currently do the
former; a close reading of the spec seems the require the latter.
Fortunately this is a pretty narrow race condition since the chances
of a signal being delivered right in the middle of a non-blocking
connect are small.  But, the paranoid application writer should check
for EINTR being returned from a non-blocking connect, since that is
the what the currently deployed Linux kernels will do right now in
this rare case.

						- Ted

Re: Lack of Documentation about SA_RESTART...

[Paolo Ornati]

On Tue, 12 Jul 2005 08:04:56 -0400
Theodore Ts'o <tytso@mit.edu> wrote:

> > 
> > The logically correct behaviur with blocking connect interrupted and
> > then restarted should be to continue the blocking wait... IHMO.
> 
> I was looking at what happened with a *non-blocking* connect
> interrupted by an SA_RESTART signal.  Since it is non-blocking, it
> will never continue with the wait.  The only question is whether it
> should return with an EINTR (which is what it currently does) or
> return with whatever error code it would have returned if the signal
> had not been delievered in the first place.  We currently do the
> former; a close reading of the spec seems the require the latter.
> Fortunately this is a pretty narrow race condition since the chances
> of a signal being delivered right in the middle of a non-blocking
> connect are small.

Hmmm... no, no. A connect() on non-blocking socket will NEVER return
EINTR. SUSV3 and Linux code agree.

A syscall isn't magically interrupted if a signal arrives... it's the
syscall that must check for pending signals and do the proper action
(usually it will return with -EINTR or -ERESTARTSYS).

A connect() on a blocking socket is something like this (very
approssimative):

	1) code to activate the connection
	2) sleep waiting for something (connection ready / signal received...)
	3) if connection is ready then return 0, else if there are pending
	signals return -ERESTARTSYS

With non-blocking socket the syscall never sleeps, and never checks for
pending signals.

Look at "net/ipv4/af_inet.c": in particular at "net_wait_for_connect"
and its usage in "inet_stream_connect".

--
	Paolo Ornati
	Linux 2.6.12.2 on x86_64

Re: Lack of Documentation about SA_RESTART...

[Theodore Ts'o]

On Tue, Jul 12, 2005 at 05:25:04PM +0200, Paolo Ornati wrote:
> Hmmm... no, no. A connect() on non-blocking socket will NEVER return
> EINTR. SUSV3 and Linux code agree.
> 
> A syscall isn't magically interrupted if a signal arrives... it's the
> syscall that must check for pending signals and do the proper action
> (usually it will return with -EINTR or -ERESTARTSYS).
> 
> A connect() on a blocking socket is something like this (very
> approssimative):
> 
> 	1) code to activate the connection
> 	2) sleep waiting for something (connection ready / signal received...)
> 	3) if connection is ready then return 0, else if there are pending
> 	signals return -ERESTARTSYS
> 
> With non-blocking socket the syscall never sleeps, and never checks for
> pending signals.
> 
> Look at "net/ipv4/af_inet.c": in particular at "net_wait_for_connect"
> and its usage in "inet_stream_connect".

Yes, do look at that.  From the latest 2.6 sources:

	timeo = sock_sndtimeo(sk, flags & O_NONBLOCK);

	if ((1 << sk->sk_state) & (TCPF_SYN_SENT | TCPF_SYN_RECV)) {
		/* Error code is set above */
		if (!timeo || !inet_wait_for_connect(sk, timeo))
			goto out;

		err = sock_intr_errno(timeo);
		if (signal_pending(current))
			goto out;
	}

If the socket is non-blocking, then we don't call
inet_wiat_for_connect(), yes.  But sock_intr_errno() will set the
error code to -EINTR if the socket is set to non-nonblocking (see
include/net/sock.h), and if a signal is pending, return it.

					- Ted

Re: Lack of Documentation about SA_RESTART...

[Paolo Ornati]

On Tue, 12 Jul 2005 14:16:35 -0400
Theodore Ts'o <tytso@mit.edu> wrote:

> Yes, do look at that.  From the latest 2.6 sources:
> 
> 	timeo = sock_sndtimeo(sk, flags & O_NONBLOCK);
> 
> 	if ((1 << sk->sk_state) & (TCPF_SYN_SENT | TCPF_SYN_RECV)) {
> 		/* Error code is set above */
> 		if (!timeo || !inet_wait_for_connect(sk, timeo))
> 			goto out;
> 
> 		err = sock_intr_errno(timeo);
> 		if (signal_pending(current))
> 			goto out;
> 	}
> 
> If the socket is non-blocking, then we don't call
> inet_wiat_for_connect(), yes.  But sock_intr_errno() will set the
> error code to -EINTR if the socket is set to non-nonblocking (see
> include/net/sock.h), and if a signal is pending, return it.

No.

With non-blocking socket "timeo" is set to 0. So the instruction:

	if (!timeo || !inet_wait_for_connect(sk, timeo))
		goto out;

jumps directly to "out" label. "sock_intr_errno()" isn't called.

-- 
	Paolo Ornati
	Linux 2.6.12.2 on x86_64

Re: Lack of Documentation about SA_RESTART...

[Linus Torvalds]

On Mon, 11 Jul 2005, Theodore Ts'o wrote:
> 
> According to the Single Unix Specification V3, all functions that
> return EINTR are supposed to restart if a process receives a signal
> where signal handler has been installed with the SA_RESTART flag.  

That can't be right.

Some operations, like "select()" and "pause()" always return EINTR, and 
indeed, real applications will break if you always restart. Restarting a 
pause() would be nonsensical.

This is why the kernel has ERESTARTSYS (restart always) and ERESTARTNOHAND
(restart if the signal had no handler) which will be turned into EINTR if
SA_RESTART isn't set. And even then, there are some ops that just always
use EINTR and don't even try to restart.

> There may be a few places in the kernel where this isn't happenning,
> that's what the specification states. 

The specs are broken, and apparently don't take real life (or sanity -
pause() in particular just doesn't make sense if restarted) into account.  

Afaik, the historical BSD behaviour (which is what SA_RESTART is all
about) always was to just restart some system calls, not all of them.

		Linus

Re: Lack of Documentation about SA_RESTART...

[Paolo Ornati]

On Sat, 23 Jul 2005 17:30:42 -0700 (PDT)
Linus Torvalds <torvalds@osdl.org> wrote:

> > According to the Single Unix Specification V3, all functions that
> > return EINTR are supposed to restart if a process receives a signal
> > where signal handler has been installed with the SA_RESTART flag.  
> 
> That can't be right.


SUV3 is really lacking about it:


sigaction()
"SA_RESTART
    [XSI] [Option Start] This flag affects the behavior of interruptible
functions; that is, those specified to fail with errno set to [EINTR].
If set, and a function specified as interruptible is interrupted by this
signal, the function shall restart and shall not fail with [EINTR]
unless otherwise specified. If the flag is not set, interruptible
functions interrupted by this signal shall fail with errno set to
[EINTR]."


Note the "unless otherwise specified"... but then SA_RESTART is only
mentioned for:


select/pselect
	"[XSI] [Option Start] If SA_RESTART has been set for the interrupting
	signal, it is implementation-defined whether the function restarts or
	returns with [EINTR]."


and for other few signal handling functions like bsd_signal(),
siginterrupt(), signal()... but just to say what SA_RESTART means.



Then, reading xrat/xsh_chap02.html:

"Signal Effects on Other Functions

The most common behavior of an interrupted function after a
signal-catching function returns is for the interrupted function to give
an [EINTR] error unless the SA_RESTART flag is in effect for the signal.
However, there are a number of specific exceptions, including sleep()
and certain situations with read() and write().

The historical implementations of many functions defined by IEEE Std
1003.1-2001 are not interruptible, but delay delivery of signals
generated during their execution until after they complete. This is
never a problem for functions that are guaranteed to complete in a short
(imperceptible to a human) period of time. It is normally those
functions that can suspend a process indefinitely or for long periods of
time (for example, wait(), pause(), sigsuspend(), sleep(), or read()/
write() on a slow device like a terminal) that are interruptible. This
permits applications to respond to interactive signals or to set
timeouts on calls to most such functions with alarm(). Therefore,
implementations should generally make such functions (including ones
defined as extensions) interruptible.

Functions not mentioned explicitly as interruptible may be so on some
implementations, possibly as an extension where the function gives an
[EINTR] error. There are several functions (for example, getpid(),
getuid()) that are specified as never returning an error, which can thus
never be extended in this way.

If a signal-catching function returns while the SA_RESTART flag is in
effect, an interrupted function is restarted at the point it was
interrupted. Conforming applications cannot make assumptions about the
internal behavior of interrupted functions, even if the functions are
async-signal-safe. For example, suppose the read() function is
interrupted with SA_RESTART in effect, the signal-catching function
closes the file descriptor being read from and returns, and the read()
function is then restarted; in this case the application cannot assume
that the read() function will give an [EBADF] error, since read() might
have checked the file descriptor for validity before being interrupted."



FreeBSD doc (man page of sigaction) is much better:
http://www.gsp.com/cgi-bin/man.cgi?section=2&topic=sigaction

"If a signal is caught during the system calls listed below, the call
may be forced to terminate with the error  EINTR, the call may return
with a data transfer shorter than requested, or the call may be
restarted. Restart of pending calls is requested by setting the 
SA_RESTART bit in  sa_flags. The affected system calls include open(2),
read(2), write(2), sendto(2), recvfrom(2), sendmsg(2) and recvmsg(2) on
a communications channel or a slow device (such as a terminal, but not a
regular file) and during a wait(2) or ioctl(2). However, calls that have
already committed are not restarted, but instead return a partial
success (for example, a short read count)."


Finally, in Linux other syscalls can be restarted... like connect(), for
example.

 -- 
	Paolo Ornati
	Linux 2.6.13-rc3 on x86_64

Re: Lack of Documentation about SA_RESTART...

[Theodore Ts'o]

On Sat, Jul 23, 2005 at 05:30:42PM -0700, Linus Torvalds wrote:
> On Mon, 11 Jul 2005, Theodore Ts'o wrote:
> > 
> > According to the Single Unix Specification V3, all functions that
> > return EINTR are supposed to restart if a process receives a signal
> > where signal handler has been installed with the SA_RESTART flag.  
> 
> That can't be right.
> 
> Some operations, like "select()" and "pause()" always return EINTR, and 
> indeed, real applications will break if you always restart. Restarting a 
> pause() would be nonsensical.

The spect says "unless otherwise specified".  The description for
pause() states that the process will sleep until receiving a signal
that terminates the process or causes it to call signal-handling
function.  That would presumably count as an "otherwise specified".

						- Ted

Re: Lack of Documentation about SA_RESTART...

[Paolo Ornati]

On Sun, 24 Jul 2005 10:56:08 -0400
Theodore Ts'o <tytso@mit.edu> wrote:

> The spect says "unless otherwise specified".  The description for
> pause() states that the process will sleep until receiving a signal
> that terminates the process or causes it to call signal-handling
> function.  That would presumably count as an "otherwise specified".

I don't think that way for at least 2 reasons:


1) SA_RESTART is an XSI extension, so every exception to the rule
"everything automatically restarted" should be under an XSI section
(like it is on the "select()" page).


2) The same thing that you claim for "pause()" (that isn't restarted)
can be claimed for other syscalls that _ARE_ restarted.

Example: wait()

SUSV3 DOC: "... The wait() function shall suspend execution of the
calling thread until status information for one of the terminated child
processes of the calling process is available, or until delivery of a
signal whose action is either to execute a signal-catching function or
to terminate the process ..."

And wait() is actually RESTARTED because:

	- it makes sense
	- FreeBSD sigaction() mapage says it is retarted

	- Linux does it (see kernel/exit.c)

	...
                retval = -ERESTARTSYS;
                if (signal_pending(current))
                        goto end;
	...


See?

-- 
	Paolo Ornati
	Linux 2.6.13-rc3 on x86_64