PROBLEM: blocking read on socket repeatedly returns EAGAIN

PROBLEM: blocking read on socket repeatedly returns EAGAIN

[Kern Sibbald]

A read() on a TCP/IP socket, which should block returns -1 with errno=EAGAIN

Unless I am mistaken, the read() should block as the socket is active with no 
problems.  The only "unusual" items are that I have set the network buffer 
size to 32K (32 * 1024), IPTOS_THROUGHPUT, and keepalive.  In addition, I put 
a lot of data into the write side of the socket, then do the read().  The 
other end of the socket is perfectly alive, but does not read() the data I 
have written, nor does it write() anything.  When my read() is issued, I 
expect it to block, but it immediately returns with -1 and errno set to 
EAGAIN.  If the read() is re-issued, a CPU intensive loop results as long as 
the other end does not read() the data written to the socket.  This is a 
multi-threaded program, but the other threads are all blocked on something.

Kernel: Fedora FC4
Linux version 2.6.12-1.1398_FC4smp (bhcompile@tweety.build.redhat.com) (gcc 
version 4.0.0 20050519 (Red Hat 4.0.0-8)) #1 SMP Fri Jul 15 01:30:13 EDT 2005

This problem occurs in my network backup program, Bacula, so it would be a 
fair amount of work to send you a program that shows this behavior.

Best regards,

Kern

Re: PROBLEM: blocking read on socket repeatedly returns EAGAIN

[Alan Cox]

On Maw, 2005-08-16 at 15:19 +0200, Kern Sibbald wrote:
> have written, nor does it write() anything.  When my read() is issued, I 
> expect it to block, but it immediately returns with -1 and errno set to 
> EAGAIN.  If the read() is re-issued, a CPU intensive loop results as long as 
> the other end does not read() the data written to the socket.  This is a 
> multi-threaded program, but the other threads are all blocked on something.

You are describing behaviour as expected with nonblocking set. That
suggests to me that something or someone set or inherited the nonblock
flag on that socket. Is the strange behaviour specific to the latest
kernel ?

Alan

Re: PROBLEM: blocking read on socket repeatedly returns EAGAIN

[Kern Sibbald]

On Tuesday 16 August 2005 16:03, Alan Cox wrote:
> On Maw, 2005-08-16 at 15:19 +0200, Kern Sibbald wrote:
> > have written, nor does it write() anything.  When my read() is issued, I
> > expect it to block, but it immediately returns with -1 and errno set to
> > EAGAIN.  If the read() is re-issued, a CPU intensive loop results as long
> > as the other end does not read() the data written to the socket.  This is
> > a multi-threaded program, but the other threads are all blocked on
> > something.
>
> You are describing behaviour as expected with nonblocking set. That
> suggests to me that something or someone set or inherited the nonblock
> flag on that socket. 

I verified that I have not explicitly set nonblocking on the socket, so expect 
it to be default blocking. 

> Is the strange behaviour specific to the latest  kernel ?

This behavior, manifesting itself as a CPU loop, has been plaguing me for a 
number of months now.  It is not specific to the latest kernel since  it 
happened on FC3 and all kernels between.  Before FC3 I was on RHEL3 (2.4 
kernel) but am unsure if I saw the problem there, my best guess is that it is 
2.6 related, but I cannot guarantee that.

I have a workaround for the problem (sleep 200ms), but can repeat it and will 
be happy to provide more info as you request.  

The only thing I would correct in what I initially wrote is I believe that the 
other end of the socket did read some of the data I wrote before I did my 
read() expecting it to block.


Kern

Re: PROBLEM: blocking read on socket repeatedly returns EAGAIN

[Alan Cox]

On Maw, 2005-08-16 at 16:12 +0200, Kern Sibbald wrote:
> I verified that I have not explicitly set nonblocking on the socket, so expect 
> it to be default blocking. 

Depends where it came from and what OS. In particular the blocking state
of a socket returned from accept may be the same as the accepting
socket, or may be blocking depending on the exact system.

You could also try turning blocking on and seeing what effect that has

Re: PROBLEM: blocking read on socket repeatedly returns EAGAIN

[Kern Sibbald]

On Tuesday 16 August 2005 16:03, Alan Cox wrote:
> On Maw, 2005-08-16 at 15:19 +0200, Kern Sibbald wrote:
> > have written, nor does it write() anything.  When my read() is issued, I
> > expect it to block, but it immediately returns with -1 and errno set to
> > EAGAIN.  If the read() is re-issued, a CPU intensive loop results as long
> > as the other end does not read() the data written to the socket.  This is
> > a multi-threaded program, but the other threads are all blocked on
> > something.
>
> You are describing behaviour as expected with nonblocking set. That
> suggests to me that something or someone set or inherited the nonblock
> flag on that socket. Is the strange behaviour specific to the latest
> kernel ?

Well, it looks like I have egg on my face; it is not a kernel problem.

Return value from fcntl(F_GETFL) in decimal, hex, octal:
rufus-fd: bnet.c:84 fcntl=2050 0x802 04002

Someone is setting nonblocking on my socket ! -- perhaps some thread library?  
Anyway, it is definitely not me, and this is a thread of the main program so 
I don't see inheritance as the problem. I'll track it down and send the bug 
report to the appropriate place.

Thanks for the fast response and sorry for the inconvenience.

Kern

Re: PROBLEM: blocking read on socket repeatedly returns EAGAIN

[David S. Miller]

From: Kern Sibbald <kern@sibbald.com>
Date: Tue, 16 Aug 2005 16:38:14 +0200

> Someone is setting nonblocking on my socket !

Glad that's resolved...

Re: PROBLEM: blocking read on socket repeatedly returns EAGAIN

[Kern Sibbald]

On Tuesday 16 August 2005 20:39, David S. Miller wrote:
> From: Kern Sibbald <kern@sibbald.com>
> Date: Tue, 16 Aug 2005 16:38:14 +0200
>
> > Someone is setting nonblocking on my socket !
>
> Glad that's resolved...

Yes, my stupidity.  There was one more fcntl() in my source than I thought :-(

By the way, if a signal is delivered while blocked on a read, IMO, the OS 
should return EINTR.  If it doesn't, oh well, I'll live with it.

I wrote this code 5 years ago and was just now wondering why I bothered to 
test on EAGAIN. Your comment about a signal causing EAGAIN to be returned 
clarifies a lot.

Re: PROBLEM: blocking read on socket repeatedly returns EAGAIN

[David S. Miller]

From: Alan Cox <alan@lxorguk.ukuu.org.uk>
Date: Tue, 16 Aug 2005 15:03:11 +0100

> You are describing behaviour as expected with nonblocking set. That
> suggests to me that something or someone set or inherited the nonblock
> flag on that socket. Is the strange behaviour specific to the latest
> kernel ?

He could be receiving a signal Alan, look at tcp_recvmsg(),
it returns -EAGAIN and always has when a signal is delivered
to the reading process.

Re: PROBLEM: blocking read on socket repeatedly returns EAGAIN

[David S. Miller]

From: Kern Sibbald <kern@sibbald.com>
Subject: PROBLEM: blocking read on socket repeatedly returns EAGAIN
Date: Tue, 16 Aug 2005 15:19:39 +0200

> A read() on a TCP/IP socket, which should block returns -1 with errno=EAGAIN

If a signal is delivered to the process during the read(),
then -EAGAIN is perfectly valid.