public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
* Re: 2.2 to 2.4... serious TCP send slowdowns
@ 2002-07-19 20:09 Nivedita Singhvi
  0 siblings, 0 replies; 8+ messages in thread
From: Nivedita Singhvi @ 2002-07-19 20:09 UTC (permalink / raw)
  To: hayden; +Cc: linux-kernel

> We're finally migrating to the 2.4 kernel due to hardware
> incompatibilities with the 2.2.  The 2.2 has worked better
> for us in the past as far as our application performs.
> Our application is an adserver and becomes bogged down in 2.4
> when sending files such as images across

When you say bogged down, what exactly does that mean? Does it
hang? Can you quantify the slowdown with any measurements?
Have you looked at TCP and network statistics to check for
timeouts, drops, other errors, the like? netstat -s should
give you some extended TCP stats which might help you diagnose
that sort of problem..

> the wire.  They're in general between 20-50k in size.  I've been
> researching the differences between 2.4 and 2.2 and have noticed
> that a lot of work has gone into autotuning with 2.4 and I'm
> wondering if this is what's slowing things down.  When I do tcpdumps
> to see the traffic being sent to the client I'm noticing that the
> receiver window is almost always set to 6430 bytes.  When looking at
> the same transfer on our 2.2 boxes the receiver window is almost
> always over 31000 bytes.  I've tried to increase the size of the
> buffers using the proc settings that are provided however

Whats your interface MTU? How did you change the size of the buffers?
Note that you need to increase the tcp_rmem[1] and tcp_wmem[1] to
affect the default tcp socket buffer sizes. Also note that
approximately half that is used by the kernel, so if you really want
64K user space, try setting the size to 128K.

> this hasn't seemed to make a difference even after restarting
> servers after each change the window is still 6430 bytes.  I've
> tried manually settting the size with setsockopt calls in the server
> code but this hasn't seemed to help.  I believe the problem is
> definately with sending the files over the line.  We files are read
> into the socket to be sent across the network byte by byte.  The boss
> says this is the best way to do it but I'm curious if this is so.

You cant optimize your read() from a fd and writes to a socket fd()
simultaneously. Are you setting TCP_NODELAY?

If all you are doing is reading large files from disk and sending them
out over a socket, consider using sendfile() instead. Much more
efficient.

thanks,
Nivedita





^ permalink raw reply	[flat|nested] 8+ messages in thread
[parent not found: <Pine.LNX.4.10.10207221646120.4476-100000@compaq.skyline.net>]
[parent not found: <Pine.LNX.4.10.10207221603340.4476-100000@compaq.skyline.net>]
* 2.2 to 2.4 migration
@ 2002-07-18 23:30 Hayden Myers
  2002-07-19 17:04 ` 2.2 to 2.4... serious TCP send slowdowns Hayden Myers
  0 siblings, 1 reply; 8+ messages in thread
From: Hayden Myers @ 2002-07-18 23:30 UTC (permalink / raw)
  To: linux-kernel

We're finally migrating to the 2.4 kernel due to hardware
incompatibilities with the 2.4.  The 2.2 has worked better for us in the
past as far as our application performs.  Our application is an adserver
and becomes bogged down in 2.4 when sending files such as images across
the wire.  They're in general between 20-50k in size.  I've been
researching the differences between 2.4 and 2.2 and have noticed that a
lot of work has gone into autotuning with 2.4 and I'm wondering if this is
what's slowing things down.  When I do tcpdumps to see the traffic being
sent to the client I'm noticing that the receiver window is almost always
set to 6430 bytes.  When looking at the same transfer on our 2.2 boxes the
receiver window is almost always over 31000 bytes.  I've tried to increase
the size of the buffers using the proc settings that are provided however
this hasn't seemed to make a difference even after restarting servers
after each change the window is still 6430 bytes.  I've tried manually
settting the size with setsockopt calls in the server code but this hasn't
seemed to help.  I believe the problem is definately with sending the
files over the line.  We files are read into the socket to be sent across
the network byte by byte.  The boss says this is the best way to do it but
I'm curious if this is so.  The code that reads the file into the socket
to go across the network is below.


int output_block(int socket, char *filename)
{
int fd, count = 0;
size_t total_bytes = 0;
/*size_t buf_cnt = 1460;*/
size_t buf_cnt = 512;
char buffer[buf_cnt];
fd_set rfds;
struct timeval tv;

   if ((fd = open(filename, O_RDONLY)) < 0) {
      //fprintf(stderr, "Unable to open filename: %s\n", filename);
      return(-1);
   }

   while ((count = read(fd, &buffer, buf_cnt)) > 0) {

      FD_ZERO(&rfds);
      FD_SET(socket, &rfds);
      tv.tv_sec = 10;
      tv.tv_usec = 0;
      if (select(socket+1, NULL, &rfds, NULL, &tv) <= 0) {
         //fprintf(stderr, "Output_block timeout\n");
         break;
      }

      if (writen(socket, buffer, count) <= 0)
         break;
      total_bytes += count;
   }

   close(fd);
   return(total_bytes);

The application is a single threaded app using a multiprocess pre forking
model if that helps any.  I'm really baffled as to why using the 2.4
kernel is slowing us down.  Any help is appreciated.  Sorry if this has
come up before.  I really have been looking for help for quite some time
before posting this.

Hayden Myers	
Support Manager
Skyline Network Technologies	
hayden@spinbox.com
(410)583-1337 option 2



^ permalink raw reply	[flat|nested] 8+ messages in thread
end of thread, other threads:[~2002-07-23  7:20 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2002-07-19 20:09 2.2 to 2.4... serious TCP send slowdowns Nivedita Singhvi
     [not found] <Pine.LNX.4.10.10207221646120.4476-100000@compaq.skyline.net>
2002-07-22 21:43 ` Mika Liljeberg
     [not found] <Pine.LNX.4.10.10207221603340.4476-100000@compaq.skyline.net>
2002-07-22 20:32 ` Mika Liljeberg
  -- strict thread matches above, loose matches on Subject: below --
2002-07-18 23:30 2.2 to 2.4 migration Hayden Myers
2002-07-19 17:04 ` 2.2 to 2.4... serious TCP send slowdowns Hayden Myers
2002-07-20 20:53   ` Alan Cox
2002-07-22 18:47     ` Hayden Myers
2002-07-22 19:51       ` Mika Liljeberg
2002-07-23  7:24       ` Buddy Lumpkin

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox