Re: slow tcp acks on loopback device

Re: slow tcp acks on loopback device

[David S. Miller]

From: Steve French <smfltc@us.ibm.com>
Date: 22 Jul 2005 14:56:59 -0500

> Noticing that the loopback device (at least on RHEL4) has an unfortunate
> mtu size 16384 (which is about 50 bytes too small for SMB read
> responses), I did try increasing the MTU slightly.  Changing that to
> 18000 did avoid the fragmentation and the 40ms delay - but what puzzled
> me was why setting TCP_NODELAY after the socket was created did not
> eliminate the delay on the ack and if there is a way to avoid the huge
> tcp ack delay by either doing something else to force client acking
> immediately or to do something on the client side of the stack to get
> the server to send the whole 16K+ frame - it looks like the tcp windows
> is 32K if the value in the tcp acks in the network trace is to be
> trusted.

TCP_NODELAY does not control ACK generation, instead it modifies
the Nagle algorithm behavior when sending data packets.

Please take networking discussions to netdev@vger.kernel.org which
is where the networking developers are.

Re: slow tcp acks on loopback device

[Steve French]

> TCP_NODELAY does not control ACK generation, instead it modifies
> the Nagle algorithm behavior when sending data packets.

Looks like RHEL4 had an odd default smb.conf on the server side - the
RedHat smb.conf seems to set SO_RCVBUF and SO_SNDBUF to 8K (reducing
them from their default). This seems very odd since common TCP frame
sizes for this protocol (SMB/CIFS) will be just over 16K (the negotiated
SMB buffer size) as well as even large frames, 60K, for large read/write
requests which Windows sends.  Removing the SO_SNDBUF, SO_RCVBUF options
(as in the smb.conf for other distros), increased performance a lot. 
The Samba Howto recommends against setting them so small as well.

The alternative solution configuring the client end - altering ack
behavior to allow the client to do quick(er) acks probably would have
helped too, but of course has sideeffects (more acks) and it was not
obvious to me how to do that safely from a kernel vfs module.