Strange slow transfer behavior on local host connection

Strange slow transfer behavior on local host connection

[Jerry Jerry]

Hello everyone,

We have found a very strange behavior where the transfer is
particularly slower when sending data to a socket connecting ports on
the same host, but rather faster to the socket connecting ports on
different host.  /proc/net/sctp/assoc shows that TX_QUEUE of same host
socket buffers a lot of data but RX_QUEUE of the socket on the
receiving side is almost empty. On the contrary, both TX_QUEUE and
RX_QUEUE are almost empty for those sockets connecting different
hosts.

We first thought this might because the processing difference between
local and remote hosts, but the problem remains the same even for a
symmetric case where both hosts send same messages to both its own
local and remote receiver, and receive messages via different local
and remote connections. In our opinion, the data transfer should be
faster or at least the same when sending to a socket connecting local
ports.

This problem particularly occurs when the message size is small, but
playing with SCTP NODELAY option doesn't help.

I'm not sure if I may have missed anything, but any comments to shed
lights on this, or suggestions to tune would be appreciated very much!

Regards,
Jerry

Re: Strange slow transfer behavior on local host connection

[Vlad Yasevich]

On 03/16/2015 12:37 PM, Jerry Jerry wrote:
> Hello everyone,
> 
> We have found a very strange behavior where the transfer is
> particularly slower when sending data to a socket connecting ports on
> the same host, but rather faster to the socket connecting ports on
> different host.  /proc/net/sctp/assoc shows that TX_QUEUE of same host
> socket buffers a lot of data but RX_QUEUE of the socket on the
> receiving side is almost empty. On the contrary, both TX_QUEUE and
> RX_QUEUE are almost empty for those sockets connecting different
> hosts.
> 
> We first thought this might because the processing difference between
> local and remote hosts, but the problem remains the same even for a
> symmetric case where both hosts send same messages to both its own
> local and remote receiver, and receive messages via different local
> and remote connections. In our opinion, the data transfer should be
> faster or at least the same when sending to a socket connecting local
> ports.
> 
> This problem particularly occurs when the message size is small, but
> playing with SCTP NODELAY option doesn't help.

This is mostly likely due to the congestion window not being able to
grow because the loopback mtu is higher then amount of queued data.

You could try tweaking the loopback MTU to see if it changes things..

-vlad

> 
> I'm not sure if I may have missed anything, but any comments to shed
> lights on this, or suggestions to tune would be appreciated very much!
> 
> Regards,
> Jerry
> --
> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: Strange slow transfer behavior on local host connection

[Jerry Jerry]

Thanks a lot, vlad, I'll give it a try.

Regards,
Jerry

On Mon, Mar 16, 2015 at 3:24 PM, Vlad Yasevich <vyasevich@gmail.com> wrote:
> On 03/16/2015 12:37 PM, Jerry Jerry wrote:
>> Hello everyone,
>>
>> We have found a very strange behavior where the transfer is
>> particularly slower when sending data to a socket connecting ports on
>> the same host, but rather faster to the socket connecting ports on
>> different host.  /proc/net/sctp/assoc shows that TX_QUEUE of same host
>> socket buffers a lot of data but RX_QUEUE of the socket on the
>> receiving side is almost empty. On the contrary, both TX_QUEUE and
>> RX_QUEUE are almost empty for those sockets connecting different
>> hosts.
>>
>> We first thought this might because the processing difference between
>> local and remote hosts, but the problem remains the same even for a
>> symmetric case where both hosts send same messages to both its own
>> local and remote receiver, and receive messages via different local
>> and remote connections. In our opinion, the data transfer should be
>> faster or at least the same when sending to a socket connecting local
>> ports.
>>
>> This problem particularly occurs when the message size is small, but
>> playing with SCTP NODELAY option doesn't help.
>
> This is mostly likely due to the congestion window not being able to
> grow because the loopback mtu is higher then amount of queued data.
>
> You could try tweaking the loopback MTU to see if it changes things..
>
> -vlad
>
>>
>> I'm not sure if I may have missed anything, but any comments to shed
>> lights on this, or suggestions to tune would be appreciated very much!
>>
>> Regards,
>> Jerry
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>