is non-inheritance of congestion control algorithm from the listen socket a bug or a feature?

is non-inheritance of congestion control algorithm from the listen socket a bug or a feature?

[Rick Jones]

Dave Taht asked me if I would consider adding control/reporting of the 
congestion control algorithm to netperf.  I'm in the midst of doing that 
and have noticed that setting a valid congestion control algorithm on a 
listen endpoint does not seem to be inherited by subsequent connections. 
  Rather than rely on netperf/netserver I cobbled-together a quick test 
program to show what I mean.

raj@raj-ubuntu-guest:~$ uname -a
Linux raj-ubuntu-guest 3.2.0-rc2+ #1 SMP Mon Nov 28 16:39:21 PST 2011 
x86_64 x86_64 x86_64 GNU/Linux

raj@raj-ubuntu-guest:~$ find /proc -name "*cong*" -print -exec cat {} \; 
2>/dev/null
/proc/sys/net/ipv4/tcp_congestion_control
cubic
/proc/sys/net/ipv4/tcp_available_congestion_control
cubic reno
/proc/sys/net/ipv4/tcp_allowed_congestion_control
cubic reno


# first just report what one gets
raj@raj-ubuntu-guest:~$ ./tcp_congestion_test
Will make no request to set congestion control algorithm
active socket has '' for congestion control
passive socket has '' for congestion control
after listen() passive socket has '' for congestion control
after connect() active has 'cubic' for congestion control
after accept() accepted has 'cubic' for congestion control

# now specify a valid alternative
raj@raj-ubuntu-guest:~$ ./tcp_congestion_test  reno
Will be requesting 'reno' as the congestion control algorithm
active socket has 'reno' for congestion control
passive socket has 'reno' for congestion control
after listen() passive socket has 'reno' for congestion control
after connect() active has 'reno' for congestion control
after accept() accepted has 'cubic' for congestion control

# now specify an invalid alternative just for completeness
raj@raj-ubuntu-guest:~$ ./tcp_congestion_test  bogus
Will be requesting 'bogus' as the congestion control algorithm
active socket has '' for congestion control
passive socket has '' for congestion control
after listen() passive socket has '' for congestion control
after connect() active has 'cubic' for congestion control
after accept() accepted has 'cubic' for congestion control

Is the non-inheritance of the congestion control algorithm a bug or a 
feature?

rick jones

Re: is non-inheritance of congestion control algorithm from the listen socket a bug or a feature?

[Dave Taht]

Hey, thx for trying it!

On Tue, Nov 29, 2011 at 9:03 PM, Rick Jones <rick.jones2@hp.com> wrote:

>
> Is the non-inheritance of the congestion control algorithm a bug or a
> feature?

Hmm... It's kind of both.

Recently I submitted an RFC patch to the
rsync mailing list, and it seemed nice to be able to set it on a per
socket basis based on various configuration options in the
rsync conf file and what was explicitly allowed.

That patch is here:

https://lists.samba.org/archive/rsync/2011-November/027111.html

It was also way cool to see tcp-lp in action vs other sorts of tcp.

That said we  have a range of allowable congestion algorithms
ranging from more intense than cubic to low bandwidth to wireless,
to highly specialized (data center tcp), to some that require
both sides to be running the same alg, others that are server
only...

Your question sort of involves what should a user be able
to set.

On the one hand, in the general case an app should
not be allowed to choose a more aggressive tcp.

On the other hand, an app on a wireless box,
requesting/getting, say, westwood over a default of lp or cubic sort
of makes sense.

On the gripping hand I thiink pushing this level of decison
making out to the app is just fine by me, so inherit
or not, is the app's call to setsockopt...

And in netperf's case, that would be all the algs made
available via allowable_congestion_control.

I had another idea regarding pushing alternate tcp algos out
to more apps without requiring source code changes,
which would be to create a cgroup controller, which
would be both trendy and would provide 'inherit-ability'...

But first up I think would be for more folk to see how
cool different algos can be.

>
> rick jones
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
FR Tel: 0638645374
http://www.bufferbloat.net

Re: is non-inheritance of congestion control algorithm from the listen socket a bug or a feature?

[Eric Dumazet]

Le mardi 29 novembre 2011 à 12:03 -0800, Rick Jones a écrit :

> Is the non-inheritance of the congestion control algorithm a bug or a 
> feature?

IMHO its a bug.

Can you provide a fix ?

Thanks !

Re: is non-inheritance of congestion control algorithm from the listen socket a bug or a feature?

[Rick Jones]

On 11/29/2011 12:56 PM, Eric Dumazet wrote:
> Le mardi 29 novembre 2011 à 12:03 -0800, Rick Jones a écrit :
>
>> Is the non-inheritance of the congestion control algorithm a bug or a
>> feature?
>
> IMHO its a bug.
>
> Can you provide a fix ?

I can try once I find my way through the maze.

rick

Re: is non-inheritance of congestion control algorithm from the listen socket a bug or a feature?

[Yuchung Cheng]

On Tue, Nov 29, 2011 at 12:56 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Le mardi 29 novembre 2011 à 12:03 -0800, Rick Jones a écrit :
>
>> Is the non-inheritance of the congestion control algorithm a bug or a
>> feature?
>
> IMHO its a bug.
>
> Can you provide a fix ?
>
> Thanks !
I actually think it's a feature :)

I find it awkward to set CC on listening socket. And current document
defines the sysctl well

tcp_congestion_control - STRING
        Set the congestion control algorithm to be used for new
        connections. The algorithm "reno" is always available, but
        additional choices may be available based on kernel configuration.
        Default is set as part of kernel configuration.

Re: is non-inheritance of congestion control algorithm from the listen socket a bug or a feature?

[Eric Dumazet]

Le mardi 29 novembre 2011 à 13:20 -0800, Yuchung Cheng a écrit :
> I actually think it's a feature :)
> 
> I find it awkward to set CC on listening socket. And current document
> defines the sysctl well
> 
> tcp_congestion_control - STRING
>         Set the congestion control algorithm to be used for new
>         connections. The algorithm "reno" is always available, but
>         additional choices may be available based on kernel configuration.
>         Default is set as part of kernel configuration.

This might be a feature, but contradicts most socket options set on
listener and inherited by a child socket on accept()

tcp_congestion_control should be system wide default.

Anyway, "man 7 tcp" doesnt document TCP_CONGESTION, so we are not
supposed to play with it ;)

Oh well...

Re: is non-inheritance of congestion control algorithm from the listen socket a bug or a feature?

[David Miller]

From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Tue, 29 Nov 2011 22:46:15 +0100

> Le mardi 29 novembre 2011 à 13:20 -0800, Yuchung Cheng a écrit :
>> I actually think it's a feature :)
>> 
>> I find it awkward to set CC on listening socket. And current document
>> defines the sysctl well
>> 
>> tcp_congestion_control - STRING
>>         Set the congestion control algorithm to be used for new
>>         connections. The algorithm "reno" is always available, but
>>         additional choices may be available based on kernel configuration.
>>         Default is set as part of kernel configuration.
> 
> This might be a feature, but contradicts most socket options set on
> listener and inherited by a child socket on accept()

There is really no reason to keep the current behavior.

If an application sets the congestion control algorithm on a listening
socket to a non-default value, what effect could possibly be intended?

Congestion control doesn't even come into play at all on a listening
socket, therefore the only logical expectation is that it inherits to
the child.

The only other logical behavior would be to forbid this operation on a
listening socket, since it has no effect, but that doesn't make any
sense now does it? :-)

Re: is non-inheritance of congestion control algorithm from the listen socket a bug or a feature?

[Eric Dumazet]

Le mardi 29 novembre 2011 à 16:52 -0500, David Miller a écrit :

> There is really no reason to keep the current behavior.
> 
> If an application sets the congestion control algorithm on a listening
> socket to a non-default value, what effect could possibly be intended?
> 
> Congestion control doesn't even come into play at all on a listening
> socket, therefore the only logical expectation is that it inherits to
> the child.
> 
> The only other logical behavior would be to forbid this operation on a
> listening socket, since it has no effect, but that doesn't make any
> sense now does it? :-)

Moreover, an application can use setsockopt(TCP_CONGESTION) before
calling listen() (while socket is still in CLOSE state)

Re: is non-inheritance of congestion control algorithm from the listen socket a bug or a feature?

[Stephen Hemminger]

On Tue, 29 Nov 2011 22:56:26 +0100
Eric Dumazet <eric.dumazet@gmail.com> wrote:

> Le mardi 29 novembre 2011 à 16:52 -0500, David Miller a écrit :
> 
> > There is really no reason to keep the current behavior.
> > 
> > If an application sets the congestion control algorithm on a listening
> > socket to a non-default value, what effect could possibly be intended?
> > 
> > Congestion control doesn't even come into play at all on a listening
> > socket, therefore the only logical expectation is that it inherits to
> > the child.
> > 
> > The only other logical behavior would be to forbid this operation on a
> > listening socket, since it has no effect, but that doesn't make any
> > sense now does it? :-)
> 
> Moreover, an application can use setsockopt(TCP_CONGESTION) before
> calling listen() (while socket is still in CLOSE state)

Agreed, it was just an oversight of the initial design.
The setsockopt() on the listening socket is ignored.