Re: SO_REUSEADDR not allowing server and client to use same port

Re: SO_REUSEADDR not allowing server and client to use same port

[Willy Tarreau]

[cc'd netdev]

On Thu, Feb 28, 2008 at 12:15:06PM -0600, Nebojsa Miljanovic wrote:
> Hello all,
> I have seen similar complaints about this issue before in the list archive, but
> I have not seen any resolution. So, I am posting the question again in hope of
> getting some replies.
> 
> Currently, Linux does not allow reuse of same local port for both TCP server and
> TCP client. I understand that there may be a need to prevent two servers from
> binding to the same local port, but having 1 server and 1 outgoing client use it
> should be allowed. Other Unix like operating systems do allow it.

I've been encountering the need for SO_REUSEPORT as implemented on some BSD
for instance. It allows any process to explicitly permit another one to bind
to the same IP:port provided that it also sets its socket with SO_REUSEPORT.
This is a real requirement when you need to restart a service without any
service disruption.

In the end, I've written a trivial patch for 2.4 (but most likely 2.6 would
use nearly the same one). I noticed that the conflict detection logic already
distinguished between 3 cases: sk->reuse==0, sk->reuse==1, sk->reuse >1.
sk->reuse was set to 1 with SO_REUSEADDR. I simply had to make SO_REUSEPORT
do sk->reuse |= 2 to get everything working as expected.

I don't know yet if it is as easy to do in 2.6, but if there is a possible
acceptance of including this useful feature into mainline, I can try to
work a patch (possibly cleaner than my existing one for 2.4).

Regards,
Willy

Re: SO_REUSEADDR not allowing server and client to use same port

[Nebojsa Miljanovic]

Willy,
I agree. That would also work nicely. I do wish SO_REUSEPORT was available.

But, I am amazed that this issue has a long history (I've seen posts from 1999)
and that there was no progress with it.

Thanks,
Neb


On 2/28/2008 2:36 PM, Willy Tarreau wrote:
> [cc'd netdev]
> 
> On Thu, Feb 28, 2008 at 12:15:06PM -0600, Nebojsa Miljanovic wrote:
> 
>>Hello all,
>>I have seen similar complaints about this issue before in the list archive, but
>>I have not seen any resolution. So, I am posting the question again in hope of
>>getting some replies.
>>
>>Currently, Linux does not allow reuse of same local port for both TCP server and
>>TCP client. I understand that there may be a need to prevent two servers from
>>binding to the same local port, but having 1 server and 1 outgoing client use it
>>should be allowed. Other Unix like operating systems do allow it.
> 
> 
> I've been encountering the need for SO_REUSEPORT as implemented on some BSD
> for instance. It allows any process to explicitly permit another one to bind
> to the same IP:port provided that it also sets its socket with SO_REUSEPORT.
> This is a real requirement when you need to restart a service without any
> service disruption.
> 
> In the end, I've written a trivial patch for 2.4 (but most likely 2.6 would
> use nearly the same one). I noticed that the conflict detection logic already
> distinguished between 3 cases: sk->reuse==0, sk->reuse==1, sk->reuse >1.
> sk->reuse was set to 1 with SO_REUSEADDR. I simply had to make SO_REUSEPORT
> do sk->reuse |= 2 to get everything working as expected.
> 
> I don't know yet if it is as easy to do in 2.6, but if there is a possible
> acceptance of including this useful feature into mainline, I can try to
> work a patch (possibly cleaner than my existing one for 2.4).
> 
> Regards,
> Willy
> 

SO_REUSEADDR not allowing server and client to use same port

[Nebojsa Miljanovic]

Hello all,
I have seen similar complaints about this issue before in the list archive, but
I have not seen any resolution. So, I am posting the question again in hope of
getting some replies.

Currently, Linux does not allow reuse of same local port for both TCP server and
TCP client. I understand that there may be a need to prevent two servers from
binding to the same local port, but having 1 server and 1 outgoing client use it
should be allowed. Other Unix like operating systems do allow it. Further more,
Linux SCTP socket API allows for this to happen. And, I can't imagine why this
restriction would exists for TCP and not for SCTP.

This issue is caused by tcp_v4_get_port() function (same check is in IPV6
version of this function). It does does not allow for "tb->fastreuse" bit to be
set when passed socket is in listening state. Suggested code change below would
allow for port reuse by a single listening socket and multiple client sockets.

[neb@mvista ipv4]$ cvs diff -u20 -p -w -b tcp_ipv4.c
Index: tcp_ipv4.c
===================================================================
RCS file:
/cvs/cvsroot/Repository/TelicaRoot/components/mvlinux/cge/devkit/lsp/7xx/linux/net/ipv4/tcp_ipv4.c,v
retrieving revision 1.1.1.2
diff -u -2 -0 -p -w -b -r1.1.1.2 tcp_ipv4.c
--- tcp_ipv4.c  27 Apr 2007 12:34:39 -0000      1.1.1.2
+++ tcp_ipv4.c  28 Feb 2008 18:10:16 -0000
@@ -258,46 +258,54 @@ static int tcp_v4_get_port(struct sock *
        tb = NULL;
        goto tb_not_found;
 tb_found:
        if (!hlist_empty(&tb->owners)) {
                if (sk->sk_reuse > 1)
                        goto success;
                if (tb->fastreuse > 0 &&
                    sk->sk_reuse && sk->sk_state != TCP_LISTEN) {
                        goto success;
                } else {
                        ret = 1;
                        if (tcp_bind_conflict(sk, tb))
                                goto fail_unlock;
                }
        }
 tb_not_found:
        ret = 1;
        if (!tb && (tb = tcp_bucket_create(head, snum)) == NULL)
                goto fail_unlock;
        if (hlist_empty(&tb->owners)) {
+#if 1 /* Do not check for TCP_LISTEN state */
+               if (sk->sk_reuse)
+#else
                if (sk->sk_reuse && sk->sk_state != TCP_LISTEN)
+#endif
                        tb->fastreuse = 1;
                else
                        tb->fastreuse = 0;
        } else if (tb->fastreuse &&
+#if 1 /* Do not check for TCP_LISTEN state */
+                  (!sk->sk_reuse))
+#else
                   (!sk->sk_reuse || sk->sk_state == TCP_LISTEN))
+#endif
                tb->fastreuse = 0;
 success:
        if (!tcp_sk(sk)->bind_hash)
                tcp_bind_hash(sk, tb, snum);
        BUG_TRAP(tcp_sk(sk)->bind_hash == tb);
        ret = 0;

 fail_unlock:
        spin_unlock(&head->lock);
 fail:
        local_bh_enable();
        return ret;
 }


Thanks,
Neb Miljanovic

Alcatel-Lucent