Re: Question regarding expected behavior of two udp sockets with SO_REUSEADDR set

[Neil Horman <nhorman@tuxdriver.com>]

On Sat, Nov 20, 2010 at 05:06:55AM +0100, Eric Dumazet wrote:
> Le vendredi 19 novembre 2010 à 19:48 -0500, Neil Horman a écrit :
> > Hey all-
> > 
> > 	Got a question regarding expected/desired behavior of $SUBJECT
> > 
> > 
> > I have a report of a problem with a program that opens two sockets:
> > 
> > The first socket is UDP and binds to 127.0.0.1 on a randomly selected port
> > 
> > The second socket is UDP and calls connect, sending to the first socket
> > 
> > Both sockets are part of the same process and have SO_REUSEADDR set
> > 
> > After the connect the second socket sends a message to the first socket.  The
> > first socket waits for the message by calling select().
> > 
> > Its observed that occasionally the first socket fails to receive the message,
> > which is odd, given that the system is unloaded, and this is the only message
> > being sent.  A little investigation shows that when this happens, the client and
> > the server wind up bound to the same port.
> > 
> > This happens because the second socket calls inet_autobind during the connect
> > call, and since both it and the server have SO_REUSEADDR set, it is possible
> > that the autobind will select the same port that the first socket is bound to.
> > When this happens the sendmsg path can get confused.  Specifically, when the skb
> > is delivered to the destination socket, the hash lookup might find the wrong
> > entry and enqueue the skb to the second socket instead of the first.
> > 
> > Questions:
> > 
> > 1) Is that expected?
> > 
> 
> Is SO_REUSADDR used on both sockets ?
> 
Yes, both udp sockets have SO_REUSEADDR set on them

> May I ask why SO_REUSEADDR is set in the first place on UDP sockets ?
> 
Honestly, I don't know.  This was reported to me as part of:
https://bugzilla.redhat.com/show_bug.cgi?id=643911
At first the consensus was that this bug was fixed by your patch series that
adds a secondary hash for udp sockets, but on closer inspection it appears that
this is just a case of what I described above.  Specifically, that two sockets
are inadvertently binding to the same port/address, and as such when someone
sends from socket A to socket B, A is actually the socket that receives the
frame rather than socket B (as the program might have intended).

> I use it before a bind() on a given port (non null), but apparently your
> program doesnt bind() the 2nd socket before its connect() ?
> 
Correct, the second socket is autobound, via inet_autobind as called from
connect(), which we call on the second udp socket.  When that happens the socket
is bound to a random port.  But sometimes if the socket has SO_REUSEADDR set it
winds up binding to the same port that the first socket is bound to, resulting
in the above problem.

> 
> > 2) If not, what do you think the best way to fix it is?
> > 
> > 	a) Deny autobinds to the same port when SO_REUSEADDR is set, but allow
> > explicity binds to the same port?
> > 
> > 	b) Deny both autobinds and explicit binds to the same port/addr,
> > effectively disablind SO_REUSEADDR with UDP, kind of like with listening TCP
> > sockets
> > 
> > 	c) Add magic to udp_rcv to detect skbs originating from local sockets,
> > and _dont_ deliver to the socket it originated from
> 
> Why ? Its a valid use case IMHO, even with a single socket.
> 
> > 
> > I'm inclined to say, no this is not expected behavior, and that we should fix it
> > with option A, but I'm interested in getting other opinions before I go down any
> > particular path.
> > 
> 
> autobind certainly is a problem, we tried to 'fix' it in recent past and
> had to revert some patches. We tried to allow more sockets to be used
> but we failed.
> 
Agreed.  My thought was to add logic to udp_lib_lport_inuse such that, if
sk_reuse is set on both sockets, and the input snum is 0 (indicating autobind)
we should not allow binding sk to inet_sk(sk2)->num.  Thoughts?
Neil

> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>