Re: SO_REUSEPORT?

[Rick Jones <rick.jones2@hp.com>]

Tom Herbert wrote:
>>>We are looking at ways to scale TCP listeners.  I think we like is the
>>>ability to listen on a port from multiple threads (sockets bound to
>>>same port,  INADDR_ANY, and no interface binding) , which is what
>>>SO_REUSEPORT would seem to allow.  Has this ever been implemented for
>>>Linux or is there a good reason not to have it?
>>
>>On Linux, SO_REUSEADDR provide most of what SO_REUSEPORT provides on BSD.
>>
>>In any case, there is absolutely no point in creating multiple TCP listeners.
>>Multiple threads can accept() on the same listener - at the same time.
>>
> 
> 
> We've been doing that, but then on wakeup it would seem that we're at
> the mercy of scheduling-- basically which ever threads wakes up first
> will get to process accept queue first.  This seems to bias towards
> threads running on the same CPU as the wakeup is called, and   so this
> method doesn't give us an even distribution of new connections across
> the threads that we'd like.

How would the presence of multiple TCP LISTEN endpoints change that? 
You'd then be at the mercy of whatever "scheduling" there was inside the 
stack.

If you want to balance the threads, perhaps a dispatch thread, or a 
virtual one - each thread knows how many connections it is servicing, 
let them know how many the other threads are servicing, and if a thread 
has N more connections than the other threads have it not go into 
accept() that time around.  Might need some tweaking to handle 
pathological starvation cases like all the other threads are hung I 
suppose but the basic idea is there.

rick jones
> 
> Tom
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html