Re: [tbench regression fixes]: digging out smelly deadmen.

Eric Dumazet
2008-10-30T18:57:45Z

Eric Dumazet a écrit :
> Stephen Hemminger a écrit :
>> Has anyone looked into the impact of port randomization on this
>> benchmark.
>> If it is generating lots of sockets quickly there could be an impact:
>> * port randomization causes available port space to get filled
>> non-uniformly
>> and what was once a linear scan may have to walk over existing ports.
>> (This could be improved by a hint bitmap)
>>
>> * port randomization adds at least one modulus operation per socket
>> creation. This could be optimized by using a loop instead.
>
>
>
> tbench setups one socket per client, then send/receive lot of messages
> on this socket.
>
> Connection setup time can be ignored for the tbench regression analysis
>

Hum, re-reading your question, I feel you might have a valid point after all :)

Not because of connection setup time, but because the rwlocks used on tcp hash table.

tcp sessions used on this tbench test might now be on the same cache lines,
because of port randomization or so.

CPUS might do cache-line ping pongs on those rwlocks.

# netstat -tn|grep 7003
tcp 0 59 127.0.0.1:37248 127.0.0.1:7003 ESTABLISHED
tcp 0 71 127.0.0.1:7003 127.0.0.1:37252 ESTABLISHED
tcp 0 0 127.0.0.1:37251 127.0.0.1:7003 ESTABLISHED
tcp 0 4155 127.0.0.1:7003 127.0.0.1:37249 ESTABLISHED
tcp 0 55 127.0.0.1:7003 127.0.0.1:37248 ESTABLISHED
tcp 0 0 127.0.0.1:37252 127.0.0.1:7003 ESTABLISHED
tcp 0 0 127.0.0.1:37249 127.0.0.1:7003 ESTABLISHED
tcp 0 59 127.0.0.1:37246 127.0.0.1:7003 ESTABLISHED
tcp 0 0 127.0.0.1:37250 127.0.0.1:7003 ESTABLISHED
tcp 71 0 127.0.0.1:37245 127.0.0.1:7003 ESTABLISHED
tcp 0 0 127.0.0.1:37244 127.0.0.1:7003 ESTABLISHED
tcp 0 87 127.0.0.1:7003 127.0.0.1:37250 ESTABLISHED
tcp 0 4155 127.0.0.1:7003 127.0.0.1:37251 ESTABLISHED
tcp 0 4155 127.0.0.1:7003 127.0.0.1:37246 ESTABLISHED
tcp 0 71 127.0.0.1:7003 127.0.0.1:37245 ESTABLISHED
tcp 0 4155 127.0.0.1:7003 127.0.0.1:37244 ESTABLISHED

We use a jhash, so normally we could expect a really random split of hash values
for all these sessions, but it would be worth to check :)

You know understand why we want to avoid those rwlocks Stephen, and switch to RCU...