Re: [tbench regression fixes]: digging out smelly deadmen.

[Eric Dumazet <dada1@cosmosbay.com>]

Eric Dumazet a écrit :
> Stephen Hemminger a écrit :
>> Has anyone looked into the impact of port randomization on this 
>> benchmark.
>> If it is generating lots of sockets quickly there could be an impact:
>>   * port randomization causes available port space to get filled 
>> non-uniformly
>>     and what was once a linear scan may have to walk over existing ports.
>>     (This could be improved by a hint bitmap)
>>
>>   * port randomization adds at least one modulus operation per socket
>>     creation. This could be optimized by using a loop instead.
> 
> 
> 
> tbench setups one socket per client, then send/receive lot of messages 
> on this socket.
> 
> Connection setup time can be ignored for the tbench regression analysis
> 

Hum, re-reading your question, I feel you might have a valid point after all :)

Not because of connection setup time, but because the rwlocks used on tcp hash table.

tcp sessions used on this tbench test might now be on the same cache lines,
because of port randomization or so.

CPUS might do cache-line ping pongs on those rwlocks.

# netstat -tn|grep 7003
tcp        0     59 127.0.0.1:37248         127.0.0.1:7003          ESTABLISHED
tcp        0     71 127.0.0.1:7003          127.0.0.1:37252         ESTABLISHED
tcp        0      0 127.0.0.1:37251         127.0.0.1:7003          ESTABLISHED
tcp        0   4155 127.0.0.1:7003          127.0.0.1:37249         ESTABLISHED
tcp        0     55 127.0.0.1:7003          127.0.0.1:37248         ESTABLISHED
tcp        0      0 127.0.0.1:37252         127.0.0.1:7003          ESTABLISHED
tcp        0      0 127.0.0.1:37249         127.0.0.1:7003          ESTABLISHED
tcp        0     59 127.0.0.1:37246         127.0.0.1:7003          ESTABLISHED
tcp        0      0 127.0.0.1:37250         127.0.0.1:7003          ESTABLISHED
tcp       71      0 127.0.0.1:37245         127.0.0.1:7003          ESTABLISHED
tcp        0      0 127.0.0.1:37244         127.0.0.1:7003          ESTABLISHED
tcp        0     87 127.0.0.1:7003          127.0.0.1:37250         ESTABLISHED
tcp        0   4155 127.0.0.1:7003          127.0.0.1:37251         ESTABLISHED
tcp        0   4155 127.0.0.1:7003          127.0.0.1:37246         ESTABLISHED
tcp        0     71 127.0.0.1:7003          127.0.0.1:37245         ESTABLISHED
tcp        0   4155 127.0.0.1:7003          127.0.0.1:37244         ESTABLISHED

We use a jhash, so normally we could expect a really random split of hash values
for all these sessions, but it would be worth to check :)

You know understand why we want to avoid those rwlocks Stephen, and switch to RCU...