Re: Extensible hashing and RCU

[Eric Dumazet <dada1@cosmosbay.com>]

On Wednesday 21 February 2007 13:41, Andi Kleen wrote:
> Eric Dumazet <dada1@cosmosbay.com> writes:
> > For example, sock_wfree() uses 1.6612 % of cpu because of false sharing
> > of sk_flags (dirtied each time SOCK_QUEUE_SHRUNK is set :(
>
> Might be easily fixable by moving the fields around a bit?

For this one, it seems sk_flags is mostly read, but SOCK_QUEUE_SHRUNK is 
mostly writen. It would make sense to move it to another point, to keep 
sk_flags shared by different cpus.

Maybe using one low order bit in a related pointer ? (like the rb_color() 
trick). Maybe this is time for a new include/linux/bap.h (bits and pointer) 
include :)

>
> > If we want to optimize tcp, we should reorder fields to reduce number of
> > cache lines, not change algos. struct sock fields are currently placed to
> > reduce holes, while they should be grouped by related fields sharing
> > cache lines.
>
> Regrouping is definitely a good thing; but I'm not sure why you are so
> deadset against exploring other data structures. The promise of RCUing
> and avoiding the big hash tables seems alluding to me, even if it
> only breaks even in the end in terms of cycles.

RCU is definitely wanted, and IP routing demonstrated the wins.

rbtree was successfully plugged into epoll instead of initial hash table 
implementation. 

Now, when the rate of lookups/inserts/delete is high, with totally random 
endpoints and cache *always* cold , 'tree structures' are not welcome (not 
cache friendly)