Re: [PATCH] Cpuset: rcu optimization of page alloc hook

[Eric Dumazet <dada1@cosmosbay.com>]

Paul Jackson a écrit :
> Eric wrote:
> 
>>struct kmem_cache  itself will be about 512*8 + some bytes
>>then for each cpu a 'struct array_cache' will be allocated (count 128 bytes 
> 
> 
> Hmmm ... 'struct array_cache' looks to be about 6 integer words,
> so if that is the main per-CPU cost, the minimal cost of a slab
> cache (once created, before use) is about 24 bytes per cpu.

Nope, because struct array_cache includes a variable length table of pointers 
to hold a cache of available objects per cpu. The 'limit' (the number of 
pointer in this cache) depends on the object size.

See enable_cpucache in mm/slab.c for 'limit' determination :

         if (cachep->objsize > 131072)
                 limit = 1;
         else if (cachep->objsize > PAGE_SIZE)
                 limit = 8;
         else if (cachep->objsize > 1024)
                 limit = 24;
         else if (cachep->objsize > 256)
                 limit = 54;
         else
                 limit = 120;



Let's take an example :

grep dentry /proc/slabinfo

dentry_cache      157113 425289    224   17    1 : tunables  120   60    8 : 
slabdata  25017  25017      0


'limit' is the number following 'tunable' : 120

On a 64 bits machines, 120*sizeof(void*) = 120*8 = 960

So for small objects (<= 256 bytes), you end with a sizeof(arracy_cache) = 
1024 bytes per cpu

If 512 CPUS : 512*1024 = 512 Kbytes + all other kmem_cache structures : (If 
you have a lot of Memory Nodes, then it can be very big too)

If you know that no more than 100 objects are used in 99% of setups, then a 
dedicated cache is overkill.

> 
> But whether its 24 or 128 bytes per cpu, that's a heavier weight
> hammer than is needed here.
> 
> Time for me to learn more about rcu.
> 
> Thanks for raising this issue.
> 

You are welcome.

Eric