Re: [PATCH, netfilter] NUMA aware ipv4/netfilter/ip_tables.c

[Eric Dumazet <dada1@cosmosbay.com>]

Andi Kleen a écrit :
> On Monday 19 September 2005 19:09, Eric dumazet wrote:
> 
>>Hi
>>
>>Part of the performance problem we have with netfilter  is memory
>>allocation is not NUMA aware, but 'only' SMP aware.
> 
> 
> How do you know? Did you measure it somehow? 

Well, it seems that even a 1MB cache is not enough for this workload, on a 
dual Opterons 275 machine.

CPU: AMD64 processors, speed 2205.54 MHz (estimated)
Counted DATA_CACHE_MISSES events (Data cache misses) with a unit mask of 0x00 
(No unit mask) count 5000
samples  %        symbol name
86104    13.3027  ipt_do_table
21508     3.3229  copy_user_generic_c
18618     2.8764  try_to_wake_up
18180     2.8087  memset
18085     2.7941  tcp_v4_rcv
16949     2.6185  schedule

# iptables -nvL | wc -l
     190



> 
> Normally I would expect fire wall state to mostly fit 
> into caches (1MB+) of modern Opterons and other NUMA systems.
> 
> 
>>What do you think of this patch ?
>>
>>Note : The allocation function is quite complex (copied from
>>drivers/pci/pci-driver.c pci_call_probe())
>>because current kernel doesnt have a NUMA aware vmalloc() wrapper
> 
> 
> Normal vmalloc is NUMA aware and allocates on the local node.
> Like all memory allocations

Yes, but I want to allocate percpu (and per node) data, using vmalloc(), not 
memory on the local node.

> 
> 
>>, maybe 
>>a future kernel will export
>>such a common function ?
> 
> 
> BTW if you want to put data onto a specific node you don't need
> to schedule there - it is enough to set the process mempolicy
> with sys_set_mempolicy. This is fine from kernel too, as long
> as you save/restore the old one (current->mempolicy) 

OK thanks, I will try this.

Eric