Re: [RFC] iptables: lock free counters (v0.6)

[Rick Jones <rick.jones2@hp.com>]

Stephen Hemminger wrote:
> The reader/writer lock in ip_tables is acquired in the critical path of
> processing packets and is one of the reasons just loading iptables can cause
> a 20% performance loss. The rwlock serves two functions:
> 
> 1) it prevents changes to table state (xt_replace) while table is in use.
>    This is now handled by doing rcu on the xt_table. When table is
>    replaced, the new table(s) are put in and the old one table(s) are freed
>    after RCU period.
> 
> 2) it provides synchronization when accesing the counter values.
>    This is now handled by swapping in new table_info entries for each cpu
>    then summing the old values, and putting the result back onto one
>    cpu.  On a busy system it may cause sampling to occur at different
>    times on each cpu, but no packet/byte counts are lost in the process.

I've taken this round for a spin on the 32-core setup.  I'd not previously 
applied Patrick's patches to remove the initialization, so my kludges to compile 
may have altered things, but assuming it was OK (convert the inits to 
__MUTEX_INITIALIZER to make the compiler happy) it appears that this change does 
very good things indeed for the "empty" case.  Where the 
2.6.29-rc2/unpatchednet-next showed a 50% drop (handwaving math) in the "empty" 
case compared to the "none" case (aka none is no iptables modules loaded, empty 
being what one gets after iptables --list) this patch shows what appears to be a 
much much smaller drop of less than 6%.

The original data can be seen at:

ftp://ftp.netperf.org/iptable_scaling/  in no_iptables and empty_iptables

and the data after this patch can be seen at:

ftp://ftp.netperf.org/hemminger/hemminger6/  in none and empty

while I have none of Eric's patches in this tree, just for grins I went ahead and 
ran "full" as well.

happy benchmarking,

rick jones