Re: [PATCH] netfilter: use per-cpu spinlock rather than RCU (v3)

[Stephen Hemminger <shemminger@vyatta.com>]

On Wed, 15 Apr 2009 16:48:11 -0700 (PDT)
David Miller <davem@davemloft.net> wrote:

> From: Eric Dumazet <dada1@cosmosbay.com>
> Date: Wed, 15 Apr 2009 23:07:29 +0200
> 
> > Well, it seems original patch was not so bad after all
> > 
> > http://lists.netfilter.org/pipermail/netfilter-devel/2006-January/023175.html
> > 
> > So change per-cpu spinlocks to per-cpu rwlocks 
> > 
> > and use read_lock() in ipt_do_table() to allow recursion...
> 
> Grumble, one more barrier to getting rid of rwlocks in the whole
> tree. :-/

Hey, we are reinventing your brwlock ;-<

The other option is use spinlock, over a smaller area (only counters),
and other mechanism to synchronize on replace.

> I really think we should entertain the idea where we don't RCU quiesce
> when adding rules.  That was dismissed as not workable because the new
> rule must be "visible" as soon as we return to userspace but let's get
> real, effectively it will be.

The counters are the bigger problem, otherwise we could just free table
info via rcu.  Do we really have to support: replace where the counter
values coming out to user space are always exactly accurate, or is it
allowed to replace a rule and maybe lose some counter ticks (worst case
NCPU-1).

> If there are any stale object reference issues, we can use RCU object
> destruction to handle that kind of thing.

The problem is pulling the counter values out of the object in the
replace case. It could be changed to use some form of counting semaphore
like thing but that gets expensive.
 
> I almost cringed when the per-spinlock idea was proposed, but per-cpu
> rwlocks just takes things too far for my tastes.