Re: [PATCH] net: allow netdev_wait_allrefs() to run faster

[Ben Greear <greearb@candelatech.com>]

On 08/09/2010 10:34 AM, Benjamin LaHaise wrote:
> Hello Ben,
>
> On Mon, Aug 09, 2010 at 10:23:37AM -0700, Ben Greear wrote:
>> I was just comparing my out-of-tree patch set to .35, and it appears
>> little or none of the patches discussed in this thread are in the
>> upstream kernel yet.
>
> I was waiting on Eric's sysfs changes for namespaces to settle down, but
> ended up getting busy on other things.  I guess now is a good time to pick
> this back up and try to merge my changes for improving interface scaling.
> I'll send out a new version of the patches sometime in the next couple of
> days.  I'm also about to make a new Babylon release as well, I just need
> to write some more documentation. :-/
>
> Btw, one thing I noticed but haven't been able to come up with a fix for
> yet is that iptables has scaling issues with lots of interfaces.
> Specifically, we had to start adding one iptables rule per interface for smtp
> filtering (not all subscribers are permitted to send smtp directly out to
> the net, so it has to be per-interface).  It seems that those all get
> dumped into a giant list.  What I'd like to do is to be able to attach rules
> directly to the interface, but I haven't really had the time to do a mergable
> set of changes for that.  Thoughts anyone?

We also have a few rules per interface, and notice that it takes around 10ms
per rule when we are removing them, even when using batching in 'ip':

This is on a high-end core i7, otherwise lightly loaded.

Total IPv4 rule listings: 2097
Cleaning 2094 rules with ip -batch...
time -p ip -4 -force -batch /tmp/crr_batch_cmds_4.txt
real 17.81
user 0.05
sys 0.00


Patrick thought had an idea, but I don't think he had time to
look at it further:

"Its probably the synchronize_rcu() in fib_nl_delrule() and
the route flushing happening after rule removal."


Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com