From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Robert Olsson <robert@herjulf.net>
Cc: Andi Kleen <andi@firstfloor.org>,
robert.olsson@its.uu.se, netdev@vger.kernel.org,
Jens.Laas@its.uu.se
Subject: Re: RCU problems in fib_table_insert
Date: Tue, 6 Apr 2010 23:45:03 -0700 [thread overview]
Message-ID: <20100407064503.GD2471@linux.vnet.ibm.com> (raw)
In-Reply-To: <19388.8645.99295.495371@gargle.gargle.HOWL>
On Wed, Apr 07, 2010 at 08:10:13AM +0200, Robert Olsson wrote:
>
> Paul E. McKenney writes:
> > On Mon, Mar 22, 2010 at 07:18:34AM +0100, Robert Olsson wrote:
> > >
> > > Seems like Paul and Eric fixed this problem... We use fib_trie with
> > > major infrastructure but always disable preempt. It was unsafe w.
> > > preempt at least before Jareks P. patches about a year ago. I havn't
> > > tested w. preempt after that but maybe someone else have...
>
>
> > Though I must admit that I would be surprised if there wasn't
> > more adjustment required in net/ipv4/fib_trie.c -- lots of
> > rcu_dereference()s in there.
>
> Hi, a follow on this thread.
>
> Maybe I was to pessimistic... we've setup for stress test running during
> easter vacation
>
> Testing the fib_trie with preempt enabled. Continuesly loading/flushing
> "full BGP" via a test script, while routing without the route cache
> to further stress locking. Average load about 300 kpps. Some more test
> details below.
>
> The test was manually stopped after 6 days. So it seems like preempt/rcu
> has been improved with fib_trie.
Always happy to be pleasantly surprised. ;-)
Thanx, Paul
> Thanks
> --ro
>
> HW
> --
> CPU Opteron 2 * 6174, TYAN S8230. Intel 82599
>
> Kernel from net-next-2.6
> ------------------------
> Version 2.6.34-rc1bifrost-x86_64 (gcc version 4.3.2 (GCC) ) #4 SMP PREEMPT
> CONFIG_PREEMPT=y
> CONFIG_DEBUG_PREEMPT=y
> CONFIG_IP_FIB_TRIE=y
> CONFIG_FIB_RULES=y
> CONFIG_TREE_RCU=y
> CONFIG_RCU_FANOUT=64
>
> IP table
> --------
> "BGP" 279 k routes
>
> Test duration
> -------------
> 09:34:14 up 6 days, 20:57, 4 users, load average: 5.28, 5.87, 5.53
>
> Test script
> -----------
> #! /bin/bash
> while(true) do
> ip -batch /etc/inet_route_add.bat;
> ip route list | wc -l; ( 279 k routes )
> ifconfig eth1 down
> ifconfig eth1 10.10.11.1 netmask 255.255.255.0
> ip route list | wc -l; ( 3 routes )
> done;
>
> /proc/net/softnet_stat
>
> 0000d503 00000000 00000014 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 4cc1ecfb 00000000 02bb1bcb 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 4cb95da1 00000000 02cc6574 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 4cb17932 00000000 02e1abdf 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 4cc40ad9 00000000 02f53db0 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 4cbfe34f 00000000 0306a88d 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 4ccb10e9 00000000 03d3a11d 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 4ca84c29 00000000 03be0ca1 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 4cb710f8 00000000 04070289 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 0b505d55 00000000 038a9bc0 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 0b54a7be 00000000 03a5de6d 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 0b5be81f 00000000 03be018c 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 0b51fdd0 00000000 0559fdd6 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 0b4fa6fd 00000000 056311dc 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 0b4a61d3 00000000 056a9276 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 0b454970 00000000 0572add1 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
>
> FYI. CPU0 is resvered for BGP/ssh/stats etc
prev parent reply other threads:[~2010-04-07 6:45 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-03-21 20:25 RCU problems in fib_table_insert Andi Kleen
2010-03-21 21:25 ` Eric Dumazet
2010-03-21 21:38 ` Paul E. McKenney
2010-03-21 21:49 ` Eric Dumazet
2010-03-21 22:57 ` Paul E. McKenney
2010-03-21 21:37 ` Paul E. McKenney
2010-03-22 1:01 ` David Miller
2010-03-22 6:51 ` Paul E. McKenney
2010-03-22 6:18 ` Robert Olsson
2010-03-22 6:51 ` Paul E. McKenney
2010-04-07 6:10 ` Robert Olsson
2010-04-07 6:45 ` Paul E. McKenney [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20100407064503.GD2471@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=Jens.Laas@its.uu.se \
--cc=andi@firstfloor.org \
--cc=netdev@vger.kernel.org \
--cc=robert.olsson@its.uu.se \
--cc=robert@herjulf.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).