[PATCH] net: gen_estimator: Fix gen_kill_estimator() lookups

[PATCH] net: gen_estimator: Fix gen_kill_estimator() lookups

[Jarek Poplawski]

gen_kill_estimator() linear lists lookups are very slow, and e.g. while
deleting a large number of HTB classes soft lockups were reported. Here
is another try to fix this problem: this time internally, with rbtree,
so similarly to Jamal's hashing idea IIRC. (Looking for next hits could
be still optimized, but it's really fast as it is.)

Reported-by: Badalian Vyacheslav <slavon@bigtelecom.ru>
Reported-by: Denys Fedoryshchenko <denys@visp.net.lb>
Signed-off-by: Jarek Poplawski <jarkao2@gmail.com>
---

 net/core/gen_estimator.c |   76 ++++++++++++++++++++++++++++++++++------------
 1 files changed, 56 insertions(+), 20 deletions(-)

diff --git a/net/core/gen_estimator.c b/net/core/gen_estimator.c
index 57abe82..80aa160 100644
--- a/net/core/gen_estimator.c
+++ b/net/core/gen_estimator.c
@@ -31,6 +31,7 @@
 #include <linux/skbuff.h>
 #include <linux/rtnetlink.h>
 #include <linux/init.h>
+#include <linux/rbtree.h>
 #include <net/sock.h>
 #include <net/gen_stats.h>
 
@@ -89,6 +90,7 @@ struct gen_estimator
 	u32			avpps;
 	u32			avbps;
 	struct rcu_head		e_rcu;
+	struct rb_node		node;
 };
 
 struct gen_estimator_head
@@ -102,6 +104,9 @@ static struct gen_estimator_head elist[EST_MAX_INTERVAL+1];
 /* Protects against NULL dereference */
 static DEFINE_RWLOCK(est_lock);
 
+/* Protects against soft lockup during large deletion */
+static struct rb_root est_root = RB_ROOT;
+
 static void est_timer(unsigned long arg)
 {
 	int idx = (int)arg;
@@ -139,6 +144,45 @@ skip:
 	rcu_read_unlock();
 }
 
+static void gen_add_node(struct gen_estimator *est)
+{
+	struct rb_node **p = &est_root.rb_node, *parent = NULL;
+
+	while (*p) {
+		struct gen_estimator *e;
+
+		parent = *p;
+		e = rb_entry(parent, struct gen_estimator, node);
+
+		if (est->bstats > e->bstats)
+			p = &parent->rb_right;
+		else
+			p = &parent->rb_left;
+	}
+	rb_link_node(&est->node, parent, p);
+	rb_insert_color(&est->node, &est_root);
+}
+
+static struct gen_estimator *gen_find_node(struct gnet_stats_basic *bstats,
+					   struct gnet_stats_rate_est *rate_est)
+{
+	struct rb_node *p = est_root.rb_node;
+
+	while (p) {
+		struct gen_estimator *e;
+
+		e = rb_entry(p, struct gen_estimator, node);
+
+		if (bstats > e->bstats)
+			p = p->rb_right;
+		else if (bstats < e->bstats || rate_est != e->rate_est)
+			p = p->rb_left;
+		else
+			return e;
+	}
+	return NULL;
+}
+
 /**
  * gen_new_estimator - create a new rate estimator
  * @bstats: basic statistics
@@ -194,6 +238,8 @@ int gen_new_estimator(struct gnet_stats_basic *bstats,
 		mod_timer(&elist[idx].timer, jiffies + ((HZ/4) << idx));
 
 	list_add_rcu(&est->list, &elist[idx].list);
+	gen_add_node(est);
+
 	return 0;
 }
 
@@ -209,34 +255,24 @@ static void __gen_kill_estimator(struct rcu_head *head)
  * @bstats: basic statistics
  * @rate_est: rate estimator statistics
  *
- * Removes the rate estimator specified by &bstats and &rate_est
- * and deletes the timer.
+ * Removes the rate estimator specified by &bstats and &rate_est.
  *
  * NOTE: Called under rtnl_mutex
  */
 void gen_kill_estimator(struct gnet_stats_basic *bstats,
-	struct gnet_stats_rate_est *rate_est)
+			struct gnet_stats_rate_est *rate_est)
 {
-	int idx;
-	struct gen_estimator *e, *n;
-
-	for (idx=0; idx <= EST_MAX_INTERVAL; idx++) {
-
-		/* Skip non initialized indexes */
-		if (!elist[idx].timer.function)
-			continue;
+	struct gen_estimator *e;
 
-		list_for_each_entry_safe(e, n, &elist[idx].list, list) {
-			if (e->rate_est != rate_est || e->bstats != bstats)
-				continue;
+	while ((e = gen_find_node(bstats, rate_est))) {
+		rb_erase(&e->node, &est_root);
 
-			write_lock_bh(&est_lock);
-			e->bstats = NULL;
-			write_unlock_bh(&est_lock);
+		write_lock_bh(&est_lock);
+		e->bstats = NULL;
+		write_unlock_bh(&est_lock);
 
-			list_del_rcu(&e->list);
-			call_rcu(&e->e_rcu, __gen_kill_estimator);
-		}
+		list_del_rcu(&e->list);
+		call_rcu(&e->e_rcu, __gen_kill_estimator);
 	}
 }
 

Re: [PATCH] net: gen_estimator: Fix gen_kill_estimator() lookups

[jamal]

On Mon, 2008-11-24 at 12:04 +0000, Jarek Poplawski wrote:
> gen_kill_estimator() linear lists lookups are very slow, and e.g. while
> deleting a large number of HTB classes soft lockups were reported. Here
> is another try to fix this problem: this time internally, with rbtree,
> so similarly to Jamal's hashing idea IIRC. (Looking for next hits could
> be still optimized, but it's really fast as it is.)

Certainly a big improvement. Compared to hashing i suggested:
the deletion speed is probably equal or better than using a hash.
I think a hash would have performed better in the case of addition
than the rb tree; but you primarily concerned about deletion, so this
good.

Acked-by: Jamal Hadi Salim <hadi@cyberus.ca>

cheers,
jamal

Re: [PATCH] net: gen_estimator: Fix gen_kill_estimator() lookups

[Jarek Poplawski]

On Mon, Nov 24, 2008 at 08:18:45AM -0500, jamal wrote:
> On Mon, 2008-11-24 at 12:04 +0000, Jarek Poplawski wrote:
> > gen_kill_estimator() linear lists lookups are very slow, and e.g. while
> > deleting a large number of HTB classes soft lockups were reported. Here
> > is another try to fix this problem: this time internally, with rbtree,
> > so similarly to Jamal's hashing idea IIRC. (Looking for next hits could
> > be still optimized, but it's really fast as it is.)
> 
> Certainly a big improvement. Compared to hashing i suggested:
> the deletion speed is probably equal or better than using a hash.
> I think a hash would have performed better in the case of addition
> than the rb tree; but you primarily concerned about deletion, so this
> good.

I first thought about a hash, but alas Patrick's solution is sched
only... Anyway, I din't see too much overhead in memory use, and no
diffrence in addition times (without batching).

> 
> Acked-by: Jamal Hadi Salim <hadi@cyberus.ca>
> 

Thanks,
Jarek P.

Re: [PATCH] net: gen_estimator: Fix gen_kill_estimator() lookups

[jamal]

On Mon, 2008-11-24 at 13:37 +0000, Jarek Poplawski wrote:

> I first thought about a hash, but alas Patrick's solution is sched
> only... Anyway, I din't see too much overhead in memory use, and no
> diffrence in addition times (without batching).

Showing numbers in a commit for perf improvement IMO is always a good
thing.
BTW, I dont think it would make a noticeable difference (batching
notwithstanding) in addition or even deletion unless you have quiet a
few with the same estimate sampling time loaded.

cheers,
jamal

Re: [PATCH] net: gen_estimator: Fix gen_kill_estimator() lookups

[David Miller]

From: Jarek Poplawski <jarkao2@gmail.com>
Date: Mon, 24 Nov 2008 13:37:24 +0000

> On Mon, Nov 24, 2008 at 08:18:45AM -0500, jamal wrote:
> > On Mon, 2008-11-24 at 12:04 +0000, Jarek Poplawski wrote:
> > > gen_kill_estimator() linear lists lookups are very slow, and e.g. while
> > > deleting a large number of HTB classes soft lockups were reported. Here
> > > is another try to fix this problem: this time internally, with rbtree,
> > > so similarly to Jamal's hashing idea IIRC. (Looking for next hits could
> > > be still optimized, but it's really fast as it is.)
> > 
> > Certainly a big improvement. Compared to hashing i suggested:
> > the deletion speed is probably equal or better than using a hash.
> > I think a hash would have performed better in the case of addition
> > than the rb tree; but you primarily concerned about deletion, so this
> > good.
> 
> I first thought about a hash, but alas Patrick's solution is sched
> only... Anyway, I din't see too much overhead in memory use, and no
> diffrence in addition times (without batching).

The kinds of things that can matter in using tree vs. hash really
only occur in the data path, and all of this stuff is control plane.

> > Acked-by: Jamal Hadi Salim <hadi@cyberus.ca>

Applied to net-next-2.6, thanks everyone.

Re: [PATCH] net: gen_estimator: Fix gen_kill_estimator() lookups

[Jarek Poplawski]

On Mon, Nov 24, 2008 at 10:00:58AM -0500, jamal wrote:
> On Mon, 2008-11-24 at 13:37 +0000, Jarek Poplawski wrote:
> 
> > I first thought about a hash, but alas Patrick's solution is sched
> > only... Anyway, I din't see too much overhead in memory use, and no
> > diffrence in addition times (without batching).
> 
> Showing numbers in a commit for perf improvement IMO is always a good
> thing.

Sure, but alas I'm not a perf guy...

> BTW, I dont think it would make a noticeable difference (batching
> notwithstanding) in addition or even deletion unless you have quiet a
> few with the same estimate sampling time loaded.

My very unprofessional tests gave approximately 319s vs. 0.34s with:
"time tc qdisc del dev lo root" for 65535 htb classes, and as you
predicted (and I was surprised) no noticeable difference in addition
times with or without batching.
 
Cheers,
Jarek P.