[RFC PATCH] ipv4: release dev refcnt early when destroying inetdev

[Nicolas Dichtel <nicolas.dichtel@6wind.com>]

[-- Attachment #1: Type: text/plain, Size: 346 bytes --]

Hi all,

We got a scalability problem when we try to remove a lot of virtual interfaces. 
After analysis, we found that a refcnt on a device was released too late.
Here is a proposal patch. If we are not missing something, the refcnt can be 
release before call_rcu(). In IPv6, this is already the case.

Comments are welcome.


Regards,
Nicolas

[-- Attachment #2: 0001-ipv4-release-dev-refcnt-early-when-destroying-inetd.patch --]
[-- Type: text/x-diff, Size: 1830 bytes --]

>From 6fe291ff56b1f94599dfaa57dfb0ed4c168b603f Mon Sep 17 00:00:00 2001
From: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Date: Fri, 10 Sep 2010 14:52:15 +0200
Subject: [PATCH] ipv4: release dev refcnt early when destroying inetdev

When a virtual device is removed, refcnt on dev is released
after rcu barrier, hence we fall always in the msleep(250)
of netdev_wait_allrefs(). This causes a long delay when
a lot of interfaces are removed.
Refcnt can be released before this rcu barrier, this allows
to accelerate the removing of virtual interfaces.

Test of removing 50 ipip tunnel interfaces:
 Before the patch:
  real    0m12.804s
  user    0m0.020s
  sys     0m0.000s

 After the patch:
  real    0m0.988s
  user    0m0.004s
  sys     0m0.016s

Signed-off-by: Wang Xuefu <xuefu.wang@6wind.com>
Signed-off-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
---
 net/ipv4/devinet.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/net/ipv4/devinet.c b/net/ipv4/devinet.c
index da14c49..dd59e79 100644
--- a/net/ipv4/devinet.c
+++ b/net/ipv4/devinet.c
@@ -131,7 +131,9 @@ static inline void inet_free_ifa(struct in_ifaddr *ifa)
 
 void in_dev_finish_destroy(struct in_device *idev)
 {
+#ifdef NET_REFCNT_DEBUG
 	struct net_device *dev = idev->dev;
+#endif
 
 	WARN_ON(idev->ifa_list);
 	WARN_ON(idev->mc_list);
@@ -139,7 +141,6 @@ void in_dev_finish_destroy(struct in_device *idev)
 	printk(KERN_DEBUG "in_dev_finish_destroy: %p=%s\n",
 	       idev, dev ? dev->name : "NIL");
 #endif
-	dev_put(dev);
 	if (!idev->dead)
 		pr_err("Freeing alive in_device %p\n", idev);
 	else
@@ -215,6 +216,7 @@ static void inetdev_destroy(struct in_device *in_dev)
 	neigh_parms_release(&arp_tbl, in_dev->arp_parms);
 	arp_ifdown(dev);
 
+	dev_put(dev);
 	call_rcu(&in_dev->rcu_head, in_dev_rcu_put);
 }
 
-- 
1.5.4.5