[Patch net] act_mirred: fix a race condition on mirred_list

[Patch net] act_mirred: fix a race condition on mirred_list

[Cong Wang]

After commit 1ce87720d456 ("net: sched: make cls_u32 lockless")
we began to release tc actions in a RCU callback. However,
mirred action relies on RTNL lock to protect the global
mirred_list, therefore we could have a race condition
between RCU callback and netdevice event, which caused
a list corruption as reported by Vinson.

Instead of relying on RTNL lock, introduce a spinlock to
protect this list.

Note, in non-bind case, it is still called with RTNL lock,
therefore should disable BH too.

Reported-by: Vinson Lee <vlee@twopensource.com>
Cc: John Fastabend <john.fastabend@gmail.com>
Cc: Jamal Hadi Salim <jhs@mojatatu.com>
Signed-off-by: Cong Wang <cwang@twopensource.com>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
---
 net/sched/act_mirred.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/net/sched/act_mirred.c b/net/sched/act_mirred.c
index 2d1be4a..3e7c51a 100644
--- a/net/sched/act_mirred.c
+++ b/net/sched/act_mirred.c
@@ -31,13 +31,17 @@
 
 #define MIRRED_TAB_MASK     7
 static LIST_HEAD(mirred_list);
+static DEFINE_SPINLOCK(mirred_list_lock);
 
 static void tcf_mirred_release(struct tc_action *a, int bind)
 {
 	struct tcf_mirred *m = to_mirred(a);
 	struct net_device *dev = rcu_dereference_protected(m->tcfm_dev, 1);
 
+	/* We could be called either in a RCU callback or with RTNL lock held. */
+	spin_lock_bh(&mirred_list_lock);
 	list_del(&m->tcfm_list);
+	spin_unlock_bh(&mirred_list_lock);
 	if (dev)
 		dev_put(dev);
 }
@@ -123,7 +127,9 @@ static int tcf_mirred_init(struct net *net, struct nlattr *nla,
 	}
 
 	if (ret == ACT_P_CREATED) {
+		spin_lock_bh(&mirred_list_lock);
 		list_add(&m->tcfm_list, &mirred_list);
+		spin_unlock_bh(&mirred_list_lock);
 		tcf_hash_insert(a);
 	}
 
@@ -221,7 +227,8 @@ static int mirred_device_event(struct notifier_block *unused,
 	struct tcf_mirred *m;
 
 	ASSERT_RTNL();
-	if (event == NETDEV_UNREGISTER)
+	if (event == NETDEV_UNREGISTER) {
+		spin_lock_bh(&mirred_list_lock);
 		list_for_each_entry(m, &mirred_list, tcfm_list) {
 			if (rcu_access_pointer(m->tcfm_dev) == dev) {
 				dev_put(dev);
@@ -231,6 +238,8 @@ static int mirred_device_event(struct notifier_block *unused,
 				RCU_INIT_POINTER(m->tcfm_dev, NULL);
 			}
 		}
+		spin_unlock_bh(&mirred_list_lock);
+	}
 
 	return NOTIFY_DONE;
 }
-- 
1.8.3.1

[Patch net] act_mirred: always release tcf hash

[Cong Wang]

Align with other tc actions.

Cc: Jamal Hadi Salim <jhs@mojatatu.com>
Signed-off-by: Cong Wang <cwang@twopensource.com>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
---
 net/sched/act_mirred.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/net/sched/act_mirred.c b/net/sched/act_mirred.c
index 3e7c51a..2efaf4e 100644
--- a/net/sched/act_mirred.c
+++ b/net/sched/act_mirred.c
@@ -107,10 +107,10 @@ static int tcf_mirred_init(struct net *net, struct nlattr *nla,
 	} else {
 		if (bind)
 			return 0;
-		if (!ovr) {
-			tcf_hash_release(a, bind);
+
+		tcf_hash_release(a, bind);
+		if (!ovr)
 			return -EEXIST;
-		}
 	}
 	m = to_mirred(a);
 
-- 
1.8.3.1

Re: [Patch net] act_mirred: always release tcf hash

[Jamal Hadi Salim]

On 10/01/15 14:37, Cong Wang wrote:
> Align with other tc actions.
>
> Cc: Jamal Hadi Salim <jhs@mojatatu.com>
> Signed-off-by: Cong Wang <cwang@twopensource.com>
> Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>

Acked-by: Jamal Hadi Salim <jhs@mojatatu.com>

cheers,
jamal

Re: [Patch net] act_mirred: always release tcf hash

[David Miller]

From: Cong Wang <xiyou.wangcong@gmail.com>
Date: Thu,  1 Oct 2015 11:37:43 -0700

> Align with other tc actions.
> 
> Cc: Jamal Hadi Salim <jhs@mojatatu.com>
> Signed-off-by: Cong Wang <cwang@twopensource.com>
> Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>

Applied.

Re: [Patch net] act_mirred: fix a race condition on mirred_list

[Jamal Hadi Salim]

Hi Cong,

I am wondering if making the bindcount or refcount atomic would help?
How does this bug get created? i.e the RTNL is still around.
Why is this specific to mirred only?

cheers,
jamal


On 10/01/15 14:37, Cong Wang wrote:
> After commit 1ce87720d456 ("net: sched: make cls_u32 lockless")
> we began to release tc actions in a RCU callback. However,
> mirred action relies on RTNL lock to protect the global
> mirred_list, therefore we could have a race condition
> between RCU callback and netdevice event, which caused
> a list corruption as reported by Vinson.
>
> Instead of relying on RTNL lock, introduce a spinlock to
> protect this list.
>
> Note, in non-bind case, it is still called with RTNL lock,
> therefore should disable BH too.
>

Re: [Patch net] act_mirred: fix a race condition on mirred_list

[Jamal Hadi Salim]

Never mind. Mirred is speacial because it points to other netdevs.
So:

Acked-by: Jamal Hadi Salim <jhs@mojatatu.com>

Thanks Cong.

cheers,
jamal

On 10/05/15 07:58, Jamal Hadi Salim wrote:
> Hi Cong,
>
> I am wondering if making the bindcount or refcount atomic would help?
> How does this bug get created? i.e the RTNL is still around.
> Why is this specific to mirred only?
>
> cheers,
> jamal
>
>
> On 10/01/15 14:37, Cong Wang wrote:
>> After commit 1ce87720d456 ("net: sched: make cls_u32 lockless")
>> we began to release tc actions in a RCU callback. However,
>> mirred action relies on RTNL lock to protect the global
>> mirred_list, therefore we could have a race condition
>> between RCU callback and netdevice event, which caused
>> a list corruption as reported by Vinson.
>>
>> Instead of relying on RTNL lock, introduce a spinlock to
>> protect this list.
>>
>> Note, in non-bind case, it is still called with RTNL lock,
>> therefore should disable BH too.
>>

Re: [Patch net] act_mirred: fix a race condition on mirred_list

[David Miller]

From: Cong Wang <xiyou.wangcong@gmail.com>
Date: Thu,  1 Oct 2015 11:37:42 -0700

> After commit 1ce87720d456 ("net: sched: make cls_u32 lockless")
> we began to release tc actions in a RCU callback. However,
> mirred action relies on RTNL lock to protect the global
> mirred_list, therefore we could have a race condition
> between RCU callback and netdevice event, which caused
> a list corruption as reported by Vinson.
> 
> Instead of relying on RTNL lock, introduce a spinlock to
> protect this list.
> 
> Note, in non-bind case, it is still called with RTNL lock,
> therefore should disable BH too.
> 
> Reported-by: Vinson Lee <vlee@twopensource.com>
> Cc: John Fastabend <john.fastabend@gmail.com>
> Cc: Jamal Hadi Salim <jhs@mojatatu.com>
> Signed-off-by: Cong Wang <cwang@twopensource.com>
> Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>

Applied.