[PATCH net-next v2] rhashtable: unnecessary to use delayed work

[PATCH net-next v2] rhashtable: unnecessary to use delayed work

[Ying Xue]

When we put our declared work task in the global workqueue with
schedule_delayed_work(), its delay parameter is always zero.
Therefore, we should define a normal work in rhashtable structure
instead of a delayed work.

By the way, we add a condition to check whether resizing functions
are NULL before cancel the work, avoiding to cancel an uninitialized
work.

Lastly, while we wait for all work items we submitted before to run
to completion with cancel_delayed_work(), ht->mutex has been taken in
rhashtable_destroy(). Moreover, cancel_delayed_work() doesn't return
until all work items are accomplished, and when work items are
scheduled, the work's function - rht_deferred_worker() will be called.
However, as rht_deferred_worker() also needs to acquire the lock,
deadlock might happen at the moment as the lock is already held before.
So if the cancel work function is moved out of the lock covered scope,
this can help to avoid the deadlock.

Fixes: 97defe1 ("rhashtable: Per bucket locks & deferred expansion/shrinking")
Signed-off-by: Ying Xue <ying.xue@windriver.com>
Cc: Thomas Graf <tgraf@suug.ch>
---
v2 changes:
  Move cancel_work_sync() out of ht->mutex lock scope

 include/linux/rhashtable.h |    2 +-
 lib/rhashtable.c           |   11 +++++------
 2 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/include/linux/rhashtable.h b/include/linux/rhashtable.h
index 9570832..a2562ed 100644
--- a/include/linux/rhashtable.h
+++ b/include/linux/rhashtable.h
@@ -119,7 +119,7 @@ struct rhashtable {
 	atomic_t			nelems;
 	atomic_t			shift;
 	struct rhashtable_params	p;
-	struct delayed_work             run_work;
+	struct work_struct		run_work;
 	struct mutex                    mutex;
 	bool                            being_destroyed;
 };
diff --git a/lib/rhashtable.c b/lib/rhashtable.c
index ed6ae1a..1f56189 100644
--- a/lib/rhashtable.c
+++ b/lib/rhashtable.c
@@ -476,7 +476,7 @@ static void rht_deferred_worker(struct work_struct *work)
 	struct rhashtable *ht;
 	struct bucket_table *tbl;
 
-	ht = container_of(work, struct rhashtable, run_work.work);
+	ht = container_of(work, struct rhashtable, run_work);
 	mutex_lock(&ht->mutex);
 	tbl = rht_dereference(ht->tbl, ht);
 
@@ -498,7 +498,7 @@ static void rhashtable_wakeup_worker(struct rhashtable *ht)
 	if (tbl == new_tbl &&
 	    ((ht->p.grow_decision && ht->p.grow_decision(ht, size)) ||
 	     (ht->p.shrink_decision && ht->p.shrink_decision(ht, size))))
-		schedule_delayed_work(&ht->run_work, 0);
+		schedule_work(&ht->run_work);
 }
 
 static void __rhashtable_insert(struct rhashtable *ht, struct rhash_head *obj,
@@ -894,7 +894,7 @@ int rhashtable_init(struct rhashtable *ht, struct rhashtable_params *params)
 		get_random_bytes(&ht->p.hash_rnd, sizeof(ht->p.hash_rnd));
 
 	if (ht->p.grow_decision || ht->p.shrink_decision)
-		INIT_DEFERRABLE_WORK(&ht->run_work, rht_deferred_worker);
+		INIT_WORK(&ht->run_work, rht_deferred_worker);
 
 	return 0;
 }
@@ -911,12 +911,11 @@ EXPORT_SYMBOL_GPL(rhashtable_init);
 void rhashtable_destroy(struct rhashtable *ht)
 {
 	ht->being_destroyed = true;
+	if (ht->p.grow_decision || ht->p.shrink_decision)
+		cancel_work_sync(&ht->run_work);
 
 	mutex_lock(&ht->mutex);
-
-	cancel_delayed_work(&ht->run_work);
 	bucket_table_free(rht_dereference(ht->tbl, ht));
-
 	mutex_unlock(&ht->mutex);
 }
 EXPORT_SYMBOL_GPL(rhashtable_destroy);
-- 
1.7.9.5

Re: [PATCH net-next v2] rhashtable: unnecessary to use delayed work

[Thomas Graf]

On 01/14/15 at 11:32am, Ying Xue wrote:
> When we put our declared work task in the global workqueue with
> schedule_delayed_work(), its delay parameter is always zero.
> Therefore, we should define a normal work in rhashtable structure
> instead of a delayed work.
> 
> By the way, we add a condition to check whether resizing functions
> are NULL before cancel the work, avoiding to cancel an uninitialized
> work.
> 
> Lastly, while we wait for all work items we submitted before to run
> to completion with cancel_delayed_work(), ht->mutex has been taken in
> rhashtable_destroy(). Moreover, cancel_delayed_work() doesn't return
> until all work items are accomplished, and when work items are
> scheduled, the work's function - rht_deferred_worker() will be called.
> However, as rht_deferred_worker() also needs to acquire the lock,
> deadlock might happen at the moment as the lock is already held before.
> So if the cancel work function is moved out of the lock covered scope,
> this can help to avoid the deadlock.
  ^^^^^^^^^^^^^^^
  will avoid :-)

> 
> Fixes: 97defe1 ("rhashtable: Per bucket locks & deferred expansion/shrinking")
> Signed-off-by: Ying Xue <ying.xue@windriver.com>
> Cc: Thomas Graf <tgraf@suug.ch>

I don't want to be too picky about the commit title but since this is
fixing a previously reported race condition I would like for that to
be reflected in the title. Something like:

rhashtable: Fix race in rhashtable_destroy() and use regular work_struct

With that fixed:
Acked-by: Thomas Graf <tgraf@suug.ch>

Re: [PATCH net-next v2] rhashtable: unnecessary to use delayed work

[Ying Xue]

On 01/14/2015 05:24 PM, Thomas Graf wrote:
> On 01/14/15 at 11:32am, Ying Xue wrote:
>> When we put our declared work task in the global workqueue with
>> schedule_delayed_work(), its delay parameter is always zero.
>> Therefore, we should define a normal work in rhashtable structure
>> instead of a delayed work.
>>
>> By the way, we add a condition to check whether resizing functions
>> are NULL before cancel the work, avoiding to cancel an uninitialized
>> work.
>>
>> Lastly, while we wait for all work items we submitted before to run
>> to completion with cancel_delayed_work(), ht->mutex has been taken in
>> rhashtable_destroy(). Moreover, cancel_delayed_work() doesn't return
>> until all work items are accomplished, and when work items are
>> scheduled, the work's function - rht_deferred_worker() will be called.
>> However, as rht_deferred_worker() also needs to acquire the lock,
>> deadlock might happen at the moment as the lock is already held before.
>> So if the cancel work function is moved out of the lock covered scope,
>> this can help to avoid the deadlock.
>   ^^^^^^^^^^^^^^^
>   will avoid :-)
> 
>>
>> Fixes: 97defe1 ("rhashtable: Per bucket locks & deferred expansion/shrinking")
>> Signed-off-by: Ying Xue <ying.xue@windriver.com>
>> Cc: Thomas Graf <tgraf@suug.ch>
> 
> I don't want to be too picky about the commit title but since this is
> fixing a previously reported race condition I would like for that to
> be reflected in the title. Something like:
> 
> rhashtable: Fix race in rhashtable_destroy() and use regular work_struct
> 
> With that fixed:
> Acked-by: Thomas Graf <tgraf@suug.ch>
> 
> 

Thanks for the review, and I will deliver the v3 soon.

Regards,
Ying