Re: [RFC PATCH] rcu: introduce kfree_rcu()

[Lai Jiangshan <laijs@cn.fujitsu.com>]

How to usage kfree_rcu:

struct my_struct {
	int data;
	struct rcu_head rcu;
};

----------------original code:--------------------------
void my_struct_release_rcu(struct rcu_head *rcu)
{
	struct my_struct *p;

	item = container_of(rcu, struct my_struct, rcu);
	kfree(p);
}

void some_fuction()
{
	struct my_struct *p;
	.....;
	call_rcu(&p->rcu, my_struct_release_rcu);
	.....;
}
---end---

-----------------after use kfree_rcu:--------------------

/* my_struct_release_rcu() was removed */

void some_fuction()
{
	struct my_struct *p;
	.....;
	kfree_rcu(p, &p->rcu);
	.....;
}
---end---

1) unloadable modules:
   A) use my_struct_release_rcu():
      when we unload this modules, we need call rcu_barrier() to wait
      all my_struct_release_rcu() had called.
   B) use kfree_rcu():
      if all trivial callback are removed and kfree_rcu() are used instead,
      we do not need to wait anything. just quick finish unloading.

2) duplicate code:
   A) use my_struct_release_rcu():
      All trivial callback are very like my_struct_release_rcu(),
      all are duplicate code.
   B) use kfree_rcu():
      all trivial callback are removed, not duplicate code like
      my_struct_release_rcu().

3) cache:
   A) use my_struct_release_rcu():
      my_struct_release_rcu() is called rarely, when my_struct_release_rcu()
      is being called, cache missing will occur.
   B) use kfree_rcu():
      my_struct_release_rcu() is removed, not such cache missing.

4) future:
   A) use my_struct_release_rcu():
      when new user use rcu, the most callback is trivial callback
      like my_struct_release_rcu(). this is the common of using rcu.
      so the problems of above are more and more heavy.
   B) use kfree_rcu():
      fix these problems for ever.


Paul E. McKenney wrote:
> On Thu, Sep 18, 2008 at 12:18:28PM +0800, Lai Jiangshan wrote:
>> sometimes a rcu callback is just calling kfree() to free a struct's memory
>> (we say this callback is a trivial callback.).
>> this patch introduce kfree_rcu() to do these things directly, easily.
> 
> Interesting!  Please see questions and comments below.
> 
>> There are 4 reasons that we need kfree_rcu():
>>
>> 1) unloadable modules:
>>    a module(rcu callback is defined in this module) using rcu must
>>    call rcu_barrier() when unload. rcu_barrier() will increase
>>    the system's overhead(the more cpus the worse) and
>>    rcu_barrier() is very time-consuming. if all rcu callback defined
>>    in this module are trivial callback, we can just call kfree_rcu()
>>    instead, save a rcu_barrier() when unload.
> 
> You lost me on this one.  Suppose that the following sequence of
> events occurred:
> 
> a.	The module invokes call_rcu() or kfree_rcu().  The callback
> 	is queued on CPU 0.
> 
> b.	Perhaps a grace period completes, and the callback is therefore
> 	moved to CPU 0's donelist.  But CPU 0 is busy, so doesn't get
> 	around to invoking the callback.  (For example, ksoftirqd.)
> 
> c.	The module is unloaded, and uses kfree_rcu() instead of
> 	rcu_barrier().  The callback is queued on CPU 1.

uses kfree_rcu() instead of trivial callback, not rcu_barrier()

> 
> d.	A grace period completes, and CPU 1 is relatively idle, so
> 	invokes its callback quickly.  The module is therefore unloaded.
> 
> e.	CPU 0 finally gets around to executing its callback, but the
> 	module has been unloaded, so there is nothingness where the
> 	callback function used to be.  We get an oops.
> 

we done need wait anything if not callback is defined in this module.

> What prevents this sequence of events from happening?
>