Re: [PATCH/RFC] synchronize_rcu(): high latency on idle system

Re: [PATCH/RFC] synchronize_rcu(): high latency on idle system

[Stephen Hemminger]

On Sat, 12 Jan 2008 19:35:58 +0100
Andi Kleen <ak@suse.de> wrote:

> On Saturday 12 January 2008 18:51:35 Benjamin LaHaise wrote:
> > On Sat, Jan 12, 2008 at 03:37:59AM +0100, Andi Kleen wrote:
> > > > And yes, the 
> > > > network stack shouldn't call synchronize_rcu() quite so much, but fixing that 
> > > > is a little more involved.
> > > 
> > > ... but the correct solution.
> > 
> > There has to be at least 1 synchronize_rcu() or equivalent in the 
> > unregister_netdev() path.  I suspect the easiest way to fix it might be to 
> > use call_rcu() to actually free the network device, as anything else will 
> > limit performance of single threaded teardown (ie, when an l2tp daemon 
> > gets terminated via kill -9).  This means an API change that exposes 
> > rcu for unregister_netdev().
> 
> The call_rcu() could be in free_netdev() couldn't it?

I think it should be in netdev_unregister_kobject().  But that would
only get rid of one of the two calls to synchronize_rcu() in the unregister_netdev.
The other synchronize_rcu() is for qdisc's and not sure if that one can
be removed?


-- 
Stephen Hemminger <stephen.hemminger@vyatta.com>

Re: [PATCH/RFC] synchronize_rcu(): high latency on idle system

[Andi Kleen]

> I think it should be in netdev_unregister_kobject().  But that would
> only get rid of one of the two calls to synchronize_rcu() in the unregister_netdev.

Would be already an improvement.

> The other synchronize_rcu() is for qdisc's and not sure if that one can
> be removed?

The standard way to remove such calls is to set a "deleted" flag in the object,
then check and ignore such objects in the reader and finally remove the object with
call_rcu

I have not checked if that is really feasible for qdiscs.

-Andi

Re: [PATCH/RFC] synchronize_rcu(): high latency on idle system

[Stephen Hemminger]

On Sun, 13 Jan 2008 16:34:17 +0100
Andi Kleen <ak@suse.de> wrote:

> 
> > I think it should be in netdev_unregister_kobject().  But that would
> > only get rid of one of the two calls to synchronize_rcu() in the unregister_netdev.
> 
> Would be already an improvement.
> 
> > The other synchronize_rcu() is for qdisc's and not sure if that one can
> > be removed?
> 
> The standard way to remove such calls is to set a "deleted" flag in the object,
> then check and ignore such objects in the reader and finally remove the object with
> call_rcu
> 
> I have not checked if that is really feasible for qdiscs.
> 
> -Andi

Actually, the synchronize_rcu() is now acting a barrier between two sections
in the current unregister process. It can't be removed.

But, an alternative unregister_and_free_netdev() could be created that uses
call_rcu.  Basically:

void unregistr_and_free_netdev() {
    do stuff before barrier...
    setup rcu callback
    call_rcu();
}

static void netdev_after_rcu() {
    rtnl_lock();
    do stuff after barier
    rtnl_unlock();
    free_netdev
}

-- 
Stephen Hemminger <stephen.hemminger@vyatta.com>