Re: [Lse-tech] Re: [ckrm-tech] Re: [PATCH 00/01] Move Exit Connectors

[Keith Owens <kaos@sgi.com>]

"Paul E. McKenney" (on Wed, 11 Jan 2006 21:04:53 -0800) wrote:
>On Thu, Jan 12, 2006 at 02:29:52PM +1100, Keith Owens wrote:
>> John Hesterberg (on Wed, 11 Jan 2006 15:39:10 -0600) wrote:
>> >On Wed, Jan 11, 2006 at 01:02:10PM -0800, Matt Helsley wrote:
>> >> 	Have you looked at Alan Stern's notifier chain fix patch? Could that be
>> >> used in task_notify?
>> >
>> >I have two concerns about an all-tasks notification interface.
>> >First, we want this to scale, so don't want more global locks.
>> >One unique part of the task notify is that it doesn't use locks.
>> 
>> Neither does Alan Stern's atomic notifier chain.  Indeed it cannot use
>> locks, because the atomic notifier chains can be called from anywhere,
>> including non maskable interrupts.  The downside is that Alan's atomic
>> notifier chains require RCU.
>> 
>> An alternative patch that requires no locks and does not even require
>> RCU is in http://marc.theaimsgroup.com/?l=linux-kernel&m=113392370322545&w=2
>
>Interesting!  Missed this on the first time around...
>
>But doesn't notifier_call_chain_lockfree() need to either disable
>preemption or use atomic operations to update notifier_chain_lockfree_inuse[]
>in order to avoid problems with preemption?

OK, I have thought about it and ...

  notifier_call_chain_lockfree() must be called with preempt disabled.

The justification for this routine is to handle all the nasty
corner cases in the notify_die() and similar chains, including panic,
spinlocks being held and even non maskable interrupts.  It is silly to
try to make notifier_call_chain_lockfree() handle the preemptible case
as well.

If notifier_call_chain_lockfree() is to be called for task
notification, then the caller must disable preempt around the call to
notifier_call_chain_lockfree().  Scalability over lots of cpus should
not be an issue, especially if notifier_chain_lockfree_inuse[] is
converted to a per cpu variable.  The amount of time spent with preempt
disabled is proportional to the number of registered callbacks on the
task notifcation chain and to the amount of work performed by those
callbacks, neither of which should be high.