Re: [PATCH -v3 6/8] fsnotify: add group priorities

[Eric Paris <eparis@redhat.com>]

On Thu, 2008-11-27 at 17:25 +0100, Peter Zijlstra wrote:
> On Tue, 2008-11-25 at 12:21 -0500, Eric Paris wrote:
> > In preperation for blocking fsnotify calls group priorities must be added.
> > When multiple groups request the same event type the lowest priority group
> > will receive the notification first.
> 
> > @@ -114,9 +117,26 @@ struct fsnotify_group *fsnotify_find_group(unsigned int group_num, unsigned long
> >  
> >  	group->ops = ops;
> >  
> > -	/* add it */
> > -	list_add_rcu(&group->group_list, &fsnotify_groups);
> > +	/* Do we need to be the first entry? */
> > +	if (list_empty(&fsnotify_groups)) {
> > +		list_add_rcu(&group->group_list, &fsnotify_groups);
> > +		goto out;
> > +	}
> > +
> > +	list_for_each_entry(group_iter, &fsnotify_groups, group_list) {
> > +		/* insert in front of this one? */
> > +		if (priority < group_iter->priority) {
> > +			/* I used list_add_tail() to insert in front of group_iter...  */
> > +			list_add_tail_rcu(&group->group_list, &group_iter->group_list);
> > +			break;
> > +		}
> >  
> > +		/* are we at the end?  if so insert at end */
> > +		if (list_is_last(&group_iter->group_list, &fsnotify_groups)) {
> > +			list_add_tail_rcu(&group->group_list, &fsnotify_groups);
> > +			break;
> > +		}
> > +	}
> >  out:
> >  	mutex_unlock(&fsnotify_grp_mutex);
> >  	fsnotify_recalc_global_mask();
> 
> What priority range do you need to cater for, and how many groups? 

On a typical system I'd expect to see one group for dnotify (rpmidmapd
uses dnotify so most systems will end up having 1 group I would expect)

inotify I wouldn't expect more than 3-4 inotify_init() calls

fsnotify I wouldn't imagine more than 3 groups.

So total we are talking about maybe 10 groups on a system really making
use of fs notification?

> I can
> imagine for many groups and limit range a priority list might be better
> suited.

talking about plist.h?  Since I don't allow 2 groups with the same
priority I'd say a lot of the plist code would just be overhead (the
prio list and the node list would be the same)

That's not a big deal since I don't really care about the add/remove
code paths since they are all notification overhead/setup/teardown.  I
would think that cleaner simpler code would probably be a better idea
rather than performance for these areas especially since it looks like
the speed critical parts of plists (list_for_each_entry) would be the
exact same.

what I don't see is plists being protected by RCU and looking at
plist_del it doesn't seem like it would be rcu safe.  RCU safe plists
might be a good idea, but for now I think I should just do my own
priority listing so I don't have to hold a lock while I walk the group
list (that path is VERY hot)

-Eric