Re: Q: cgroup: Questions about possible issues in cgroup locking

[Mandeep Singh Baines <msb-F7+t8E8rja9g9hUCZPvPmw@public.gmane.org>]

Oleg Nesterov (oleg-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org) wrote:
> On 01/12, Mandeep Singh Baines wrote:
> >
> > Oleg Nesterov (oleg-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org) wrote:
> > >
> > > Still can't understand... Lets look at this trivial example again.
> > >
> > > We start from the main thread M, it is ->group_leader. There is
> > > another thread T in this thread group. We are doing
> > >
> > > 	OLD = M;
> > >
> > > 	t = M;
> > > 	do {
> > > 		do_smth(t);
> > > 	}
> > > 	while (t->group_leader == OLD && ((t = next_thread(t)) != M);
> > >
> > > The first iteration does do_smth(M).
> > >
> > > T calls de_thread() and, in particular, it does M->group_leader = T
> > > (see "leader->group_leader = tsk" in de_thread).
> > >
> > > after that t->group_leader == OLD fails. t == M, its group_leader == T.
> > > do_smth(T) won't be called.
> > >
> > > No?
> > >
> >
> > I think we can handle this by removing the assignment. So in de_thread():
> >
> > -		leader->group_leader = tsk;
> 
> Ah, so that was you plan. I was confused by the 3rd argument, why
> it is needed?
> 

Good question. On second thought, I don't think its needed as shown
in you're solution below.

> Yes, I thought about this too. Suppose we remove this assignment,
> then we can simply do
> 
> 	#define while_each_thread(g, t) \
> 		while (t->group_leader == g->group_leader && (t = next_thread(t)) != g)
> 
> with the same effect. (to remind, currently I ignore the barriers/etc).
> 

Nice! I think this works.

> But this can _only_ help if we start at the group leader!

But I don't think this solution requires we start at the group leader.

My thinking:

Case 1: g is the exec thread

The only condition under which g->group_leader would change is if g is the
exec thread. If you are the exec the only requirement is that you visit the
exec thread. Visiting any other threads is optional. Since g is the exec
thread, you've already visited it and can safely stop once
g->group_leader is re-assigned to g.

Case 2: g is the group leader

If g is the group leader and a subthread execs, you'll terminate just
after visiting the exec thread.

Case 3: g is some other thread

In this case, g MUST be current so you don't really need to worry
about de_thread() since current can't be de_threaded.

> 
> May be we should enforce this rule (for the lockless case), I dunno...
> In that case I'd prefer to add the new while_each_thread_rcu() helper.
> But! in this case we do not need to change de_thread(), we can simply do
> 
> 	#define while_each_thread_rcu(t) \
> 		while (({ t = next_thread(t); !thread_group_leader(t); }))
> 

Won't this terminate just before visiting the exec thread?

> The definition above was one of the possibilities I considered, but
> I wasn't able to convince myself this is the best option.
> 
> See? Or do you think I missed something?
> 
> Just in case... note that while_each_thread_rcu() doesn't use 'g'
> at all. May be it makes sense to keep the old "t != g &&", but this
> is minor.
> 
> Oleg.
> 

Regards,
Mandeep