Re: [PATCH v3 1/1] rcu: Reduce synchronize_rcu() waiting time

[Uladzislau Rezki <urezki@gmail.com>]

On Wed, Oct 18, 2023 at 10:32:22AM -0400, Joel Fernandes wrote:
> On Tue, Oct 17, 2023 at 10:06 AM Uladzislau Rezki <urezki@gmail.com> wrote:
> [...]
> > > > +
> > > > +       /* Finally. */
> > > > +       complete(&rs->completion);
> > > > +}
> > > > +
> > > > +static void rcu_sr_normal_gp_cleanup_work(struct work_struct *work)
> > > > +{
> > > > +       struct llist_node *done, *rcu, *next;
> > > > +
> > > > +       done = llist_del_all(&sr.done);
> > > > +       if (!done)
> > > > +               return;
> > > > +
> > > > +       llist_for_each_safe(rcu, next, done)
> > > > +               rcu_sr_normal_complete(rcu);
> > > > +}
> > > [...]
> > > > +static void rcu_sr_normal_add_req(struct rcu_synchronize *rs)
> > > > +{
> > > > +       atomic_inc(&sr.active);
> > > > +       if (llist_add((struct llist_node *) &rs->head, &sr.curr))
> > > > +               /* Set the tail. Only first and one user can do that. */
> > > > +               WRITE_ONCE(sr.curr_tail, (struct llist_node *) &rs->head);
> > > > +       atomic_dec(&sr.active);
> > >
> > > Here there is no memory ordering provided by the atomic ops. Is that really Ok?
> > >
> > This needs to be reworked since there is no ordering guaranteed. I think
> > there is a version of "atomic_inc_something" that guarantees it?
> 
> Yeah there is atomic_fetch_{inc,dec}{_acquire,_release}()
> 
> Or:
>   atomic_inc(&sr.active);
>   smp_mb__after_atomic();
> 
>   smp_mb__before_atomic();
>   atomic_dec(&sr.active);
> 
> ?
>
> That's probably better because we don't need ordering before the inc
> or after the dec, AFAICS.
> 
There are two variants, atomic_inc_return() and atomic_dec_return()
which are fully ordered. Any thoughts about them? One line instead of
two as in your case.

Your concern is about, that atomic_inc() can be reodered? There is a
llist_add() that has inside the try_cmpxchg() that should have barrier.

Any thoughts?

Thank you for the review and help, Joel!

--
Uladzislau Rezki