Re: synchronize with a non-atomic flag

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Fri, Oct 06, 2017 at 08:35:00PM +0800, Yubin Ruan wrote:
> 2017-10-06 20:03 GMT+08:00 Akira Yokosawa <akiyks@gmail.com>:
> > On 2017/10/06 14:52, Yubin Ruan wrote:

[ . . . ]

> > I/O operations in printf() might make the situation trickier.
> 
> printf(3) is claimed to be thread-safe, so I think this issue will not
> concern us.
> 
> > In a more realistic case where you do something meaningful in
> > do_something() in both threads:
> >
> >     //process 1
> >     while(1) {
> >         if(READ_ONCE(flag) == 0) {
> >             do_something();
> >             WRITE_ONCE(flag, 1); // let another process to run
> >         } else {
> >             continue;
> >         }
> >     }
> >
> >     //process 2
> >     while(1) {
> >         if(READ_ONCE(flag) == 1) {
> >             do_something();
> >             WRITE_ONCE(flag, 0); // let another process to run
> >         } else {
> >             continue;
> >         }
> >     }

In the Linux kernel, there is control-dependency ordering between
the READ_ONCE(flag) and any stores in either the then-clause or
the else-clause.  However, I see no ordering between do_something()
and the WRITE_ONCE().

> > and if do_something() uses some shared variables other than "flag",
> > you need a couple of memory barriers to ensure the ordering of
> > READ_ONCE(), do_something(), and WRITE_ONCE() something like:
> >
> >     //process 1
> >     while(1) {
> >         if(READ_ONCE(flag) == 0) {
> >             smp_rmb();
> >             do_something();
> >             smp_wmb();
> >             WRITE_ONCE(flag, 1); // let another process to run
> >         } else {
> >             continue;
> >         }
> >     }
> >
> >     //process 2
> >     while(1) {
> >         if(READ_ONCE(flag) == 1) {
> >             smp_rmb();
> >             do_something();
> >             smp_wmb();
> >             WRITE_ONCE(flag, 0); // let another process to run
> >         } else {
> >             continue;
> >         }
> >     }

Here, the control dependency again orders the READ_ONCE() against later
stores, and the smp_rmb() orders the READ_ONCE() against any later
loads.  The smp_wmb() orders do_something()'s writes (but not its reads!)
against the WRITE_ONCE().

> > In Linux kernel memory model, you can use acquire/release APIs instead:
> >
> >     //process 1
> >     while(1) {
> >         if(smp_load_acquire(&flag) == 0) {
> >             do_something();
> >             smp_store_release(&flag, 1); // let another process to run
> >         } else {
> >             continue;
> >         }
> >     }
> >
> >     //process 2
> >     while(1) {
> >         if(smp_load_acquire(&flag) == 1) {
> >             do_something();
> >             smp_store_release(&flag, 0); // let another process to run
> >         } else {
> >             continue;
> >         }
> >     }

This is probably the most straightforward of the above approaches.

That said, if you really want a series of things to execute in a
particular order, why not just put them into the same process?

							Thanx, Paul

> Yes it could be tricky when `do_something()' really do something that
> involved other shared variable.
> 
> Yubin
> 
> > The intention of the code is easier to see when you use well-defined APIs.
> > Just my two cents.
> >
> >               Thanks, Akira
> >
> >>                                                That is because:
> >>
> >>     1) on X86/X64, load/store on 32-bits variable are atomic
> >>     2) I use READ_ONCE/WRITE_ONCE to prevent possibly harmful compiler
> >> optimization on `flag'.
> >>     3) I use only one variable to communicate between two processes,
> >> so there is no need for any kind of barrier.
> >>
> >> Does anyone have any objection at that?
> >>
> >> I know using a lock or atomic operation will save me a lot of
> >> argument, but I think those things are unnecessary at this
> >> circumstance, and it matter where performance matter, so I am picky
> >> here...
> >>
> >> Yubin
> >>
> >> [1]: https://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong
> >> [2]: https://www.usenix.org/conference/osdi10/ad-hoc-synchronization-considered-harmful
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe perfbook" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>
> >
>