Re: Internal vs. external barriers (was: Re: Interesting LKMM litmus test)

Alan Stern
2023-01-25T19:09:45Z

On Wed, Jan 25, 2023 at 09:18:32AM -0800, Paul E. McKenney wrote:
> ------------------------------------------------------------------------
>
> C C-srcu-observed-4
>
> (*
> * Result: Sometimes
> *
> * The Linux-kernel implementation is suspected to forbid this.
> *)
>
> {}
>
> P0(int *x, int *y, int *z, struct srcu_struct *s)
> {
> int r1;
>
> r1 = srcu_read_lock(s);
> WRITE_ONCE(*y, 2);
> WRITE_ONCE(*x, 1);
> srcu_read_unlock(s, r1);
> }
>
> P1(int *x, int *y, int *z, struct srcu_struct *s)
> {
> int r1;
>
> WRITE_ONCE(*y, 1);
> synchronize_srcu(s);
> WRITE_ONCE(*z, 2);
> }
>
> P2(int *x, int *y, int *z, struct srcu_struct *s)
> {
> WRITE_ONCE(*z, 1);
> smp_store_release(x, 2);
> }
>
> exists (x=1 /\ y=1 /\ z=1)
>
> ------------------------------------------------------------------------
>
> We get the following from herd7:
>
> ------------------------------------------------------------------------
>
> $ herd7 -conf linux-kernel.cfg C-srcu-observed-4.litmus
> Test C-srcu-observed-4 Allowed
> States 8
> x=1; y=1; z=1;
> x=1; y=1; z=2;
> x=1; y=2; z=1;
> x=1; y=2; z=2;
> x=2; y=1; z=1;
> x=2; y=1; z=2;
> x=2; y=2; z=1;
> x=2; y=2; z=2;
> Ok
> Witnesses
> Positive: 1 Negative: 7
> Condition exists (x=1 /\ y=1 /\ z=1)
> Observation C-srcu-observed-4 Sometimes 1 7
> Time C-srcu-observed-4 0.02
> Hash=8b6020369b73ac19070864a9db00bbf8
>
> ------------------------------------------------------------------------
>
> This does not seem to me to be consistent with your "The RCU guarantee
> about writes in a read-side critical section becoming visible to all
> CPUs before a later grace period ends".

Let's see. That guarantee requires only that x=1 and y=2 become visible
to P1 and P2 before the grace period ends. And since synchronize_srcu
is a strong fence, y=1 must become visible to P0 and P2 before the grace
period ends. Presumably after y=2 does, because it overwrites y=2.
Okay so far.

Now at some point P2 executes x=2. If this were to happen after the
grace period ended, it would overwrite x=1. Therefore it must happen
before the grace period ends, and therefore P2 must also write z=1
before the grace period ends.

So we have P2 writing z=1 before P1 writes z=2. But this doesn't mean
z=2 has to overwrite z=1! (You had a diagram illustrating this point in
one of your own slides for a talk about the LKMM.) Overwriting is
required only when the earlier write becomes visible to the later
write's CPU before the later write occurs, and nothing in this test
forces z=2 to propagate to P1 before the z=1 write executes.

So the litmus test's outcome can happen without violating my guarantee.

> So what am I missing here?

Can't tell. I'm not sure why you think the litmus test isn't consistent
with the guarantee.

> Again, I am OK with LKMM allowing C-srcu-observed-4.litmus, as long as
> the actual Linux-kernel implementation forbids it.

Why do you want the implementation to forbid it? The pattern of the
litmus test resembles 3+3W, and you don't care whether the kernel allows
that pattern. Do you?

Alan