Re: C aggregate passing (Rust kernel policy)

[Boqun Feng <boqun.feng@gmail.com>]

On Fri, Feb 28, 2025 at 11:21:47AM -0500, Kent Overstreet wrote:
> On Fri, Feb 28, 2025 at 08:13:09AM -0800, Boqun Feng wrote:
> > On Fri, Feb 28, 2025 at 11:04:28AM -0500, Kent Overstreet wrote:
> > > On Fri, Feb 28, 2025 at 07:46:23AM -0800, Boqun Feng wrote:
> > > > On Fri, Feb 28, 2025 at 10:41:12AM -0500, Kent Overstreet wrote:
> > > > > On Fri, Feb 28, 2025 at 08:44:58AM +0100, Ralf Jung wrote:
> > > > > > Hi,
> > > > > > 
> > > > > > > > I guess you can sum this up to:
> > > > > > > > 
> > > > > > > >    The compiler should never assume it's safe to read a global more than the
> > > > > > > >    code specifies, but if the code reads a global more than once, it's fine
> > > > > > > >    to cache the multiple reads.
> > > > > > > > 
> > > > > > > > Same for writes, but I find WRITE_ONCE() used less often than READ_ONCE().
> > > > > > > > And when I do use it, it is more to prevent write tearing as you mentioned.
> > > > > > > 
> > > > > > > Except that (IIRC) it is actually valid for the compiler to write something
> > > > > > > entirely unrelated to a memory location before writing the expected value.
> > > > > > > (eg use it instead of stack for a register spill+reload.)
> > > > > > > Not gcc doesn't do that - but the standard lets it do it.
> > > > > > 
> > > > > > Whether the compiler is permitted to do that depends heavily on what exactly
> > > > > > the code looks like, so it's hard to discuss this in the abstract.
> > > > > > If inside some function, *all* writes to a given location are atomic (I
> > > > > > think that's what you call WRITE_ONCE?), then the compiler is *not* allowed
> > > > > > to invent any new writes to that memory. The compiler has to assume that
> > > > > > there might be concurrent reads from other threads, whose behavior could
> > > > > > change from the extra compiler-introduced writes. The spec (in C, C++, and
> > > > > > Rust) already works like that.
> > > > > > 
> > > > > > OTOH, the moment you do a single non-atomic write (i.e., a regular "*ptr =
> > > > > > val;" or memcpy or so), that is a signal to the compiler that there cannot
> > > > > > be any concurrent accesses happening at the moment, and therefore it can
> > > > > > (and likely will) introduce extra writes to that memory.
> > > > > 
> > > > > Is that how it really works?
> > > > > 
> > > > > I'd expect the atomic writes to have what we call "compiler barriers"
> > > > > before and after; IOW, the compiler can do whatever it wants with non
> > > > 
> > > > If the atomic writes are relaxed, they shouldn't have "compiler
> > > > barriers" before or after, e.g. our kernel atomics don't have such
> > > > compiler barriers. And WRITE_ONCE() is basically relaxed atomic writes.
> > > 
> > > Then perhaps we need a better definition of ATOMIC_RELAXED?
> > > 
> > > I've always taken ATOMIC_RELAXED to mean "may be reordered with accesses
> > > to other memory locations". What you're describing seems likely to cause
> > 
> > You lost me on this one. if RELAXED means "reordering are allowed", then
> > why the compiler barriers implied from it?
> 
> yes, compiler barrier is the wrong language here
> 
> > > e.g. if you allocate a struct, memset() it to zero it out, then publish
> > > it, then do a WRITE_ONCE()...
> > 
> > How do you publish it? If you mean:
> > 
> > 	// assume gp == NULL initially.
> > 
> > 	*x = 0;
> > 	smp_store_release(gp, x);
> > 
> > 	WRITE_ONCE(*x, 1);
> > 
> > and the other thread does
> > 
> > 	x = smp_load_acquire(gp);
> > 	if (p) {
> > 		r1 = READ_ONCE(*x);
> > 	}
> > 
> > r1 can be either 0 or 1.
> 
> So if the compiler does obey the store_release barrier, then we're ok.
> 
> IOW, that has to override the "compiler sees the non-atomic store as a
> hint..." - but the thing is, since we're moving more to type system

This might be a bad example, but I think that means if you add another
*x = 2 after WRITE_ONCE(*x, 1):

 	*x = 0;
 	smp_store_release(gp, x);
 
 	WRITE_ONCE(*x, 1);
	*x = 2;

then compilers in-theory can do anything they seems fit. I.e. r1 can be
anything. Because it's a data race.

> described concurrency than helpers, I wonder if that will actually be
> the case.
> 
> Also, what's the situation with reads? Can we end up in a situation
> where a non-atomic read causes the compiler do erronious things with an
> atomic_load(..., relaxed)?

For LKMM, no, because our data races requires at least one access
being write[1], this applies to both C and Rust. For Rust native memory
model, no, because Ralf fixed it:

	https://github.com/rust-lang/rust/pull/128778

[1]: "PLAIN ACCESSES AND DATA RACES" in tools/memory-model/Documentation/explanation.txt

Regards,
Boqun