Re: [RFC][PATCH] pipe: Convert ring to head/tail

Will Deacon
2019-09-17T17:07:23Z

Hi David,

On Tue, Sep 17, 2019 at 02:51:35PM +0100, David Howells wrote:
> Will Deacon <will@kernel.org> wrote:
>
> > > + /* Barrier: head belongs to the write side, so order reading
> > > + * the data after reading the head pointer.
> > > + */
> > > + unsigned int head = READ_ONCE(pipe->head);
> >
> > Hmm, I don't understand this. Since READ_ONCE() doesn't imply a barrier,
> > how are you enforcing the read-read ordering in the CPU?
>
> It does imply a barrier: smp_read_barrier_depends(). I believe that's

I fed your incomplete sentence to https://talktotransformer.com/ :

It does imply a barrier: smp_read_barrier_depends(). I believe that's
correct. (I'm not a coder so I assume it just means it's a dependency. Maybe
this works for other languages too.)

but I have a feeling that's not what you meant. I guess AI isn't quite
ready to rule the world.

> > What is the purpose of saying "This may need to insert a barrier"? Can this
> > function be overridden or something?
>
> I mean it's arch-dependent whether READ_ONCE() inserts a barrier or not.

Ok, but why would the caller care?

> > Saying that "This inserts a barrier" feels misleading, because READ_ONCE()
> > doesn't do that.
>
> Yes it does - on the Alpha:
>
> [arch/alpha/include/asm/barrier.h]
> #define read_barrier_depends() __asm__ __volatile__("mb": : :"memory")
>
> [include/asm-generic/barrier.h]
> #ifndef __smp_read_barrier_depends
> #define __smp_read_barrier_depends() read_barrier_depends()
> #endif
> ...
> #ifndef smp_read_barrier_depends
> #define smp_read_barrier_depends() __smp_read_barrier_depends()
> #endif
>
> [include/linux/compiler.h]
> #define __READ_ONCE(x, check) \
> ({ \
> union { typeof(x) __val; char __c[1]; } __u; \
> if (check) \
> __read_once_size(&(x), __u.__c, sizeof(x)); \
> else \
> __read_once_size_nocheck(&(x), __u.__c, sizeof(x)); \
> smp_read_barrier_depends(); /* Enforce dependency ordering from x */ \
> __u.__val; \
> })
> #define READ_ONCE(x) __READ_ONCE(x, 1)
>
> See:
>
> commit 76ebbe78f7390aee075a7f3768af197ded1bdfbb
> Author: Will Deacon <will.deacon@arm.com>
> Date: Tue Oct 24 11:22:47 2017 +0100
> locking/barriers: Add implicit smp_read_barrier_depends() to READ_ONCE()

Ah, that guy. I tried emailing him but he didn't reply.

Seriously though, READ_ONCE() implies a barrier on Alpha, but its portable
barrier semantics are only that it can be used to head an address
dependency, a bit like rcu_dereference(). You shouldn't be relying on the
stronger ordering provided by Alpha, and I doubt that you really are.

If I'm understanding your code correctly (big 'if'), then you have things
like this in pipe_read():

unsigned int head = READ_ONCE(pipe->head);
unsigned int tail = pipe->tail;
unsigned int mask = pipe->buffers - 1;

if (tail != head) {
struct pipe_buffer *buf = &pipe->bufs[tail & mask];

[...]

written = copy_page_to_iter(buf->page, buf->offset, chars, to);

where you want to make sure you don't read from 'buf->page' until after
you've read the updated head index. Is that right? If so, then READ_ONCE()
will not give you that guarantee on architectures such as Power and Arm,
because the 'if (tail != head)' branch can be speculated and the buffer
can be read before we've got around to looking at the head index.

So I reckon you need smp_load_acquire() in this case. pipe_write() might be ok
with the control dependency because CPUs don't tend to make speculative writes
visible, but I didn't check it carefully and the compiler can do crazy stuff in
this area, so I'd be inclined to use smp_load_acquire() here too unless you
really need the last ounce of performance.

Will