Re: [PATCH 1/2] percpu-rw-semaphores: use light/heavy barriers

[Oleg Nesterov <oleg@redhat.com>]

Not really the comment, but the question...

On 10/22, Mikulas Patocka wrote:
>
>  static inline void percpu_down_read(struct percpu_rw_semaphore *p)
>  {
>  	rcu_read_lock();
> @@ -24,22 +27,12 @@ static inline void percpu_down_read(stru
>  	}
>  	this_cpu_inc(*p->counters);
>  	rcu_read_unlock();
> +	light_mb(); /* A, between read of p->locked and read of data, paired with D */
>  }

rcu_read_unlock() (or even preempt_enable) should have compiler barrier
semantics... But I agree, this adds more documentation for free.

>  static inline void percpu_up_read(struct percpu_rw_semaphore *p)
>  {
> -	/*
> -	 * On X86, write operation in this_cpu_dec serves as a memory unlock
> -	 * barrier (i.e. memory accesses may be moved before the write, but
> -	 * no memory accesses are moved past the write).
> -	 * On other architectures this may not be the case, so we need smp_mb()
> -	 * there.
> -	 */
> -#if defined(CONFIG_X86) && (!defined(CONFIG_X86_PPRO_FENCE) && !defined(CONFIG_X86_OOSTORE))
> -	barrier();
> -#else
> -	smp_mb();
> -#endif
> +	light_mb(); /* B, between read of the data and write to p->counter, paired with C */
>  	this_cpu_dec(*p->counters);
>  }
>
> @@ -61,11 +54,12 @@ static inline void percpu_down_write(str
>  	synchronize_rcu();
>  	while (__percpu_count(p->counters))
>  		msleep(1);
> -	smp_rmb(); /* paired with smp_mb() in percpu_sem_up_read() */
> +	heavy_mb(); /* C, between read of p->counter and write to data, paired with B */

I _think_ this is correct.


Just I am wondering if this is strongly correct in theory, I would
really like to know what Paul thinks.

Ignoring the current implementation, according to the documentation
synchronize_sched() has all rights to return immediately if there is
no active rcu_read_lock_sched() section. If this were possible, than
percpu_up_read() lacks mb.

So _perhaps_ it makes sense to document that synchronize_sched() also
guarantees that all pending loads/stores on other CPUs should be
completed upon return? Or I misunderstood the patch?

Oleg.