Re: [RFC PATCH] introduce sys_membarrier(): process-wide memory barrier

Steven Rostedt
2010-01-10T16:23:13Z

On Sun, 2010-01-10 at 11:03 -0500, Mathieu Desnoyers wrote:
> * Steven Rostedt (rostedt@goodmis.org) wrote:

> The way I see it, TLB can be seen as read-only elements (a local
> read-only cache) on the processors. Therefore, we don't care if they are
> in a stale state while performing the cpumask update, because the fact
> that we are executing switch_mm() means that these TLB entries are not
> being used locally anyway and will be dropped shortly. So we have the
> equivalent of a full memory barrier (load_cr3()) _after_ the cpumask
> updates.
>
> However, in sys_membarrier(), we also need to flush the write buffers
> present on each processor running threads which belong to our current
> process. Therefore, we would need, in addition, a smp_mb() before the
> mm cpumask modification. For x86, cpumask_clear_cpu/cpumask_set_cpu
> implies a LOCK-prefixed operation, and hence does not need any added
> barrier, but this could be different for other architectures.
>
> So, AFAIK, doing a flush_tlb() would not guarantee the kind of
> synchronization we are looking for because an uncommitted write buffer
> could still sit on the remote CPU when we return from sys_membarrier().

Ah, so you are saying we can have this:

CPU 0 CPU 1
---------- --------------
obj = list->obj;
<user space>
rcu_read_lock();
obj = rcu_dereference(list->obj);
obj->foo = bar;

<preempt>
<kernel space>

schedule();
cpumask_clear(mm_cpumask, cpu);

sys_membarrier();
free(obj);

<store to obj->foo goes to memory> <- corruption

So, if there's no smp_wmb() between the <preempt> and cpumask_clear()
then we have an issue?

-- Steve