Re: [RFC PATCH] introduce sys_membarrier(): process-wide memory barrier

[Steven Rostedt <rostedt@goodmis.org>]

On Sun, 2010-01-10 at 11:03 -0500, Mathieu Desnoyers wrote:
> * Steven Rostedt (rostedt@goodmis.org) wrote:

> The way I see it, TLB can be seen as read-only elements (a local
> read-only cache) on the processors. Therefore, we don't care if they are
> in a stale state while performing the cpumask update, because the fact
> that we are executing switch_mm() means that these TLB entries are not
> being used locally anyway and will be dropped shortly. So we have the
> equivalent of a full memory barrier (load_cr3()) _after_ the cpumask
> updates.
> 
> However, in sys_membarrier(), we also need to flush the write buffers
> present on each processor running threads which belong to our current
> process. Therefore, we would need, in addition, a smp_mb() before the
> mm cpumask modification. For x86, cpumask_clear_cpu/cpumask_set_cpu
> implies a LOCK-prefixed operation, and hence does not need any added
> barrier, but this could be different for other architectures.
> 
> So, AFAIK, doing a flush_tlb() would not guarantee the kind of
> synchronization we are looking for because an uncommitted write buffer
> could still sit on the remote CPU when we return from sys_membarrier().

Ah, so you are saying we can have this:


	CPU 0			CPU 1
     ----------		    --------------
	obj = list->obj;
				<user space>
				rcu_read_lock();
				obj = rcu_dereference(list->obj);
				obj->foo = bar;

				<preempt>
				<kernel space>

				schedule();
				cpumask_clear(mm_cpumask, cpu);

	sys_membarrier();
	free(obj);

				<store to obj->foo goes to memory>  <- corruption

		

So, if there's no smp_wmb() between the <preempt> and cpumask_clear()
then we have an issue?

-- Steve