Re: [RFC PATCH] introduce sys_membarrier(): process-wide memory barrier (v3a)

Paul E. McKenney
2010-01-11T22:48:16Z

On Mon, Jan 11, 2010 at 11:20:16PM +0100, Peter Zijlstra wrote:
> On Mon, 2010-01-11 at 17:04 -0500, Mathieu Desnoyers wrote:
> > * Peter Zijlstra (peterz@infradead.org) wrote:
> > > On Mon, 2010-01-11 at 15:52 -0500, Mathieu Desnoyers wrote:
> > > >
> > > > So the clear bit can occur far, far away in the future, we don't care.
> > > > We'll just send extra IPIs when unneeded in this time-frame.
> > >
> > > I think we should try harder not to disturb CPUs, particularly in the
> > > face of RT tasks and DoS scenarios. Therefore I don't think we should
> > > just wildly send to mm_cpumask(), but verify (although speculatively)
> > > that the remote tasks' mm matches ours.
> >
> > Well, my point of view is that if IPI TLB shootdown does not care about
> > disturbing CPUs running other processes in the time window of the lazy
> > removal, why should we ?
>
> while (1)
> sys_membarrier();
>
> is a very good reason, TLB shootdown doesn't have that problem.

You can get a similar effect by doing mmap() to a fixed virtual address
in a tight loop, right? Of course, mmap() has quite a bit more overhead
than sys_membarrier(), so the resulting IPIs probably won't hit the
other CPUs quite as hard, but it will hit them repeatedly.

> > We're adding an overhead very close to that of
> > an unrequired IPI shootdown which returns immediately without doing
> > anything.
>
> Except we don't clear the mask.
>
> > The tradeoff here seems to be:
> > - more overhead within switch_mm() for more precise mm_cpumask.
> > vs
> > - lazy removal of the cpumask, which implies that some processors
> > running a different process can receive the IPI for nothing.
> >
> > I really doubt we could create an IPI DoS based on such a small
> > time window.
>
> What small window? When there's less runnable tasks than available mm
> contexts some architectures can go quite a long while without
> invalidating TLBs.
>
> So what again is wrong with:
>
> int cpu, this_cpu = get_cpu();
>
> smp_mb();
>
> for_each_cpu(cpu, mm_cpumask(current->mm)) {
> if (cpu == this_cpu)
> continue;
> if (cpu_curr(cpu)->mm != current->mm)
> continue;
> smp_send_call_function_single(cpu, do_mb, NULL, 1);
> }
>
> put_cpu();
>
> ?

Well, if you have lots of CPUs, you will have disabled preemption for
quite some time. Not that there aren't already numerous similar
problems throughout the Linux kernel...

Thanx, Paul