Re: smp.c && barriers (Was: [PATCH 1/4] generic-smp: remove single ipi fallback for smp_call_function_many())

Ingo Molnar
2009-02-18T16:23:49Z

* Linus Torvalds <torvalds@linux-foundation.org> wrote:

> On Wed, 18 Feb 2009, Nick Piggin wrote:
> > >
> > > x2apic register reads/writes don't have serializing semantics, as
> > > opposed to uncached xapic accesses, which are inherently serializing.
> > >
> > > With this patch, we need to fix the corresponding x2apic IPI operations.
> > > I will take a look at it.
> >
> > You're saying the problem is in generic_exec_single because I've
> > removed the smp_mb that inadvertently also serialises memory with
> > the x2apic on x86?
>
> I think Suresh is wrong on this.
>
> The x2apic is using "wrmsr" to write events, and that's a
> serializing instruction.
>
> I really don't know of any way to get unordered information
> out of a x86 core, except for playing games with WC memory,
> and WC memory would not be appropriate for something like an
> interrupt controller.
>
> Of course, it's possible that Intel made the x2apic MSR's
> magic, and that they don't serialize, but that's very much
> against some very explicit Intel documentation. wrmsr is one
> of the (few) instructions that is mentioned all ove the
> documentation as being serializing.

heh, i just went through all those codepaths to figure out the
SMP ordering semantics. I didnt find anything but the MSR write,
so maybe the MSR writes did get weakened on certain CPUs.

Serializing is a serious performance penalty - and it would not
be totally out of question to optimize xAPIC MSR accesses. If
that's the case it's not quite nice to not document it though.

Ingo