Re: Q: smp.c && barriers (Was: [PATCH 1/4] generic-smp: remove single ipi fallback for smp_call_function_many())

Linus Torvalds
2009-02-18T16:34:30Z

On Wed, 18 Feb 2009, Ingo Molnar wrote:
>
> But ... WRMSR should already be serializing - it is documented
> as a serializing instruction.

Hmm. I was thinking about this some more, and I think I've come up with an
explanation.

"wrmsr" probably serializes _after_ doing the write. After all, it's
historically used for changing internal CPU state, so you want to do the
write, and then wait until the effects of the write are "stable" in the
core.

That would explain how x2apic can use both a serializing instruction
(wrmsr) and still effectively cause the IPI to happen out of sequence: the
IPI can reach the destination CPU before the source CPU has flushed its
store buffers, because the IPI is actually sent before serializing the
core.

But I would very strongly put this in the "x2apic code bug" column. If
this is a true issue (and your TLB patch does imply it is), then we should
just make sure that the x2apic IPI calls always do a 'sfence' before they
happen - regardless of whether they are for TLB flushes or for generic
kernel cross-calls, or for anything else.

Linus