Re: x86 memory barrier: why does Linux prefer MFENCE to Locked ADD?

[Peter Zijlstra <peterz@infradead.org>]

On Thu, Mar 03, 2016 at 04:27:39PM +0100, Ingo Molnar wrote:
> 
> * Dexuan Cui <decui@microsoft.com> wrote:
> 
> > Hi,
> > My understanding about arch/x86/include/asm/barrier.h is: obviously Linux
> > more likes {L,S,M}FENCE -- Locked ADD is only used in x86_32 platforms that
> > don't support XMM2.
> > 
> > However, it looks people say Locked Add is much faster than the FENCE
> > instructions, even on modern Intel CPUs like Haswell, e.g., please see
> > the three sources:
> > 
> > " 11.5.1 Locked Instructions as Memory Barriers
> > Optimization
> > Use locked instructions to implement Store/Store and Store/Load barriers.
> > "
> > http://support.amd.com/TechDocs/47414_15h_sw_opt_guide.pdf
> > 
> > "lock addl %(rsp), 0 is a better solution for StoreLoad barrier ":
> > http://shipilev.net/blog/2014/on-the-fence-with-dependencies/
> > 
> > "...locked instruction are more efficient barriers...":
> > http://www.pvk.ca/Blog/2014/10/19/performance-optimisation-~-writing-an-essay/
> > 
> > I also found that FreeBSD prefers Locked Add.
> > 
> > So, I'm curious why Linux prefers MFENCE.
> > I guess I may be missing something.
> > 
> > I tried to google the question, but didn't find an answer.
> 
> It's being worked on, see this thread on lkml from a few weeks ago:
> 
>    C Jan 13 Michael S. Tsir    | [PATCH v3 0/4] x86: faster mb()+documentation tweaks
>    C Jan 13 Michael S. Tsir    | ├─>[PATCH v3 1/4] x86: add cc clobber for addl
>    C Jan 13 Michael S. Tsir    | ├─>[PATCH v3 2/4] x86: drop a comment left over from X86_OOSTORE
>    C Jan 13 Michael S. Tsir    | ├─>[PATCH v3 3/4] x86: tweak the comment about use of wmb for IO
>    C Jan 13 Michael S. Tsir    | ├─>[PATCH v3 4/4] x86: drop mfence in favor of lock+addl
> 
> The 4th patch changes MFENCE to a LOCK ADDL locked instruction.

Lots of additional chatter here:

  lkml.kernel.org/r/20160112150032-mutt-send-email-mst@redhat.com

And some useful bits here:

  lkml.kernel.org/r/56957D54.5000602@zytor.com

latest version here:

  lkml.kernel.org/r/1453921746-16178-1-git-send-email-mst@redhat.com