x86: rdtsc_barrier possibly missing serializing instruction

[Mathieu Desnoyers <mathieu.desnoyers@efficios.com>]

Hi,

I noticed that rdtsc_barrier is possibly missing serializing instructions, so I
am bringing this to your attention. As the comment states above rdtsc_barrier():

 * Stop RDTSC speculation. This is needed when you need to use RDTSC
 * (or get_cycles or vread that possibly accesses the TSC) in a defined
 * code region.

So it seems to imply that the rdtsc will not be reordered wrt other instructions
due to speculative execution, which does not seem to be taken care of here.

rdtsc_barrier() as found in recent (2.6.36) kernel translates into either a
mfence or lfence (or nothing), depending on the architecture. Now the Intel
manuals state the following:

"The MFENCE instruction is ordered with respect to all load and store
instructions, other MFENCE instructions, any SFENCE and LFENCE instructions, and
any serializing instructions (such as the CPUID instruction)."

and...

- Privileged serializing instructions—MOV (to control register), MOV (to debug
  register), WRMSR, INVD, INVLPG, WBINVD, LGDT, LLDT, LIDT, and LTR.
- Nonprivileged serializing instructions—CPUID, IRET, and RSM.

So rdtsc does not appear in the list of serializing instruction, which makes me
wonder how mfence or lfence can ensure that rdtsc does not spill outside of the
memory barrier ? I feel I might be missing an undocumented feature here.

If I get it right, surrounding TSC read with these barriers should end up with
something like:

rdtsc_barrier_acquire()
  m/lfence
  cpuid
get_cycles()
  rdtsc
rdtsc_barrier_release()
  cpuid
  m/lfence

Thoughts ?

Thanks,

Mathieu

-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com