Re: [kvm-devel] [PATCH] KVM: Avoid using vmx instruction directly

[Jeremy Fitzhardinge <jeremy@goop.org>]

H. Peter Anvin wrote:
> I think you're wrong about that; in particular, I'm pretty sure "asm
> volatiles" are ordered among themselves.  What the "volatile" means is
> "this has side effects you (the compiler) don't understand", and gcc
> can't assume that it can reorder such side effects. 
That's not how I read the manual (quoted below).  "asm volatile" is much
weaker than people seem to think it is; the "volatile" puts fewer
constraints on the compiler than it would for a variable.  While the
manual doesn't say that "asm volatiles" could be reordered with respect
to each other, it doesn't say that they won't, and I don't see anything
in this description which could be read to imply it (indeed "can be
moved relative to other code" includes other asm volatiles).

Like "volatile" variables, I think "asm volatile" is probably overused. 
If you want to guarantee specific ordering of asms, it's probably better
to add an explicit dependency between them rather than rely on asm
volatile; this could either be a "memory" clobber, or something more
fine-grained.  For example:

    /* need never be instansiated; never actually referenced */
    extern int spin_sequencer;

    /* %0 never referenced */
    asm("take spinlock" : "+m" (spin_sequencer)...);

    ...

    /* again, %0 never referenced */
    asm("release spinlock" : "+m" (spin_sequencer)...);

This is example is a bit contrived since a real spinlock would also have
to have a memory clobber - or some other barrier - but you get the
idea.  It has the nice property of allowing you to define precise
dependencies between various asm()s, without having to set up
unnecessary dependencies between unrelated asms; and by making use of
in/out/inout asm parameters, you can express different kinds of
dependencies which give gcc a better chance of understanding what's
going on.

The relevent bit of the manual:

    The `volatile' keyword indicates that the instruction has important
    side-effects.  GCC will not delete a volatile `asm' if it is reachable.
    (The instruction can still be deleted if GCC can prove that
    control-flow will never reach the location of the instruction.)  Note
    that even a volatile `asm' instruction can be moved relative to other
    code, including across jump instructions.  For example, on many targets
    there is a system register which can be set to control the rounding
    mode of floating point operations.  You might try setting it with a
    volatile `asm', like this PowerPC example:

                asm volatile("mtfsf 255,%0" : : "f" (fpenv));
                sum = x + y;

    This will not work reliably, as the compiler may move the addition back
    before the volatile `asm'.  To make it work you need to add an
    artificial dependency to the `asm' referencing a variable in the code
    you don't want moved, for example:

             asm volatile ("mtfsf 255,%1" : "=X"(sum): "f"(fpenv));
             sum = x + y;

     Similarly, you can't expect a sequence of volatile `asm' instructions
    to remain perfectly consecutive.  If you want consecutive output, use a
    single `asm'.  Also, GCC will perform some optimizations across a
    volatile `asm' instruction; GCC does not "forget everything" when it
    encounters a volatile `asm' instruction the way some other compilers do.

     An `asm' instruction without any output operands will be treated
    identically to a volatile `asm' instruction.