Re: [PATCH] Define flush_wc, a way to flush write combining store buffers

["Bryan O'Sullivan" <bos@pathscale.com>]

On Thu, 2006-03-09 at 10:40 +1100, Benjamin Herrenschmidt wrote:

> What bothers me is that because of that exact same argument "it makes
> substantial difference for us", we end up with basically barriers
> tailored for architectures... that is as many kind of barriers as we
> have architectuers... like mmiowb :)

Unfortunately, it does express an untidy facet of reality, which is that
you really *do* want about as many different kinds of barrier as you
have kinds of distinct semantics, so that you have some hope of
expressing fairly portable concepts in a somewhat high-performance way.

I can see four problems at hand.  You can choose the order of their
importance yourself :-)

     1. Linux has a set of barriers that is too small to write
        performant code in a portable manner.
     2. The semantics of the existing barriers are not well understood.
     3. People don't understand when barriers are appropriate in the
        first place.  Witness the current thread on this topic.
     4. If you add more subtle barriers, they'll get misused (see #3)
        and inevitably introduce bugs that are a nightmare to find.

Regarding #2, it's not obvious to lots of people why e.g. the
atomic_*_return kinds of routines need to have memory barrier semantics,
in some cases even after reading Documentation/atomic_ops.txt.  So this
is a nasty education problem.

Regardless, in the case of the ipath driver, we squeeze every cycle out
of the hardware that we can, and this is our selling point.  I'm fine
with a driver-specific hack in this case because it suits my immediate
needs, but it would be nice to have something portable and well-defined
to rely on.

	<b