Re: Opinion on ordering of writel vs. stores to RAM

[Jeff Garzik <jeff@garzik.org>]

David Miller wrote:
> From: Paul Mackerras <paulus@samba.org>
> Date: Sat, 9 Sep 2006 13:02:27 +1000
> 
>> I suspect the best thing at this point is to move the sync in writeX()
>> before the store, as you suggest, and add an "eieio" before the load
>> in readX().  That does mean that we are then relying on driver writers
>> putting in the mmiowb() between a writeX() and a spin_unlock, but at
>> least that is documented.
> 
> I think not matching what PC systems do is, at least from one
> perspective, a very bad engineering decision for 2 reasons.
> 
> 1) You will be chasing down these kinds of problems forever,
>    you will fix tg3 today, but tomorrow it will be another driver
>    for which you will invest weeks of delicate debugging that
>    could have been spent on much more useful coding
> 
> 2) Driver authors will not get these memory barriers right,
>    you can say they will because it will be "documented" but
>    that does not change reality which is that driver folks
>    will get simple interfaces right but these memory barriers
>    are relatively advanced concepts, which they thus will get
>    wrong half the time
> 
> Sure it's more expensive, but at least on sparc64 I'd much rather
> spend my time working on more interesting things than "today's
> missing memory barrier" :-)
> 
> I also don't want to see all of these memory barriers crapping up our
> drivers.  I do a MMIO, then I access a descriptor, or vice versa, then
> those should be ordered because they are both technically accesses to
> "physical device state".  Having to say this explicitly seems really
> the wrong thing to do, at least to me.

Agreed.

As (I think) BenH mentioned in another email, the normal way Linux 
handles these interfaces is for the primary API (readX, writeX) to be 
strongly ordered, strongly coherent, etc.  And then there is a relaxed 
version without barriers and syncs, for the smart guys who know what 
they're doing.  We used do this for fbdev drivers, I dunno what happened 
to that interface.

We want to make it tough for driver writers to screw up, unless they are 
really trying...

	Jeff