Re: RFH: What are the semantics of writeb() and friends?

[Alan Cox <alan@lxorguk.ukuu.org.uk>]

On Gwe, 2005-07-01 at 10:33, Maciej W. Rozycki wrote:
> > al.?  I would assume that the effects of these must be in the same order 
> > that they were issued, and that any hardware write back queue cannot 
> > combine or merge them in any way.  Is that correct?
> 
>  No it's not.  You need to insert appropriate barriers, one of: wmb(), 
> mb() or rmb().  In rare cases you may need to use iob(), which is 
> currently non-portable (which reminds me I should really push it 
> upstream).

Its even more complicated than that 8)

writeb/writel may be merged in some cases (but not re-ordered) for I/O
devices but a simple mb() will only synchronize them as viewed from
cpu/memory interface. There are two other synchronization points. From
the bridge with the I/O device (typically the PCI root bridge) which is
not enforced automatically across processors on some large numa boxes
but is not usually a problem and on the PCI bus itself.

PCI permits posting (delaying writes) and some forms of merging (but not
re-ordering). Thus if you need an I/O to hit a device on the PCI bus and
know it arrived you must follow it by a read from the same device. So
for example if you want to shut down a DMA transfer and free the buffer
for a PCI device you
need to do

		writel(TURN_DMA_OFF, dev->control);
		readl(dev->something);
		/* Only now is the free safe */

Alan