Re: [PATCH] Define wc_wmb, a write barrier for PCI write combining

[Jeremy Higdon <jeremy@sgi.com>]

On Tue, Feb 28, 2006 at 06:20:32PM +0100, Jes Sorensen wrote:
> Roland Dreier wrote:
> >    Jes> Not quite correct as far as I understand it. mmiowb() is
> >    Jes> supposed to guarantee that writes to MMIO space have
> >    Jes> completed before continuing.  That of course covers the
> >    Jes> multi-CPU case, but it should also cover the write-combining
> >    Jes> case.
> >
> >I don't believe this is correct.  mmiowb() does not guarantee that
> >writes have completed -- they may still be pending in a buffer in a
> >bridge somewhere.  The _only_ effect of mmiowb() is to make sure that
> >writes which have been ordered between CPUs using some other mechanism
> >(i.e. a lock) are properly ordered by the rest of the system.  This
> >only has an effect systems like very large ia64 systems, where (as I
> >understand it), writes can pass each other on the way to the PCI bus.
> >In fact, mmiowb() is a NOP on essentially every architecture.
> 
> Hmmmm
> 
> That could be, seems like Jesse agrees that it could all be in the
> pipeline somewhere. Considering Jesse was responsible for mmiowb() I'll
> take his word for it ;-)

Right.  On the Altix, the mmiowb ensures that the write is into a
FIFO on the destination node (node I/O device is attached to), so
that later writes from other nodes will be ordered after it.  But
it doesn't actually force it to the bus.  That's one reason why it's
so much quicker than a using a read for ordering.

jeremy