Re: I/O write ordering

[James Bottomley <James.Bottomley@SteelEye.com>]

On Wed, 2004-09-22 at 21:58, Matthew Wilcox wrote:
> On Thu, Sep 23, 2004 at 10:19:57AM +1000, Benjamin Herrenschmidt wrote:
> > And that wouldn't help vs. spinlocks which are in real RAM
> > (we have a similar issue with IO leaking potentially out of locks, 
> > if we don't use some heavy barriers there too).
> 
> Yep, that was the problem that SGI wanted to solve.

Yes, but the ppc introduces an additional issue with the consistent
memory problem.

The io memory barrier is used to prevent writes to PCI space leaking
around synchronisation primitives.  However, there's also the ordering
of writes to consistent memory vs writes to PCI space.

The ownership model for consistent memory solves this.

Think of it this way: writes to consistent memory are normal memory
writes (they don't trigger actions in the device), it's the writes to
the device space that trigger actions.  All you want to ensure is that
all the consistent memory writes are issued before the device is
triggered into action.

So, in the ownership model you transfer the ownership of the consistent
memory area to the device (which would be the heavyweight barrier on
ppc) then write to the device space to trigger it.

synchronisation of consistent memory accesses can be done via normal
spin locks; synchronisation of device space needs the the iomb() to
prevent reordering leaks around the spinlock.

James