Re: [PATCH] Document Linux's memory barriers [try #2]

[Jesse Barnes <jbarnes@virtuousgeek.org>]

On Wednesday, March 08, 2006 5:27 pm, Linus Torvalds wrote:
> That said, when I heard of the NUMA IO issues on the SGI platform, I
> was initially pretty horrified. It seems to have worked out ok, and
> as long as we're talking about machines where you can concentrate on
> validating just a few drivers, it seems to be a good tradeoff.

It's actually not too bad.  We tried hard to make the arch code support 
the semantics that Linux drivers expect.  mmiowb() was an optimization 
we added (though it's much less of an optimization than read_relaxed() 
was) to make things a little faster.  Like you say, the alternative was 
to embed the same functionality into spin_unlock or something (IRIX 
actually had an io_spin_unlock that did that iirc), but that would mean 
an MMIO access on every unlock, which would be bad.

So ultimately mmiowb() is the only thing drivers really have to care 
about on Altix (assuming they do DMA mapping correctly), and the rules 
for that are fairly simple.  Then they can additionally use 
read_relaxed() to optimize performance a bit (quite a bit on big 
systems).

> Would I want the hard-to-think-about IO ordering on a regular desktop
> platform? No.

I guess you don't want anyone to send you an O2 then? :)

Jesse