Re: wmb vs mmiowb

Brent Casavant
2007-08-28T21:21:04Z

On Thu, 23 Aug 2007, Jesse Barnes wrote:

> On Thursday, August 23, 2007 12:27 am Benjamin Herrenschmidt wrote:
> > > Of course, the normal memory barrier would usually be a
> > > "spin_unlock()" or something like that, not a "wmb()". In fact, I
> > > don't think the powerpc implementation (as an example of this) will
> > > actually synchronize with anything *but* a spin_unlock().
> >
> > We are even more sneaky in the sense that we set a per-cpu flag on
> > any MMIO write and do the sync automatically in spin_unlock() :-)
>
> Yeah, that's a reasonable thing to do, and in fact I think there's code
> to do something similar when a task is switched out (this keeps user
> level drivers from having do mmiowb() type things).

Yes there is, git commit e08e6c521355cd33e647b2f739885bc3050eead6.

On SN2 any user process performing memory-mapped IO directly to a
device needs something like mmiowb() to be performed at the node of
the CPU it last ran on when the task context switches onto a new CPU.

The current code performs this action for all inter-CPU context
switches, but we had discussed the possibility of targetting the
action only when the user process has actually mapped a device for
IO. I believe it was decided that this level of complexity wasn't
warranted unless this simple solution was found to cause a problem.

That reminds me. Are the people who are working on the user-level
driver effort including a capability similar to mmiowb()? If we
had that capability we could eventually do away with the change
mentioned above. But that would come after all user-level drivers
were coded to include the mmiowb()-like calls, and existing drivers
which provide mmap() capability directly to hardware go away.

Brent

--
Brent Casavant All music is folk music. I ain't
bcasavan@sgi.com never heard a horse sing a song.
Silicon Graphics, Inc. -- Louis Armstrong