Re: [RFC] MMIO accessors & barriers documentation

[Benjamin Herrenschmidt <benh@kernel.crashing.org>]

> ioremap_wc is actually the easy half.  I have an old patch that handles
> that.  The trick is to make certain multiple people don't map the same
> thing with different attributes.  Unfortunately I haven't had time to
> work through that one yet.

Actually, that's interesting because I need the exactly oposite on
PowerPC I think.... That is people will -need- to do both a wc and a
non-wc mapping if they want to be able to issue stores that are
guaranteed not to be combined.

The problem I've seen is that at least one processor (the Cell) and
maybe more seem to be combining between threads on the same CPU (unless
the stores are issues to a guarded mapping which prevents combining
completely, that is the sort of mapping we currently do with ioremap).

That means that it's impossible to prevent combining with explicit
barriers. For example:

Thread 0          Thread 1
store to A        store to A+1
barrier           barrier
  \                / 
   \             /
    \          /
   Store unit might sees:
      store to A
      store to A+1
      barrier
      barrier

That is the stores aren't tagged with their source thread and thus the
non cacheable store unit will not prevent combining between them.

Again, it might just be a Cell CPU bug in which case we may have to just
disable use of WC on that processor, period. But it might be a more
generic problem too, we need to investigate.

If the problem ends up being widespread, the only ways I see to prevent
the combining from happening are to do a dual mapping as I explained
earlier, or maybe to have drivers always do the stores that must not be
combine as part of spinlocks, with appropriate use of
io_to_lock_barrier() (mmiowb()).

Anyway, let's not pollute this discussion with that too much now :)

Ben.