Re: cacheble to uncachble change

[David Mosberger <davidm@napali.hpl.hp.com>]

>>>>> On Tue, 27 Apr 2004 09:24:41 -0500, Jack Steiner <steiner@sgi.com> said:

  Jack> On Tue, Apr 27, 2004 at 05:52:28AM -0500, Robin Holt wrote:
  >> On Mon, Apr 26, 2004 at 05:03:23PM -0700, David Mosberger wrote:
  >> > >>>>> On Mon, 26 Apr 2004 16:35:55 -0500, Robin Holt <holt@sgi.com> said:
  >> > Are you just re-stating my caveat about memory-attribute-aliasing or
  >> > are you saying something else?  If the latter, I'm not following.  If
  >> > the former, I certainly agree: memory attribute-aliasing leads to
  >> > really nasty-to-track-down bugs.  Hence, you want to make sure
  >> > _upfront_ that it doesn't occur.

  >> Restating.  Don't you love the person who plays the master of the obvious
  >> role.  I started writing the email and was at the same time looking for
  >> examples of kernel code from 2.4 which we had found that was speculating

  Jack> IIRC, one place that got us in trouble in 2.4 was in
  Jack> free_one_pgd(). The code prefetches a dirty cacheline that is
  Jack> one cache line BEYOND the end of the PT page. The line is
  Jack> marked dirty (prefetchw()) in the cache even though the
  Jack> function does not actually modify it. The line will
  Jack> subsequently be written back to memory. If the following page
  Jack> is in the same granule & is being used uncached
  Jack> (memory-attribute-aliasing), bad things will happen......

Yes, of course.  Violating a correctness requirement (no
memory-attribute aliasing) can cause bad things to happen.  Duh.

I don't mean this as a criticism.  I only want to be clear that the
bug here was caused by ignoring a correctness requirement ("no
attribute aliasing").  The kernel is perfectly entitled to prefetch
any address of its choosing.  In fact, even without explicit data
prefetches, you'll get implicit code prefetching (not to mention
speculative loads when using a modern compiler).

The moral of the story: we're not kidding when we say memory-attribute
aliasing needs to be taken seriously.  (And it's not an ia64-only
concern, as the nasty AGP-related attribute-aliasing bug revealed on
Linux for certain x86 CPUs.)

	--david