Re: [patch] linux 2.4.17: An mb() rework

[Dominic Sweetman <dom@algor.co.uk>]

> > > No, a sync will still empty the write buffer. It may halt the
> > >pipeline for many (~80 perhaps) cycles while posted write data is
> > >dumped to the system controller.
> > 
> > Then it's an implementation bug.  For a CPU in the UP mode "sync"
> > is only defined to prevent write and read reordering -- there is
> > nothing about flushing.
> 
> Did some more research + phoning.. RM7K is definately documented to dump
> the write buffer on 'sync'.

The RM7000 sync does more than is required by the ISA.  However, since
the write-buffer is not part of the architecture, this is not a bug.
Though it might well be held to be undesirable.

And there has to be *some* way to force the write buffer to empty, and
this is cheaper than the standard alternative of an uncached read.

I believe the RM7000 write buffer can hold one pending cache-line
writeback or up to four uncached writes.  Emptying the write buffer
can only occupy more than a handful of cycles when the system
controller is already backed-up with pending writes; under those
circumstances the system was probably about to stall anyway, so I
wouldn't be too worried about the performance implications of a
'sync'.

> The RM7K guide even has an example (7.8.5)
> where it implies that sync also forces a write back of any dirty cache
> lines - gah! (Hard to belive though..)

There's not a word of truth in that (there just aren't the right wires
in the hardware to make that possible).  But I think what it means is
that it stops the CPU until a pending cache line write back is
complete.  It's hard to see why that would be useful, but since it
uses the same write buffer hardware it's easy to see why it would happen.

Of course (as everyone has been piling in and pointing out) the real
problems are:

1. The MIPS people who invented 'sync' had a much more sophisticated
   understanding of read/write behaviour than is common in those 
   who support, document and program uniprocessor "embedded" CPUs; so
   there's lots of scope for misunderstanding 'sync'.

2. When write-posting bites you, it's usually not in the CPU but
   somewhere further down the system.  You should never believe a write
   has actually arrived at the device you sent it to, until you see
   the device itself change state...

And a plea: the word "flush" in Linux is already abused for caches
(where it means invalidate, write back, or possibly both).  That's
enough trouble: if you also "flush" write buffers, your confusion will
be complete.  Maybe we should leave "flush" to plumbers and use more
precise terminology for computers...

-- 
Dominic Sweetman, 
Algorithmics Ltd
The Fruit Farm, Ely Road, Chittering, CAMBS CB5 9PH, ENGLAND
phone: +44 1223 706200 / fax: +44 1223 706250 / direct: +44 1223 706205
http://www.algor.co.uk