[U-Boot-Users] PPC sync/eieio (was cfi_flash.c and lost volatile qualifier)

[Jerry Van Baren <gerald.vanbaren@ge.com>]

Joakim Tjernlund wrote:
>> [1] Sync is a big hammer, eieio is a medium size hammer.  Don't use 
>> both, PPC people will know you don't know what you are doing.  ;-)
> 
> Yet the in_bex()/out_bex() functions in PowerPC linux uses sync and all
> SOC drivers are encouraged to use them. What a waste :(
> 
>  Jocke

Well, I was a little terse because I was cross-applying PPC instructions 
in a ARM discussion.  Personally, I prefer to use sync vs. eieio.  The 
size of the hammer isn't that different, I don't believe.  The advantage 
of sync is that it flushes the read/write operation out on the bus 
*now*.  When I'm writing to hardware to control the hardware, *now* is 
what I want.

The eieio merely guarantees the preceding operations will go out on the 
bus before following bus operations.  The preceding operations could be 
hung up in the bus interface unit *indefinitely* if no "following" bus 
operations occur.  This is an unlikely occurrence, but could be the 
result of running out of cache in a tight loop.

For instance, if you do a write to a hardware register, a eieio, and 
then wait in a tight loop until time goes by (reading the decrementer 
register) followed by another write, the write1/delay/write2 sequence 
could actually be delay/write1/write2.  Note that the order of the 
writes on the bus is correct per the eieio, but it isn't what the 
hardware *needed*.

Illustration - what you did in code and intended to occur:
   write1 (eieio)
   delay
   write2 (eieio)

What actually may happen on the bus:
   delay
   write1 (eieio)
   write2 (eieio)

By using a sync, you guarantee the write isn't delayed:
   write1 (sync)
   delay
   write2 (sync)

Disclaimer: the above explanation is from my fertile imagination.  It 
may or may not happen and *will* happen differently on different PPC 
processors.  For instance, the eieio instruction is actually a NOP in 
the 603e core because it doesn't reorder bus operations, but it *does* 
have a BIU that can buffer and delay the bus operations, causing the 
above timing problem.

I contend using the "sync" instruction will always work correctly and 
the "eieio" instruction is at best a false economy and at worst a lot of 
very difficult, mysterious bugs to find, so I'm in agreement with the 
linux in_bex/out_bex recommendation.

Side note: I don't know if I communicated it properly, but when you see 
"eieio ; sync" or "sync ; eieio", you know the author of that code 
doesn't understand sync and eieio.  "isync ; sync" is occasionally a 
valid combination, but I don't believe it is necessary other than when 
called for by the Users Manual in conjunction with writing to special 
purpose registers.

Best regards,
gvb