[PATCH V2 1/5] arm: mvebu: Added support for coherency fabric in mach-mvebu

[gregory.clement@free-electrons.com (Gregory CLEMENT)]

On 11/15/2012 05:21 PM, Will Deacon wrote:
> Hi Gregory,
> 
> On Thu, Nov 15, 2012 at 03:54:39PM +0000, Gregory CLEMENT wrote:
>> On 11/15/2012 11:17 AM, Will Deacon wrote:
>>> Interesting, thanks for asking them about this. Does this mean that:
>>
>> Here come the answers to your new questions
> 
> Great, thanks for the quick turn-around!
> 
>>> 	1. When not running coherently (i.e. before initialising the
>>> 	   coherency fabric), memory is treated as non-shareable,
>>> 	   non-cacheable?
>>
>> It can be cacheable. The shared memory (as defined on the page table)
>> will NOT be coherent by HW.
> 
> Ok, so we really are incoherent before enabling the fabric.
> 
>>> 	2. If (1), then are exclusive accesses the only way to achieve
>>> 	   coherent memory accesses in this scenario?
>>
>> I quote: "I suspect there is terminology miss-use: exclusive accesses
>> are NOT used to achieve memory coherency - they are used to achieve
>> atomicity. To achieve memory coherency while fabric is configured to
>> be non-coherent, SW should use maintenance operations over the L1
>> caches."
> 
> Ok, so if I'm understanding correctly then I don't really see the usefulness
> of having working exclusives that are incoherent. Surely it means that you
> can guarantee mutual exclusion on a lock variable, but the value you actually
> end up reading from the lock is junk unless you litter the accessors with cache
> clean operations?
> 
> Anyway, that's by-the-by as this is all called early enough that we
> shouldn't care. The thing I don't like now is that the fabric initialisation
> is done entirely differently on the primary CPU than the secondaries. The
> primary probes the device-tree (well, it's also now hard-coded for v2) and
> accesses the registers from a C function(armada_370_xp_set_cpu_coherent) whilst
> the secondaries have hardcoded addresses and access via asm
> (armada_xp_secondary_startup).


Now it is hardcoded in both case as you pointed it. So the last
difference is setup from a C function or via asm.

The differences between primary and secondary CPU when they enable the
coherency, is due to the fact that we really are in a different
situation. For primary CPU, as it is the only CPU online it doesn't
need to enable the coherency from the beginning, so we can wait to
have MMU enable and convenient feature. Whereas for the secondary CPU
they need the coherency from the very beginning are by definition they
won't be alone. That's why this very first instruction are written in
asm and they use physical address.

I don't see how to handle it in a different way.

Gregory