[U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush

[Mark Rutland <mark.rutland@arm.com>]

> > > Now in the flush_dcache_all we are invoking the actual asm call to
> > > flush dcache which may wipeout the stored return value in stack with
> > > cahe contents(main memory). Hence the return from the flush_dcahe_all
> > > will fail.
> > >
> > > To confirm this I modified the dcache_disable in the below way and it
> > worked fine.
> > > 1. Disable the dcache.
> > > 2. Now I called the __asm_flush_dcache_all(); and then flush_l3_cache();
> > instead of calling the flush_dcache_all().
> > 
> > That also is unsafe; implicit (e.g. stack) accesses at any point after SCTLR.C is
> > cleared and before flush_l3_cache() has completed may see stale data, or
> > get overwritten by stale data.
> > 
> > Set/Way ops only flush the CPU-local caches, so you only guarantee that
> > these are clean (and potentially dirty cache lines for the stack could be sat in
> > L3 and written back at any time). So your flush_l3_cache() function might not
> > work.
> > 
> > Per ARMv8 the L3 _must_ respect maintenance by VA, so after disabling the
> > MMU you can flush the memory region corresponding to your stack (and any
> > other data you need) by VA to the PoC before executing flush_l3_cache(), in
> > addition to the Set/Way ops used to empty the CPU-local caches.
> Thanks for explanation.
> So in that case, the flushing of the required stack or any other data
> which needs to be flushed should be part of board specific. Am I
> correct?

It could be done in generic code, assuming we know the bounds of memory
which will be used, because maintenance by VA should always work.

Do we know which memory U-Boot might use (e.g. does it all fall within
some static carveout?), or can it dynamically allocate from anywhere in
memory?

> If yes, then this disable_dcache() should contain a asm call to a
> routine() (which might be board specific) after disabling the cache to
> flush the required data and then flush_dcache_all() followed by flush
> L3 cache.. 

You could probably get away with:

* Load the memory bounds that we need to flush into some registers, or
  flush some datastructure containing these to memory.
* In assembly:
  - disable the MMU.
  - flush the PA range(s) we need to use to be able to use C safely.
  - flush by Set/Way to empry the CPU-local caches
* Implementation-specific L3 flushing for anything else.

If we only map a small amount of memory, we could simply flush this by
VA (knowing that this will drain the CPU and L3 caches, without any
special maintenance).

Mark.