From mboxrd@z Thu Jan  1 00:00:00 1970
From: Mark Rutland <mark.rutland@arm.com>
Date: Fri, 17 Apr 2015 11:06:11 +0100
Subject: [U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush
In-Reply-To: <9320328e-0daa-4843-a899-8f39aab2e0cf@BY2FFO11FD017.protection.gbl>
References: <b57c5d00d9ca21c0d00a269010aea7860832e70e.1429097578.git.michal.simek@xilinx.com>
 <20150415131058.GG2866@leverpostej>
 <1169d5ee-ade7-4759-b59c-0e7a17f6652b@BN1AFFO11FD021.protection.gbl>
 <20150416095836.GI2866@leverpostej>
 <9320328e-0daa-4843-a899-8f39aab2e0cf@BY2FFO11FD017.protection.gbl>
Message-ID: <20150417100611.GA21904@leverpostej>
List-Id: <u-boot.lists.denx.de>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
To: u-boot@lists.denx.de

> > > Now in the flush_dcache_all we are invoking the actual asm call to
> > > flush dcache which may wipeout the stored return value in stack with
> > > cahe contents(main memory). Hence the return from the flush_dcahe_all
> > > will fail.
> > >
> > > To confirm this I modified the dcache_disable in the below way and it
> > worked fine.
> > > 1. Disable the dcache.
> > > 2. Now I called the __asm_flush_dcache_all(); and then flush_l3_cache();
> > instead of calling the flush_dcache_all().
> > 
> > That also is unsafe; implicit (e.g. stack) accesses at any point after SCTLR.C is
> > cleared and before flush_l3_cache() has completed may see stale data, or
> > get overwritten by stale data.
> > 
> > Set/Way ops only flush the CPU-local caches, so you only guarantee that
> > these are clean (and potentially dirty cache lines for the stack could be sat in
> > L3 and written back at any time). So your flush_l3_cache() function might not
> > work.
> > 
> > Per ARMv8 the L3 _must_ respect maintenance by VA, so after disabling the
> > MMU you can flush the memory region corresponding to your stack (and any
> > other data you need) by VA to the PoC before executing flush_l3_cache(), in
> > addition to the Set/Way ops used to empty the CPU-local caches.
> Thanks for explanation.
> So in that case, the flushing of the required stack or any other data
> which needs to be flushed should be part of board specific. Am I
> correct?

It could be done in generic code, assuming we know the bounds of memory
which will be used, because maintenance by VA should always work.

Do we know which memory U-Boot might use (e.g. does it all fall within
some static carveout?), or can it dynamically allocate from anywhere in
memory?

> If yes, then this disable_dcache() should contain a asm call to a
> routine() (which might be board specific) after disabling the cache to
> flush the required data and then flush_dcache_all() followed by flush
> L3 cache.. 

You could probably get away with:

* Load the memory bounds that we need to flush into some registers, or
  flush some datastructure containing these to memory.
* In assembly:
  - disable the MMU.
  - flush the PA range(s) we need to use to be able to use C safely.
  - flush by Set/Way to empry the CPU-local caches
* Implementation-specific L3 flushing for anything else.

If we only map a small amount of memory, we could simply flush this by
VA (knowing that this will drain the CPU and L3 caches, without any
special maintenance).

Mark.