[U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush

[U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush

[Michal Simek]

From: Siva Durga Prasad Paladugu <siva.durga.paladugu@xilinx.com>

Always disable dcache after the flush operation
The following sequence is advisable while disabling d-cache:
1. disable_dcache() - flushes and disables d-cache
2. invalidate_dcache_all() - invalid any entry that came to the cache
   in the short period after the cache was flushed but before the
   cache got disabled

Signed-off-by: Siva Durga Prasad Paladugu <sivadur@xilinx.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
---

 arch/arm/cpu/armv8/cache_v8.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm/cpu/armv8/cache_v8.c b/arch/arm/cpu/armv8/cache_v8.c
index c5ec5297cd39..2a0492fbef52 100644
--- a/arch/arm/cpu/armv8/cache_v8.c
+++ b/arch/arm/cpu/armv8/cache_v8.c
@@ -128,10 +128,10 @@ void dcache_disable(void)
 	if (!(sctlr & CR_C))
 		return;
 
-	set_sctlr(sctlr & ~(CR_C|CR_M));
-
 	flush_dcache_all();
 	__asm_invalidate_tlb_all();
+
+	set_sctlr(sctlr & ~(CR_C|CR_M));
 }
 
 int dcache_status(void)
-- 
2.3.5

[U-Boot] [PATCH 2/2] armv8: caches: Added routine to set non cacheable region zzz

[Michal Simek]

Added routine mmu_set_region_dcache_behaviour() to set a
particular region as non cacheable.

Define dummy routine for mmu_set_region_dcache_behaviour()
to handle incase of dcache off.

Signed-off-by: Siva Durga Prasad Paladugu <sivadur@xilinx.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
---

 arch/arm/cpu/armv8/cache_v8.c | 23 +++++++++++++++++++++++
 arch/arm/include/asm/system.h | 28 ++++++++++++++++++----------
 2 files changed, 41 insertions(+), 10 deletions(-)

diff --git a/arch/arm/cpu/armv8/cache_v8.c b/arch/arm/cpu/armv8/cache_v8.c
index 2a0492fbef52..b4f6c6b977a2 100644
--- a/arch/arm/cpu/armv8/cache_v8.c
+++ b/arch/arm/cpu/armv8/cache_v8.c
@@ -139,6 +139,24 @@ int dcache_status(void)
 	return (get_sctlr() & CR_C) != 0;
 }
 
+void mmu_set_region_dcache_behaviour(phys_addr_t start, size_t size,
+				     enum dcache_option option)
+{
+	/* get the level2_table0 start address */
+	u64 *page_table = (u64 *)(gd->arch.tlb_addr + 0x3000);
+	u64 upto, end;
+
+	end = ALIGN(start + size, (1 << MMU_SECTION_SHIFT)) >>
+	      MMU_SECTION_SHIFT;
+	start = start >> MMU_SECTION_SHIFT;
+	for (upto = start; upto < end; upto++) {
+		page_table[upto] &= ~PMD_ATTRINDX_MASK;
+		page_table[upto] |= PMD_ATTRINDX(option);
+	}
+
+	flush_dcache_range(page_table[start], page_table[end]);
+	__asm_invalidate_tlb_all();
+}
 #else	/* CONFIG_SYS_DCACHE_OFF */
 
 void invalidate_dcache_all(void)
@@ -170,6 +188,11 @@ int dcache_status(void)
 	return 0;
 }
 
+void mmu_set_region_dcache_behaviour(phys_addr_t start, size_t size,
+				     enum dcache_option option)
+{
+}
+
 #endif	/* CONFIG_SYS_DCACHE_OFF */
 
 #ifndef CONFIG_SYS_ICACHE_OFF
diff --git a/arch/arm/include/asm/system.h b/arch/arm/include/asm/system.h
index 2a5bed2e46b6..c88687860ec1 100644
--- a/arch/arm/include/asm/system.h
+++ b/arch/arm/include/asm/system.h
@@ -15,9 +15,15 @@
 #define CR_EE		(1 << 25)	/* Exception (Big) Endian	*/
 
 #define PGTABLE_SIZE	(0x10000)
+/* 2M granularity */
+#define MMU_SECTION_SHIFT	21
 
 #ifndef __ASSEMBLY__
 
+enum dcache_option {
+	DCACHE_OFF = 0x3,
+};
+
 #define isb()				\
 	({asm volatile(			\
 	"isb" : : : "memory");		\
@@ -211,16 +217,6 @@ enum {
 };
 
 /**
- * Change the cache settings for a region.
- *
- * \param start		start address of memory region to change
- * \param size		size of memory region to change
- * \param option	dcache option to select
- */
-void mmu_set_region_dcache_behaviour(phys_addr_t start, size_t size,
-				     enum dcache_option option);
-
-/**
  * Register an update to the page tables, and flush the TLB
  *
  * \param start		start address of update in page table
@@ -241,4 +237,16 @@ phys_addr_t noncached_alloc(size_t size, size_t align);
 
 #endif /* CONFIG_ARM64 */
 
+#ifndef __ASSEMBLY__
+/**
+ * Change the cache settings for a region.
+ *
+ * \param start		start address of memory region to change
+ * \param size		size of memory region to change
+ * \param option	dcache option to select
+ */
+void mmu_set_region_dcache_behaviour(phys_addr_t start, size_t size,
+				     enum dcache_option option);
+#endif /* __ASSEMBLY__ */
+
 #endif
-- 
2.3.5

[U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush

[Mark Rutland]

On Wed, Apr 15, 2015 at 12:33:00PM +0100, Michal Simek wrote:
> From: Siva Durga Prasad Paladugu <siva.durga.paladugu@xilinx.com>
> 
> Always disable dcache after the flush operation
> The following sequence is advisable while disabling d-cache:
> 1. disable_dcache() - flushes and disables d-cache
> 2. invalidate_dcache_all() - invalid any entry that came to the cache
>    in the short period after the cache was flushed but before the
>    cache got disabled

For reasons I have described previously (see [1,2,3]), this is unsafe.
The first cache flush may achieve nothing.

If you need data out at the PoC before disabling the cache, then you
should first use maintenance by VA to push that data out.

Thanks,
Mark.

[1] http://lists.denx.de/pipermail/u-boot/2015-February/204403.html
[2] http://lists.denx.de/pipermail/u-boot/2015-February/204407.html
[3] http://lists.denx.de/pipermail/u-boot/2015-February/204702.html

> 
> Signed-off-by: Siva Durga Prasad Paladugu <sivadur@xilinx.com>
> Signed-off-by: Michal Simek <michal.simek@xilinx.com>
> ---
> 
>  arch/arm/cpu/armv8/cache_v8.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/arm/cpu/armv8/cache_v8.c b/arch/arm/cpu/armv8/cache_v8.c
> index c5ec5297cd39..2a0492fbef52 100644
> --- a/arch/arm/cpu/armv8/cache_v8.c
> +++ b/arch/arm/cpu/armv8/cache_v8.c
> @@ -128,10 +128,10 @@ void dcache_disable(void)
>  	if (!(sctlr & CR_C))
>  		return;
>  
> -	set_sctlr(sctlr & ~(CR_C|CR_M));
> -
>  	flush_dcache_all();
>  	__asm_invalidate_tlb_all();
> +
> +	set_sctlr(sctlr & ~(CR_C|CR_M));
>  }
>  
>  int dcache_status(void)
> -- 
> 2.3.5
> 
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> http://lists.denx.de/mailman/listinfo/u-boot
> 

[U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush

[Siva Durga Prasad Paladugu]

Hi Mark.

> -----Original Message-----
> From: Mark Rutland [mailto:mark.rutland at arm.com]
> Sent: Wednesday, April 15, 2015 6:41 PM
> To: Michal Simek
> Cc: u-boot at lists.denx.de; Tom Rini; Siva Durga Prasad Paladugu; Varun Sethi;
> Arnab Basu; York Sun
> Subject: Re: [U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush
> 
> On Wed, Apr 15, 2015 at 12:33:00PM +0100, Michal Simek wrote:
> > From: Siva Durga Prasad Paladugu <siva.durga.paladugu@xilinx.com>
> >
> > Always disable dcache after the flush operation The following sequence
> > is advisable while disabling d-cache:
> > 1. disable_dcache() - flushes and disables d-cache 2.
> > invalidate_dcache_all() - invalid any entry that came to the cache
> >    in the short period after the cache was flushed but before the
> >    cache got disabled
> 
> For reasons I have described previously (see [1,2,3]), this is unsafe.
> The first cache flush may achieve nothing.
> 
> If you need data out at the PoC before disabling the cache, then you should
> first use maintenance by VA to push that data out.
> 
> Thanks,
> Mark.
> 
> [1] http://lists.denx.de/pipermail/u-boot/2015-February/204403.html
> [2] http://lists.denx.de/pipermail/u-boot/2015-February/204407.html
> [3] http://lists.denx.de/pipermail/u-boot/2015-February/204702.html
> 
> >
> > Signed-off-by: Siva Durga Prasad Paladugu <sivadur@xilinx.com>
> > Signed-off-by: Michal Simek <michal.simek@xilinx.com>
> > ---
> >
> >  arch/arm/cpu/armv8/cache_v8.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/arch/arm/cpu/armv8/cache_v8.c
> > b/arch/arm/cpu/armv8/cache_v8.c index c5ec5297cd39..2a0492fbef52
> > 100644
> > --- a/arch/arm/cpu/armv8/cache_v8.c
> > +++ b/arch/arm/cpu/armv8/cache_v8.c
> > @@ -128,10 +128,10 @@ void dcache_disable(void)
> >  	if (!(sctlr & CR_C))
> >  		return;
> >
> > -	set_sctlr(sctlr & ~(CR_C|CR_M));
> > -
> >  	flush_dcache_all();
> >  	__asm_invalidate_tlb_all();
> > +
> > +	set_sctlr(sctlr & ~(CR_C|CR_M));

I got your point. But here in this scenario also there is an issue with disable first and then flush_dcache_all().
This is because when we disable the cache and invoke the c routine flush_dcache_all() then the return address of this is stored in a stack(in memory as dcache is disabled).
Now in the flush_dcache_all we are invoking the actual asm call to flush dcache which may wipeout the stored return value in stack with cahe contents(main memory). Hence the return from the flush_dcahe_all will fail.

To confirm this I modified the dcache_disable in the below way and it worked fine.
1. Disable the dcache.
2. Now I called the __asm_flush_dcache_all(); and then flush_l3_cache();  instead of calling the flush_dcache_all().

Regards,
Siva

> >  }
> >
> >  int dcache_status(void)
> > --
> > 2.3.5
> >
> > _______________________________________________
> > U-Boot mailing list
> > U-Boot at lists.denx.de
> > http://lists.denx.de/mailman/listinfo/u-boot
> >

[U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush

[Mark Rutland]

On Thu, Apr 16, 2015 at 06:17:59AM +0100, Siva Durga Prasad Paladugu wrote:
> Hi Mark.
> 
> > -----Original Message-----
> > From: Mark Rutland [mailto:mark.rutland at arm.com]
> > Sent: Wednesday, April 15, 2015 6:41 PM
> > To: Michal Simek
> > Cc: u-boot at lists.denx.de; Tom Rini; Siva Durga Prasad Paladugu; Varun Sethi;
> > Arnab Basu; York Sun
> > Subject: Re: [U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush
> > 
> > On Wed, Apr 15, 2015 at 12:33:00PM +0100, Michal Simek wrote:
> > > From: Siva Durga Prasad Paladugu <siva.durga.paladugu@xilinx.com>
> > >
> > > Always disable dcache after the flush operation The following sequence
> > > is advisable while disabling d-cache:
> > > 1. disable_dcache() - flushes and disables d-cache 2.
> > > invalidate_dcache_all() - invalid any entry that came to the cache
> > >    in the short period after the cache was flushed but before the
> > >    cache got disabled
> > 
> > For reasons I have described previously (see [1,2,3]), this is unsafe.
> > The first cache flush may achieve nothing.
> > 
> > If you need data out at the PoC before disabling the cache, then you should
> > first use maintenance by VA to push that data out.
> > 
> > Thanks,
> > Mark.
> > 
> > [1] http://lists.denx.de/pipermail/u-boot/2015-February/204403.html
> > [2] http://lists.denx.de/pipermail/u-boot/2015-February/204407.html
> > [3] http://lists.denx.de/pipermail/u-boot/2015-February/204702.html
> > 
> > >
> > > Signed-off-by: Siva Durga Prasad Paladugu <sivadur@xilinx.com>
> > > Signed-off-by: Michal Simek <michal.simek@xilinx.com>
> > > ---
> > >
> > >  arch/arm/cpu/armv8/cache_v8.c | 4 ++--
> > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/arch/arm/cpu/armv8/cache_v8.c
> > > b/arch/arm/cpu/armv8/cache_v8.c index c5ec5297cd39..2a0492fbef52
> > > 100644
> > > --- a/arch/arm/cpu/armv8/cache_v8.c
> > > +++ b/arch/arm/cpu/armv8/cache_v8.c
> > > @@ -128,10 +128,10 @@ void dcache_disable(void)
> > >  	if (!(sctlr & CR_C))
> > >  		return;
> > >
> > > -	set_sctlr(sctlr & ~(CR_C|CR_M));
> > > -
> > >  	flush_dcache_all();
> > >  	__asm_invalidate_tlb_all();
> > > +
> > > +	set_sctlr(sctlr & ~(CR_C|CR_M));
> 
> I got your point. But here in this scenario also there is an issue
> with disable first and then flush_dcache_all().  This is because when
> we disable the cache and invoke the c routine flush_dcache_all() then
> the return address of this is stored in a stack(in memory as dcache is
> disabled).

Which is why this sequence cannot be written in C, and needs to be
performed in assembly, without any memory accesses between the write to
the SCTLR and the cache flush.

> Now in the flush_dcache_all we are invoking the actual asm call to
> flush dcache which may wipeout the stored return value in stack with
> cahe contents(main memory). Hence the return from the flush_dcahe_all
> will fail.
> 
> To confirm this I modified the dcache_disable in the below way and it worked fine.
> 1. Disable the dcache.
> 2. Now I called the __asm_flush_dcache_all(); and then flush_l3_cache();  instead of calling the flush_dcache_all().

That also is unsafe; implicit (e.g. stack) accesses at any point after
SCTLR.C is cleared and before flush_l3_cache() has completed may see
stale data, or get overwritten by stale data.

Set/Way ops only flush the CPU-local caches, so you only guarantee that
these are clean (and potentially dirty cache lines for the stack could
be sat in L3 and written back at any time). So your flush_l3_cache()
function might not work.

Per ARMv8 the L3 _must_ respect maintenance by VA, so after disabling
the MMU you can flush the memory region corresponding to your stack (and
any other data you need) by VA to the PoC before executing
flush_l3_cache(), in addition to the Set/Way ops used to empty the
CPU-local caches.

Thanks,
Mark.

[U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush

[Siva Durga Prasad Paladugu]

Hi Mark,

> -----Original Message-----
> From: Mark Rutland [mailto:mark.rutland at arm.com]
> Sent: Thursday, April 16, 2015 3:29 PM
> To: Siva Durga Prasad Paladugu
> Cc: Michal Simek; u-boot at lists.denx.de; Tom Rini; Varun Sethi; Arnab Basu;
> York Sun
> Subject: Re: [U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush
> 
> On Thu, Apr 16, 2015 at 06:17:59AM +0100, Siva Durga Prasad Paladugu
> wrote:
> > Hi Mark.
> >
> > > -----Original Message-----
> > > From: Mark Rutland [mailto:mark.rutland at arm.com]
> > > Sent: Wednesday, April 15, 2015 6:41 PM
> > > To: Michal Simek
> > > Cc: u-boot at lists.denx.de; Tom Rini; Siva Durga Prasad Paladugu;
> > > Varun Sethi; Arnab Basu; York Sun
> > > Subject: Re: [U-Boot] [PATCH 1/2] armv8: caches: Disable dcache
> > > after flush
> > >
> > > On Wed, Apr 15, 2015 at 12:33:00PM +0100, Michal Simek wrote:
> > > > From: Siva Durga Prasad Paladugu <siva.durga.paladugu@xilinx.com>
> > > >
> > > > Always disable dcache after the flush operation The following
> > > > sequence is advisable while disabling d-cache:
> > > > 1. disable_dcache() - flushes and disables d-cache 2.
> > > > invalidate_dcache_all() - invalid any entry that came to the cache
> > > >    in the short period after the cache was flushed but before the
> > > >    cache got disabled
> > >
> > > For reasons I have described previously (see [1,2,3]), this is unsafe.
> > > The first cache flush may achieve nothing.
> > >
> > > If you need data out at the PoC before disabling the cache, then you
> > > should first use maintenance by VA to push that data out.
> > >
> > > Thanks,
> > > Mark.
> > >
> > > [1] http://lists.denx.de/pipermail/u-boot/2015-February/204403.html
> > > [2] http://lists.denx.de/pipermail/u-boot/2015-February/204407.html
> > > [3] http://lists.denx.de/pipermail/u-boot/2015-February/204702.html
> > >
> > > >
> > > > Signed-off-by: Siva Durga Prasad Paladugu <sivadur@xilinx.com>
> > > > Signed-off-by: Michal Simek <michal.simek@xilinx.com>
> > > > ---
> > > >
> > > >  arch/arm/cpu/armv8/cache_v8.c | 4 ++--
> > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/arch/arm/cpu/armv8/cache_v8.c
> > > > b/arch/arm/cpu/armv8/cache_v8.c index c5ec5297cd39..2a0492fbef52
> > > > 100644
> > > > --- a/arch/arm/cpu/armv8/cache_v8.c
> > > > +++ b/arch/arm/cpu/armv8/cache_v8.c
> > > > @@ -128,10 +128,10 @@ void dcache_disable(void)
> > > >  	if (!(sctlr & CR_C))
> > > >  		return;
> > > >
> > > > -	set_sctlr(sctlr & ~(CR_C|CR_M));
> > > > -
> > > >  	flush_dcache_all();
> > > >  	__asm_invalidate_tlb_all();
> > > > +
> > > > +	set_sctlr(sctlr & ~(CR_C|CR_M));
> >
> > I got your point. But here in this scenario also there is an issue
> > with disable first and then flush_dcache_all().  This is because when
> > we disable the cache and invoke the c routine flush_dcache_all() then
> > the return address of this is stored in a stack(in memory as dcache is
> > disabled).
> 
> Which is why this sequence cannot be written in C, and needs to be
> performed in assembly, without any memory accesses between the write to
> the SCTLR and the cache flush.
> 
> > Now in the flush_dcache_all we are invoking the actual asm call to
> > flush dcache which may wipeout the stored return value in stack with
> > cahe contents(main memory). Hence the return from the flush_dcahe_all
> > will fail.
> >
> > To confirm this I modified the dcache_disable in the below way and it
> worked fine.
> > 1. Disable the dcache.
> > 2. Now I called the __asm_flush_dcache_all(); and then flush_l3_cache();
> instead of calling the flush_dcache_all().
> 
> That also is unsafe; implicit (e.g. stack) accesses at any point after SCTLR.C is
> cleared and before flush_l3_cache() has completed may see stale data, or
> get overwritten by stale data.
> 
> Set/Way ops only flush the CPU-local caches, so you only guarantee that
> these are clean (and potentially dirty cache lines for the stack could be sat in
> L3 and written back at any time). So your flush_l3_cache() function might not
> work.
> 
> Per ARMv8 the L3 _must_ respect maintenance by VA, so after disabling the
> MMU you can flush the memory region corresponding to your stack (and any
> other data you need) by VA to the PoC before executing flush_l3_cache(), in
> addition to the Set/Way ops used to empty the CPU-local caches.
Thanks for explanation.
So in that case, the flushing of the required stack or any other data which needs to be flushed should be part of board specific. Am I correct?
If yes, then this disable_dcache() should contain a asm call to a routine() (which might be board specific) after disabling the cache to flush the required data and then flush_dcache_all() followed by flush L3 cache.. 
The current implementation just didn't work.
It worked for me with the changes i mentioned in my last mail as I don't have L3 cache in my case.

Regards,
Siva

> 
> Thanks,
> Mark.

[U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush

[Mark Rutland]

> > > Now in the flush_dcache_all we are invoking the actual asm call to
> > > flush dcache which may wipeout the stored return value in stack with
> > > cahe contents(main memory). Hence the return from the flush_dcahe_all
> > > will fail.
> > >
> > > To confirm this I modified the dcache_disable in the below way and it
> > worked fine.
> > > 1. Disable the dcache.
> > > 2. Now I called the __asm_flush_dcache_all(); and then flush_l3_cache();
> > instead of calling the flush_dcache_all().
> > 
> > That also is unsafe; implicit (e.g. stack) accesses at any point after SCTLR.C is
> > cleared and before flush_l3_cache() has completed may see stale data, or
> > get overwritten by stale data.
> > 
> > Set/Way ops only flush the CPU-local caches, so you only guarantee that
> > these are clean (and potentially dirty cache lines for the stack could be sat in
> > L3 and written back at any time). So your flush_l3_cache() function might not
> > work.
> > 
> > Per ARMv8 the L3 _must_ respect maintenance by VA, so after disabling the
> > MMU you can flush the memory region corresponding to your stack (and any
> > other data you need) by VA to the PoC before executing flush_l3_cache(), in
> > addition to the Set/Way ops used to empty the CPU-local caches.
> Thanks for explanation.
> So in that case, the flushing of the required stack or any other data
> which needs to be flushed should be part of board specific. Am I
> correct?

It could be done in generic code, assuming we know the bounds of memory
which will be used, because maintenance by VA should always work.

Do we know which memory U-Boot might use (e.g. does it all fall within
some static carveout?), or can it dynamically allocate from anywhere in
memory?

> If yes, then this disable_dcache() should contain a asm call to a
> routine() (which might be board specific) after disabling the cache to
> flush the required data and then flush_dcache_all() followed by flush
> L3 cache.. 

You could probably get away with:

* Load the memory bounds that we need to flush into some registers, or
  flush some datastructure containing these to memory.
* In assembly:
  - disable the MMU.
  - flush the PA range(s) we need to use to be able to use C safely.
  - flush by Set/Way to empry the CPU-local caches
* Implementation-specific L3 flushing for anything else.

If we only map a small amount of memory, we could simply flush this by
VA (knowing that this will drain the CPU and L3 caches, without any
special maintenance).

Mark.

[U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush

[Siva Durga Prasad Paladugu]

Hi Mark,

> -----Original Message-----
> From: Mark Rutland [mailto:mark.rutland at arm.com]
> Sent: Friday, April 17, 2015 3:36 PM
> To: Siva Durga Prasad Paladugu
> Cc: Michal Simek; u-boot at lists.denx.de; Tom Rini; Varun Sethi; Arnab Basu;
> York Sun
> Subject: Re: [U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush
>
> > > > Now in the flush_dcache_all we are invoking the actual asm call to
> > > > flush dcache which may wipeout the stored return value in stack
> > > > with cahe contents(main memory). Hence the return from the
> > > > flush_dcahe_all will fail.
> > > >
> > > > To confirm this I modified the dcache_disable in the below way and
> > > > it
> > > worked fine.
> > > > 1. Disable the dcache.
> > > > 2. Now I called the __asm_flush_dcache_all(); and then
> > > > flush_l3_cache();
> > > instead of calling the flush_dcache_all().
> > >
> > > That also is unsafe; implicit (e.g. stack) accesses at any point
> > > after SCTLR.C is cleared and before flush_l3_cache() has completed
> > > may see stale data, or get overwritten by stale data.
> > >
> > > Set/Way ops only flush the CPU-local caches, so you only guarantee
> > > that these are clean (and potentially dirty cache lines for the
> > > stack could be sat in
> > > L3 and written back at any time). So your flush_l3_cache() function
> > > might not work.
> > >
> > > Per ARMv8 the L3 _must_ respect maintenance by VA, so after
> > > disabling the MMU you can flush the memory region corresponding to
> > > your stack (and any other data you need) by VA to the PoC before
> > > executing flush_l3_cache(), in addition to the Set/Way ops used to empty
> the CPU-local caches.
> > Thanks for explanation.
> > So in that case, the flushing of the required stack or any other data
> > which needs to be flushed should be part of board specific. Am I
> > correct?
>
> It could be done in generic code, assuming we know the bounds of memory
> which will be used, because maintenance by VA should always work.
>
> Do we know which memory U-Boot might use (e.g. does it all fall within some
> static carveout?), or can it dynamically allocate from anywhere in memory?
>
> > If yes, then this disable_dcache() should contain a asm call to a
> > routine() (which might be board specific) after disabling the cache to
> > flush the required data and then flush_dcache_all() followed by flush
> > L3 cache..
>
> You could probably get away with:
>
> * Load the memory bounds that we need to flush into some registers, or
>   flush some datastructure containing these to memory.
> * In assembly:
>   - disable the MMU.
>   - flush the PA range(s) we need to use to be able to use C safely.
>   - flush by Set/Way to empry the CPU-local caches
> * Implementation-specific L3 flushing for anything else.
>
> If we only map a small amount of memory, we could simply flush this by VA
> (knowing that this will drain the CPU and L3 caches, without any special
> maintenance).
I just looked at one of the old patch from York in the link below. Can you look at this.
http://lists.denx.de/pipermail/u-boot/2015-January/200514.html

Regards,
DP
>
> Mark.


This email and any attachments are intended for the sole use of the named recipient(s) and contain(s) confidential information that may be proprietary, privileged or copyrighted under applicable law. If you are not the intended recipient, do not read, copy, or forward this email message or any attachments. Delete this email message and any attachments immediately.

[U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush

[Mark Rutland]

> > > Thanks for explanation.
> > > So in that case, the flushing of the required stack or any other data
> > > which needs to be flushed should be part of board specific. Am I
> > > correct?
> >
> > It could be done in generic code, assuming we know the bounds of memory
> > which will be used, because maintenance by VA should always work.
> >
> > Do we know which memory U-Boot might use (e.g. does it all fall within some
> > static carveout?), or can it dynamically allocate from anywhere in memory?
> >
> > > If yes, then this disable_dcache() should contain a asm call to a
> > > routine() (which might be board specific) after disabling the cache to
> > > flush the required data and then flush_dcache_all() followed by flush
> > > L3 cache..
> >
> > You could probably get away with:
> >
> > * Load the memory bounds that we need to flush into some registers, or
> >   flush some datastructure containing these to memory.
> > * In assembly:
> >   - disable the MMU.
> >   - flush the PA range(s) we need to use to be able to use C safely.
> >   - flush by Set/Way to empry the CPU-local caches
> > * Implementation-specific L3 flushing for anything else.
> >
> > If we only map a small amount of memory, we could simply flush this by VA
> > (knowing that this will drain the CPU and L3 caches, without any special
> > maintenance).
> I just looked at one of the old patch from York in the link below. Can you look at this.
> http://lists.denx.de/pipermail/u-boot/2015-January/200514.html

I'm not sure what you're expecting me to say w.r.t. that. Is there a
particular question you have?

Thanks,
Mark.

[U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush

[Siva Durga Prasad Paladugu]

Hi Mark,

> -----Original Message-----
> From: Mark Rutland [mailto:mark.rutland at arm.com]
> Sent: Monday, April 20, 2015 3:48 PM
> To: Siva Durga Prasad Paladugu
> Cc: Michal Simek; u-boot at lists.denx.de; Tom Rini; Varun Sethi; Arnab Basu;
> York Sun
> Subject: Re: [U-Boot] [PATCH 1/2] armv8: caches: Disable dcache after flush
> 
> > > > Thanks for explanation.
> > > > So in that case, the flushing of the required stack or any other
> > > > data which needs to be flushed should be part of board specific.
> > > > Am I correct?
> > >
> > > It could be done in generic code, assuming we know the bounds of
> > > memory which will be used, because maintenance by VA should always
> work.
> > >
> > > Do we know which memory U-Boot might use (e.g. does it all fall
> > > within some static carveout?), or can it dynamically allocate from
> anywhere in memory?
> > >
> > > > If yes, then this disable_dcache() should contain a asm call to a
> > > > routine() (which might be board specific) after disabling the
> > > > cache to flush the required data and then flush_dcache_all()
> > > > followed by flush
> > > > L3 cache..
> > >
> > > You could probably get away with:
> > >
> > > * Load the memory bounds that we need to flush into some registers, or
> > >   flush some datastructure containing these to memory.
> > > * In assembly:
> > >   - disable the MMU.
> > >   - flush the PA range(s) we need to use to be able to use C safely.
> > >   - flush by Set/Way to empry the CPU-local caches
> > > * Implementation-specific L3 flushing for anything else.
> > >
> > > If we only map a small amount of memory, we could simply flush this
> > > by VA (knowing that this will drain the CPU and L3 caches, without
> > > any special maintenance).
> > I just looked at one of the old patch from York in the link below. Can you
> look at this.
> > http://lists.denx.de/pipermail/u-boot/2015-January/200514.html
> 
> I'm not sure what you're expecting me to say w.r.t. that. Is there a particular
> question you have?


I am mentioning that I looked at the patch in the below link which is same as this scenario.. Here the cleaning of cpu caches and L3 caches were modified for asm.
Do you think that we can apply this patch and can live with that? I didn't see any review comments for this patch. 
http://lists.denx.de/pipermail/u-boot/2015-January/200514.html

Regards,
Siva
> 
> Thanks,
> Mark.