[PATCH] common: spl: Enable Instruction cache after relocation in board_init_r

[PATCH] common: spl: Enable Instruction cache after relocation in board_init_r

[Prasanth Babu Mantena]

ICACHE is enabled in board_init_f which executes only before relocation.
Instruction cache invalidation is needed after relocation as well in the
common spl, which is taken care in the u-boot init_sequence, but missing
for the spl. So, enable it at the start of board_init_r for spl, which
invalidates icache needed after instruction relocation.

Fixes: 52a86e69e20 ("arm: k3: Enable instruction cache for main domain SPL")
Signed-off-by: Prasanth Babu Mantena <p-mantena@ti.com>
---
 arch/arm/mach-k3/common.c | 1 +
 common/spl/spl.c          | 1 +
 2 files changed, 2 insertions(+)

diff --git a/arch/arm/mach-k3/common.c b/arch/arm/mach-k3/common.c
index fa8cd93d664..e4eb33512e7 100644
--- a/arch/arm/mach-k3/common.c
+++ b/arch/arm/mach-k3/common.c
@@ -266,6 +266,7 @@ static __maybe_unused void k3_dma_remove(void)
 void spl_board_prepare_for_boot(void)
 {
 #if !(defined(CONFIG_SYS_ICACHE_OFF) && defined(CONFIG_SYS_DCACHE_OFF))
+	icache_disable();
 	dcache_disable();
 #endif
 #if IS_ENABLED(CONFIG_SPL_DMA) && IS_ENABLED(CONFIG_SPL_DM_DEVICE_REMOVE)
diff --git a/common/spl/spl.c b/common/spl/spl.c
index 76fd56dfe4b..1db9a258e35 100644
--- a/common/spl/spl.c
+++ b/common/spl/spl.c
@@ -696,6 +696,7 @@ void board_init_r(gd_t *dummy1, ulong dummy2)
 	debug(">>" PHASE_PROMPT "board_init_r()\n");
 
 	spl_set_bd();
+	enable_caches();
 
 	if (IS_ENABLED(CONFIG_SPL_SYS_MALLOC)) {
 		mem_malloc_init(SPL_SYS_MALLOC_START, SPL_SYS_MALLOC_SIZE);
-- 
2.34.1

Re: [PATCH] common: spl: Enable Instruction cache after relocation in board_init_r

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 921 bytes --]

On Mon, Mar 17, 2025 at 12:15:07PM +0530, Prasanth Babu Mantena wrote:

> ICACHE is enabled in board_init_f which executes only before relocation.
> Instruction cache invalidation is needed after relocation as well in the
> common spl, which is taken care in the u-boot init_sequence, but missing
> for the spl. So, enable it at the start of board_init_r for spl, which
> invalidates icache needed after instruction relocation.
> 
> Fixes: 52a86e69e20 ("arm: k3: Enable instruction cache for main domain SPL")
> Signed-off-by: Prasanth Babu Mantena <p-mantena@ti.com>
> ---
>  arch/arm/mach-k3/common.c | 1 +
>  common/spl/spl.c          | 1 +
>  2 files changed, 2 insertions(+)

What's missing from spl_enable_cache() in K3 already? And looking more
at this, since Rockchip does this slightly differently I wonder if we
need to think harder about making some of these hook points generic.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH] common: spl: Enable Instruction cache after relocation in board_init_r

[Prasanth Mantena]

On 09:13, Tom Rini wrote:
> On Mon, Mar 17, 2025 at 12:15:07PM +0530, Prasanth Babu Mantena wrote:
> 
> > ICACHE is enabled in board_init_f which executes only before relocation.
> > Instruction cache invalidation is needed after relocation as well in the
> > common spl, which is taken care in the u-boot init_sequence, but missing
> > for the spl. So, enable it at the start of board_init_r for spl, which
> > invalidates icache needed after instruction relocation.
> > 
> > Fixes: 52a86e69e20 ("arm: k3: Enable instruction cache for main domain SPL")
> > Signed-off-by: Prasanth Babu Mantena <p-mantena@ti.com>
> > ---
> >  arch/arm/mach-k3/common.c | 1 +
> >  common/spl/spl.c          | 1 +
> >  2 files changed, 2 insertions(+)
> 
> What's missing from spl_enable_cache() in K3 already? And looking more
> at this, since Rockchip does this slightly differently I wonder if we
> need to think harder about making some of these hook points generic.

spl_enable_cache is getting called in the board_init_f and the
enable_cache here is getting called in the board_init_r, which is
basically doing the cache invalidate after the relocation. If this is
the case, I doubt, if we have to enable this directly in board_init_r
only, instead doing it in board_init_f.

Regards,
Prasanth


> 
> -- 
> Tom

Re: [PATCH] common: spl: Enable Instruction cache after relocation in board_init_r

[Ernest Van Hoecke]

On Tue, Mar 25, 2025 at 11:04:20AM +0530, Prasanth Mantena wrote:
> On 09:13, Tom Rini wrote:
> > On Mon, Mar 17, 2025 at 12:15:07PM +0530, Prasanth Babu Mantena wrote:
> > 
> > > ICACHE is enabled in board_init_f which executes only before relocation.
> > > Instruction cache invalidation is needed after relocation as well in the
> > > common spl, which is taken care in the u-boot init_sequence, but missing
> > > for the spl. So, enable it at the start of board_init_r for spl, which
> > > invalidates icache needed after instruction relocation.
> > > 
> > > Fixes: 52a86e69e20 ("arm: k3: Enable instruction cache for main domain SPL")
> > > Signed-off-by: Prasanth Babu Mantena <p-mantena@ti.com>
> > > ---
> > >  arch/arm/mach-k3/common.c | 1 +
> > >  common/spl/spl.c          | 1 +
> > >  2 files changed, 2 insertions(+)
> > 
> > What's missing from spl_enable_cache() in K3 already? And looking more
> > at this, since Rockchip does this slightly differently I wonder if we
> > need to think harder about making some of these hook points generic.
> 
> spl_enable_cache is getting called in the board_init_f and the
> enable_cache here is getting called in the board_init_r, which is
> basically doing the cache invalidate after the relocation. If this is
> the case, I doubt, if we have to enable this directly in board_init_r
> only, instead doing it in board_init_f.

Hi all,

We have seen intermittent boot failures on the AM69, fixed by applying
this patch. How can we fix this up to get it upstream? Thanks for your
efforts here.

Kind regards,
Ernest

Re: [PATCH] common: spl: Enable Instruction cache after relocation in board_init_r

[Kumar, Udit]

On 11/26/2025 5:04 PM, Ernest Van Hoecke wrote:
> On Tue, Mar 25, 2025 at 11:04:20AM +0530, Prasanth Mantena wrote:
>> On 09:13, Tom Rini wrote:
>>> On Mon, Mar 17, 2025 at 12:15:07PM +0530, Prasanth Babu Mantena wrote:
>>>
>>>> ICACHE is enabled in board_init_f which executes only before relocation.
>>>> Instruction cache invalidation is needed after relocation as well in the
>>>> common spl, which is taken care in the u-boot init_sequence, but missing
>>>> for the spl. So, enable it at the start of board_init_r for spl, which
>>>> invalidates icache needed after instruction relocation.
>>>>
>>>> Fixes: 52a86e69e20 ("arm: k3: Enable instruction cache for main domain SPL")
>>>> Signed-off-by: Prasanth Babu Mantena <p-mantena@ti.com>
>>>> ---
>>>>   arch/arm/mach-k3/common.c | 1 +
>>>>   common/spl/spl.c          | 1 +
>>>>   2 files changed, 2 insertions(+)
>>> What's missing from spl_enable_cache() in K3 already? And looking more
>>> at this, since Rockchip does this slightly differently I wonder if we
>>> need to think harder about making some of these hook points generic.
>> spl_enable_cache is getting called in the board_init_f and the
>> enable_cache here is getting called in the board_init_r, which is
>> basically doing the cache invalidate after the relocation. If this is
>> the case, I doubt, if we have to enable this directly in board_init_r
>> only, instead doing it in board_init_f.

I think, you just need to invalidate D-cache only


> Hi all,
>
> We have seen intermittent boot failures on the AM69, fixed by applying
> this patch. How can we fix this up to get it upstream? Thanks for your
> efforts here.
>
> Kind regards,
> Ernest

Thanks Ernest for reviving this patch ,

Changes in arch/arm/mach-k3/common.c seems good to me.

but common/spl/spl.c can not pushed in generic way, I can think of few 
platforms running SPL w/o caches on.

Re: [PATCH] common: spl: Enable Instruction cache after relocation in board_init_r

[Ernest Van Hoecke]

On Wed, Nov 26, 2025 at 06:34:15PM +0530, Kumar, Udit wrote:
> Thanks Ernest for reviving this patch ,
> 
> Changes in arch/arm/mach-k3/common.c seems good to me.
> 
> but common/spl/spl.c can not pushed in generic way, I can think of few
> platforms running SPL w/o caches on.
> 

Hi Udit,

Thanks for your prompt reply and input. I have now tested this patch but
with the enable_caches in common/spl/spl.c dropped, and everything looks
great to me. I will continue running the tests over the weekend, but can
confirm that this change seems unnecessary to fix the issue we saw with
sporadic boot failures.

A v2 that only touches mach-k3/common.c would be suitable. If you
prefer, I have a patch ready that we can send. I don't believe much
functionality is lost by simply dropping the spl.c line, platforms can
enable caches themselves as they want.

Kind regards,
Ernest

Re: [PATCH] common: spl: Enable Instruction cache after relocation in board_init_r

[Kumar, Udit]

On 11/29/2025 12:54 AM, Ernest Van Hoecke wrote:
> On Wed, Nov 26, 2025 at 06:34:15PM +0530, Kumar, Udit wrote:
>> Thanks Ernest for reviving this patch ,
>>
>> Changes in arch/arm/mach-k3/common.c seems good to me.
>>
>> but common/spl/spl.c can not pushed in generic way, I can think of few
>> platforms running SPL w/o caches on.
>>
> Hi Udit,
>
> Thanks for your prompt reply and input. I have now tested this patch but
> with the enable_caches in common/spl/spl.c dropped, and everything looks
> great to me. I will continue running the tests over the weekend, but can
> confirm that this change seems unnecessary to fix the issue we saw with
> sporadic boot failures.

Thanks again Ernest,

Ideally, I prefer to revert commit sha 52a86e69e20.

Adding more TI folks for any objection on reverting sha 52a86e69e20

>
> A v2 that only touches mach-k3/common.c would be suitable. If you
> prefer, I have a patch ready that we can send. I don't believe much

Unfortunately, only change in mach-k3/common.c will not be enough to fix 
the issue.

we need to clean-caches at board_init_r stage, and which is common code.


> functionality is lost by simply dropping the spl.c line, platforms can
> enable caches themselves as they want.
>
> Kind regards,
> Ernest

Re: [PATCH] common: spl: Enable Instruction cache after relocation in board_init_r

[Ernest Van Hoecke]

On Sat, Nov 29, 2025 at 09:56:44AM +0530, Kumar, Udit wrote:
> 
> On 11/29/2025 12:54 AM, Ernest Van Hoecke wrote:
> > On Wed, Nov 26, 2025 at 06:34:15PM +0530, Kumar, Udit wrote:
> > > Thanks Ernest for reviving this patch ,
> > > 
> > > Changes in arch/arm/mach-k3/common.c seems good to me.
> > > 
> > > but common/spl/spl.c can not pushed in generic way, I can think of few
> > > platforms running SPL w/o caches on.
> > > 
> > Hi Udit,
> > 
> > Thanks for your prompt reply and input. I have now tested this patch but
> > with the enable_caches in common/spl/spl.c dropped, and everything looks
> > great to me. I will continue running the tests over the weekend, but can
> > confirm that this change seems unnecessary to fix the issue we saw with
> > sporadic boot failures.
> 
> Thanks again Ernest,
> 
> Ideally, I prefer to revert commit sha 52a86e69e20.
> 
> Adding more TI folks for any objection on reverting sha 52a86e69e20
> 
> > 
> > A v2 that only touches mach-k3/common.c would be suitable. If you
> > prefer, I have a patch ready that we can send. I don't believe much
> 
> Unfortunately, only change in mach-k3/common.c will not be enough to fix the
> issue.
> 
> we need to clean-caches at board_init_r stage, and which is common code.
> 

Thanks Kumar, I'm assuming the issue is that there are still a number of
instructions executing after the SPL relocation until we finally disable
icache in spl_board_prepare_for_boot?

With 50k boots over the weekend I can confirm that, for the AM69A with
our test setup here on a TI AM69 SDK, the issue did not occur again.

I agree that we might need to find a better place to invalidate the
caches if necessary, but commit 52a86e69e20 was necessary to not have a
1.3 second delay between the A53 SPL and A53 U-Boot on the AM62 as
reported on E2E. [1]

[1] https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1217839/am625-boot-time-between-a53-spl-and-a53-u-boot

Thanks for your help here.

Kind regards,
Ernest