[U-Boot] [issue report] mainline u-boot was stuck before booting kernel

[Ziyuan Xu <xzy.xu@rock-chips.com>]

Hi Sandy,

I'm pleased to learn that it does fix for you.

On 2016?07?26? 21:16, Sandy Patterson wrote:
> Hi Ziyuan, good work!
>
> This does in fact fix it for me.
>
> I followed through your notes. What is isr() supposed to do? it looks 
> like it just tells the compiler that the memory cache has been 
> invalidated. But in the
isb()?
I'm not very enrich knowledgeable in cache, this issue I'm doing also is 
a big learning experience for me. I catch something from TRM, and review 
u-boot again and again in order to figure out the root cause. I suspect 
the former disable_dcache/mmu operation didn't complete, or branch 
prediction was abnormal?
I'm not really sure.

> TRM it says we need to actually do an ISB.
>
> Would fixing the dcache flush so it does an ISB when it disables the 
> MMU maybe be an alternative solution? (I couldn't figure out which 
> macro I should be using. isb() is defined for arm64 to do the actual op.
>
> Anyway, the following also fixes for me.
>
> diff --git a/arch/arm/lib/cache-cp15.c b/arch/arm/lib/cache-cp15.c
> index 1121dc3..3494a5c 100644
> --- a/arch/arm/lib/cache-cp15.c
> +++ b/arch/arm/lib/cache-cp15.c
> @@ -217,6 +217,8 @@ static void cache_disable(uint32_t cache_bit)
>         if (cache_bit == (CR_C | CR_M))
>                 flush_dcache_all();
>         set_cr(reg & ~cache_bit);
> +       if(cache_bit & CR_M)
> +               asm volatile ("isb" : : : "memory");
>  }
>  #endif
>
>
> I'm not sure what kinds of side effects for other boards these changes 
> would have.
Let's listen to the opinions of others.
>
> On Tue, Jul 26, 2016 at 8:15 AM, Ziyuan Xu <xzy.xu@rock-chips.com 
> <mailto:xzy.xu@rock-chips.com>> wrote:
>
>     + Simon and heiko
>
>     On 2016?07?26? 14:30, Ziyuan Xu wrote:
>
>         Dear All,
>
>         I add the ISB operation after dcache_disable(), and I can jump
>         to linux kernel entry.:-)
>
>         diff --git a/arch/arm/cpu/armv7/cpu.c b/arch/arm/cpu/armv7/cpu.c
>         index 6eac5ef..5cc09ba 100644
>         --- a/arch/arm/cpu/armv7/cpu.c
>         +++ b/arch/arm/cpu/armv7/cpu.c
>         @@ -43,6 +43,7 @@ int cleanup_before_linux_select(int flags)
>                         */
>                         dcache_disable();
>                         v7_outer_cache_disable();
>         +               ISB;
>
>                         /*
>                         * After D-cache is flushed and before it is
>         disabled there may
>
>         Sounds crazy. In fact, there is already an 'ISB' operation in
>         set_cr() which to disable dcache and MMU. But it just tell
>         processor that  memory had change, not real ISB for armv7.
>
>         dcache_disable
>                 ==>flush_dcache_all()
>                 ==>set_cr()   disable dcache and MMU.
>
>         #define isb() __asm__ __volatile__ ("" : : : "memory")
>         static inline void set_cr(unsigned int val)
>         {
>             if (is_hyp())
>                 asm volatile("mcr p15, 4, %0, c1, c0, 0    @ set CR" :
>                                           : "r" (val)
>                                           : "cc");
>             else
>                 asm volatile("mcr p15, 0, %0, c1, c0, 0    @ set CR" :
>                                           : "r" (val)
>                                           : "cc");
>             isb();
>         }
>
>         I also find something in ARM cortex-A17 TRM. ==>"Disable the
>         MMU from the each processor followed by an ISB to ensure the
>         MMU disable operation is complete, then followed by a DSB to
>         drain previous memory transactions."
>
>         In my humble opinion, maybe instructions tlb had alter, and
>         cause running away??? I'm not sure about it.
>         @Sandy, could you have a try with my update?
>
>         @Simon,  did you hit this glitch? I hope you can help ...:-)
>
>         On 2016?07?25? 23:00, Sandy Patterson wrote:
>
>             Ah, thanks. Your debugging looks the same as what I've
>             seen from printf debugging. I'll try to verify though.
>
>             On Mon, Jul 25, 2016 at 10:58 AM, Ziyuan Xu
>             <xzy.xu at rock-chips.com <mailto:xzy.xu@rock-chips.com>
>             <mailto:xzy.xu@rock-chips.com
>             <mailto:xzy.xu@rock-chips.com>>> wrote:
>
>                 hi Stany,
>
>                 The difference is that you print out assertion log.
>
>                 Reset not supported on this platform
>                 ### ERROR ### Please RESET the board ###
>
>                 You can add show_boot_progress() function in your bsp
>             file to find
>                 out something.
>                 #define CONFIG_SHOW_BOOT_PROGRESS
>                 void show_boot_progress(int progress)
>                 {
>                     printf("Boot reached stage %d\n", progress);
>
>                 }
>
>
>                 On 2016?07?25? 22:12, Ziyuan Xu wrote:
>
>                     Hi All,
>
>                     I'm sorry to tell you that I failed to boot linux
>             kernel with
>                     the mainline u-boot on rk3288 board(both evb-rk3288 &
>                     fennec-rk3288). It was stuck in
>             cleanup_before_linux() before
>                     jumping to linux, and the boot_stage_flag is
>                     BOOTSTAGE_ID_BOOTM_HANDOFF.
>
>                     ## Current stack ends at 0x7df638b0 * kernel:
>             cmdline image
>                     address = 0x02000000
>                     ## No init Ramdisk
>                        ramdisk start = 0x00000000, ramdisk end =
>             0x00000000
>                     ## No Flattened Device Tree
>                     Continuing to boot without FDT
>                     Initial value for argc=3
>                     Final value for argc=3
>                     using: ATAGS
>                     ## Transferring control to Linux (at address
>             02000000)...
>
>                     Starting kernel ...
>
>                     With the further investigation, it never returnned
>             back from
>                     invalidate_dcache_all(). I mean that I can't reach
>             stage 4 as
>                     below.
>
>                     cleanup_before_linux
>                         ==>cleanup_before_linux_select
>                             ==>1.disable_interrupts
>                             ==>2.dcache_disable
>                             ==>3.invalidate_dcache_all
>                             ==>4.icache_disable
>
>                     Debug further, invalidate_dcache_all invalidate
>             all cache
>                     one-by-one which cache type is DATA_ONLY,
>             INSTRUCTION_DATA or
>                     UNIFIED. And invalidate way from one set to
>             another set in
>                     order. The problem is that the PC ran away in
>             invalidate way
>                     loop  [cache level L1!!!].
>
>                     I add some serial output code in
>             __v7_flush_dcache_all to
>                     figure out the bog.
>                     I expect:
>                     Print the value of r9 in sequence, e.g 0x7f, 0x7e,
>             0x7d.....
>                     0x01, 0x00 (hex)
>                     In fact, print the value of r9 in sequence at
>             first, but print
>                     unexpected value afterwards.   e.g 0x7f, 0x7e,
>             0x7d, ..,0x73,
>                     0x40, 0x85,....
>
>                     ENTRY(__v7_flush_dcache_all)
>                     [snip]
>                     loop1:
>                         mov    r9, r7                @ create working
>             copy of max
>                     index
>                     loop2:
>                     +      stmfd    sp!, {r0}
>                     +      ldr     r0, =0xff690000
>                     +      str     r9, [r0]
>                     +      ldmfd    sp!, {r0}
>                      ARM(    orr    r11, r10, r4, lsl r5    ) @ factor
>             way and
>                     cache number into r11
>                      THUMB(    lsl    r6, r4, r5        )
>                      THUMB(    orr    r11, r10, r6        )    @
>             factor way and
>                     cache number into r11
>                      ARM(    orr    r11, r11, r9, lsl r2    ) @ factor
>             index
>                     number into r11
>                      THUMB(    lsl    r6, r9, r2        )
>                      THUMB(    orr    r11, r11, r6        )    @
>             factor index
>                     number into r11
>                         mcr    p15, 0, r11, c7, c14, 2        @ clean
>             & invalidate
>                     by set/way
>                         subs    r9, r9, #1            @ decrement the
>             index
>                         bge    loop2
>                         subs    r4, r4, #1            @ decrement the way
>                         bge    loop1
>                     skip:
>                     ENDPROC(__v7_flush_dcache_all)
>
>                     I don't have the jtag, hence I can't address the
>             current pc. I
>                     have no doubt that if any glitches in
>             __v7_flush_dcache_all, I
>                     reviewed several times, also copy kernel's
>             implement to here.
>                     :-( No effect.
>                     A more interesting thing is that Sandy had report
>             it. He and I
>                     have similar problem. Everything work fine after I
>             applied his
>                     patches, or disable dcache(active
>             CONFIG_SYS_DCACHE_OFF).
>                     @Stany, I'm sorry that I disable dcache during
>             hack.. That was
>                     a mistake:-(
>
>                     @Simon & hieko,
>                     Can you boot linux with the mainline u-boot? have
>             a try?
>
>
>             _______________________________________________
>                     U-Boot mailing list
>             U-Boot at lists.denx.de <mailto:U-Boot@lists.denx.de>
>             <mailto:U-Boot at lists.denx.de <mailto:U-Boot@lists.denx.de>>
>             http://lists.denx.de/mailman/listinfo/u-boot
>
>
>
>
>
>
>
>
>
>         _______________________________________________
>         U-Boot mailing list
>         U-Boot at lists.denx.de <mailto:U-Boot@lists.denx.de>
>         http://lists.denx.de/mailman/listinfo/u-boot
>
>
>
>