[U-Boot] MIPS cpu has problem detecting CFI

[myuboot at fastmail.fm <myuboot@fastmail.fm>]

On Sat, 10 Oct 2009 15:47 -0500, "Andrew Dyer" <amdyer@gmail.com> wrote:
> On Sat, Oct 10, 2009 at 12:27 PM,  <myuboot@fastmail.fm> wrote:
> > On Fri, 09 Oct 2009 12:16 -0500, "Andrew Dyer" <amdyer@gmail.com> wrote:
> >> On Fri, Oct 9, 2009 at 11:21 AM, ?<myuboot@fastmail.fm> wrote:
> >> > I think I found a problem in cpu/mips/start.S.
> >> >
> >> > gp register is used to point to the SDRAM. But after gp is adjusted to
> >> > proper location, a C function flush_cache is called. But this function
> >> > actually changes gp register before gp is used to jump to SDRAM. That
> >> > makes the u-boot run from flash and fails to detect CFI.
> >> >
> >> > Here is the assembler code for flush_cache.c
> >> > (gdb) disassem 0xb0000798
> >> > Dump of assembler code for function flush_cache:
> >> > 0xb000078c <flush_cache+0>: ? ? lui ? ? gp,0x3
> >> > 0xb0000790 <flush_cache+4>: ? ? addiu ? gp,gp,-32300
> >> > 0xb0000794 <flush_cache+8>: ? ? addu ? ?gp,gp,t9
> >> >
> >> > And here is how I fixed the issue -
> >> >
> >> >
> >> > diff --git a/u-boot-2009.08/cpu/mips/start.S
> >> > b/u-boot-2009.08/cpu/mips/start.S
> >> > index 57db589..0e8f8ed 100644
> >> > --- a/u-boot-2009.08/cpu/mips/start.S
> >> > +++ b/u-boot-2009.08/cpu/mips/start.S
> >> > @@ -321,6 +321,7 @@ relocate_code:
> >> > ? ? ? ?move ? ?t6, gp
> >> > ? ? ? ?sub ? ? gp, CONFIG_SYS_MONITOR_BASE
> >> > ? ? ? ?add ? ? gp, a2 ? ? ? ? ?/* gp now adjusted ? ? ? ? ? ? ?*/
> >> > + ? ? ? ?move ? ?t8, gp
> >> > ? ? ? ?sub ? ? s1, gp, t6 ? ? ?/* s1 <-- relocation offset ? ? */
> >> >
> >> > ? ? ? ?/*
> >> > @@ -358,6 +359,7 @@ relocate_code:
> >> >
> >> > ? ? ? ?/* Jump to where we've relocated ourselves.
> >> > ? ? ? ? */
> >> > + ? ? ? ?move ? ?gp, t8
> >> > ? ? ? ?addi ? ?t0, s2, in_ram - _start
> >> > ? ? ? ?jr ? ? ?t0
> >> > ? ? ? ?nop
> >>
> >> Something seems weird here - IIRC, $gp is supposed to be saved in the
> >> stack frame during the function prologue and restored from the frame
> >> on exit. ?Also I think relying on $t8 is risky, as I don't believe it
> >> is guaranteed by the ABI to be preserved across function calls.
> >>
> >> What compiler are you using? ?Can you post the whole flush_cache()
> >> disassembly?
> >
> > You may be right. The c function should get the input arguments through
> > stack. I just use the same toolchain(with -S option) to generate the
> > assembler code and the assembler code does not show the modification on
> > the gp register. I am attaching the whole assembler code file here too.
> > But the assembler code I sent in my previous mail did show the gp
> > pointer is changed. In that email, I got that assembler using  gdb
> > through a BDI debugger. I don't have BDI with me today so I cann't give
> > you the whole assembler code using BDI.
> >
> > Could it be the gcc toolchain  generates the wrong code when it is
> > creating object file, but generates correct assembler code? Or the BDI
> > is changing the assembler code at run time such that I see the wrong
> > assembler code? But if that is the case, I don't know how to explain
> > with my workaround/fix the board did come up even without bdi and it
> > seems to work every time. I am so confused now. Any suggestion on
> > anything I can try?
> 
> 1) please don't top post, put replies on the bottom, I fixed this in this
> post.
> 
> 2) reply to the list, there are a lot of smart people (better versed
> in toolchains than I) who read it.
> I've cc'd the list on this one.
> 
> 3) the assembler looks fine, but it doesn't have the function prologue
> that the gdb dump shows.  I doubt the bdi is modifying the asm.  I
> /think/ the linker might be the part that inserts that or maybe the
> compile after it generates the asm.  Try looking at the output of
> '${CROSS_COMPILE}objdump --source cpu.o' and then
> '${CROSS_COMPILE}objdump --source u-boot'.
> 
> 4) where did the toolchain come from, what version, etc.?  Maybe run
> gcc with the -v option to see the subprograms and options it sets.
> 
> 5) try downloading the mips ELDK and build u-boot with that.

Thanks for 1) and 2).
For 3), both results show GP register is modified.
the result of ${CROSS_COMPILE}objdump --source cpu.o is:

void flush_cache(ulong start_addr, ulong size)
{
  7c:   3c1c0000        lui     gp,0x0
  80:   279c0000        addiu   gp,gp,0
  84:   0399e021        addu    gp,gp,t9
        unsigned long lsize = CONFIG_SYS_CACHELINE_SIZE;
        unsigned long addr = start_addr & ~(lsize - 1);
  88:   2403fff0        li      v1,-16
  8c:   00831024        and     v0,a0,v1
        unsigned long aend = (start_addr + size - 1) & ~(lsize - 1);
  90:   2484ffff        addiu   a0,a0,-1
  94:   00852021        addu    a0,a0,a1
  98:   00831824        and     v1,a0,v1

        while (1) {
                cache_op(Hit_Writeback_Inv_D, addr);
  9c:   bc550000        cache   0x15,0(v0)
                cache_op(Hit_Invalidate_I, addr);
  a0:   bc500000        cache   0x10,0(v0)
                if (addr == aend)
  a4:   1443fffd        bne     v0,v1,9c <flush_cache+0x20>
  a8:   24420010        addiu   v0,v0,16
                        break;
                addr += lsize;
        }
#if defined(CONFIG_TNETVxxxx)
        invalidate_dcache();
  ac:   8f990000        lw      t9,0(gp)
  b0:   03200008        jr      t9
  b4:   00000000        nop

000000b8 <do_reset>:
void __attribute__((weak)) _machine_restart(void)
{
}

void flush_cache(ulong start_addr, ulong size)
{
  7c:   3c1c0000        lui     gp,0x0
  80:   279c0000        addiu   gp,gp,0
  84:   0399e021        addu    gp,gp,t9
        unsigned long lsize = CONFIG_SYS_CACHELINE_SIZE;
        unsigned long addr = start_addr & ~(lsize - 1);
  88:   2403fff0        li      v1,-16
  8c:   00831024        and     v0,a0,v1
        unsigned long aend = (start_addr + size - 1) & ~(lsize - 1);
  90:   2484ffff        addiu   a0,a0,-1
  94:   00852021        addu    a0,a0,a1
  98:   00831824        and     v1,a0,v1

        while (1) {
                cache_op(Hit_Writeback_Inv_D, addr);
  9c:   bc550000        cache   0x15,0(v0)
                cache_op(Hit_Invalidate_I, addr);
  a0:   bc500000        cache   0x10,0(v0)
                if (addr == aend)
  a4:   1443fffd        bne     v0,v1,9c <flush_cache+0x20>
  a8:   24420010        addiu   v0,v0,16
                        break;
                addr += lsize;
        }
#if defined(CONFIG_TNETVxxxx)
        invalidate_dcache();
  ac:   8f990000        lw      t9,0(gp)
  b0:   03200008        jr      t9
  b4:   00000000        nop

000000b8 <do_reset>:
void __attribute__((weak)) _machine_restart(void)
{
}


For 4), I generated the toolchain by myself using buildroot 2009.08. 
Using built-in specs. And the result of gcc -v is:
Target: mips-linux-uclibc
Configured with:
/home/root123/sources/buildroot-2009.08-k/toolchain_build_mips/gcc-4.4.1/configure
--prefix=/usr --build=i386-pc-linux-gnu --host=i386-pc-linux-gnu
--target=mips-linux-uclibc --enable-languages=c
--with-sysroot=/home/root123/sources/buildroot-2009.08-k/build_mips/staging_dir
--with-build-time-tools=/home/root123/sources/buildroot-2009.08-k/build_mips/staging_dir/usr/mips-linux-uclibc/bin
--disable-__cxa_atexit --enable-target-optspace --with-gnu-ld
--disable-libssp --disable-tls --enable-shared
--with-gmp=/home/root123/sources/buildroot-2009.08-k/toolchain_build_mips/gmp
--with-mpfr=/home/root123/sources/buildroot-2009.08-k/toolchain_build_mips/mpfr
--disable-nls --enable-threads --disable-multilib
--disable-decimal-float --with-float=soft --with-abi=32
--with-tune=mips32 : (reconfigured)
/home/root123/sources/buildroot-2009.08-k/toolchain_build_mips/gcc-4.4.1/configure
--prefix=/usr --build=i386-pc-linux-gnu --host=i386-pc-linux-gnu
--target=mips-linux-uclibc --enable-languages=c
--with-sysroot=/home/root123/sources/buildroot-2009.08-k/build_mips/staging_dir
--with-build-time-tools=/home/root123/sources/buildroot-2009.08-k/build_mips/staging_dir/usr/mips-linux-uclibc/bin
--disable-__cxa_atexit --enable-target-optspace --with-gnu-ld
--disable-libssp --disable-tls --enable-shared
--with-gmp=/home/root123/sources/buildroot-2009.08-k/toolchain_build_mips/gmp
--with-mpfr=/home/root123/sources/buildroot-2009.08-k/toolchain_build_mips/mpfr
--disable-nls --enable-threads --disable-multilib
--disable-decimal-float --with-float=soft --with-abi=32
--with-tune=mips32
Thread model: posix
gcc version 4.4.1 (GCC)

For most of the flags above, they are automatically added by buildroot.
Please let me know if there is anything wrong with the setting. 

4) I am not familiar with ELDK. It seems to be able to create toolchain
quickly. I will give it a try. How is the toolchain created by ELDK
different from buildroot or openembedded?

Thanks.