RE: do_ri failure in cache flushing routines

RE: do_ri failure in cache flushing routines

[G H]

[-- Attachment #1: Type: text/plain, Size: 790 bytes --]

I've not had much response to this question so I would like to rephrase it :
 
Can anyone think of any possible scenario where do_ri could occur in blast_icache32() ??
 
Is this possibly a cache synchronisation problem ??
 
TIA
 
>While testing out an amd au1500 based board I have been getting " do_ri " exceptions >that always occur in the cache flushing routines. More often than not in >blast_icache_32().
 
>So far this has mainly happened after running the board for days on end while running >multiple telnet sessions to it. It has sometimes ( quite rarely ) happened after a few >hours to a day of multiple telnet session use.


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

[-- Attachment #2: Type: text/html, Size: 985 bytes --]

Re: do_ri failure in cache flushing routines

[Pete Popov]

G H wrote:

> I've not had much response to this question so I would like to 
> rephrase it :
>  
> Can anyone think of any possible scenario where do_ri could occur in 
> blast_icache32() ??
>  
> Is this possibly a cache synchronisation problem ??
>  

Could be a hardware memory glitch. I would use kgdb to put a breakpoint 
there and see what the data in memory looks like when this happens -- 
look for memory corruption, etc.

Pete

> TIA
>  
> >While testing out an amd au1500 based board I have been getting " 
> do_ri " exceptions >that always occur in the cache flushing routines. 
> More often than not in >blast_icache_32().
>  
> >So far this has mainly happened after running the board for days on 
> end while running >multiple telnet sessions to it. It has sometimes ( 
> quite rarely ) happened after a few >hours to a day of multiple telnet 
> session use.
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>

Re: do_ri failure in cache flushing routines

[Pete Popov]

Pete Popov wrote:

> G H wrote:
>
>> I've not had much response to this question so I would like to 
>> rephrase it :
>>  
>> Can anyone think of any possible scenario where do_ri could occur in 
>> blast_icache32() ??
>>  
>> Is this possibly a cache synchronisation problem ??
>>  
>
>
> Could be a hardware memory glitch. I would use kgdb to put a 
> breakpoint there and see what the data in memory looks like when this 
> happens -- look for memory corruption, etc.

Another thought, what other stress testing have you done on your board? 
Does it complete a full native kernel compile without crashing?

Pete

>
> Pete
>
>> TIA
>>  
>> >While testing out an amd au1500 based board I have been getting " 
>> do_ri " exceptions >that always occur in the cache flushing routines. 
>> More often than not in >blast_icache_32().
>>  
>> >So far this has mainly happened after running the board for days on 
>> end while running >multiple telnet sessions to it. It has sometimes ( 
>> quite rarely ) happened after a few >hours to a day of multiple 
>> telnet session use.
>>
>> __________________________________________________
>> Do You Yahoo!?
>> Tired of spam? Yahoo! Mail has the best spam protection around
>> http://mail.yahoo.com
>>
>
>

Re: do_ri failure in cache flushing routines

[G H]

[-- Attachment #1: Type: text/plain, Size: 1717 bytes --]

At the moment I don't have the board set up for using kgdb and it's complicated by the fact that we only have one serial console port. But I am looking into setting it up for kgdb now.
 
As far as stressing the system, it doesn't have enough resources ( disk space ) to be able to compile the kernel, but we did write a simple program that would stress the system by spawning multiple threads, each one performing floating point calculations. With this test, top reported a load average of over 400 and we have seen no failure so far.

Pete Popov <ppopov@mvista.com> wrote:
G H wrote:

> I've not had much response to this question so I would like to 
> rephrase it :
> 
> Can anyone think of any possible scenario where do_ri could occur in 
> blast_icache32() ??
> 
> Is this possibly a cache synchronisation problem ??
> 

Could be a hardware memory glitch. I would use kgdb to put a breakpoint 
there and see what the data in memory looks like when this happens -- 
look for memory corruption, etc.

Pete

> TIA
> 
> >While testing out an amd au1500 based board I have been getting " 
> do_ri " exceptions >that always occur in the cache flushing routines. 
> More often than not in >blast_icache_32().
> 
> >So far this has mainly happened after running the board for days on 
> end while running >multiple telnet sessions to it. It has sometimes ( 
> quite rarely ) happened after a few >hours to a day of multiple telnet 
> session use.
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>


		
---------------------------------
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.

[-- Attachment #2: Type: text/html, Size: 2199 bytes --]

Re: do_ri failure in cache flushing routines

[Pete Popov]

G H wrote:

> At the moment I don't have the board set up for using kgdb and it's 
> complicated by the fact that we only have one serial console port. But 
> I am looking into setting it up for kgdb now.

A single serial port will work fine.

>  
> As far as stressing the system, it doesn't have enough resources ( 
> disk space ) to be able to compile the kernel, but we did write a 
> simple program that would stress the system by spawning multiple 
> threads, each one performing floating point calculations. With this 
> test, top reported a load average of over 400 and we have 
> seen no failure so far.

You can use an NFS mounted root fs to do native builds, assuming you 
have a native toolchain. That's an excellent stress test.

Pete

>
> */Pete Popov <ppopov@mvista.com>/* wrote:
>
>     G H wrote:
>
>     > I've not had much response to this question so I would like to
>     > rephrase it :
>     >
>     > Can anyone think of any possible scenario where do_ri could
>     occur in
>     > blast_icache32() ??
>     >
>     > Is this possibly a cache synchronisation problem ??
>     >
>
>     Could be a hardware memory glitch. I would use kgdb to put a
>     breakpoint
>     there and see what the data in memory looks like when this happens --
>     look for memory corruption, etc.
>
>     Pete
>
>     > TIA
>     >
>     > >While testing out an amd au1500 based board I have been getting "
>     > do_ri " exceptions >that always occur in the cache flushing
>     routines.
>     > More often than not in >blast_icache_32().
>     >
>     > >So far this has mainly happened after running the board for
>     days on
>     > end while running >multiple telnet sessions t! o it. It has
>     sometimes (
>     > quite rarely ) happened after a few >hours to a day of multiple
>     telnet
>     > session use.
>     >
>     > __________________________________________________
>     > Do You Yahoo!?
>     > Tired of spam? Yahoo! Mail has the best spam protection around
>     > http://mail.yahoo.com
>     >
>
> ------------------------------------------------------------------------
> Do you Yahoo!?
> Yahoo! Mail Address AutoComplete 
> <http://us.rd.yahoo.com/mail_us/taglines/aac/*http://promotions.yahoo.com/new_mail/static/ease.html> 
> - You start. We finish. 

Re: do_ri failure in cache flushing routines

[Jun Sun]

On Thu, Aug 05, 2004 at 11:04:27AM -0700, G H wrote:
> I've not had much response to this question so I would like to rephrase it :
>  
> Can anyone think of any possible scenario where do_ri could occur in blast_icache32() ??
>  

One possibility _could_ be the "instruction flushing itself" problem on
MIPS32.  However, as far as I know au1x00 CPUs don't suffer from this problem.
Anybody knows for sure?

You could try to use the two phase cache flushing (such as the one used
by tx47xx and also see an earlier related discussion thread) and see if
the problem goes away.

Jun

Re: do_ri failure in cache flushing routines

[G H]

[-- Attachment #1: Type: text/plain, Size: 1189 bytes --]

I was also thinking that maybe it could be related to the MIPS32 instruction cache flushing routine, so I tried applying the patch Ralf posted. The system still functioned OK for me, but one other in my group had their board lock up hard ( no oops produced ), and when I asked Ralf if that patch was ready for applying to CVS, he said it needed to be reworked before doing that. As a result we didn't follow up too closely on that avenue of investigation.
 
So basically what I am concluding from the responses so far , is that do_ri should NEVER occur in blast_icache32() and for it to do so, it could be either a hardware problem, or possibly the MIPS32 icache flushing problem.
Anyone agree / disagree ?
 
Jun Sun <jsun@mvista.com> wrote:


One possibility _could_ be the "instruction flushing itself" problem on
MIPS32. However, as far as I know au1x00 CPUs don't suffer from this problem.
Anybody knows for sure?


You could try to use the two phase cache flushing (such as the one used
by tx47xx and also see an earlier related discussion thread) and see if
the problem goes away.


		
---------------------------------
Do you Yahoo!?
Yahoo! Mail - You care about security. So do we.

[-- Attachment #2: Type: text/html, Size: 1539 bytes --]

Re: do_ri failure in cache flushing routines

[G H]

[-- Attachment #1: Type: text/plain, Size: 1189 bytes --]

I was also thinking that maybe it could be related to the MIPS32 instruction cache flushing routine, so I tried applying the patch Ralf posted. The system still functioned OK for me, but one other in my group had their board lock up hard ( no oops produced ), and when I asked Ralf if that patch was ready for applying to CVS, he said it needed to be reworked before doing that. As a result we didn't follow up too closely on that avenue of investigation.
 
So basically what I am concluding from the responses so far , is that do_ri should NEVER occur in blast_icache32() and for it to do so, it could be either a hardware problem, or possibly the MIPS32 icache flushing problem.
Anyone agree / disagree ?
 
Jun Sun <jsun@mvista.com> wrote:


One possibility _could_ be the "instruction flushing itself" problem on
MIPS32. However, as far as I know au1x00 CPUs don't suffer from this problem.
Anybody knows for sure?


You could try to use the two phase cache flushing (such as the one used
by tx47xx and also see an earlier related discussion thread) and see if
the problem goes away.


		
---------------------------------
Do you Yahoo!?
Yahoo! Mail - You care about security. So do we.

[-- Attachment #2: Type: text/html, Size: 1539 bytes --]

do_ri failure in cache flushing routines

[G H]

[-- Attachment #1: Type: text/plain, Size: 11086 bytes --]

While testing out an amd au1500 based board I have been getting " do_ri " exceptions that always occur in the cache flushing routines. More often than not in blast_icache_32().
 
So far this has mainly happened after running the board for days on end while running multiple telnet sessions to it. It has sometimes ( quite rarely ) happened after a few hours to a day of multiple telnet session use.
 
If left to run for days on end the the do_ri always occurs in a lcd daemon we have for updating a simple 40 row by 2 column I2c dispay and the oops trace is :
 
Reading Oops report from the terminal
$0 : 00000000 1000fc01 7fff76e0 ffffffe0 7fff76e0 8034880c ffffffff 00000000
$8 : ffffffff ffffffff 00000000 00000002 80313ae4 0005800b 00000340 849dbe20
$16: 7fff76d8 7fff7ea4 00416050 0000000e 8437ff30 00000000 8436da5c 8437e3d0
$24: 00000000 2afb0900                   8437e000 8437fe30 8437ff08 80105510
Hi : 00000000
Lo : 00000001
epc   : 8010cd50    Not tainted
Using defaults from ksymoops -t elf32-tradlittlemips -a mips:3000
Status: 1000fc03
Cause : 00800028
Process lcd (pid: 106, stackpage=8437e000)
Stack:    1000fc01 00000001 00000000 00000001 0000000e 00000080 00000000
 00000000 00000000 00000003 00000000 0000000a 00000001 802feda0 8437e000
 802fe920 84cec000 00000000 802fe920 803487e0 00000000 2ab34390 00000010
 80156614 8437e000 8437feb8 8437feb8 80104a98 1000fc03 b3b77d7a 000006c5
 00000021 00801000 8010f690 ffffffff 10015d08 7fff76e8 7fff76f0 1000fc01
 00000000 ...
Call Trace:   [<80156614>] [<80104a98>] [<8010f690>] [<80104aa4>] [<801097a0>]
 [<80162038>]
 Code: bc550000  00031823  00832024 <bc900000> 03e00008  00000000  3c028035  8c42881c  27bdffe8

>>$5; 8034880c <cpu_data+2c/80>
>>$12; 80313ae4 <contig_page_data+0/3ac>
>>$31; 80105510 <do_signal+2a8/df8>
>>PC;  8010cd50 <r4k_flush_cache_sigtramp+24/30>   <=====
Trace; 80156614 <iput+128/2b4>
Trace; 80104a98 <_sys_rt_sigsuspend+164/180>
Trace; 8010f690 <schedule+2d8/4fc>
Trace; 80104aa4 <_sys_rt_sigsuspend+170/180>
Trace; 801097a0 <stack_done+1c/38>
Trace; 80162038 <create_proc_entry+58/d0>
Code;  8010cd44 <r4k_flush_cache_sigtramp+18/30>
00000000 <_PC>:
Code;  8010cd44 <r4k_flush_cache_sigtramp+18/30>
   0:   bc550000  0xbc550000
Code;  8010cd48 <r4k_flush_cache_sigtramp+1c/30>
   4:   00031823  negu    v1,v1
Code;  8010cd4c <r4k_flush_cache_sigtramp+20/30>
   8:   00832024  and     a0,a0,v1
Code;  8010cd50 <r4k_flush_cache_sigtramp+24/30>   <=====
   c:   bc900000  0xbc900000   <=====
Code;  8010cd54 <r4k_flush_cache_sigtramp+28/30>
  10:   03e00008  jr      ra
Code;  8010cd58 <r4k_flush_cache_sigtramp+2c/30>
  14:   00000000  nop
Code;  8010cd5c <r4k_flush_icache_all+0/3c>
  18:   3c028035  lui     v0,0x8035
Code;  8010cd60 <r4k_flush_icache_all+4/3c>
  1c:   8c42881c  lw      v0,-30692(v0)
Code;  8010cd64 <r4k_flush_icache_all+8/3c>
  20:   27bdffe8  addiu   sp,sp,-24
 
But on the times it has happened after a few hours / a day or so it has happened due to activity on the serial console when running general linux commands the oops traces are at the end of this email.
 
I am using a 2.4.24 kernel from linux-mips cvs. 
 
Does anyone have any idea what could be causing this ? And possible ways to fix it ?
 
Can someone explain to me why the cache flushing routines are triggering a "reserved instruction" at random like this when up to the point of failure the same code must have been executed millions of times without triggering the exception ?
 
Any help in debugging this is greatly appreciated.
 
 
Failure in sshd :
 
Reserved instruction in kernel code in traps.c::do_ri, line 663:
 $0 : 00000000 80000000 80001c00 80001000 80000c00 00001000 00000001 00004000
 $8 : 00001000 00000000 820700f0 00000000 2afaee50 2afaee9c fffffff8 fffffffa
 $16: 810517c8 823acf60 0041ee48 820700f0 0041ee48 00000000 802feda0 00000000
 $24: fffffffe 0041ee48                   81dd4000 81dd5d98 00000004 8010cabc
 Hi : ffffa71d
 Lo : 00001da1
 epc   : 8010d66c    Not tainted
Using defaults from ksymoops -t elf32-tradlittlemips -a mips:3000
 Status: 1000fc03
 Cause : 00800028
 Process sshd (pid: 631, stackpage=81dd4000)
 Stack:    0041ee48 820700f0 0041ee48 00000000 810517c8 823acf60 0041ee48
  80125138 74737271 78777675 000000d8 0015a298 100151fc 0007831e 000000d8
  0015a298 00000000 00000000 000000d8 0015a298 000007de 0007831e 802feda0
  00000000 0041ee48 823acf60 81dd5f30 00000000 802fedbc 00000001 80125460
  801253f4 826d5cf0 82030a20 81dd5ed0 81dd5ed0 820700f0 0007831a 7fff78b0
  00000005 ...
 Call Trace:   [<80125138>] [<80125460>] [<801253f4>] [<80133d74>] [<8010c1d8>]
  [<8023112c>] [<80231510>] [<80126d88>] [<80125a0c>] [<8010e8bc>]
 Code: bc400260  bc400280  bc4002a0 <bc4002c0> bc4002e0  bc400300  bc400320  bc400340  bc400360

>>$22; 802feda0 <pci_socket_init+20/58>
>>$31; 8010cabc <r4k_flush_icache_page+138/218>
>>PC;  8010d66c <blast_icache32+a0/f0>   <=====
Trace; 80125138 <do_no_page+144/3b8>
Trace; 80125460 <handle_mm_fault+b4/278>
Trace; 801253f4 <handle_mm_fault+48/278>
Trace; 80133d74 <free_pages+48/50>
Trace; 8010c1d8 <do_page_fault+160/3c8>
Trace; 8023112c <sock_recvmsg+48/104>
Trace; 80231510 <sock_poll+28/34>
Trace; 80126d88 <do_brk+16c/23c>
Trace; 80125a0c <sys_brk+c8/140>
Trace; 8010e8bc <nopage_tlbl+f0/114>
Code;  8010d660 <blast_icache32+94/f0>
00000000 <_PC>:
Code;  8010d660 <blast_icache32+94/f0>
   0:   bc400260  0xbc400260
Code;  8010d664 <blast_icache32+98/f0>
   4:   bc400280  0xbc400280
Code;  8010d668 <blast_icache32+9c/f0>
   8:   bc4002a0  0xbc4002a0
Code;  8010d66c <blast_icache32+a0/f0>   <=====
   c:   bc4002c0  0xbc4002c0   <=====
Code;  8010d670 <blast_icache32+a4/f0>
  10:   bc4002e0  0xbc4002e0
Code;  8010d674 <blast_icache32+a8/f0>
  14:   bc400300  0xbc400300
Code;  8010d678 <blast_icache32+ac/f0>
  18:   bc400320  0xbc400320
Code;  8010d67c <blast_icache32+b0/f0>
  1c:   bc400340  0xbc400340
Code;  8010d680 <blast_icache32+b4/f0>
  20:   bc400360  0xbc400360

Failure in ls :
 

Reading Oops report from the terminal
Reserved instruction in kernel code in traps.c::do_ri, line 663:
$0 : 00000000 80000000 80002000 80001000 80000000 00002000 000079d8 00004000
$8 : 00000001 00000000 8349a0e8 00000b3b 7f1c0300 00000001 65726168 00000115
$16: 810e7d2c 834cb740 0041da54 8349a0e8 0041da54 00000000 83a96460 00000000
$24: 00000000 2abede20                   8010f58b 83481d98 00000000 8010cabc
Hi : ffff031c
Lo : 0000544c
epc   : 8010d634    Not tainted
Using defaults from ksymoops -t elf32-tradlittlemips -a mips:3000
Status: 1000fc03
Cause : 00800028
Process ls (pid: 1335, stackpage=83480000)
Stack:    0041da54 8349a0e8 0041da54 00000000 810e7d2c 834cb740 0041da54
 80125138 4c4b4a49 504f4e4d 000000d8 0010b2d8 62615a59 66656463 000000d8
 0010b2d8 00000000 00000000 000000d8 0010b2d8 48474645 4c4b4a49 83a96460
 00000000 0041da54 834cb740 83481f30 00000000 83a9647c 00000000 80125460
 801253f4 44434241 48474645 7fff7ce0 00000000 8349a0e8 83481eb8 80312cd0
 2ad2d520 ...
Call Trace:   [<80125138>] [<80125460>] [<801253f4>] [<801198e8>] [<8010c1d8>]
 [<801bf480>] [<80125b4c>] [<80126e08>] [<801ae43c>] [<8014de3c>] [<80125a0c>]
 [<8010e8bc>]
Code: bc4000a0  bc4000c0  bc4000e0 <bc400100> bc400120  bc400140  bc400160  bc400180  bc4001a0

>>$28; 8010f58b <schedule+1d3/4fc>
>>$31; 8010cabc <r4k_flush_icache_page+138/218>
>>PC;  8010d634 <blast_icache32+68/f0>   <=====
Trace; 80125138 <do_no_page+144/3b8>
Trace; 80125460 <handle_mm_fault+b4/278>
Trace; 801253f4 <handle_mm_fault+48/278>
Trace; 801198e8 <parse_table+16c/174>
Trace; 8010c1d8 <do_page_fault+160/3c8>
Trace; 801bf480 <ltxser_interrupt+204/32c>
Trace; 80125b4c <__vma_link+44/dc>
Trace; 80126e08 <do_brk+1ec/23c>
Trace; 801ae43c <tty_ioctl+15c/51c>
Trace; 8014de3c <sys_ioctl+a0/2e4>
Trace; 80125a0c <sys_brk+c8/140>
Trace; 8010e8bc <nopage_tlbl+f0/114>
Code;  8010d628 <blast_icache32+5c/f0>
00000000 <_PC>:
Code;  8010d628 <blast_icache32+5c/f0>
   0:   bc4000a0  0xbc4000a0
Code;  8010d62c <blast_icache32+60/f0>
   4:   bc4000c0  0xbc4000c0
Code;  8010d630 <blast_icache32+64/f0>
   8:   bc4000e0  0xbc4000e0
Code;  8010d634 <blast_icache32+68/f0>   <=====
   c:   bc400100  0xbc400100   <=====
Code;  8010d638 <blast_icache32+6c/f0>
  10:   bc400120  0xbc400120
Code;  8010d63c <blast_icache32+70/f0>
  14:   bc400140  0xbc400140
Code;  8010d640 <blast_icache32+74/f0>
  18:   bc400160  0xbc400160
Code;  8010d644 <blast_icache32+78/f0>
  1c:   bc400180  0xbc400180
Code;  8010d648 <blast_icache32+7c/f0>
  20:   bc4001a0  0xbc4001a0

Failure in stat:
 
Reserved instruction in kernel code in traps.c::do_ri, line 663:
$0 : 00000000 80000000 80002400 80001000 80000400 00002000 000079d8 00004000
$8 : 00000001 00000000 82ea8828 6ffffeff 6fffff72 00000063 40f2d4f7 00000000
$16: 810eeb84 834cb920 2ab051ac 82ea8828 2ab051ac 00000000 83a96460 00000000
$24: 00000000 2aabcd10                   8010f58b 82eb5d98 7fff7678 8010cabc
Hi : fffefb96
Lo : 000056ce
epc   : 8010d650    Not tainted
Using defaults from ksymoops -t elf32-tradlittlemips -a mips:3000
Status: 1000fc03
Cause : 00800028
Process stat (pid: 1376, stackpage=82eb4000)
Stack:    2ab051ac 82ea8828 2ab051ac 00000000 810eeb84 834cb920 2ab051ac
 80125138 00000000 856ff768 000000d8 0015b458 000005dc 000baa5c 000000d8
 0015b458 00000000 00000000 000000d8 0015b458 000000d8 000baa58 83a96460
 00000000 2ab051ac 834cb920 82eb5f30 00000000 83a9647c 00000001 80125460
 801253f4 834cb938 834cb920 834cbbc0 834cbbc0 82ea8828 0015bb1a 801267c8
 8010e8bc ...
Call Trace:   [<80125138>] [<80125460>] [<801253f4>] [<801267c8>] [<8010e8bc>]
 [<8010c1d8>] [<80126068>] [<8012d008>] [<801a074c>] [<80108034>] [<8010e8bc>]
Code: bc400180  bc4001a0  bc4001c0 <bc4001e0> bc400200  bc400220  bc400240  bc400260  bc400280

>>$28; 8010f58b <schedule+1d3/4fc>
>>$31; 8010cabc <r4k_flush_icache_page+138/218>
>>PC;  8010d650 <blast_icache32+84/f0>   <=====
Trace; 80125138 <do_no_page+144/3b8>
Trace; 80125460 <handle_mm_fault+b4/278>
Trace; 801253f4 <handle_mm_fault+48/278>
Trace; 801267c8 <unmap_fixup+1e4/1fc>
Trace; 8010e8bc <nopage_tlbl+f0/114>
Trace; 8010c1d8 <do_page_fault+160/3c8>
Trace; 80126068 <do_mmap_pgoff+350/5b0>
Trace; 8012d008 <mprotect_fixup+1f0/534>
Trace; 801a074c <fpu_emulator_cop1Handler+198/1bc>
Trace; 80108034 <do_cpu+2e0/334>
Trace; 8010e8bc <nopage_tlbl+f0/114>
Code;  8010d644 <blast_icache32+78/f0>
00000000 <_PC>:
Code;  8010d644 <blast_icache32+78/f0>
   0:   bc400180  0xbc400180
Code;  8010d648 <blast_icache32+7c/f0>
   4:   bc4001a0  0xbc4001a0
Code;  8010d64c <blast_icache32+80/f0>
   8:   bc4001c0  0xbc4001c0
Code;  8010d650 <blast_icache32+84/f0>   <=====
   c:   bc4001e0  0xbc4001e0   <=====
Code;  8010d654 <blast_icache32+88/f0>
  10:   bc400200  0xbc400200
Code;  8010d658 <blast_icache32+8c/f0>
  14:   bc400220  0xbc400220
Code;  8010d65c <blast_icache32+90/f0>
  18:   bc400240  0xbc400240
Code;  8010d660 <blast_icache32+94/f0>
  1c:   bc400260  0xbc400260
Code;  8010d664 <blast_icache32+98/f0>
  20:   bc400280  0xbc400280

  

		
---------------------------------
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.

[-- Attachment #2: Type: text/html, Size: 15478 bytes --]