Unknown ARCS message/hang

Unknown ARCS message/hang

[Erik J. Green]

Hello all;

I'm working on getting mips-linux up and running on my Octane, which may be an
impossible task.  I am new to the MIPS architecture, and I'm absorbing a lot of
information rapidly from MIPS Inc. manuals, See MIPS Run, et. al., but I have
been banging my head against the wall for a couple days trying to get a simple
problem fixed.

First, my Octane works just fine with Irix 6.5.17.  Configuration is a R12K
CPU/300 mhz, 256MB RAM, 1 18G disk, ethernet connection.  I boot using bootp. 
I've used the network to boot fx, a copy of the irix kernel, etc.

I get the following messages when I try to boot the (very slightly) modified
linux kernel I am working with:

--start messages

Obtaining /vmlinux.64 from server
1813568+1150976+172144 entry: 0xa8000000211c4000

*** PROM write error on cacheline 0x1fcd3b00 at PC=0x211c4018 RA=0xffffffff9fc5ace4

--end messages

The PC address is the first instruction in head.S (mips64) that touches the
control register.  I've tried multiple fixes, including initializing the whole
TLB before the error occurs.  Same error.

Can anyone tell me:

1) What does this error text mean exactly? 

2) What is "RA"?  The address is a location in the PROM text/stack section.

3) Am I missing something simple?  An initialization, a rule I'm not following?

objdump of the head.S I am using is below.

Thanks for any clues you can provide.  I'm excited to get working on this
machine if I can just learn MIPS =\

Erik

Disassembly of section .text.init:

a8000000211c4000 <kernel_entry>:
a8000000211c4000:       37bd000f        ori     sp,sp,0xf
a8000000211c4004:       3bbd000f        xori    sp,sp,0xf
a8000000211c4008:       3c0c211c        lui     t0,0x211c
a8000000211c400c:       258c4018        addiu   t0,t0,16408
a8000000211c4010:       01800008        jr      t0
a8000000211c4014:       00000000        nop
a8000000211c4018:       400c6000        mfc0    t0,$12
a8000000211c401c:       3c0d1000        lui     t1,0x1000
a8000000211c4020:       35ad001f        ori     t1,t1,0x1f
a8000000211c4024:       018d6025        or      t0,t0,t1
a8000000211c4028:       398c001f        xori    t0,t0,0x1f
a8000000211c402c:       408c6000        mtc0    t0,$12
a8000000211c4030:       3c1c211c        lui     gp,0x211c
a8000000211c4034:       279c0000        addiu   gp,gp,0
a8000000211c4038:       679d3fe0        daddiu  sp,gp,16352
a8000000211c403c:       8f8c0060        lw      t0,96(gp)



-- 
Erik J. Green
erik@greendragon.org

Re: Unknown ARCS message/hang

[Maciej W. Rozycki]

On Fri, 4 Apr 2003, Erik J. Green wrote:

> I get the following messages when I try to boot the (very slightly) modified
> linux kernel I am working with:
> 
> --start messages
> 
> Obtaining /vmlinux.64 from server
> 1813568+1150976+172144 entry: 0xa8000000211c4000
> 
> *** PROM write error on cacheline 0x1fcd3b00 at PC=0x211c4018 RA=0xffffffff9fc5ace4
> 
> --end messages
> 
> The PC address is the first instruction in head.S (mips64) that touches the
> control register.  I've tried multiple fixes, including initializing the whole
> TLB before the error occurs.  Same error.

 0x211c4018 is a mapped address, which you can't use that early in a boot.

> Can anyone tell me:
> 
> 1) What does this error text mean exactly? 

 An unhandled exception happened due to using a mapped address.  The PROM
caught it and reported. 

> 2) What is "RA"?  The address is a location in the PROM text/stack section.

 It's a CPU register, otherwise known as $31, where the return address is
stored by most of the jump-and-link and branch-and-link instructions.
Here it's an address in the PROM following the "jalr" instruction that
invoked the kernel.

> 3) Am I missing something simple?  An initialization, a rule I'm not following?

 You really want to link your kernel at a KSEG0 address (otherwise you'd
need to struggle with the kernel and the tools to get an unsupported yet
configuration to work).  Basically this means setting LOADADDR in
arch/mips64/Makefile appropriately.  See how it's done for other
platforms.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: Unknown ARCS message/hang

[Guido Guenther]

On Fri, Apr 04, 2003 at 02:58:02PM +0200, Maciej W. Rozycki wrote:
[..snip..] 
> > Obtaining /vmlinux.64 from server
> > 1813568+1150976+172144 entry: 0xa8000000211c4000
> > 
> > *** PROM write error on cacheline 0x1fcd3b00 at PC=0x211c4018 RA=0xffffffff9fc5ace4
[..snip..] 
> 
>  0x211c4018 is a mapped address, which you can't use that early in a boot.
Isn't 0xa8000000211c4000 in xkphys and therefore unmapped? The PROM only
seems to look at the lower 32bits of PC though.
Puzzled,
 -- Guido

Re: Unknown ARCS message/hang

[Maciej W. Rozycki]

On Fri, 4 Apr 2003, Guido Guenther wrote:

> > > Obtaining /vmlinux.64 from server
> > > 1813568+1150976+172144 entry: 0xa8000000211c4000
> > > 
> > > *** PROM write error on cacheline 0x1fcd3b00 at PC=0x211c4018 RA=0xffffffff9fc5ace4
> [..snip..] 
> > 
> >  0x211c4018 is a mapped address, which you can't use that early in a boot.
> Isn't 0xa8000000211c4000 in xkphys and therefore unmapped? The PROM only
> seems to look at the lower 32bits of PC though.

 0xa8000000211c4000 is indeed in XKPHYS but the code jumps to 0x211c4018.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: Unknown ARCS message/hang

[Guido Guenther]

On Fri, Apr 04, 2003 at 04:19:44PM +0200, Maciej W. Rozycki wrote:
> On Fri, 4 Apr 2003, Guido Guenther wrote:
> 
> > > > Obtaining /vmlinux.64 from server
> > > > 1813568+1150976+172144 entry: 0xa8000000211c4000
> > > > 
> > > > *** PROM write error on cacheline 0x1fcd3b00 at PC=0x211c4018 RA=0xffffffff9fc5ace4
> > [..snip..] 
> > > 
> > >  0x211c4018 is a mapped address, which you can't use that early in a boot.
> > Isn't 0xa8000000211c4000 in xkphys and therefore unmapped? The PROM only
> > seems to look at the lower 32bits of PC though.
> 
>  0xa8000000211c4000 is indeed in XKPHYS but the code jumps to 0x211c4018.
So either the PROM or (more likely) the start address in the kernel is
bogus. What does
 objdump -x vmlinux | grep 'start address'
say?
Regards,
 -- Guido

Re: Unknown ARCS message/hang

[Guido Guenther]

On Fri, Apr 04, 2003 at 04:35:34PM +0200, Guido Guenther wrote:
> So either the PROM or (more likely) the start address in the kernel is
> bogus. What does
Forget that. Looking at the disassembly in the first mail it's clearly
the code that's wrong.
Regards,
 -- Guido

Re: Unknown ARCS message/hang

[Erik J. Green]

Quoting "Maciej W. Rozycki" <macro@ds2.pg.gda.pl>:

> On Fri, 4 Apr 2003, Guido Guenther wrote:
> 
> > > > Obtaining /vmlinux.64 from server
> > > > 1813568+1150976+172144 entry: 0xa8000000211c4000
> > > >
> > > > *** PROM write error on cacheline 0x1fcd3b00 at PC=0x211c4018
> RA=0xffffffff9fc5ace4
> > [..snip..]
> > >
> > >  0x211c4018 is a mapped address, which you can't use that early in a
> boot.
> > Isn't 0xa8000000211c4000 in xkphys and therefore unmapped? The PROM only
> > seems to look at the lower 32bits of PC though.
> 
>  0xa8000000211c4000 is indeed in XKPHYS but the code jumps to 0x211c4018.

Ah, okay.  I didn't understand the jr instruction there.  That's generated as 
part of a macro:

	.macro	ARC64_TWIDDLE_PC
#if defined(CONFIG_ARC64) || defined(CONFIG_MAPPED_KERNEL)
	/* We get launched at a XKPHYS address but the kernel is linked to
	   run at a KSEG0 address, so jump there.  */
	PTR_LA	t0, \@f
	jr	t0
\@:

I was under the assumption that Octane is ARC64, I may be wrong.

Clearly then, the kernel is linked at the wrong address to have this work.  The
question I have is, why is kseg0 used in this case?  Is it due to the 32 to 64
bit conversion that happens later on in the build?  It looks like the IP27 load
address was originally  0xa80000000001c000, but was amended to 0x8001c000 for
the current CVS(2.4) kernel.  Again, due to the conversion?

Erik


-- 
Erik J. Green
erik@greendragon.org

Re: Unknown ARCS message/hang

[Maciej W. Rozycki]

On Fri, 4 Apr 2003, Erik J. Green wrote:

> Clearly then, the kernel is linked at the wrong address to have this work.  The
> question I have is, why is kseg0 used in this case?  Is it due to the 32 to 64
> bit conversion that happens later on in the build?  It looks like the IP27 load
> address was originally  0xa80000000001c000, but was amended to 0x8001c000 for
> the current CVS(2.4) kernel.  Again, due to the conversion?

 Not all parts of the MIPS64/Linux kernel are 64-bit clean when it comes
to addressing.  There used to be troubles with 64-bit tools until
recently, too.  That's why the kernel is built with 32-bit addressing and
only after the final link converted to a 64-bit object to satisfy firmware
that needs such for a load.

 It should be fixed one day, but that's not necessarily a starter's task. 

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

64 to 32 bit jr

[Erik J. Green]

Quoting "Maciej W. Rozycki" <macro@ds2.pg.gda.pl>:
[deletions]
> > > > *** PROM write error on cacheline 0x1fcd3b00 at PC=0x211c4018
> RA=0xffffffff9fc5ace4
> > [..snip..]
> > >
> > >  0x211c4018 is a mapped address, which you can't use that early in a
> boot.
> > Isn't 0xa8000000211c4000 in xkphys and therefore unmapped? The PROM only
> > seems to look at the lower 32bits of PC though.
> 
>  0xa8000000211c4000 is indeed in XKPHYS but the code jumps to 0x211c4018.

Okay, I want to make sure I understand the addressing correctly for the 64 to 32
bit jump.  The existing code for the IP27 (seems to load at about
a800000000000000, which is one of the segments in xkphys, corresponding to
physical memory starting at address 0.  Head.S then jumps to the 32-bit part of
the xkphys address, which happens to be arranged so that it matches the correct
(next instruction) address in kseg0.

I am unable to arrange my addresses similarly neatly, mostly I think due to
fighting with the toolchain I have.  Is it "legal" for me to load a kernel using
the xkphys address and then do something like:

lui    t0,0x8000
addiu  t0,t0,@next
jr     t0
nop
next:


to jump to the next instruction but in kseg0 instead of xkphys?  I believe the
jump target should be word aligned in this case because it's the start of an
instruction.  I'm assuming if I generate a jr to a 32 bit address that the cpu
will assume I'm jumping to a compatibility segment, am I wrong?

Erik




-- 
Erik J. Green
erik@greendragon.org

Re: 64 to 32 bit jr

[Maciej W. Rozycki]

On Mon, 7 Apr 2003, Erik J. Green wrote:

> >  0xa8000000211c4000 is indeed in XKPHYS but the code jumps to 0x211c4018.
> 
> Okay, I want to make sure I understand the addressing correctly for the 64 to 32
> bit jump.  The existing code for the IP27 (seems to load at about
> a800000000000000, which is one of the segments in xkphys, corresponding to
> physical memory starting at address 0.  Head.S then jumps to the 32-bit part of
> the xkphys address, which happens to be arranged so that it matches the correct
> (next instruction) address in kseg0.

 Just see how these virtual addresses map to physical ones.

> I am unable to arrange my addresses similarly neatly, mostly I think due to
> fighting with the toolchain I have.  Is it "legal" for me to load a kernel using
> the xkphys address and then do something like:
> 
> lui    t0,0x8000
> addiu  t0,t0,@next
> jr     t0
> nop
> next:

 You need to do:

lui	t0,%hi(next)
addiu	t0,%lo(next)

or use the "la" macro instead, to get both low halfwords right (the "lui" 
instruction will implicitly set the high word of t0 appropriately by
sign-extending the low word).

> to jump to the next instruction but in kseg0 instead of xkphys?  I believe the

 Obviously you have to make sure that the binary is linked such that 
"next" is within KSEG0.

> jump target should be word aligned in this case because it's the start of an

 It normally is unless you try to play tricks with the assembler.

> instruction.  I'm assuming if I generate a jr to a 32 bit address that the cpu
> will assume I'm jumping to a compatibility segment, am I wrong?

 Internally there is no such thing as a 32-bit address -- all addresses
are 64-bit in 64-bit processors (as that is the width of registers).  It's
simply that using 32-bit operations to calculate addresses limits the
range of results to these that have their top 33 bits set to the same
value.

 There is nothing special about the jump above and the CPU assumes nothing
specific about it.  The target address is just like any other one.  Once
the jump is executed, the following instruction will be fetched using
addressing rules defined for the segment it lies within (or an address
error exception happens if the address happens to be outside defined
segments). So after a jump to 0x00000000211c4018, the rules for XKUSEG
apply. 

 HTH,

  Maciej

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: 64 to 32 bit jr

[Erik J. Green]

Quoting "Maciej W. Rozycki" <macro@ds2.pg.gda.pl>:

[deletions]
> > physical memory starting at address 0.  Head.S then jumps to the 32-bit
> part of
> > the xkphys address, which happens to be arranged so that it matches the
> correct
> > (next instruction) address in kseg0.
> 
>  Just see how these virtual addresses map to physical ones.

According to my current understanding, the base of each of 8 segments in xkphys
maps to the start of physical memory, so offset 0 in kseg0 should be the same
data as at offset 0 of the a800...0000 segment in xkphys.  So, if I load code
starting at offset 0 in xkphys, I should be able to jump to the 32-bit part of
the xkphys address and end up at the same offset in kseg0, provided the target
address is sign-extended properly.

The actual difficulty I'm having is that the address my code is loading at is
computed at link time by adding the xkphys base to the LOADADDRESS value using
mips64-linux-objcopy.  I haven't quite worked out the address math yet to  make
the code in xkphys and kseg0 have the same offsets.  Objcopy seems to have some
non-obvious rules for doing address calculations, IE objcopy using
--change-addresses=X

0xa800000000000000 + 0x20004000

gives something close to (not near my MIPS system atm)

0xa7ffffff2001c000

So, I'm thinking constructing the address in a register might be easier for now.

> 
>  HTH,
> 
>   Maciej


It does, thanks.

Erik
-- 
Erik J. Green
erik@greendragon.org

Re: 64 to 32 bit jr

[Maciej W. Rozycki]

On Mon, 7 Apr 2003, Erik J. Green wrote:

> According to my current understanding, the base of each of 8 segments in xkphys
> maps to the start of physical memory, so offset 0 in kseg0 should be the same
> data as at offset 0 of the a800...0000 segment in xkphys.  So, if I load code
> starting at offset 0 in xkphys, I should be able to jump to the 32-bit part of
> the xkphys address and end up at the same offset in kseg0, provided the target
> address is sign-extended properly.

 As long as your offset into XPHYS fits within the KSEG0 size.

> the code in xkphys and kseg0 have the same offsets.  Objcopy seems to have some
> non-obvious rules for doing address calculations, IE objcopy using
> --change-addresses=X
> 
> 0xa800000000000000 + 0x20004000
> 
> gives something close to (not near my MIPS system atm)
> 
> 0xa7ffffff2001c000

 Hmm, the option seems to work for me as expected.  What version of
objcopy?  What do you use for "X"?  What does `readelf -l' report before
and after copying? 

> So, I'm thinking constructing the address in a register might be easier for now.

 But your kernel really needs to be linked at a KSEG0 address -- if you
are to construct the address manually, the resulting kernel won't work. 

  Maciej

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: 64 to 32 bit jr

[Ralf Baechle]

On Mon, Apr 07, 2003 at 03:01:59PM +0000, Erik J. Green wrote:

> I am unable to arrange my addresses similarly neatly, mostly I think due to
> fighting with the toolchain I have.  Is it "legal" for me to load a
> kernel using the xkphys address and then do something like:

You may map the kernel into XKSEG (0xc000000000000000).  The kernel
already has an option for that, CONFIG_MAPPED_KERNEL.  It's used as a
ccNUMA optimization on large Origin configurations to map a local copy
of the kernel code to that address range.

  Ralf

Re: Unknown ARCS message/hang

[Erik J. Green]

.. and just to display my complete technical mastery, I failed to echo this
message to the list the first time around =\


Quoting Guido Guenther <agx@sigxcpu.org>:
> >  0x211c4018 is a mapped address, which you can't use that early in a boot.
> Isn't 0xa8000000211c4000 in xkphys and therefore unmapped? The PROM only
> seems to look at the lower 32bits of PC though.
> Puzzled,
>  -- Guido

That's what I thought too.  I did notice that the 64 bit kernel seems to refer
to some 32 bit compatibility address spaces, so I'm probably confused on what
gets used when.

FYI, the load address I'm using (0xa800000020004000) is the one specified in the
irix headers for an IP30 kernel (as I read it anyway) and is very close to the
entry point IRIX uses on the same machine.

Erik




-- 
Erik J. Green
erik@greendragon.org

Re: Unknown ARCS message/hang

[Thiemo Seufer]

Erik J. Green wrote:
> 
> .. and just to display my complete technical mastery, I failed to echo this
> message to the list the first time around =\
> 
> 
> Quoting Guido Guenther <agx@sigxcpu.org>:
> > >  0x211c4018 is a mapped address, which you can't use that early in a boot.
> > Isn't 0xa8000000211c4000 in xkphys and therefore unmapped? The PROM only
> > seems to look at the lower 32bits of PC though.
> > Puzzled,
> >  -- Guido
> 
> That's what I thought too.  I did notice that the 64 bit kernel seems to refer
> to some 32 bit compatibility address spaces, so I'm probably confused on what
> gets used when.
> 
> FYI, the load address I'm using (0xa800000020004000) is the one specified in the
> irix headers for an IP30 kernel (as I read it anyway) and is very close to the
> entry point IRIX uses on the same machine.

Current 64-bit kernels are loaded in the 32-bit compatibility space, i.e. the
load address is 0xffffffff8???????. If you want to load to 64-bit space (the
firmware of r10k ip28 needs this, too), you'll have to fix several macro
expansions in the kernel. I did this once for my ip28, but haven't found the
time to make it really work yet.

An outdated patch which covers this and some other issues is available at
http://www.csv.ica.uni-stuttgart.de/homes/ths/linux-mips/kernel/oss-linux-2002-06-05.diff


Thiemo