Troubles with TLB refills

Troubles with TLB refills

[Liam Davies]

I am trying to get the 2.4 kernel up and running on the Cobalt boxes.
At the moment I am trying to get the initial transition from kernel
to user mode working.

The elf loader is trying to put stuff in the stack for the new user
process and *each* call to NEW_AUX_ENTRY is generating a page fault
that cannot be resolved. The address generated by the BadVAddr on the
TLBS exception is not correct. Also, I never receive a TLBrefill
exception on the accesses. It is using the except_vec0_nevada handler.

All that I can think of is that the handler at 0x80000000 is getting
smashed after it is copied in -- ala troubles with _kd_mksound last
year.
Any ideas on what is happening?? how to fix it??


Thanks
Liam


elf_entry=4000b0
elf_bss=10010130
elf_brk=10010130
start_code=400000
end_code =400130
start_data=10010130
end_data =10010130

[sh:1:10004f4c:1:8005f504]
Dump TLB all:

Dump TLB wired:
Wired: 0
Dump TLB address 10004f4c :
No entry for address 0x10004f4c in TLB

vma: start=10010000
vma: flags=    1873
Bad_Area:
no_context:

Re: Troubles with TLB refills

[Ralf Baechle]

On Mon, Mar 05, 2001 at 01:40:01PM +1000, Liam Davies wrote:

> I am trying to get the 2.4 kernel up and running on the Cobalt boxes.
> At the moment I am trying to get the initial transition from kernel
> to user mode working.
> 
> The elf loader is trying to put stuff in the stack for the new user
> process and *each* call to NEW_AUX_ENTRY is generating a page fault
> that cannot be resolved. The address generated by the BadVAddr on the
> TLBS exception is not correct. Also, I never receive a TLBrefill
> exception on the accesses. It is using the except_vec0_nevada handler.

The fault address 0x10004f4c looks wrong; NEW_AUX_ENTRY should access
something near the top of stack, that is a value a few bytes below
0x80000000.  I wonder if this behaviour is related to this CPU bug -
one which btw. was never acknowledged by QED but identified independantly
by sever Cobalt people back at the time.

[head.S]
	/* TLB refill, EXL == 0, R52x0 "Nevada" version */
        /*
         * This version has a bug workaround for the Nevada.  It seems
         * as if under certain circumstances the move from cp0_context
         * might produce a bogus result when the mfc0 instruction and
         * it's consumer are in a different cacheline or a load instruction,
         * probably any memory reference, is between them.  This is
         * potencially slower than the R4000 version, so we use this
         * special version.
         */
	.set	noreorder
	.set	noat
	LEAF(except_vec0_nevada)
	.set	mips3
	mfc0	k0, CP0_BADVADDR		# Get faulting address
	srl	k0, k0, 22			# get pgd only bits
	lw	k1, current_pgd			# get pgd pointer
	sll	k0, k0, 2
	addu	k1, k1, k0			# add in pgd offset
	lw	k1, (k1)
	mfc0	k0, CP0_CONTEXT			# get context reg
	srl	k0, k0, 1			# get pte offset
	and	k0, k0, 0xff8
	addu	k1, k1, k0			# add in offset
	lw	k0, 0(k1)			# get even pte
	lw	k1, 4(k1)			# get odd pte
	srl	k0, k0, 6			# convert to entrylo0
	mtc0	k0, CP0_ENTRYLO0		# load it
	srl	k1, k1, 6			# convert to entrylo1
	mtc0	k1, CP0_ENTRYLO1		# load it
	nop					# QED specified nops
	nop
	tlbwr					# write random tlb entry
	nop					# traditional nop
	eret					# return from trap
	END(except_vec0_nevada)
[...]

This exception handler has been modified since the version tested in the
Cobalt Qube and I'm not sure if the bug workaround actually got tested
since then.

(Bonus points to whoever writes a good probe for this bug!)

handle_page_fault got called and printed something; therefore the
exception handler cannot possibly have been trashed.  do_page_fault gets
called by via the generic exception handler.  The TLB vectors there are only
taken if there is a TLB entry matching the address in the TLB.  Therefore
your theory about no tlb refill exception cannot be right.  The TLB
dump only displays entries where at least on of the entry0 / entry1
entries is valid, therefore you get an empty dump; maybe that made you
believe you didn't get a TLB reload exception.

  Ralf

Re: Troubles with TLB refills

[Liam Davies]

Ralf Baechle wrote:

> This exception handler has been modified since the version tested in the
> Cobalt Qube and I'm not sure if the bug workaround actually got tested
> since then.
>

It seems to work now.

> handle_page_fault got called and printed something; therefore the
> exception handler cannot possibly have been trashed.  do_page_fault gets
> called by via the generic exception handler.  The TLB vectors there are only
> taken if there is a TLB entry matching the address in the TLB.  Therefore
> your theory about no tlb refill exception cannot be right.

And this is the way I understood the workings as well.

> The TLB
> dump only displays entries where at least on of the entry0 / entry1
> entries is valid, therefore you get an empty dump; maybe that made you
> believe you didn't get a TLB reload exception.

No, this is what I expected in the TLB since this was the first kuseg access.

Not long after posting this message I found the problem.  The exception handler

had been trashed, *before* it was copied to 0x80000000. I'm not sure what
trashed it yet, that is a job for later. What made this whole thing more
puzzling
is the actual stuff that was in 0x80000000 was absolute trash, it made no sense

in terms of instruction encodings. I would have thought the cpu would have
crapped out when it hit bad instructions. So it would seem the
exceptions were occurring but the code that it was executing wasn't even code.
Hence my assumption that we never got a TLB refill., even though the fault
handler was being called.

Liam

Re: Troubles with TLB refills

[Liam Davies]

Ralf,

>From the change 1.42 to 1.43 on file arch/mips/kernel/traps.c some code was
added
to copy the EJTAG exception vector

+ /*
+  * Copy the EJTAG debug exception vector handler code to it's final
+  * destination.
+  */
+ memcpy((void *)(KSEG0 + 0x300), &except_vec_ejtag_debug, 0x80);

This code indescriminatly smashes the end of except_vec0_r4600 and
all of except_vec0_nevada handlers with the .fill set to only 0x280
00000000800002d4 T except_vec0_r4600
0000000080000328 T except_vec0_nevada
0000000080000380 T except_vec0_r45k_bvahwbug

I'm not sure under what platform we need to load JTAG support, but we can
just increase the fill area in head.S to say 0x400

Cheers
Liam

=-------------=
Agile TV Corporation

Re: Troubles with TLB refills

[Liam Davies]

Ralf,

From the change 1.42 to 1.43 on file arch/mips/kernel/traps.c some code was
added
to copy the EJTAG exception vector

+ /*
+  * Copy the EJTAG debug exception vector handler code to it's final
+  * destination.
+  */
+ memcpy((void *)(KSEG0 + 0x300), &except_vec_ejtag_debug, 0x80);

This code indescriminatly smashes the end of except_vec0_r4600 and
all of except_vec0_nevada handlers with the .fill set to only 0x280
00000000800002d4 T except_vec0_r4600
0000000080000328 T except_vec0_nevada
0000000080000380 T except_vec0_r45k_bvahwbug

I'm not sure under what platform we need to load JTAG support, but we can
just increase the fill area in head.S to say 0x400

Cheers
Liam

=-------------=
Agile TV Corporation

Re: Troubles with TLB refills

[Ralf Baechle]

On Tue, Mar 06, 2001 at 01:10:27PM +1000, Liam Davies wrote:

> in terms of instruction encodings. I would have thought the cpu would have
> crapped out when it hit bad instructions. So it would seem the
> exceptions were occurring but the code that it was executing wasn't even code.
> Hence my assumption that we never got a TLB refill., even though the fault
> handler was being called.

Probably somewhere in the garbage there was another memory reference
which resulted in a second TLB exception, at that time a store to 0x10004f4c
which then got handled via the general exception handler and resulted in
do_page_fault being called.

  Ralf

Problem makeing an O2 run bootp

[nick]

I've got an o2 that I'm trying to make netboot, and it seems to work,
however the o2 never acks the tftp packets.  The tcpdump is attached.  If
anyone has suggestions/ideas I'd love to hear them.  I booted the o2 and
ran "bootp():" from the arc prompt.
	Thanks
		Nick
22:31:26.697102 arp who-has 10.1.1.3 tell 10.1.1.3
22:31:26.697348 10.1.1.3.bootpc > 255.255.255.255.bootps: xid:0xbbb2
secs:5 C:10.1.1.3 [|bootp]
22:31:26.698900 10.1.1.1.bootps > 10.1.1.3.bootpc: xid:0xbbb2 secs:5
C:10.1.1.3 Y:10.1.1.3 S:10.1.1.1 [|bootp] [tos 0x10]
22:31:26.699191 arp who-has 10.1.1.3 tell 10.1.1.3
22:31:26.710600 10.1.1.3.15283 > 10.1.1.1.tftp: 22 RRQ "vmlinuxo2"
22:31:26.741123 10.1.1.1.35447 > 10.1.1.3.15283: udp 516 (DF)
22:31:31.734006 arp who-has 10.1.1.3 tell 10.1.1.1
22:31:31.734156 arp reply 10.1.1.3 is-at 8:0:69:5:b8:cb
22:31:31.734291 10.1.1.1.35447 > 10.1.1.3.15283: udp 516 (DF)
22:31:36.734065 10.1.1.1.35447 > 10.1.1.3.15283: udp 516 (DF)
22:31:41.734046 10.1.1.1.35447 > 10.1.1.3.15283: udp 516 (DF)
22:31:46.734050 10.1.1.1.35447 > 10.1.1.3.15283: udp 516 (DF)
22:31:56.072778 10.1.1.3.15284 > 10.1.1.1.tftp: 22 RRQ "vmlinuxo2"
22:31:56.102821 10.1.1.1.35447 > 10.1.1.3.15284: udp 516 (DF)
22:32:01.094154 10.1.1.1.35447 > 10.1.1.3.15284: udp 516 (DF)
22:32:06.094045 10.1.1.1.35447 > 10.1.1.3.15284: udp 516 (DF)
22:32:11.094043 10.1.1.1.35447 > 10.1.1.3.15284: udp 516 (DF)
22:32:16.094071 10.1.1.1.35447 > 10.1.1.3.15284: udp 516 (DF)
22:32:21.094002 arp who-has 10.1.1.3 tell 10.1.1.1
22:32:21.094142 arp reply 10.1.1.3 is-at 8:0:69:5:b8:cb
22:32:26.000316 10.1.1.3.15285 > 10.1.1.1.tftp: 22 RRQ "vmlinuxo2"
22:32:26.030369 10.1.1.1.35447 > 10.1.1.3.15285: udp 516 (DF)
22:32:31.024092 10.1.1.1.35447 > 10.1.1.3.15285: udp 516 (DF)
22:32:36.024044 10.1.1.1.35447 > 10.1.1.3.15285: udp 516 (DF)
22:32:41.024045 10.1.1.1.35447 > 10.1.1.3.15285: udp 516 (DF)
22:32:46.024055 10.1.1.1.35447 > 10.1.1.3.15285: udp 516 (DF)
22:32:51.023999 arp who-has 10.1.1.3 tell 10.1.1.1
22:32:51.024138 arp reply 10.1.1.3 is-at 8:0:69:5:b8:cb
22:32:55.927938 10.1.1.3.15286 > 10.1.1.1.tftp: 22 RRQ "vmlinuxo2"
22:32:55.957167 10.1.1.1.35447 > 10.1.1.3.15286: udp 516 (DF)
22:33:00.954096 10.1.1.1.35447 > 10.1.1.3.15286: udp 516 (DF)
22:33:05.954047 10.1.1.1.35447 > 10.1.1.3.15286: udp 516 (DF)
22:33:10.954056 10.1.1.1.35447 > 10.1.1.3.15286: udp 516 (DF)
22:33:15.954051 10.1.1.1.35447 > 10.1.1.3.15286: udp 516 (DF)
22:33:20.954008 arp who-has 10.1.1.3 tell 10.1.1.1
22:33:20.954142 arp reply 10.1.1.3 is-at 8:0:69:5:b8:cb

Problem makeing an O2 run bootp

[nick]

I've got an o2 that I'm trying to make netboot, and it seems to work,
however the o2 never acks the tftp packets.  The tcpdump is attached.  If
anyone has suggestions/ideas I'd love to hear them.  I booted the o2 and
ran "bootp():" from the arc prompt.
	Thanks
		Nick
22:31:26.697102 arp who-has 10.1.1.3 tell 10.1.1.3
22:31:26.697348 10.1.1.3.bootpc > 255.255.255.255.bootps: xid:0xbbb2
secs:5 C:10.1.1.3 [|bootp]
22:31:26.698900 10.1.1.1.bootps > 10.1.1.3.bootpc: xid:0xbbb2 secs:5
C:10.1.1.3 Y:10.1.1.3 S:10.1.1.1 [|bootp] [tos 0x10]
22:31:26.699191 arp who-has 10.1.1.3 tell 10.1.1.3
22:31:26.710600 10.1.1.3.15283 > 10.1.1.1.tftp: 22 RRQ "vmlinuxo2"
22:31:26.741123 10.1.1.1.35447 > 10.1.1.3.15283: udp 516 (DF)
22:31:31.734006 arp who-has 10.1.1.3 tell 10.1.1.1
22:31:31.734156 arp reply 10.1.1.3 is-at 8:0:69:5:b8:cb
22:31:31.734291 10.1.1.1.35447 > 10.1.1.3.15283: udp 516 (DF)
22:31:36.734065 10.1.1.1.35447 > 10.1.1.3.15283: udp 516 (DF)
22:31:41.734046 10.1.1.1.35447 > 10.1.1.3.15283: udp 516 (DF)
22:31:46.734050 10.1.1.1.35447 > 10.1.1.3.15283: udp 516 (DF)
22:31:56.072778 10.1.1.3.15284 > 10.1.1.1.tftp: 22 RRQ "vmlinuxo2"
22:31:56.102821 10.1.1.1.35447 > 10.1.1.3.15284: udp 516 (DF)
22:32:01.094154 10.1.1.1.35447 > 10.1.1.3.15284: udp 516 (DF)
22:32:06.094045 10.1.1.1.35447 > 10.1.1.3.15284: udp 516 (DF)
22:32:11.094043 10.1.1.1.35447 > 10.1.1.3.15284: udp 516 (DF)
22:32:16.094071 10.1.1.1.35447 > 10.1.1.3.15284: udp 516 (DF)
22:32:21.094002 arp who-has 10.1.1.3 tell 10.1.1.1
22:32:21.094142 arp reply 10.1.1.3 is-at 8:0:69:5:b8:cb
22:32:26.000316 10.1.1.3.15285 > 10.1.1.1.tftp: 22 RRQ "vmlinuxo2"
22:32:26.030369 10.1.1.1.35447 > 10.1.1.3.15285: udp 516 (DF)
22:32:31.024092 10.1.1.1.35447 > 10.1.1.3.15285: udp 516 (DF)
22:32:36.024044 10.1.1.1.35447 > 10.1.1.3.15285: udp 516 (DF)
22:32:41.024045 10.1.1.1.35447 > 10.1.1.3.15285: udp 516 (DF)
22:32:46.024055 10.1.1.1.35447 > 10.1.1.3.15285: udp 516 (DF)
22:32:51.023999 arp who-has 10.1.1.3 tell 10.1.1.1
22:32:51.024138 arp reply 10.1.1.3 is-at 8:0:69:5:b8:cb
22:32:55.927938 10.1.1.3.15286 > 10.1.1.1.tftp: 22 RRQ "vmlinuxo2"
22:32:55.957167 10.1.1.1.35447 > 10.1.1.3.15286: udp 516 (DF)
22:33:00.954096 10.1.1.1.35447 > 10.1.1.3.15286: udp 516 (DF)
22:33:05.954047 10.1.1.1.35447 > 10.1.1.3.15286: udp 516 (DF)
22:33:10.954056 10.1.1.1.35447 > 10.1.1.3.15286: udp 516 (DF)
22:33:15.954051 10.1.1.1.35447 > 10.1.1.3.15286: udp 516 (DF)
22:33:20.954008 arp who-has 10.1.1.3 tell 10.1.1.1
22:33:20.954142 arp reply 10.1.1.3 is-at 8:0:69:5:b8:cb

Re: Problem makeing an O2 run bootp

[Rafal Boni]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Content-Type: text/plain; charset=us-ascii

In message <Pine.LNX.4.21.0103062231010.23542-100000@ns>, you write: 

- -> I've got an o2 that I'm trying to make netboot, and it seems to work,
- -> however the o2 never acks the tftp packets.  The tcpdump is attached.  If
- -> anyone has suggestions/ideas I'd love to hear them.  I booted the o2 and
- -> ran "bootp():" from the arc prompt.

I had issues with my Indigo2 where the PROM rejected all TFTP packets 
from ports > 32767 (but that's with a stone-age PROM, I imagine O2's 
would have a somewhat more modern PROM).  

Also, try 'setenv DEBUG 1' (dunno if that does or doesn't work on the
O2, it does on the Indigo2) in the PROM and see if it gives any tell-
tale output.

- --rafal

- ----
Rafal Boni                                              rafal.boni@eDial.com
 PGP key C7D3024C, print EA49 160D F5E4 C46A 9E91  524E 11E0 7133 C7D3 024C
    Need to get a hold of me?  http://800.edial.com/rafal.boni@eDial.com

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.0 (GNU/Linux)
Comment: Exmh version 2.1.1 10/15/1999

iD8DBQE6pofVEeBxM8fTAkwRAmNfAKDsC3yF1WmIZUK9i7Pu+VCR/iy3DACfQjIq
NlO8XXf87pp0cJkVDTw/tAY=
=gc8M
-----END PGP SIGNATURE-----

Re: Problem makeing an O2 run bootp

[nick]

Thanks much.  I'll try that.  It has also been suggested that O2s probably
refuse all packets with DF set, so I will attempt to fix that as well.
	Nick

On Wed, 7 Mar 2001, Rafal Boni wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Content-Type: text/plain; charset=us-ascii
> 
> In message <Pine.LNX.4.21.0103062231010.23542-100000@ns>, you write: 
> 
> - -> I've got an o2 that I'm trying to make netboot, and it seems to work,
> - -> however the o2 never acks the tftp packets.  The tcpdump is attached.  If
> - -> anyone has suggestions/ideas I'd love to hear them.  I booted the o2 and
> - -> ran "bootp():" from the arc prompt.
> 
> I had issues with my Indigo2 where the PROM rejected all TFTP packets 
> from ports > 32767 (but that's with a stone-age PROM, I imagine O2's 
> would have a somewhat more modern PROM).  
> 
> Also, try 'setenv DEBUG 1' (dunno if that does or doesn't work on the
> O2, it does on the Indigo2) in the PROM and see if it gives any tell-
> tale output.
> 
> - --rafal
> 
> - ----
> Rafal Boni                                              rafal.boni@eDial.com
>  PGP key C7D3024C, print EA49 160D F5E4 C46A 9E91  524E 11E0 7133 C7D3 024C
>     Need to get a hold of me?  http://800.edial.com/rafal.boni@eDial.com
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.0.0 (GNU/Linux)
> Comment: Exmh version 2.1.1 10/15/1999
> 
> iD8DBQE6pofVEeBxM8fTAkwRAmNfAKDsC3yF1WmIZUK9i7Pu+VCR/iy3DACfQjIq
> NlO8XXf87pp0cJkVDTw/tAY=
> =gc8M
> -----END PGP SIGNATURE-----
> 

Re: Problem makeing an O2 run bootp

[Chris Ruvolo]

[-- Attachment #1: Type: text/plain, Size: 561 bytes --]

On Tue, Mar 06, 2001 at 10:36:45PM -0500, nick@snowman.net wrote:
> I've got an o2 that I'm trying to make netboot, and it seems to work,
> however the o2 never acks the tftp packets.  The tcpdump is attached.  If
> anyone has suggestions/ideas I'd love to hear them.  I booted the o2 and
> ran "bootp():" from the arc prompt.

I had the same problem with my Indy.  I think this is in the HOWTO now, but
in case you missed it..  If you are running your tftpd on Linux >= 2.3.x 

echo 1 > /proc/sys/net/ipv4/ip_no_pmtu_disc

Worked for me.

-Chris

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: Problem makeing an O2 run bootp

[Ralf Baechle]

On Thu, Mar 08, 2001 at 02:57:51AM -0500, Chris Ruvolo wrote:

> On Tue, Mar 06, 2001 at 10:36:45PM -0500, nick@snowman.net wrote:
> > I've got an o2 that I'm trying to make netboot, and it seems to work,
> > however the o2 never acks the tftp packets.  The tcpdump is attached.  If
> > anyone has suggestions/ideas I'd love to hear them.  I booted the o2 and
> > ran "bootp():" from the arc prompt.
> 
> I had the same problem with my Indy.  I think this is in the HOWTO now, but
> in case you missed it..  If you are running your tftpd on Linux >= 2.3.x 
> 
> echo 1 > /proc/sys/net/ipv4/ip_no_pmtu_disc
> 
> Worked for me.

Yep, it's in there:

  7.6.  My machine doesn't download the kernel when I try to netboot


  Your machine is replying to the BOOTP packets (you may verify this
  using a packet sniffer like tcpdump or ethereal), but doesn't download
  the kernel from your BOOTP server. This happens if your boot server is
  running a kernel of the 2.3 series or higher. The problem may be
  circumvented by doing a "echo 1 > /proc/sys/net/ipv4/ip_no_pmtu_disc"
  as root on your boot server.

  Ralf

IP32 (O2) diff

[nick]

http://www.snowman.net/~nick/ip32.diff.bz2

The above is a link to my patch against the current CVS source that sort
of works on the SGI O2.  It boots, but does not support any consoles, SCSI
drivers, or ethernet drivers.  Any assistance is greatly apreiciated.
	Nick