newest kernel

newest kernel

[George Gensure]

I build the 2.4.3 #5 kernel from the CVS source, and have had some 
trouble with specific programs.  I get illegal instruction errors on 
find and tar.  Is there anything I can do to correct this?

George
werkt@csh.rit.edu

Re: newest kernel

[Pete Popov]

[-- Attachment #1: Type: text/plain, Size: 441 bytes --]

George Gensure wrote:

> I build the 2.4.3 #5 kernel from the CVS source, and have had some  
> trouble with specific programs.  I get illegal instruction errors on  
> find and tar.  Is there anything I can do to correct this?

I'm guessing you're running into the sys_sysmips problem. A patch has 
not yet been applied to the oss.sgi.com kernel.  I've attached a patch 
from Florian that seems to work well and should apply cleanly.

Pete

[-- Attachment #2: sysmips_02.patch --]
[-- Type: text/plain, Size: 3604 bytes --]

diff -Nur linux.orig/arch/mips/kernel/Makefile linux/arch/mips/kernel/Makefile
--- linux.orig/arch/mips/kernel/Makefile	Mon Apr  9 00:23:08 2001
+++ linux/arch/mips/kernel/Makefile	Mon Apr  9 00:23:34 2001
@@ -20,7 +20,7 @@
 obj-y				+= branch.o process.o signal.o entry.o \
 				   traps.o ptrace.o vm86.o ioport.o reset.o \
 				   semaphore.o setup.o syscall.o sysmips.o \
-				   ipc.o scall_o32.o unaligned.o
+				   ipc.o scall_o32.o unaligned.o fast-sysmips.o
 obj-$(CONFIG_MODULES)		+= mips_ksyms.o
 
 ifdef CONFIG_CPU_R3000
@@ -69,5 +69,6 @@
 
 entry.o: entry.S
 head.o: head.S
+fast-sysmips.o: fast-sysmips.S
 
 include $(TOPDIR)/Rules.make
diff -Nur linux.orig/arch/mips/kernel/fast-sysmips.S linux/arch/mips/kernel/fast-sysmips.S
--- linux.orig/arch/mips/kernel/fast-sysmips.S	Thu Jan  1 01:00:00 1970
+++ linux/arch/mips/kernel/fast-sysmips.S	Mon Apr  9 00:28:20 2001
@@ -0,0 +1,85 @@
+/*
+ * MIPS_ATOMIC_SET asm implementation for ll/sc capable cpus
+ *
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License.  See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2001 Florian Lohoff <flo@rfc822.org>
+ *
+ */
+#include <asm/asm.h>
+#include <asm/mipsregs.h>
+#include <asm/regdef.h>
+#include <asm/stackframe.h>
+#include <asm/isadep.h>
+#include <asm/unistd.h>
+#include <asm/sysmips.h>
+#include <asm/offset.h>
+#include <asm/errno.h>
+
+#define PT_TRACESYS     0x00000002
+
+	EXPORT(fast_sysmips)
+
+	.set	noreorder
+
+	li	t0, MIPS_ATOMIC_SET
+	beq	a0, t0, 1f
+	 nop
+	j	sys_sysmips
+	 nop
+
+1:
+
+	# a0 - MIPS_ATOMIC_SET
+	# a1 - mem ptr
+	# a2 - value
+
+	addiu	sp, sp, -8			# Reserve space
+	sw	a0, (sp)			# Save arg0
+
+	addiu	a0, a1, 4			# addr+size
+	ori	v0, a1, 4			# addr | size
+	lw	v1, THREAD_CURDS(gp)		# current->thread.current_ds
+	or	v0, v0, a0			# addr | size | (addr+size)
+	and	v1, v1, v0			# (mask)&(addr | size | (addr+size)
+	bltz	v1, 5f
+	 nop
+
+2:
+	ll	v0, (a1)
+	move	t0, a2
+	sc	t0, (a1)
+	beqz	t0, 2b
+	 nop
+
+	sw	v0, PT_R2+8(sp)			# Result value
+	sw	zero, PT_R7+8(sp)		# Success indicator
+
+	lw      t0, TASK_PTRACE(gp)		# syscall tracing enabled?
+	andi    t0, PT_TRACESYS
+	bnez    t0, 3f
+	 nop
+
+4:
+	lw	a0, (sp)			# Restore arg0
+	addiu	sp, sp, 8			# Restore sp
+
+	j	o32_ret_from_sys_call
+	 nop
+
+3:
+	sw	ra, 4(sp)
+	jal	syscall_trace
+	 nop
+	lw	ra, 4(sp)
+	j	4b
+	 nop
+
+5:
+	lw	a0, (sp)			# Restore arg0
+	addiu	sp, sp, 8			# Restore sp
+	j	ra
+	 li	v0, -EFAULT
+
diff -Nur linux.orig/arch/mips/kernel/irix5sys.h linux/arch/mips/kernel/irix5sys.h
--- linux.orig/arch/mips/kernel/irix5sys.h	Mon Apr  9 00:16:29 2001
+++ linux/arch/mips/kernel/irix5sys.h	Sun Apr  8 21:21:16 2001
@@ -69,7 +69,7 @@
 SYS(irix_getgid, 0)			/* 1047  getgid()	       V*/
 SYS(irix_unimp, 0)			/* 1048  (XXX IRIX 4 ssig)     V*/
 SYS(irix_msgsys, 6)			/* 1049  sys_msgsys	       V*/
-SYS(sys_sysmips, 4)			/* 1050  sysmips()	      HV*/
+SYS(fast_sysmips, 4)			/* 1050  sysmips()	      HV*/
 SYS(irix_unimp, 0)			/* 1051	 XXX sysacct()	      IV*/
 SYS(irix_shmsys, 5)			/* 1052  sys_shmsys	       V*/
 SYS(irix_semsys, 0)			/* 1053  sys_semsys	       V*/
diff -Nur linux.orig/arch/mips/kernel/syscalls.h linux/arch/mips/kernel/syscalls.h
--- linux.orig/arch/mips/kernel/syscalls.h	Mon Apr  9 00:16:30 2001
+++ linux/arch/mips/kernel/syscalls.h	Sun Apr  8 21:21:43 2001
@@ -163,7 +163,7 @@
 SYS(sys_writev, 3)
 SYS(sys_cacheflush, 3)
 SYS(sys_cachectl, 3)
-SYS(sys_sysmips, 4)
+SYS(fast_sysmips, 4)
 SYS(sys_ni_syscall, 0)				/* 4150 */
 SYS(sys_getsid, 1)
 SYS(sys_fdatasync, 0)

Re: newest kernel

[Jun Sun]

The patch seems to be just a fast implementation of sysmips().  Why would it
solve an otherwise illegal instruction problem?

George, what was exactly the error and the faulty instruction?

Jun

Pete Popov wrote:
> 
> George Gensure wrote:
> 
> > I build the 2.4.3 #5 kernel from the CVS source, and have had some
> > trouble with specific programs.  I get illegal instruction errors on
> > find and tar.  Is there anything I can do to correct this?
> 
> I'm guessing you're running into the sys_sysmips problem. A patch has
> not yet been applied to the oss.sgi.com kernel.  I've attached a patch
> from Florian that seems to work well and should apply cleanly.
> 
> Pete
> 
>   ------------------------------------------------------------------------------
> diff -Nur linux.orig/arch/mips/kernel/Makefile linux/arch/mips/kernel/Makefile
> --- linux.orig/arch/mips/kernel/Makefile        Mon Apr  9 00:23:08 2001
> +++ linux/arch/mips/kernel/Makefile     Mon Apr  9 00:23:34 2001
> @@ -20,7 +20,7 @@
>  obj-y                          += branch.o process.o signal.o entry.o \
>                                    traps.o ptrace.o vm86.o ioport.o reset.o \
>                                    semaphore.o setup.o syscall.o sysmips.o \
> -                                  ipc.o scall_o32.o unaligned.o
> +                                  ipc.o scall_o32.o unaligned.o fast-sysmips.o
>  obj-$(CONFIG_MODULES)          += mips_ksyms.o
> 
>  ifdef CONFIG_CPU_R3000
> @@ -69,5 +69,6 @@
> 
>  entry.o: entry.S
>  head.o: head.S
> +fast-sysmips.o: fast-sysmips.S
> 
>  include $(TOPDIR)/Rules.make
> diff -Nur linux.orig/arch/mips/kernel/fast-sysmips.S linux/arch/mips/kernel/fast-sysmips.S
> --- linux.orig/arch/mips/kernel/fast-sysmips.S  Thu Jan  1 01:00:00 1970
> +++ linux/arch/mips/kernel/fast-sysmips.S       Mon Apr  9 00:28:20 2001
> @@ -0,0 +1,85 @@
> +/*
> + * MIPS_ATOMIC_SET asm implementation for ll/sc capable cpus
> + *
> + * This file is subject to the terms and conditions of the GNU General Public
> + * License.  See the file "COPYING" in the main directory of this archive
> + * for more details.
> + *
> + * Copyright (C) 2001 Florian Lohoff <flo@rfc822.org>
> + *
> + */
> +#include <asm/asm.h>
> +#include <asm/mipsregs.h>
> +#include <asm/regdef.h>
> +#include <asm/stackframe.h>
> +#include <asm/isadep.h>
> +#include <asm/unistd.h>
> +#include <asm/sysmips.h>
> +#include <asm/offset.h>
> +#include <asm/errno.h>
> +
> +#define PT_TRACESYS     0x00000002
> +
> +       EXPORT(fast_sysmips)
> +
> +       .set    noreorder
> +
> +       li      t0, MIPS_ATOMIC_SET
> +       beq     a0, t0, 1f
> +        nop
> +       j       sys_sysmips
> +        nop
> +
> +1:
> +
> +       # a0 - MIPS_ATOMIC_SET
> +       # a1 - mem ptr
> +       # a2 - value
> +
> +       addiu   sp, sp, -8                      # Reserve space
> +       sw      a0, (sp)                        # Save arg0
> +
> +       addiu   a0, a1, 4                       # addr+size
> +       ori     v0, a1, 4                       # addr | size
> +       lw      v1, THREAD_CURDS(gp)            # current->thread.current_ds
> +       or      v0, v0, a0                      # addr | size | (addr+size)
> +       and     v1, v1, v0                      # (mask)&(addr | size | (addr+size)
> +       bltz    v1, 5f
> +        nop
> +
> +2:
> +       ll      v0, (a1)
> +       move    t0, a2
> +       sc      t0, (a1)
> +       beqz    t0, 2b
> +        nop
> +
> +       sw      v0, PT_R2+8(sp)                 # Result value
> +       sw      zero, PT_R7+8(sp)               # Success indicator
> +
> +       lw      t0, TASK_PTRACE(gp)             # syscall tracing enabled?
> +       andi    t0, PT_TRACESYS
> +       bnez    t0, 3f
> +        nop
> +
> +4:
> +       lw      a0, (sp)                        # Restore arg0
> +       addiu   sp, sp, 8                       # Restore sp
> +
> +       j       o32_ret_from_sys_call
> +        nop
> +
> +3:
> +       sw      ra, 4(sp)
> +       jal     syscall_trace
> +        nop
> +       lw      ra, 4(sp)
> +       j       4b
> +        nop
> +
> +5:
> +       lw      a0, (sp)                        # Restore arg0
> +       addiu   sp, sp, 8                       # Restore sp
> +       j       ra
> +        li     v0, -EFAULT
> +
> diff -Nur linux.orig/arch/mips/kernel/irix5sys.h linux/arch/mips/kernel/irix5sys.h
> --- linux.orig/arch/mips/kernel/irix5sys.h      Mon Apr  9 00:16:29 2001
> +++ linux/arch/mips/kernel/irix5sys.h   Sun Apr  8 21:21:16 2001
> @@ -69,7 +69,7 @@
>  SYS(irix_getgid, 0)                    /* 1047  getgid()              V*/
>  SYS(irix_unimp, 0)                     /* 1048  (XXX IRIX 4 ssig)     V*/
>  SYS(irix_msgsys, 6)                    /* 1049  sys_msgsys            V*/
> -SYS(sys_sysmips, 4)                    /* 1050  sysmips()            HV*/
> +SYS(fast_sysmips, 4)                   /* 1050  sysmips()            HV*/
>  SYS(irix_unimp, 0)                     /* 1051  XXX sysacct()        IV*/
>  SYS(irix_shmsys, 5)                    /* 1052  sys_shmsys            V*/
>  SYS(irix_semsys, 0)                    /* 1053  sys_semsys            V*/
> diff -Nur linux.orig/arch/mips/kernel/syscalls.h linux/arch/mips/kernel/syscalls.h
> --- linux.orig/arch/mips/kernel/syscalls.h      Mon Apr  9 00:16:30 2001
> +++ linux/arch/mips/kernel/syscalls.h   Sun Apr  8 21:21:43 2001
> @@ -163,7 +163,7 @@
>  SYS(sys_writev, 3)
>  SYS(sys_cacheflush, 3)
>  SYS(sys_cachectl, 3)
> -SYS(sys_sysmips, 4)
> +SYS(fast_sysmips, 4)
>  SYS(sys_ni_syscall, 0)                         /* 4150 */
>  SYS(sys_getsid, 1)
>  SYS(sys_fdatasync, 0)

Re: newest kernel

[Pete Popov]

Jun Sun wrote:

>The patch seems to be just a fast implementation of sysmips().  Why would it
>solve an otherwise illegal instruction problem?
>
>George, what was exactly the error and the faulty instruction?
>
This is the pthread patch, so it's not just fast sysmips.  I think 
George's tar and find are probably linked with pthread, thus causing the 
problem. 

Pete

>Jun
>
>Pete Popov wrote:
>
>>George Gensure wrote:
>>
>>>I build the 2.4.3 #5 kernel from the CVS source, and have had some
>>>trouble with specific programs.  I get illegal instruction errors on
>>>find and tar.  Is there anything I can do to correct this?
>>>
>>I'm guessing you're running into the sys_sysmips problem. A patch has
>>not yet been applied to the oss.sgi.com kernel.  I've attached a patch
>>from Florian that seems to work well and should apply cleanly.
>>
>>Pete
>>
>>  ------------------------------------------------------------------------------
>>diff -Nur linux.orig/arch/mips/kernel/Makefile linux/arch/mips/kernel/Makefile
>>--- linux.orig/arch/mips/kernel/Makefile        Mon Apr  9 00:23:08 2001
>>+++ linux/arch/mips/kernel/Makefile     Mon Apr  9 00:23:34 2001
>>@@ -20,7 +20,7 @@
>> obj-y                          += branch.o process.o signal.o entry.o \
>>                                   traps.o ptrace.o vm86.o ioport.o reset.o \
>>                                   semaphore.o setup.o syscall.o sysmips.o \
>>-                                  ipc.o scall_o32.o unaligned.o
>>+                                  ipc.o scall_o32.o unaligned.o fast-sysmips.o
>> obj-$(CONFIG_MODULES)          += mips_ksyms.o
>>
>> ifdef CONFIG_CPU_R3000
>>@@ -69,5 +69,6 @@
>>
>> entry.o: entry.S
>> head.o: head.S
>>+fast-sysmips.o: fast-sysmips.S
>>
>> include $(TOPDIR)/Rules.make
>>diff -Nur linux.orig/arch/mips/kernel/fast-sysmips.S linux/arch/mips/kernel/fast-sysmips.S
>>--- linux.orig/arch/mips/kernel/fast-sysmips.S  Thu Jan  1 01:00:00 1970
>>+++ linux/arch/mips/kernel/fast-sysmips.S       Mon Apr  9 00:28:20 2001
>>@@ -0,0 +1,85 @@
>>+/*
>>+ * MIPS_ATOMIC_SET asm implementation for ll/sc capable cpus
>>+ *
>>+ * This file is subject to the terms and conditions of the GNU General Public
>>+ * License.  See the file "COPYING" in the main directory of this archive
>>+ * for more details.
>>+ *
>>+ * Copyright (C) 2001 Florian Lohoff <flo@rfc822.org>
>>+ *
>>+ */
>>+#include <asm/asm.h>
>>+#include <asm/mipsregs.h>
>>+#include <asm/regdef.h>
>>+#include <asm/stackframe.h>
>>+#include <asm/isadep.h>
>>+#include <asm/unistd.h>
>>+#include <asm/sysmips.h>
>>+#include <asm/offset.h>
>>+#include <asm/errno.h>
>>+
>>+#define PT_TRACESYS     0x00000002
>>+
>>+       EXPORT(fast_sysmips)
>>+
>>+       .set    noreorder
>>+
>>+       li      t0, MIPS_ATOMIC_SET
>>+       beq     a0, t0, 1f
>>+        nop
>>+       j       sys_sysmips
>>+        nop
>>+
>>+1:
>>+
>>+       # a0 - MIPS_ATOMIC_SET
>>+       # a1 - mem ptr
>>+       # a2 - value
>>+
>>+       addiu   sp, sp, -8                      # Reserve space
>>+       sw      a0, (sp)                        # Save arg0
>>+
>>+       addiu   a0, a1, 4                       # addr+size
>>+       ori     v0, a1, 4                       # addr | size
>>+       lw      v1, THREAD_CURDS(gp)            # current->thread.current_ds
>>+       or      v0, v0, a0                      # addr | size | (addr+size)
>>+       and     v1, v1, v0                      # (mask)&(addr | size | (addr+size)
>>+       bltz    v1, 5f
>>+        nop
>>+
>>+2:
>>+       ll      v0, (a1)
>>+       move    t0, a2
>>+       sc      t0, (a1)
>>+       beqz    t0, 2b
>>+        nop
>>+
>>+       sw      v0, PT_R2+8(sp)                 # Result value
>>+       sw      zero, PT_R7+8(sp)               # Success indicator
>>+
>>+       lw      t0, TASK_PTRACE(gp)             # syscall tracing enabled?
>>+       andi    t0, PT_TRACESYS
>>+       bnez    t0, 3f
>>+        nop
>>+
>>+4:
>>+       lw      a0, (sp)                        # Restore arg0
>>+       addiu   sp, sp, 8                       # Restore sp
>>+
>>+       j       o32_ret_from_sys_call
>>+        nop
>>+
>>+3:
>>+       sw      ra, 4(sp)
>>+       jal     syscall_trace
>>+        nop
>>+       lw      ra, 4(sp)
>>+       j       4b
>>+        nop
>>+
>>+5:
>>+       lw      a0, (sp)                        # Restore arg0
>>+       addiu   sp, sp, 8                       # Restore sp
>>+       j       ra
>>+        li     v0, -EFAULT
>>+
>>diff -Nur linux.orig/arch/mips/kernel/irix5sys.h linux/arch/mips/kernel/irix5sys.h
>>--- linux.orig/arch/mips/kernel/irix5sys.h      Mon Apr  9 00:16:29 2001
>>+++ linux/arch/mips/kernel/irix5sys.h   Sun Apr  8 21:21:16 2001
>>@@ -69,7 +69,7 @@
>> SYS(irix_getgid, 0)                    /* 1047  getgid()              V*/
>> SYS(irix_unimp, 0)                     /* 1048  (XXX IRIX 4 ssig)     V*/
>> SYS(irix_msgsys, 6)                    /* 1049  sys_msgsys            V*/
>>-SYS(sys_sysmips, 4)                    /* 1050  sysmips()            HV*/
>>+SYS(fast_sysmips, 4)                   /* 1050  sysmips()            HV*/
>> SYS(irix_unimp, 0)                     /* 1051  XXX sysacct()        IV*/
>> SYS(irix_shmsys, 5)                    /* 1052  sys_shmsys            V*/
>> SYS(irix_semsys, 0)                    /* 1053  sys_semsys            V*/
>>diff -Nur linux.orig/arch/mips/kernel/syscalls.h linux/arch/mips/kernel/syscalls.h
>>--- linux.orig/arch/mips/kernel/syscalls.h      Mon Apr  9 00:16:30 2001
>>+++ linux/arch/mips/kernel/syscalls.h   Sun Apr  8 21:21:43 2001
>>@@ -163,7 +163,7 @@
>> SYS(sys_writev, 3)
>> SYS(sys_cacheflush, 3)
>> SYS(sys_cachectl, 3)
>>-SYS(sys_sysmips, 4)
>>+SYS(fast_sysmips, 4)
>> SYS(sys_ni_syscall, 0)                         /* 4150 */
>> SYS(sys_getsid, 1)
>> SYS(sys_fdatasync, 0)
>>

Re: newest kernel

[Florian Lohoff]

On Mon, May 21, 2001 at 04:23:52PM -0700, Jun Sun wrote:
> The patch seems to be just a fast implementation of sysmips().  Why would it
> solve an otherwise illegal instruction problem?
> 
> George, what was exactly the error and the faulty instruction?

Wrong - Its not only a "fast" path sysmips. It solves the illegal instruction
case as it carefully doesnt touch registers it should not touch.

The sysmips illegal instruction stuff came from the early exit
needed to skip the -EXXXX case in the scall32.S which did not
restore the modified registers. This needed fixing and there was
no clean way of doing this in C thus i wrote an asm sysmips/MIPS_ATOMIC_SET
and called it "fast_sysmips" which itself would go into the old
sysmips function when not MIPS_ATOMIC_SET.

Flo
-- 
Florian Lohoff                  flo@rfc822.org             +49-5201-669912
     Why is it called "common sense" when nobody seems to have any?

Re: newest kernel

[George Gensure]

Good call.  Thanks everyone, that did it. now if i could just get my cross compiled gcc 3.1 to realize that crti.o exists and should be linked in with libgcc...

George
werkt@csh.rit.edu

On Tue, 22 May 2001, Florian Lohoff wrote:

> On Mon, May 21, 2001 at 04:23:52PM -0700, Jun Sun wrote:
> > The patch seems to be just a fast implementation of sysmips().  Why would it
> > solve an otherwise illegal instruction problem?
> >
> > George, what was exactly the error and the faulty instruction?
>
> Wrong - Its not only a "fast" path sysmips. It solves the illegal instruction
> case as it carefully doesnt touch registers it should not touch.
>
> The sysmips illegal instruction stuff came from the early exit
> needed to skip the -EXXXX case in the scall32.S which did not
> restore the modified registers. This needed fixing and there was
> no clean way of doing this in C thus i wrote an asm sysmips/MIPS_ATOMIC_SET
> and called it "fast_sysmips" which itself would go into the old
> sysmips function when not MIPS_ATOMIC_SET.
>
> Flo
>

Re: newest kernel

[George Gensure]

Good call.  Thanks everyone, that did it. now if i could just get my cross compiled gcc 3.1 to realize that crti.o exists and should be linked in with libgcc...

George
werkt@csh.rit.edu

On Tue, 22 May 2001, Florian Lohoff wrote:

> On Mon, May 21, 2001 at 04:23:52PM -0700, Jun Sun wrote:
> > The patch seems to be just a fast implementation of sysmips().  Why would it
> > solve an otherwise illegal instruction problem?
> >
> > George, what was exactly the error and the faulty instruction?
>
> Wrong - Its not only a "fast" path sysmips. It solves the illegal instruction
> case as it carefully doesnt touch registers it should not touch.
>
> The sysmips illegal instruction stuff came from the early exit
> needed to skip the -EXXXX case in the scall32.S which did not
> restore the modified registers. This needed fixing and there was
> no clean way of doing this in C thus i wrote an asm sysmips/MIPS_ATOMIC_SET
> and called it "fast_sysmips" which itself would go into the old
> sysmips function when not MIPS_ATOMIC_SET.
>
> Flo
>

MIPS_ATOMIC_SET again (Re: newest kernel

[Jun Sun]

Florian Lohoff wrote:
> 
> On Mon, May 21, 2001 at 04:23:52PM -0700, Jun Sun wrote:
> > The patch seems to be just a fast implementation of sysmips().  Why would it
> > solve an otherwise illegal instruction problem?
> >
> > George, what was exactly the error and the faulty instruction?
> 
> Wrong - Its not only a "fast" path sysmips. It solves the illegal instruction
> case as it carefully doesnt touch registers it should not touch.
> 
> The sysmips illegal instruction stuff came from the early exit
> needed to skip the -EXXXX case in the scall32.S which did not
> restore the modified registers. This needed fixing and there was
> no clean way of doing this in C thus i wrote an asm sysmips/MIPS_ATOMIC_SET
> and called it "fast_sysmips" which itself would go into the old
> sysmips function when not MIPS_ATOMIC_SET.
> 

I see.

I took a look of MIPS ABI in system V and found that the spec only specifies
this extended call in C prototype:

int _test_and_set(int *p, int v);

It seems perfectly legal for us to add one more argument to store the return
value while still have the function returns error.  Of course, doing that will
break binary compatibility.

Otherwise, I think Flo's patch is the best fix to satisfy the spec and binary
compatibility although it is a little clunky.

A third possibility is the have a MIPS_NEW_ATOMIC_SET that take three
arguments.  If that approach is taken, I would take out the inline assembly
that jumps to o32_ret_from_sys_call and documents MIPS_ATOMIC_SET as
deprecated and valnerable.

My preference, in the decreasing order, is 3), 2) and 1).

Ralf, what do you think?  We cannot let the bug sit around in the CVS tree for
long.  Have to have some fix.

Jun

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Joe deBlaquiere]

I would vote for option #4 - make sure the ll/sc emulation stuff works 
and use ll/sc in glibc instead of sysmips. Beyond the pthreads mutex 
stuff in glibc I have yet to come across usage of sysmips. Of course you 
still need sys_sysmips to function correctly (in case somebody did a 
silly thing like call sysmips directly just for the fun of it), so I 
like like Florian's solution. Adding a parameter is a silly thing to do, 
and we don't need to be adding functionality to sys_sysmips through 
NEW_MO_BETTER_AS_SEEN_ON_TV_ATOMIC_SET or what have you...

Joe

Jun Sun wrote:

> Florian Lohoff wrote:
> 
>> On Mon, May 21, 2001 at 04:23:52PM -0700, Jun Sun wrote:
>> 
>>> The patch seems to be just a fast implementation of sysmips().  Why would it
>>> solve an otherwise illegal instruction problem?
>>> 
>>> George, what was exactly the error and the faulty instruction?
>> 
>> Wrong - Its not only a "fast" path sysmips. It solves the illegal instruction
>> case as it carefully doesnt touch registers it should not touch.
>> 
>> The sysmips illegal instruction stuff came from the early exit
>> needed to skip the -EXXXX case in the scall32.S which did not
>> restore the modified registers. This needed fixing and there was
>> no clean way of doing this in C thus i wrote an asm sysmips/MIPS_ATOMIC_SET
>> and called it "fast_sysmips" which itself would go into the old
>> sysmips function when not MIPS_ATOMIC_SET.
>> 
> 
> 
> I see.
> 
> I took a look of MIPS ABI in system V and found that the spec only specifies
> this extended call in C prototype:
> 
> int _test_and_set(int *p, int v);
> 
> It seems perfectly legal for us to add one more argument to store the return
> value while still have the function returns error.  Of course, doing that will
> break binary compatibility.
> 
> Otherwise, I think Flo's patch is the best fix to satisfy the spec and binary
> compatibility although it is a little clunky.
> 
> A third possibility is the have a MIPS_NEW_ATOMIC_SET that take three
> arguments.  If that approach is taken, I would take out the inline assembly
> that jumps to o32_ret_from_sys_call and documents MIPS_ATOMIC_SET as
> deprecated and valnerable.
> 
> My preference, in the decreasing order, is 3), 2) and 1).
> 
> Ralf, what do you think?  We cannot let the bug sit around in the CVS tree for
> long.  Have to have some fix.
> 
> Jun


-- 
Joe deBlaquiere
Red Hat, Inc.
307 Wynn Drive
Huntsville AL, 35805
voice : (256)-704-9200
fax   : (256)-837-3839

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Tue, 22 May 2001, Jun Sun wrote:

> I took a look of MIPS ABI in system V and found that the spec only specifies
> this extended call in C prototype:
> 
> int _test_and_set(int *p, int v);
> 
> It seems perfectly legal for us to add one more argument to store the return
> value while still have the function returns error.  Of course, doing that will
> break binary compatibility.

 We have _test_and_set() in the library.  Implementing a clean underlying
_test_and_set() syscall is very high on my to-do list -- lack of time
prevents me from finishing it, unfortunately.

 There is no point to mess with sysmips() any further, I think.  The
library's _test_and_set() only calls sysmips() if the library was compiled
for ISA I systems.  As I guess from reports on the list, ISA I systems are
a minority, mostly DECstations and possibly a few embedded systems.  Most
people have ISA II+ and they do not need to call any syscall from
_test_and_set() at all.  For ISA II+ systems the library implements
_test_and_set() in the userland, using ll/sc appropriately. 

 Anyone having ISA II+ systems only, please do yourself a favour and set
"-mips2" (or maybe even "-mips3") somewhere in your CFLAGS for all
compilations -- not only you'll get better optimized binaries, but you'll
get rid of this sysmips() problem as well. 

> Otherwise, I think Flo's patch is the best fix to satisfy the spec and binary
> compatibility although it is a little clunky.

 I'll have yet to look at the patch, but what I think is, we should get
sysmips() work as defined by the original spec (or as reverse-engineered,
as the real spec seems to be out of reach).  Everything else belong to a
separate implementation. 

 Binary compatibility is not a necessity here.  The only sysmips() client
is glibc at the moment, and it can be updated to use a new syscall at any
time. 

 In short: let's leave sysmips() semantics alone.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Wed, 23 May 2001, Joe deBlaquiere wrote:

> I would vote for option #4 - make sure the ll/sc emulation stuff works 
> and use ll/sc in glibc instead of sysmips. Beyond the pthreads mutex 

 Please don't.  The emulation is an overkill here and the overhead is
painful for ISA I systems, which are usually not the fastest ones.  This
has already been discussed here.

 If you want to go for speed and use ll/sc on an ISA II+ system, then
compile glibc for ISA II or better.  It will never call sysmips() then. 

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

wrt irc

[joshua]

The other day on irc.openprojects.net #mipslinux, someone said that the
initial O2 port was done using the irix header files to glean enough
hardware info to make the needed changes.  How would this process
work?  Could someone post examples of how parts of the header files
indicate how to get something else to boot on the hardware?

Re: porting from headers

[Keith M Wesolowski]

On Wed, May 23, 2001 at 09:45:54AM -0400, joshua@babbage.millersv.edu wrote:

> The other day on irc.openprojects.net #mipslinux, someone said that the
> initial O2 port was done using the irix header files to glean enough
> hardware info to make the needed changes.  How would this process
> work?  Could someone post examples of how parts of the header files
> indicate how to get something else to boot on the hardware?

Start with common sense: 

1. It's ARCS, so prom_printf and the ARCS memory map work.  2. It's
got an already-supported MIPS CPU.

That's enough if you don't screw it up to get to the having no root
and must scream point.  Then, you look at the headers and see that it
has two 16550 serial ports, and you know their addresses.  Plug those
into the standard peecee serial driver and you have a serial console.
Likewise the RTC.  You look at crime.h and see the CRIME_TIME
mechanism, along with its frequency, and you guess that you can
calibrate your system timer from that.  Now your timers are right.

You want PCI, that's another matter; you can pull out the config space
pointers and read/write the config space rather easily from the
headers, and likewise it gives you the CPU physical addresses for
device BARs.  But nobody will tell you that most of these spaces need
to be swapped... :-)

Basically the headers give you the addresses of registers and
occasionally their formats.  If you're clever you can figure out how
to go from there.  I'm not so clever so it's been something like 8
months I've been fooling with this.

-- 
Keith M Wesolowski <wesolows@foobazco.org> http://foobazco.org/~wesolows
------(( Project Foobazco Coordinator and Network Administrator ))------
	"Nothing motivates a man more than to see his boss put
	 in an honest day's work." -- The fortune file

Re: wrt irc

[nick]

Most of the important addresses are listed in the irix headers.  With
enough hardware and knowledge you can infer how those addresses are used,
and I belive he also decompiled the o2's irix kernel.
	Nick

On Wed, 23 May 2001 joshua@babbage.millersv.edu wrote:

> The other day on irc.openprojects.net #mipslinux, someone said that the
> initial O2 port was done using the irix header files to glean enough
> hardware info to make the needed changes.  How would this process
> work?  Could someone post examples of how parts of the header files
> indicate how to get something else to boot on the hardware?
> 
> 

Re: wrt irc

[Keith M Wesolowski]

On Wed, May 23, 2001 at 11:55:29AM -0400, nick@snowman.net wrote:

> Most of the important addresses are listed in the irix headers.  With
> enough hardware and knowledge you can infer how those addresses are used,
> and I belive he also decompiled the o2's irix kernel.

Yes, I did; it was a great way to waste a few hours.  Nothing of
interest came of that.  IRIX seems to have many layers of indirection
before you actually find the code you want.  Not recommended.

-- 
Keith M Wesolowski <wesolows@foobazco.org> http://foobazco.org/~wesolows
------(( Project Foobazco Coordinator and Network Administrator ))------
	"Nothing motivates a man more than to see his boss put
	 in an honest day's work." -- The fortune file

Re: wrt irc

[nick]

I agree that the IRIX kernel is horribly obfuscated, however it still
contains valuable information, and in theory can be decoded.  I strongly
suspect this is where harold got much of his information, and if one is
demented enough, and has enough time much can be learned from
it.  Curiousity, what is your goal in asking this?  There is no magic
answer, and it will probably take you months to years if you want to port
something useing just the irix headers.
	Nick

On Wed, 23 May 2001, Keith M Wesolowski wrote:

> On Wed, May 23, 2001 at 11:55:29AM -0400, nick@snowman.net wrote:
> 
> > Most of the important addresses are listed in the irix headers.  With
> > enough hardware and knowledge you can infer how those addresses are used,
> > and I belive he also decompiled the o2's irix kernel.
> 
> Yes, I did; it was a great way to waste a few hours.  Nothing of
> interest came of that.  IRIX seems to have many layers of indirection
> before you actually find the code you want.  Not recommended.
> 
> -- 
> Keith M Wesolowski <wesolows@foobazco.org> http://foobazco.org/~wesolows
> ------(( Project Foobazco Coordinator and Network Administrator ))------
> 	"Nothing motivates a man more than to see his boss put
> 	 in an honest day's work." -- The fortune file
> 

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Jun Sun]

Joe deBlaquiere wrote:
> 
> I would vote for option #4 - make sure the ll/sc emulation stuff works
> and use ll/sc in glibc instead of sysmips. Beyond the pthreads mutex
> stuff in glibc I have yet to come across usage of sysmips. Of course you
> still need sys_sysmips to function correctly (in case somebody did a
> silly thing like call sysmips directly just for the fun of it), so I
> like like Florian's solution.

> Adding a parameter is a silly thing to do,
> and we don't need to be adding functionality to sys_sysmips through
> NEW_MO_BETTER_AS_SEEN_ON_TV_ATOMIC_SET or what have you...
> 

I disagree with the ll/sc emulation approach.

. ll/sc is difficult to emulate  (as anybody who has tried will know)

. ll/sc takes a much bigger performance hit.   It takes at least two syscalls
to complete a sequence of ll/sc instructions.  In addition, since each system
call takes so much time, it increases the likelihood for the failure of sc
instruction, which again decreases the performance.

. Although not a strange argument, but sysmips() implementation in pthread
provides MIPS I comptability which can be important for, for example, a
desktop distribution.


> Adding a parameter is a silly thing to do,
> and we don't need to be adding functionality to sys_sysmips through
> NEW_MO_BETTER_AS_SEEN_ON_TV_ATOMIC_SET or what have you...
> 

Please forgive my silliness again - can you illustrate why this idea sound so
silly to you?  


Jun

> Jun Sun wrote:
> 
> > Florian Lohoff wrote:
> >
> >> On Mon, May 21, 2001 at 04:23:52PM -0700, Jun Sun wrote:
> >>
> >>> The patch seems to be just a fast implementation of sysmips().  Why would it
> >>> solve an otherwise illegal instruction problem?
> >>>
> >>> George, what was exactly the error and the faulty instruction?
> >>
> >> Wrong - Its not only a "fast" path sysmips. It solves the illegal instruction
> >> case as it carefully doesnt touch registers it should not touch.
> >>
> >> The sysmips illegal instruction stuff came from the early exit
> >> needed to skip the -EXXXX case in the scall32.S which did not
> >> restore the modified registers. This needed fixing and there was
> >> no clean way of doing this in C thus i wrote an asm sysmips/MIPS_ATOMIC_SET
> >> and called it "fast_sysmips" which itself would go into the old
> >> sysmips function when not MIPS_ATOMIC_SET.
> >>
> >
> >
> > I see.
> >
> > I took a look of MIPS ABI in system V and found that the spec only specifies
> > this extended call in C prototype:
> >
> > int _test_and_set(int *p, int v);
> >
> > It seems perfectly legal for us to add one more argument to store the return
> > value while still have the function returns error.  Of course, doing that will
> > break binary compatibility.
> >
> > Otherwise, I think Flo's patch is the best fix to satisfy the spec and binary
> > compatibility although it is a little clunky.
> >
> > A third possibility is the have a MIPS_NEW_ATOMIC_SET that take three
> > arguments.  If that approach is taken, I would take out the inline assembly
> > that jumps to o32_ret_from_sys_call and documents MIPS_ATOMIC_SET as
> > deprecated and valnerable.
> >
> > My preference, in the decreasing order, is 3), 2) and 1).
> >
> > Ralf, what do you think?  We cannot let the bug sit around in the CVS tree for
> > long.  Have to have some fix.
> >
> > Jun
> 
> --
> Joe deBlaquiere
> Red Hat, Inc.
> 307 Wynn Drive
> Huntsville AL, 35805
> voice : (256)-704-9200
> fax   : (256)-837-3839

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Jun Sun]

"Maciej W. Rozycki" wrote:
> 
> On Tue, 22 May 2001, Jun Sun wrote:
> 
> > I took a look of MIPS ABI in system V and found that the spec only specifies
> > this extended call in C prototype:
> >
> > int _test_and_set(int *p, int v);
> >
> > It seems perfectly legal for us to add one more argument to store the return
> > value while still have the function returns error.  Of course, doing that will
> > break binary compatibility.
> 
>  We have _test_and_set() in the library.  Implementing a clean underlying
> _test_and_set() syscall is very high on my to-do list -- lack of time
> prevents me from finishing it, unfortunately.
> 
>  There is no point to mess with sysmips() any further, I think.  The
> library's _test_and_set() only calls sysmips() if the library was compiled
> for ISA I systems.  As I guess from reports on the list, ISA I systems are
> a minority, mostly DECstations and possibly a few embedded systems.  Most
> people have ISA II+ and they do not need to call any syscall from
> _test_and_set() at all.  For ISA II+ systems the library implements
> _test_and_set() in the userland, using ll/sc appropriately.
> 
>  Anyone having ISA II+ systems only, please do yourself a favour and set
> "-mips2" (or maybe even "-mips3") somewhere in your CFLAGS for all
> compilations -- not only you'll get better optimized binaries, but you'll
> get rid of this sysmips() problem as well.
> 
> > Otherwise, I think Flo's patch is the best fix to satisfy the spec and binary
> > compatibility although it is a little clunky.
> 
>  I'll have yet to look at the patch, but what I think is, we should get
> sysmips() work as defined by the original spec (or as reverse-engineered,
> as the real spec seems to be out of reach).  Everything else belong to a
> separate implementation.
> 
>  Binary compatibility is not a necessity here.  The only sysmips() client
> is glibc at the moment, and it can be updated to use a new syscall at any
> time.
> 
>  In short: let's leave sysmips() semantics alone.
> 

Same old questions : Where is the definition of sysmips()?  What is considered
standard implementation of sysmips() so that we can do reverse-engineering? 
Irix?  

Even if Irix is considered standard implementation of sysmips(), I wonder if
we need to mirror it.  Here is my reasoning.  

The sytem V ABI specifies _test_and_set() as the exntended system call. 
Exactly how we achieve that is up to each implementation of the OS. 
Therefore, for MIPS I system, we simply choose to call sysmips(NEW_ATOMIC_SET,
....) with three arguments.  Any application that bypasses _test_and_set() and
calls sysmips() directly is running at its own risk of not being portable.

Of course, we need to fix glibc, which seems to be the only client that
matters at this moment.

In short, I think we DON"T have to maitain the semantic of sysmips().  My
understanding is that it is justed used to implement extended MIPS ABS calls. 
As long as we get those calls implemented correctly, the exact sysmips()
semantic does not matter.

Did I misundertand something?

Along this line, I think I even favor "just change the current ATOMIC_SET"
approach.

Jun

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Joe deBlaquiere]

Hi Maciej, Jun,

	Didn't mean to bring up a sore point, but it seems that we haven't yet 
come to a consensus on what policy to have here. I think you both make 
valid points that I don't necesarily disagree with, but I would like to 
follow the counterpoint a little further.

Maciej W. Rozycki wrote:

> On Wed, 23 May 2001, Joe deBlaquiere wrote:
> 
> 
>> I would vote for option #4 - make sure the ll/sc emulation stuff works 
>> and use ll/sc in glibc instead of sysmips. Beyond the pthreads mutex 
> 
> 
>  Please don't.  The emulation is an overkill here and the overhead is
> painful for ISA I systems, which are usually not the fastest ones.  This
> has already been discussed here.
> 

There's overhead to sysmips also, so neither one is going to give 
stunning performance. All out performance isn't likely an issue on one 
of these systems anyways.


>  If you want to go for speed and use ll/sc on an ISA II+ system, then
> compile glibc for ISA II or better.  It will never call sysmips() then. 

	The problem here is that now I have mips, mipsel, and mipselnollsc 
configurations of the cross-tools, the c library and the binary 
applications. It's one extra configuration to maintain.

	There's no way to solve the endianness issues, but using emulation to 
handle missing instructions (be they floating point or ll/sc, or 
what-have-you) solves the minor differences in the instruction set from 
the library/application standpoint. If all MIPS processors used the same 
instruction set then we wouldn't have the problem at all. Of course 
there are very good reasons (and probably some silly ones too...) why 
ISAs are tailored.

	The kernel is already going to have to adjust some anyway, so keeping the 
differences in the kernel doesn't increase the testing burden. Throwing 
the problem over to glibc (and the toolchain) does increase the number 
of active configurations.

Just my opinion.

-- 
Joe deBlaquiere
Red Hat, Inc.
307 Wynn Drive
Huntsville AL, 35805
voice : (256)-704-9200
fax   : (256)-837-3839

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Kevin D. Kissell]

> There's no way to solve the endianness issues, but using emulation to
> handle missing instructions (be they floating point or ll/sc, or
> what-have-you) solves the minor differences in the instruction set from
> the library/application standpoint. If all MIPS processors used the same
> instruction set then we wouldn't have the problem at all. Of course
> there are very good reasons (and probably some silly ones too...) why
> ISAs are tailored.
>
> The kernel is already going to have to adjust some anyway, so keeping the
> differences in the kernel doesn't increase the testing burden. Throwing
> the problem over to glibc (and the toolchain) does increase the number
> of active configurations.

And for the sake of beating a dead horse that perhaps only
I can see, there is a philosophical choice that must sometimes
be made whether to achieve binary portability by compiling to
the lowest-common denominator, or by emulating the missing
operations in the OS.  The former is more efficient for downrev
parts, the latter is more efficient for contemporary parts.  Those
of us who work on recent and future designs will always tend
to favor the latter - what's the point of using MIPS32/MIPS64
and beyond CPUs if gnu/Linux binaries are going to treat them
like R3000s?

            Kevin K.

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Kevin D. Kissell]

> There's no way to solve the endianness issues, but using emulation to
> handle missing instructions (be they floating point or ll/sc, or
> what-have-you) solves the minor differences in the instruction set from
> the library/application standpoint. If all MIPS processors used the same
> instruction set then we wouldn't have the problem at all. Of course
> there are very good reasons (and probably some silly ones too...) why
> ISAs are tailored.
>
> The kernel is already going to have to adjust some anyway, so keeping the
> differences in the kernel doesn't increase the testing burden. Throwing
> the problem over to glibc (and the toolchain) does increase the number
> of active configurations.

And for the sake of beating a dead horse that perhaps only
I can see, there is a philosophical choice that must sometimes
be made whether to achieve binary portability by compiling to
the lowest-common denominator, or by emulating the missing
operations in the OS.  The former is more efficient for downrev
parts, the latter is more efficient for contemporary parts.  Those
of us who work on recent and future designs will always tend
to favor the latter - what's the point of using MIPS32/MIPS64
and beyond CPUs if gnu/Linux binaries are going to treat them
like R3000s?

            Kevin K.

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Jun Sun]

Joe deBlaquiere wrote:
> 
 >  Please don't.  The emulation is an overkill here and the overhead is
> > painful for ISA I systems, which are usually not the fastest ones.  This
> > has already been discussed here.
> >
> 
> There's overhead to sysmips also, so neither one is going to give
> stunning performance. All out performance isn't likely an issue on one
> of these systems anyways.
> 

Like I said in the previous email, ll/sc emulation is at least twice as bad as
sysmips().  The likely failure of sc will make the performance even worse.  In
addition, the new glibc starts to pthread massively now (try 'ls' and you will
see). I do think performance is a factor here.

> >  If you want to go for speed and use ll/sc on an ISA II+ system, then
> > compile glibc for ISA II or better.  It will never call sysmips() then.
> 
>         The problem here is that now I have mips, mipsel, and mipselnollsc
> configurations of the cross-tools, the c library and the binary
> applications. It's one extra configuration to maintain.
>

I see the trouble of having extra configurations.  If you were planning to
have separate support for MIPS I and MIPS II systems, you should be covered. 
After all there are only limited number of variants anyway - so far. :-)

Jun

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Wed, 23 May 2001, Jun Sun wrote:

> Same old questions : Where is the definition of sysmips()?  What is considered
> standard implementation of sysmips() so that we can do reverse-engineering? 
> Irix?  

 I think Ralf can comment it.

> Even if Irix is considered standard implementation of sysmips(), I wonder if
> we need to mirror it.  Here is my reasoning.  
> 
> The sytem V ABI specifies _test_and_set() as the exntended system call. 

 I think we want to execute IRIX binaries.  The idea of providing
sysmips() was to provide compatibility to other systems, AFAIK.  I reused
sysmips() for pthreads support in glibc as it was non-existent for ISA I
CPUs and I needeed librt to run on my R3k.

 Much of discussion happened later and the conclusion was it's better to
make a new call than to try to fiddle with sysmips().  I was not aware of
the problems with sysmips() when I wrote the glibc code as sysmips() was
nowhere defined and the interface seemed to be reasonable as implemented
at that time.  Later I was told sysmips(MIPS_ATOMIC_SET) is expected to
return an error code which makes it incompatible to _test_and_set().

> Therefore, for MIPS I system, we simply choose to call sysmips(NEW_ATOMIC_SET,
> ....) with three arguments.  Any application that bypasses _test_and_set() and
> calls sysmips() directly is running at its own risk of not being portable.

 Well, I now see using sysmips() for this purpose is ugly, it complicates
immplementation a lot and hits performance of these slow systems where we
want to save as many CPU cycles as possible.  And still the problem of the
error code remains.

> Of course, we need to fix glibc, which seems to be the only client that
> matters at this moment.

 That is not a problem, as I already stated.  I'll see if I can cook the
new syscall I'm writing of, soon.  You'll see what I mean, then.

> In short, I think we DON"T have to maitain the semantic of sysmips().  My
> understanding is that it is justed used to implement extended MIPS ABS calls. 

 Adding a new subfunction of sysmips() is not a problem.  The cleanness on
the code and the performance is.  Note that other subfunctions of
sysmips() are not performance-critical. 

 The whole idea of sysmips() appears hackish to me.

  Maciej

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Florian Lohoff]

On Wed, May 23, 2001 at 11:41:57AM -0700, Jun Sun wrote:
> Like I said in the previous email, ll/sc emulation is at least twice as bad as
> sysmips().  The likely failure of sc will make the performance even worse.  In
> addition, the new glibc starts to pthread massively now (try 'ls' and you will
> see). I do think performance is a factor here.

There are a lot of glibc issues to have a look at - Try issueing a "sleep"
compiled against glibc 2.2 and you'll see at least 20-30 sysmips/shed_yield
calls. As for sleep this is completely unecessary but i guess this
is common glibc startup code and on most architectures atomic test/set
instructions are not as painful as on non ll/sc mips cpus.

> I see the trouble of having extra configurations.  If you were planning to
> have separate support for MIPS I and MIPS II systems, you should be covered. 
> After all there are only limited number of variants anyway - so far. :-)

My favourit would be to let the glibc on runtime decide whether
to use sysmips or ll/sc in the atomic test_and_set stuff. This would
lead to an common atom op in the userspace which is fast on ll/sc 
cpus and gives much lesser performance penaltys in the sysmips case
than emulating ll/sc.

Flo
-- 
Florian Lohoff                  flo@rfc822.org             +49-5201-669912
     Why is it called "common sense" when nobody seems to have any?

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Florian Lohoff]

On Wed, May 23, 2001 at 08:54:12PM +0200, Florian Lohoff wrote:
> My favourit would be to let the glibc on runtime decide whether
> to use sysmips or ll/sc in the atomic test_and_set stuff. This would
> lead to an common atom op in the userspace which is fast on ll/sc 
> cpus and gives much lesser performance penaltys in the sysmips case
> than emulating ll/sc.

But again - I tried to run this discussion again and again - As long
as there is no code to use there is no point in taking a discussion.
I needed a working sysmips for debian as we compile the glibc with
sysmips (performance penalty but for now works everywhere) thus
i fixed the sysmips.

Let the code speak

Flo
-- 
Florian Lohoff                  flo@rfc822.org             +49-5201-669912
     Why is it called "common sense" when nobody seems to have any?

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Wed, 23 May 2001, Joe deBlaquiere wrote:

> 	Didn't mean to bring up a sore point, but it seems that we haven't yet 
> come to a consensus on what policy to have here. I think you both make 
> valid points that I don't necesarily disagree with, but I would like to 
> follow the counterpoint a little further.

 The consensus was come to in January, IIRC.  I should have coded an
implementation long time ago, actually, but various events keep
distracting me. 

> There's overhead to sysmips also, so neither one is going to give 
> stunning performance. All out performance isn't likely an issue on one 
> of these systems anyways.

 There is a big advantage of invoking a single syscall vs faulting twice
or more times on invalid opcodes.  An the performance is an issue here --
wasting a few cycles may remain unnoticed on a decent system while it is
extremely painful on slow systems.

> 	The problem here is that now I have mips, mipsel, and mipselnollsc 
> configurations of the cross-tools, the c library and the binary 
> applications. It's one extra configuration to maintain.

 How often do you build glibc?

 For other programs it really doesn't matter which version of glibc you
link against (given the same endianness).  The linker does not fetch any
code from shared objects when linking.  Install binaries of glibc as
appropriate but just choose any single copy of glibc for development.

> 	There's no way to solve the endianness issues, but using emulation to
> handle missing instructions (be they floating point or ll/sc, or
> what-have-you) solves the minor differences in the instruction set from
> the library/application standpoint. If all MIPS processors used the same

 Well, do you really have an ISA I CPU which implements ll/sc?  No?  What
about other missing instructions?  Emulating instructions (as well as e.g. 
unaligned memory accesses) might be good for debugging, but for good
performance you need to target your binaries to a specific system anyway.

> the library/application standpoint. If all MIPS processors used the same 
> instruction set then we wouldn't have the problem at all. Of course 
> there are very good reasons (and probably some silly ones too...) why 
> ISAs are tailored.

 The same is true for other processors.  E.g. there is a noticeable
performance advantage for certain code when compiled for the EV56
variation of Alpha (due to the BWX extension, i.e. instructions for byte-
and halfword-wide memory accesses).  Yet the resulting code does not run
on an EV5 or EV4.  You have to choose: either performance or
compatibility.

> 	The kernel is already going to have to adjust some anyway, so keeping the 
> differences in the kernel doesn't increase the testing burden. Throwing 

 The kernel gets adjusted at the build time.  It's unrelated to the topic,
though -- the syscall interface remains the same.

> the problem over to glibc (and the toolchain) does increase the number 
> of active configurations.

 The same can be said of other platforms.  For example, glibc built for
i386 does not make use of i686's cmov instructions and is thus highly
inefficient on such systems.  So if you care of performance you choose an
appropriate glibc build-time configuration which suits your system. 

  Maciej

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Wed, 23 May 2001, Kevin D. Kissell wrote:

> parts, the latter is more efficient for contemporary parts.  Those
> of us who work on recent and future designs will always tend
> to favor the latter - what's the point of using MIPS32/MIPS64
> and beyond CPUs if gnu/Linux binaries are going to treat them
> like R3000s?

 If you work on new processors only, then there is no problem.  You
configure your tools to build binaries for systems you use and you'll
never see R3k compatibility code.

 Please do yourself a favor and look at the relevant part of glibc.  If
you build glibc (and any other other program that makes use of
_test_and_set()) for ISA II+, the code gets actually inlined with ll/sc
used as expected.

 So the problem is?

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Wed, 23 May 2001, Florian Lohoff wrote:

> My favourit would be to let the glibc on runtime decide whether
> to use sysmips or ll/sc in the atomic test_and_set stuff. This would
> lead to an common atom op in the userspace which is fast on ll/sc 
> cpus and gives much lesser performance penaltys in the sysmips case
> than emulating ll/sc.

 How would you do it for inlined code?

 Anyway, I consider run-time detection of ll/sc an overkill.  You only
solve a single inefficiency of ISA I code when run on better CPUs.  You
really want to recompile code to make use of new instructions if you care
about performance.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Joe deBlaquiere]

Florian Lohoff wrote:

> On Wed, May 23, 2001 at 08:54:12PM +0200, Florian Lohoff wrote:
> 
>> My favourit would be to let the glibc on runtime decide whether
>> to use sysmips or ll/sc in the atomic test_and_set stuff. This would
>> lead to an common atom op in the userspace which is fast on ll/sc 
>> cpus and gives much lesser performance penaltys in the sysmips case
>> than emulating ll/sc.
> 
> 
> But again - I tried to run this discussion again and again - As long
> as there is no code to use there is no point in taking a discussion.
> I needed a working sysmips for debian as we compile the glibc with
> sysmips (performance penalty but for now works everywhere) thus
> i fixed the sysmips.
> 

I have to agree with the pragmatism first, philosophy second approach. If it is beautiful but doesn't work it dosn't help anyone.

The thing I don't understand is how glibc is going to cleanly decide at 
runtime which code to use. It's relatively easy to do something like 
that in the kernel, but I can't come up with an elegant solution to make 
such a choice at runtime in glibc.

Assuming that we're moving forward (as Kevin points out) the percentage 
of systems without ll/sc is going down. While these systems don't have 
much CPU power to spare, we should make the baseline implementation have 
ll/sc emulation. If somebody wants to make a MIPS I optimized glibc, 
then that's fine, but allowing the 'standard' MIPSII glibc to work on 
all systems simplifies life ( mine at least ;) ).

-- 
Joe deBlaquiere
Red Hat, Inc.
307 Wynn Drive
Huntsville AL, 35805
voice : (256)-704-9200
fax   : (256)-837-3839

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Jun Sun]

"Maciej W. Rozycki" wrote:
> 
> On Wed, 23 May 2001, Jun Sun wrote:
> 
> > Same old questions : Where is the definition of sysmips()?  What is considered
> > standard implementation of sysmips() so that we can do reverse-engineering?
> > Irix?
> 
>  I think Ralf can comment it.
> 
> > Even if Irix is considered standard implementation of sysmips(), I wonder if
> > we need to mirror it.  Here is my reasoning.
> >
> > The sytem V ABI specifies _test_and_set() as the exntended system call.
> 
>  I think we want to execute IRIX binaries. 

That would make sense to keep sysmips() as it is if your statement is true. 
But I thought the binary comptability with IRIX has long been broken.  Can
someone confirm that?

If binary compatbility with IRIX is broken now, I don't think we should care
to fix it in the future - obviously. :-)

Jun

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Jun Sun]

Florian Lohoff wrote:

> My favourit would be to let the glibc on runtime decide whether
> to use sysmips or ll/sc in the atomic test_and_set stuff. This would
> lead to an common atom op in the userspace which is fast on ll/sc
> cpus and gives much lesser performance penaltys in the sysmips case
> than emulating ll/sc.
> 

Do you have an idea how this run-time detection might be implemented?  I am
very curious.  

For example, how will you figure out the existence of ll/sc instruction?  Is
that a test which is a part of every process startup procedure?  Or do you
introduce a new syscall?

Jun

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Kevin D. Kissell]

> > parts, the latter is more efficient for contemporary parts.  Those
> > of us who work on recent and future designs will always tend
> > to favor the latter - what's the point of using MIPS32/MIPS64
> > and beyond CPUs if gnu/Linux binaries are going to treat them
> > like R3000s?
> 
>  If you work on new processors only, then there is no problem.  You
> configure your tools to build binaries for systems you use and you'll
> never see R3k compatibility code.
> 
>  Please do yourself a favor and look at the relevant part of glibc.  If
> you build glibc (and any other other program that makes use of
> _test_and_set()) for ISA II+, the code gets actually inlined with ll/sc
> used as expected.
> 
>  So the problem is?

The problem is that, out in industry, not everyone wants to
build their entire userland from source, and nobody particularly 
wants to deal with  the product management problems of making, 
maintaining,  testing, and distributing all the permutations of BE/LE, 
FP/noFP, LLSC/noLLSC, etc, etc.

            Kevin K.

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Kevin D. Kissell]

> > parts, the latter is more efficient for contemporary parts.  Those
> > of us who work on recent and future designs will always tend
> > to favor the latter - what's the point of using MIPS32/MIPS64
> > and beyond CPUs if gnu/Linux binaries are going to treat them
> > like R3000s?
> 
>  If you work on new processors only, then there is no problem.  You
> configure your tools to build binaries for systems you use and you'll
> never see R3k compatibility code.
> 
>  Please do yourself a favor and look at the relevant part of glibc.  If
> you build glibc (and any other other program that makes use of
> _test_and_set()) for ISA II+, the code gets actually inlined with ll/sc
> used as expected.
> 
>  So the problem is?

The problem is that, out in industry, not everyone wants to
build their entire userland from source, and nobody particularly 
wants to deal with  the product management problems of making, 
maintaining,  testing, and distributing all the permutations of BE/LE, 
FP/noFP, LLSC/noLLSC, etc, etc.

            Kevin K.

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Joe deBlaquiere]

Kevin D. Kissell wrote:


>> 
>>  So the problem is?
> 
> 
> The problem is that, out in industry, not everyone wants to
> build their entire userland from source, and nobody particularly 
> wants to deal with  the product management problems of making, 
> maintaining,  testing, and distributing all the permutations of BE/LE, 
> FP/noFP, LLSC/noLLSC, etc, etc.
> 
Could not have said it better myself. If you have the emulation then you 
can always use a noLLSC version of glibc if you are performance-driven. 
Otherwise you can _also_ use the generic LLSC version. The overhead of 
having a few hundreds of words of code is pretty small (compared with 
70+k of filenames via the BUG() macro) and ensures that either glibc 
will work. It's the best of both worlds.

-- 
Joe deBlaquiere
Red Hat, Inc.
307 Wynn Drive
Huntsville AL, 35805
voice : (256)-704-9200
fax   : (256)-837-3839

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Keith M Wesolowski]

On Wed, May 23, 2001 at 09:44:36PM +0200, Maciej W. Rozycki wrote:

> solve a single inefficiency of ISA I code when run on better CPUs.  You
> really want to recompile code to make use of new instructions if you care
> about performance.

I agree.  Therefore I am sure some MIPS I users will be happy to build
special glibcs for their ISA.

-- 
Keith M Wesolowski <wesolows@foobazco.org> http://foobazco.org/~wesolows
------(( Project Foobazco Coordinator and Network Administrator ))------
	"Nothing motivates a man more than to see his boss put
	 in an honest day's work." -- The fortune file

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Wed, 23 May 2001, Joe deBlaquiere wrote:

> ll/sc emulation. If somebody wants to make a MIPS I optimized glibc, 
> then that's fine, but allowing the 'standard' MIPSII glibc to work on 
> all systems simplifies life ( mine at least ;) ).

 I have no objections against providing an ll/sc emulation -- I have never
had and certainly haven't expressed them.  What I insist on is to keep
ISA-I-compiled glibc not making use of ll/sc.  Anyone feel free to finish
the emulation we have.

 Anyway, an ISA-II-compiled glibc won't work on an ISA I system even if
the ll/sc emulation works.  An ISA II is more than just an addition of the
ll and sc instructions.  There were also branch likely, trap and
doubleword coprocessor load instructions added in ISA II.  Do you want to
emulate these, too? 

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Thu, 24 May 2001, Kevin D. Kissell wrote:

> The problem is that, out in industry, not everyone wants to
> build their entire userland from source, and nobody particularly 
> wants to deal with  the product management problems of making, 
> maintaining,  testing, and distributing all the permutations of BE/LE, 
> FP/noFP, LLSC/noLLSC, etc, etc.

 First, we are talking about glibc and not the entire userland.  I insist
on having the performance-wise implementation in glibc.

 Second, do you expect everyone compiling the entire userland from
sources?  I don't.  The normal approach is to take a distribution and
build only these pieces which are not satisfying for one reason or
another.  Just take an ISA I, ISA II or whatever version you need.

 Third, an ISA-II-hosted glibc still contains an _test_and_set() function
which makes use of ll/sc, independently from an inlined version in a
header.

 Fourth, maintaining differently optimized distributions is not that
troublesome.  It's mostly a matter of disk space (which is hardly an issue
these days).  Once you have one version ready (prepared manually), all
others can be built automatically with no intervention.  With RPM it's as
easy as having different optflags settings in an rc file and having an
autobuilder perform the boring work.  It's not a theory, this is for
example how the PLD distribution is being developed -- see
http://www.pld.org.pl/ for details.  It just works.

 Fifth, I don't object having an ll/sc emulation per se -- as long as you
use the ABI-defined _test_and_set() function, everyone is free to
recompile sources to suite their needs. 

  Maciej

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Wed, 23 May 2001, Joe deBlaquiere wrote:

> Could not have said it better myself. If you have the emulation then you 
> can always use a noLLSC version of glibc if you are performance-driven. 

 I think there is some misunderstanding here -- I thought you are
recommending to drop the non-ll/sc code from glibc.

> Otherwise you can _also_ use the generic LLSC version. The overhead of 
> having a few hundreds of words of code is pretty small (compared with 
> 70+k of filenames via the BUG() macro) and ensures that either glibc 
> will work. It's the best of both worlds.

 Can't agree more.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Kevin D. Kissell]

> > The problem is that, out in industry, not everyone wants to
> > build their entire userland from source, and nobody particularly 
> > wants to deal with  the product management problems of making, 
> > maintaining,  testing, and distributing all the permutations of BE/LE, 
> > FP/noFP, LLSC/noLLSC, etc, etc.
> 
>  First, we are talking about glibc and not the entire userland.

But since essentially the entire userland is linked with glibc,
I'm not sure how much that distinction matters, in practical
terms.

> I insist on having the performance-wise implementation in glibc.

Joe and I likewise insist on having a high-performance
implementation of glibc as the default.  The question is
whether it's to be one optimised for performance on R3000's
or on contemporary parts.  As has been stated by others,
of *course* one wants to be able to build it either way.
I'm simply saying that if one just does "./configure"
and "make" for glibc with a mips target, it should default
to use ll/sc, and that if one simply builds RPMs for binary
distribution of MIPS/Linux binaries, they should be linked
with a glibc that uses ll/sc.  And that therefore the MIPS/Linux
kernel for R3000's (and R4100's) should have ll/sc emulation 
support built in by default.

>  Second, do you expect everyone compiling the entire userland from
> sources?  I don't.  The normal approach is to take a distribution and
> build only these pieces which are not satisfying for one reason or
> another.  Just take an ISA I, ISA II or whatever version you need.

>From where?  I'd love to find a decently complete (with compilers,
networking tools, X, etc.) mipsel distribution of any MIPS ISA level 
less that MIPS V to replace the antique crud that runs on my mipsel 
platform.

>  Fourth, maintaining differently optimized distributions is not that
> troublesome.

Please don't get me wrong here, because I tremendously
respect the work that you've been doing for MIPS/Linux,
but how many differently optimised distributions do you
presonally build, distribute, support, and maintain?

            Regards,

            Kevin K.

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Kevin D. Kissell]

> > The problem is that, out in industry, not everyone wants to
> > build their entire userland from source, and nobody particularly 
> > wants to deal with  the product management problems of making, 
> > maintaining,  testing, and distributing all the permutations of BE/LE, 
> > FP/noFP, LLSC/noLLSC, etc, etc.
> 
>  First, we are talking about glibc and not the entire userland.

But since essentially the entire userland is linked with glibc,
I'm not sure how much that distinction matters, in practical
terms.

> I insist on having the performance-wise implementation in glibc.

Joe and I likewise insist on having a high-performance
implementation of glibc as the default.  The question is
whether it's to be one optimised for performance on R3000's
or on contemporary parts.  As has been stated by others,
of *course* one wants to be able to build it either way.
I'm simply saying that if one just does "./configure"
and "make" for glibc with a mips target, it should default
to use ll/sc, and that if one simply builds RPMs for binary
distribution of MIPS/Linux binaries, they should be linked
with a glibc that uses ll/sc.  And that therefore the MIPS/Linux
kernel for R3000's (and R4100's) should have ll/sc emulation 
support built in by default.

>  Second, do you expect everyone compiling the entire userland from
> sources?  I don't.  The normal approach is to take a distribution and
> build only these pieces which are not satisfying for one reason or
> another.  Just take an ISA I, ISA II or whatever version you need.

From where?  I'd love to find a decently complete (with compilers,
networking tools, X, etc.) mipsel distribution of any MIPS ISA level 
less that MIPS V to replace the antique crud that runs on my mipsel 
platform.

>  Fourth, maintaining differently optimized distributions is not that
> troublesome.

Please don't get me wrong here, because I tremendously
respect the work that you've been doing for MIPS/Linux,
but how many differently optimised distributions do you
presonally build, distribute, support, and maintain?

            Regards,

            Kevin K.

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Thu, 24 May 2001, Kevin D. Kissell wrote:

> >  First, we are talking about glibc and not the entire userland.
> 
> But since essentially the entire userland is linked with glibc,
> I'm not sure how much that distinction matters, in practical
> terms.

 Do you link statically?  If not, then almost no code from glibc is
incorporated into your binaries, with an unimportant exception of a few
functions from libc_nonshared.

> I'm simply saying that if one just does "./configure"
> and "make" for glibc with a mips target, it should default
> to use ll/sc, and that if one simply builds RPMs for binary
> distribution of MIPS/Linux binaries, they should be linked
> with a glibc that uses ll/sc.  And that therefore the MIPS/Linux
> kernel for R3000's (and R4100's) should have ll/sc emulation 
> support built in by default.

 Then you need to change your gcc configuration so that it assumes
"-mips2" (or whatever level you want to) by default.  That's typically
done in the gcc's specs file.  Nothing about glibc here. 

 For RPM you might put "-mips2" into optflags in its rc file.

> >  Second, do you expect everyone compiling the entire userland from
> > sources?  I don't.  The normal approach is to take a distribution and
> > build only these pieces which are not satisfying for one reason or
> > another.  Just take an ISA I, ISA II or whatever version you need.
> 
> >From where?  I'd love to find a decently complete (with compilers,
> networking tools, X, etc.) mipsel distribution of any MIPS ISA level 
> less that MIPS V to replace the antique crud that runs on my mipsel 
> platform.

 That's a real problem, but there is Debian available for MIPS -- is it
missing anything?  The main problem was to get glibc 2.2 right.  Since it
was done, building other programs should be easy.

 Then building other variants is just a matter of disk space and CPU time.

> >  Fourth, maintaining differently optimized distributions is not that
> > troublesome.
> 
> Please don't get me wrong here, because I tremendously
> respect the work that you've been doing for MIPS/Linux,
> but how many differently optimised distributions do you
> presonally build, distribute, support, and maintain?

 I understand maintaining a distribution is serious task, but with
appropriate tools maintaining two or more distributions with a common
source set is not much harder than a single one.

 For example, using the source set as available at my FTP site, building
differently optimized binaries is as easy as running a script looking
more or less as follows:

for rc in .rpmrc-mipsel .rpmrc-mipsel-II .rpmrc-mipsel-III
.rpmrc-mipsel-IV; do
	for name in *.src.rpm; do
		rpm --rcfile=$rc --rebuild $name
	done
done

with different optflags set in .rpmrc-mipsel* files.  It's boring and
disk-consuming but easy and automatic.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Joe deBlaquiere]

Maciej W. Rozycki wrote:

> On Thu, 24 May 2001, Kevin D. Kissell wrote:
> 
> 
>>>  First, we are talking about glibc and not the entire userland.
>> 
>> But since essentially the entire userland is linked with glibc,
>> I'm not sure how much that distinction matters, in practical
>> terms.
> 
> 
>  Do you link statically?  If not, then almost no code from glibc is
> incorporated into your binaries, with an unimportant exception of a few
> functions from libc_nonshared.
> 

and those pesky little inlined code snippets...

#define PT_EI extern inline

PT_EI long int
testandset (int *spinlock)

which of course uses ll/sc if your world is built for _MIPS_ISA >= 
_MIPS_ISA_MIPS2


This can be fixed, of course. 

-- 
Joe deBlaquiere
Red Hat, Inc.
307 Wynn Drive
Huntsville AL, 35805
voice : (256)-704-9200
fax   : (256)-837-3839

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Thu, 24 May 2001, Joe deBlaquiere wrote:

> and those pesky little inlined code snippets...
> 
> #define PT_EI extern inline
> 
> PT_EI long int
> testandset (int *spinlock)
> 
> which of course uses ll/sc if your world is built for _MIPS_ISA >= 
> _MIPS_ISA_MIPS2

 The glibc's non-inlined _test_and_set() also uses ll/sc, if built for
_MIPS_ISA >= _MIPS_ISA_MIPS2.  We might remove the inline version of
_test_and_set() for _MIPS_ISA == _MIPS_ISA_MIPS1 (I forgot about this one
previously, sorry) from <sys/tas.h>, but at a cost of an additional
function call.  I'm not sure if that's fine performance-wise at this
moment...

 However, when I finish my implementation of a _test_and_set() syscall, it
will be perfectly fine and even necessary to remove the inline wrapper for
_MIPS_ISA == _MIPS_ISA_MIPS1 -- the only reason the wrapper is needed now
is the incompatibility of the arguments of sysmips() and _test_and_set(). 
The good news is I already started the implementation -- hopefully it'll
be ready over this weekend and the never-ending discussion about
sysmips(MIPS_ATOMIC_SET) will be over.

  Maciej

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Surprise! (Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Jun Sun]

Alright, I rolled my sleeve and digged into IRIX 6.5, and guess what? 
sysmips() does NOT have MIPS_ATOMIC_SET (2001) on IRIX!  See the header below.

So apparently MIPS_ATOMIC_SET was invented for Linux only, probably just to
implement _test_and_set().  (It would be interesting to see how IRIX implement
_test_and_set() on MIPS I machines.  However, the machine I have access uses
ll/sc instructions).

Given the above understanding, I think we are free to add a new sysmips
subcall or even just change the current one if we want to.

Through the discussions, I hope we have achived the following concensus:

1. We need a correct syscall support to implement _test_and_set() on MIPS I
machine.  ll/sc emulation is not considered good enough to eliminate this need
due to performance concerns.
  
People who take this route understand that they may have to create extra user
land configurations.

2. Nobody seems to object the idea of ll/sc emulation per se, althought it is
not currently fully implemented based on my understanding.

People who prefer this route will enjoy the same ll/sc-enabled userland but
presumably take a performance hit on machines without ll/sc.

Anybody still have objections to the above conclusions?

-------------

Now back to the fix for syscall: the cvs tree is buggy as it is, as pointed
out by Florian.

I see several possibilities, which more or less the same as I brought up at
the beginning:

1) Florian's fix - introduce a new assembly routine to intercept
MIPS_ATOMIC_SET call so that correct return value is returned when there is no
error and error value is returned when there is an error.

1.a) A slight modification to 1) - we send a seg fault when there is access
error.  This solves two problems in 1).  

With 1), _test_and_set() will not be able to distinguish whether an error has
happened or a negative value is returned.  So in effect, error is mistakenly
ignored.  

The second benenfit is that with seg fault MIPS I implementation will have the
same behavior as MIPS II implementation which uses ll/sc.

2) Add a new subcall to sysmips, MIPS_NEW_ATOMIC_SET.  It takes three
arguments with the third one being the address to hold return value.  Again, I
think we should send seg fault on access errors.

The advantage of 2) is that we can still use the same sysmips() call without
introducing any new files.  The disadvantage is that people who use
MIPS_ATOMIC_SET directly is still subject to error, either in one form or
another.

To me, either 1.a) or 2) is fine with me, although I have a slight faovr over
2) (perhaps because I don't like assembly code and the extra "vertical"
calling layer introduced in 1.a)

--------------

So, please do us all a favor, can everybody who cares let us know :

1) can we agree on the concensus?

2) which fix do you like (naturally assuming you agree with the concensus)?

Let us work together and put a closure on this recurring matter!

Jun

Re: Surprise! (Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Jun Sun]

Oops!  I guess I forgot to include the sysmips.h header file from IRIX.

Jun

/*      Copyright (c) 1984 AT&T */
/*        All Rights Reserved   */

/*      THIS IS UNPUBLISHED PROPRIETARY SOURCE CODE OF AT&T     */
/*      The copyright notice above does not evidence any        */
/*      actual or intended publication of such source code.     */

#ident "$Revision: 3.12 $"

/*
 * Sysmips() system call commands.
 */

#define SETNAME         1       /* rename the system */
#define STIME           2       /* set time */
#define FLUSH_CACHE     3       /* flush caches */
#define MIPS_FPSIGINTR  5       /* generate a SIGFPE on every fp interrupt */
#define MIPS_FPU        6       /* turn on/off the fpu */
#define MIPS_FIXADE     7       /* fix address error (unaligned) */
#define POSTWAIT        8       /* post wait driver for Oracle */
#define PPHYSIO         9       /* parallel IO */
#define PPHYSIO64       10      /* parallel IO - big offsets */

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Florian Lohoff]

On Thu, May 24, 2001 at 11:32:29AM +0200, Maciej W. Rozycki wrote:
> On Wed, 23 May 2001, Joe deBlaquiere wrote:
> 
> > ll/sc emulation. If somebody wants to make a MIPS I optimized glibc, 
> > then that's fine, but allowing the 'standard' MIPSII glibc to work on 
> > all systems simplifies life ( mine at least ;) ).
> 
>  I have no objections against providing an ll/sc emulation -- I have never
> had and certainly haven't expressed them.  What I insist on is to keep
> ISA-I-compiled glibc not making use of ll/sc.  Anyone feel free to finish
> the emulation we have.
> 
>  Anyway, an ISA-II-compiled glibc won't work on an ISA I system even if
> the ll/sc emulation works.  An ISA II is more than just an addition of the
> ll and sc instructions.  There were also branch likely, trap and
> doubleword coprocessor load instructions added in ISA II.  Do you want to
> emulate these, too? 

Isnt this the reason why Linux/Mips userspace is compiles with ISA I + ll/sc ?

Flo
-- 
Florian Lohoff                  flo@rfc822.org             +49-5201-669912
     Why is it called "common sense" when nobody seems to have any?

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Florian Lohoff]

On Wed, May 23, 2001 at 03:04:41PM -0500, Joe deBlaquiere wrote:
> The thing I don't understand is how glibc is going to cleanly decide at 
> runtime which code to use. It's relatively easy to do something like 
> that in the kernel, but I can't come up with an elegant solution to make 
> such a choice at runtime in glibc.

Export the existance of ll/sc via /proc/cpuinfo or whatever.

> Assuming that we're moving forward (as Kevin points out) the percentage 
> of systems without ll/sc is going down. While these systems don't have 
> much CPU power to spare, we should make the baseline implementation have 
> ll/sc emulation. If somebody wants to make a MIPS I optimized glibc, 
> then that's fine, but allowing the 'standard' MIPSII glibc to work on 
> all systems simplifies life ( mine at least ;) ).

I dont think this is true necessarly - There are still people building
embedded x86 systems based on 386 cores. Look at the vr41xx systems - They
do also lack the ll/sc afaik. This is nowadays the most commonly
used embedded/pda cpu.

Flo
-- 
Florian Lohoff                  flo@rfc822.org             +49-5201-669912
     Why is it called "common sense" when nobody seems to have any?

Re: Surprise! (Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Fri, 25 May 2001, Jun Sun wrote:

> Alright, I rolled my sleeve and digged into IRIX 6.5, and guess what? 
> sysmips() does NOT have MIPS_ATOMIC_SET (2001) on IRIX!  See the header below.

 I remember Ralf writing of this being a compatibility call with RISC/OS
(is it the original OS of MIPS, Inc.?), IIRC.  Ralf: am I right? 

> So apparently MIPS_ATOMIC_SET was invented for Linux only, probably just to
> implement _test_and_set().  (It would be interesting to see how IRIX implement
> _test_and_set() on MIPS I machines.  However, the machine I have access uses
> ll/sc instructions).

 Does IRIX actually run on anything below ISA II?

> To me, either 1.a) or 2) is fine with me, although I have a slight faovr over
> 2) (perhaps because I don't like assembly code and the extra "vertical"
> calling layer introduced in 1.a)

 What about 3) -- a new syscall with a different semantics and no need to
care about limitations of current implementations (especially the
sysmips() bag).  I've just sent a proposal for discussion.  I'm looking
forward for constructive feedback.

  Maciej

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Sat, 26 May 2001, Florian Lohoff wrote:

> >  Anyway, an ISA-II-compiled glibc won't work on an ISA I system even if
> > the ll/sc emulation works.  An ISA II is more than just an addition of the
> > ll and sc instructions.  There were also branch likely, trap and
> > doubleword coprocessor load instructions added in ISA II.  Do you want to
> > emulate these, too? 
> 
> Isnt this the reason why Linux/Mips userspace is compiles with ISA I + ll/sc ?

 Is it?  Mine certainly is not. 

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Sat, 26 May 2001, Florian Lohoff wrote:

> Export the existance of ll/sc via /proc/cpuinfo or whatever.

 That's a valid approach and also nothing new to glibc -- see Alpha and
in()/out() support.  But do we want an extra overhead due to an indirect
call?  Especially as _test_and_set() gets usually inlined?

> I dont think this is true necessarly - There are still people building
> embedded x86 systems based on 386 cores. Look at the vr41xx systems - They
> do also lack the ll/sc afaik. This is nowadays the most commonly
> used embedded/pda cpu.

 Are vr41xx plain ISA I or crippled ISA II+ CPUs? 

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Kevin D. Kissell]

> > Export the existance of ll/sc via /proc/cpuinfo or whatever.
>
>  That's a valid approach and also nothing new to glibc -- see Alpha and
> in()/out() support.  But do we want an extra overhead due to an indirect
> call?  Especially as _test_and_set() gets usually inlined?

Use a global variable testable by the inline code?

> > I dont think this is true necessarly - There are still people building
> > embedded x86 systems based on 386 cores. Look at the vr41xx systems -
They
> > do also lack the ll/sc afaik. This is nowadays the most commonly
> > used embedded/pda cpu.
>
>  Are vr41xx plain ISA I or crippled ISA II+ CPUs?

Actually, they are crippled MIPS III+ 64-bit CPUs
(with added stuff like16x16 bit MAC instructions!).

            Kevin K.

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Kevin D. Kissell]

> > Export the existance of ll/sc via /proc/cpuinfo or whatever.
>
>  That's a valid approach and also nothing new to glibc -- see Alpha and
> in()/out() support.  But do we want an extra overhead due to an indirect
> call?  Especially as _test_and_set() gets usually inlined?

Use a global variable testable by the inline code?

> > I dont think this is true necessarly - There are still people building
> > embedded x86 systems based on 386 cores. Look at the vr41xx systems -
They
> > do also lack the ll/sc afaik. This is nowadays the most commonly
> > used embedded/pda cpu.
>
>  Are vr41xx plain ISA I or crippled ISA II+ CPUs?

Actually, they are crippled MIPS III+ 64-bit CPUs
(with added stuff like16x16 bit MAC instructions!).

            Kevin K.

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Mon, 28 May 2001, Kevin D. Kissell wrote:

> Use a global variable testable by the inline code?

 With both variants inlined?  Now that's really ugly.

> >  Are vr41xx plain ISA I or crippled ISA II+ CPUs?
> 
> Actually, they are crippled MIPS III+ 64-bit CPUs

 Then an ll/sc and lld/scd emulation seems to be most appropriate here.  I
don't think we want to add _test_and_set() to mips64*-linux. 

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Geert Uytterhoeven]

On Mon, 28 May 2001, Maciej W. Rozycki wrote:
> On Mon, 28 May 2001, Kevin D. Kissell wrote:
> > >  Are vr41xx plain ISA I or crippled ISA II+ CPUs?
> > 
> > Actually, they are crippled MIPS III+ 64-bit CPUs
> 
>  Then an ll/sc and lld/scd emulation seems to be most appropriate here.  I
> don't think we want to add _test_and_set() to mips64*-linux. 

Do you want to run a 64-bit kernel on a Vr41xx? Most of these are used in
embedded systems, where the amount of RAM is a few orders of magnitude smaller
than the 32-bit RAM limit.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven ------------- Sony Software Development Center Europe (SDCE)
Geert.Uytterhoeven@sonycom.com ------------------- Sint-Stevens-Woluwestraat 55
Voice +32-2-7248626 Fax +32-2-7262686 ---------------- B-1130 Brussels, Belgium

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Tue, 29 May 2001, Geert Uytterhoeven wrote:

> >  Then an ll/sc and lld/scd emulation seems to be most appropriate here.  I
> > don't think we want to add _test_and_set() to mips64*-linux. 
> 
> Do you want to run a 64-bit kernel on a Vr41xx? Most of these are used in
> embedded systems, where the amount of RAM is a few orders of magnitude smaller
> than the 32-bit RAM limit.

 I have no preference on a Vr41xx, at least as long as I don't have one. 
You need an ll/sc emulation for it anyway, as you don't want to run ISA I
binaries on it, I suppose. 

 I wonder why ll/sc was skipped from it -- its hardware implementation is
quite simple... 

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Joe deBlaquiere]

Maciej W. Rozycki wrote:

> On Mon, 28 May 2001, Kevin D. Kissell wrote:
> 
> 
>> Use a global variable testable by the inline code?
> 
> 
>  With both variants inlined?  Now that's really ugly.
> 
> 
>>>  Are vr41xx plain ISA I or crippled ISA II+ CPUs?
>> 
>> Actually, they are crippled MIPS III+ 64-bit CPUs
> 
> 
>  Then an ll/sc and lld/scd emulation seems to be most appropriate here.  I
> don't think we want to add _test_and_set() to mips64*-linux. 

All the cases I've seen have been for 32-bit kernels. A 64-bit PDA kernel seems like a wee tiny bit of overkill



-- 
Joe deBlaquiere
Red Hat, Inc.
307 Wynn Drive
Huntsville AL, 35805
voice : (256)-704-9200
fax   : (256)-837-3839

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Mike McDonald]

>Date: Tue, 29 May 2001 08:02:46 -0500
>From: Joe deBlaquiere <jadb@redhat.com>
>To: "Maciej W. Rozycki" <macro@ds2.pg.gda.pl>
>Subject: Re: MIPS_ATOMIC_SET again (Re: newest kernel
>
>
>
>Maciej W. Rozycki wrote:
>
>> On Mon, 28 May 2001, Kevin D. Kissell wrote:
>> 
>> 
>>> Use a global variable testable by the inline code?
>> 
>> 
>>  With both variants inlined?  Now that's really ugly.
>> 
>> 
>>>>  Are vr41xx plain ISA I or crippled ISA II+ CPUs?
>>> 
>>> Actually, they are crippled MIPS III+ 64-bit CPUs
>> 
>> 
>>  Then an ll/sc and lld/scd emulation seems to be most appropriate here.  I
>> don't think we want to add _test_and_set() to mips64*-linux. 
>
>All the cases I've seen have been for 32-bit kernels. A 64-bit PDA kernel seems like a wee tiny bit of overkill

  I've been asked about running 64 bit binaries on a VR4121.

  Mike McDonald
  mikemac@mikemac.com

Re: Surprise! (Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Jun Sun]

"Maciej W. Rozycki" wrote:
> 
> On Fri, 25 May 2001, Jun Sun wrote:
> 
> > Alright, I rolled my sleeve and digged into IRIX 6.5, and guess what?
> > sysmips() does NOT have MIPS_ATOMIC_SET (2001) on IRIX!  See the header below.
> 
>  I remember Ralf writing of this being a compatibility call with RISC/OS
> (is it the original OS of MIPS, Inc.?), IIRC.  Ralf: am I right?
> 
> > So apparently MIPS_ATOMIC_SET was invented for Linux only, probably just to
> > implement _test_and_set().  (It would be interesting to see how IRIX implement
> > _test_and_set() on MIPS I machines.  However, the machine I have access uses
> > ll/sc instructions).
> 
>  Does IRIX actually run on anything below ISA II?
>

I assume nobody answering the above questions means nobody really care.  So we
can safely move ahead without worrying about them. :-)
 
> > To me, either 1.a) or 2) is fine with me, although I have a slight faovr over
> > 2) (perhaps because I don't like assembly code and the extra "vertical"
> > calling layer introduced in 1.a)
> 
>  What about 3) -- a new syscall with a different semantics and no need to
> care about limitations of current implementations (especially the
> sysmips() bag).  

Having a new syscall is fine with me, although seems a little more instrusive
than adding a subcall to sysmips().

> I've just sent a proposal for discussion.  I'm looking
> forward for constructive feedback.
> 

The patch looks good to me.

BTW, why wouldn't you choose to have three arguments in the syscall, where the
last one is a pointer to the variable to hold the return value?  Doing that
would avoid tricky register manipulation on both calling side (fetching return
value from $v1) and kernel side (setting regs.regs[3]).

Jun

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Jun Sun]

"Maciej W. Rozycki" wrote:
> 
> On Mon, 28 May 2001, Kevin D. Kissell wrote:
> 
> > Use a global variable testable by the inline code?
> 
>  With both variants inlined?  Now that's really ugly.

I think system V requires _test_and_set() being included in the libsys dynamic
library.  Does Linux want to be sysv compatible?  If so, we should removed the
inlined _test_and_set().

> 
> > >  Are vr41xx plain ISA I or crippled ISA II+ CPUs?
> >
> > Actually, they are crippled MIPS III+ 64-bit CPUs
> 
>  Then an ll/sc and lld/scd emulation seems to be most appropriate here.  I
> don't think we want to add _test_and_set() to mips64*-linux.
> 

64 bit is a overkill for the pityful vr41xx CPUs.

The need for kernel emulated test_and_set() in 64bit kernel is not obvious
yet, and hopefully will never come.

Jun

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Mike McDonald]

>To: Joe deBlaquiere <jadb@redhat.com>
>Subject: Re: MIPS_ATOMIC_SET again (Re: newest kernel 
>Date: Tue, 29 May 2001 08:45:20 -0700
>From: Mike McDonald <mikemac@mikemac.com>

>  I've been asked about running 64 bit binaries on a VR4121.
						      ^^^^^^
						      VR4122!

  I shouldn't try writing before I have my tea in the morning!

  Mike "Banned from sgi.bad-attitude" McDonald
  mikemac@mikemac.com

Re: Surprise! (Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Kevin D. Kissell]

> > > So apparently MIPS_ATOMIC_SET was invented for Linux only, probably
just to
> > > implement _test_and_set().  (It would be interesting to see how IRIX
implement
> > > _test_and_set() on MIPS I machines.  However, the machine I have
access uses
> > > ll/sc instructions).
> >
> >  Does IRIX actually run on anything below ISA II?
> >
>
> I assume nobody answering the above questions means nobody
> really care.  So we can safely move ahead without worrying about
> them. :-)

I was waiting for someone to give a more authoritative
answer, but IIRC from my days at SGI, IRIX did of course
run on the R3000-based products, but support for them
was dropped in IRIX 6.x.  I couldn't give you the version
number of the last IRIX version that did support the R3K,
though.

And as others have observed, it was in any case
not IRIX, but MIPS' own RiscOS that drove the ABI.

            Keivn K.

Re: Surprise! (Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Kevin D. Kissell]

> > > So apparently MIPS_ATOMIC_SET was invented for Linux only, probably
just to
> > > implement _test_and_set().  (It would be interesting to see how IRIX
implement
> > > _test_and_set() on MIPS I machines.  However, the machine I have
access uses
> > > ll/sc instructions).
> >
> >  Does IRIX actually run on anything below ISA II?
> >
>
> I assume nobody answering the above questions means nobody
> really care.  So we can safely move ahead without worrying about
> them. :-)

I was waiting for someone to give a more authoritative
answer, but IIRC from my days at SGI, IRIX did of course
run on the R3000-based products, but support for them
was dropped in IRIX 6.x.  I couldn't give you the version
number of the last IRIX version that did support the R3K,
though.

And as others have observed, it was in any case
not IRIX, but MIPS' own RiscOS that drove the ABI.

            Keivn K.

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Kevin D. Kissell]

> >>>>  Are vr41xx plain ISA I or crippled ISA II+ CPUs?
> >>>
> >>> Actually, they are crippled MIPS III+ 64-bit CPUs
> >>
> >>
> >>  Then an ll/sc and lld/scd emulation seems to be most appropriate here.
I
> >> don't think we want to add _test_and_set() to mips64*-linux.
> >
> >All the cases I've seen have been for 32-bit kernels. A 64-bit PDA kernel
seems like a wee tiny bit of overkill
>
>   I've been asked about running 64 bit binaries on a VR4121.

Being able to use all 64-bits of the registers is a huge win for
certain embedded/handheld applications, even if 64-bit addressing
is worse than useless.  It's unfortunate that the original R4000 merged
the enabling of 64-bit data types with the generation of 64-bit addresses.
The newer MIPS64 CPUs have a bit in the Status register to
enable the data types without enabling the addresses.
I don't know that NEC has implemented this for the VR41xx
family, but they really should.  And fix ll/sc while they are at it.  ;-)

            Kevin K.

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Kevin D. Kissell]

> >>>>  Are vr41xx plain ISA I or crippled ISA II+ CPUs?
> >>>
> >>> Actually, they are crippled MIPS III+ 64-bit CPUs
> >>
> >>
> >>  Then an ll/sc and lld/scd emulation seems to be most appropriate here.
I
> >> don't think we want to add _test_and_set() to mips64*-linux.
> >
> >All the cases I've seen have been for 32-bit kernels. A 64-bit PDA kernel
seems like a wee tiny bit of overkill
>
>   I've been asked about running 64 bit binaries on a VR4121.

Being able to use all 64-bits of the registers is a huge win for
certain embedded/handheld applications, even if 64-bit addressing
is worse than useless.  It's unfortunate that the original R4000 merged
the enabling of 64-bit data types with the generation of 64-bit addresses.
The newer MIPS64 CPUs have a bit in the Status register to
enable the data types without enabling the addresses.
I don't know that NEC has implemented this for the VR41xx
family, but they really should.  And fix ll/sc while they are at it.  ;-)

            Kevin K.

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Tue, 29 May 2001, Jun Sun wrote:

> I think system V requires _test_and_set() being included in the libsys dynamic
> library.  Does Linux want to be sysv compatible?  If so, we should removed the
> inlined _test_and_set().

 Why should we remove the inlined _test_and_set()?  We do have a number of
other inlined functions in glibc, e.g. memcpy() and friends in
<bits/string.h> (not for MIPS, actually, but for other hosts), yet it does
not make glibc SVR4 incompatible.  Of course we always provide non-inlined
versions of such functions as well -- check with objdump if unsure. 

 Note they are *extern* inline.

> The need for kernel emulated test_and_set() in 64bit kernel is not obvious
> yet, and hopefully will never come.

 Agreed.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: Surprise! (Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Tue, 29 May 2001, Jun Sun wrote:

> >  What about 3) -- a new syscall with a different semantics and no need to
> > care about limitations of current implementations (especially the
> > sysmips() bag).  
> 
> Having a new syscall is fine with me, although seems a little more instrusive
> than adding a subcall to sysmips().

 Actually whole sysmips() looks like a crazy hack, much like ioctl(), but
even worse (passing a pointer in an integer argument, even if it works...
yuck!).  And it is weird, to say at least, to have different
interpretations of the return value -- sometimes it's errno and sometimes 
it's something different.

 A separate syscall has the advantage we can write it cleanly and
optimally, not caring of the calling convention of the incorporating
syscall as in the case of sysmips().  And the overhead is a bit smaller. 

> The patch looks good to me.

 Great!

> BTW, why wouldn't you choose to have three arguments in the syscall, where the
> last one is a pointer to the variable to hold the return value?  Doing that
> would avoid tricky register manipulation on both calling side (fetching return
> value from $v1) and kernel side (setting regs.regs[3]).

 You may treat this as a syscall returning long long. ;-)

 A few other syscalls have non-standard register allocations as well.  I
think it's cleaner to return a value in a register -- no need to handle
passing an invalid pointer.  And the overhead is a bit smaller -- no need
for two memory accesses, which require an auxillary register anyway and
which may cause a TLB fault and/or a cache line reload. 

 This is to be discussed though.  

 I have the glibc patch basically ready.  It's nice -- the target
sys__test_and_set() wrapper is only seven instructions long (of which
three are the gp PIC preamble, sigh).  The v1 to v0 move is free -- it
fits in the delay slot of "jr $ra" nicely. :-)

 Unfortunately, for glibc 2.2.x we should probably fall back to
sysmips(MIPS_ATOMIC_SET) if the sys__test_and_set() fails (with ENOSYS),
for compatibility.  This lengthens the function to 27 instructions.  We
can hopefully remove the crap in glibc 2.3. 

  Maciej

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[J. Scott Kasten]

> >All the cases I've seen have been for 32-bit kernels. A 64-bit PDA kernel seems like a wee tiny bit of overkill
>
>   I've been asked about running 64 bit binaries on a VR4121.

Unless you're doing encryption in an embedded device, then 64 bit regs are
quite nice...

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[J. Scott Kasten]

> >All the cases I've seen have been for 32-bit kernels. A 64-bit PDA kernel seems like a wee tiny bit of overkill
>
>   I've been asked about running 64 bit binaries on a VR4121.

Unless you're doing encryption in an embedded device, then 64 bit regs are
quite nice...

Re: Surprise! (Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Jun Sun]

"Maciej W. Rozycki" wrote:
> 
> On Tue, 29 May 2001, Jun Sun wrote:
> 
> > >  What about 3) -- a new syscall with a different semantics and no need to
> > > care about limitations of current implementations (especially the
> > > sysmips() bag).
> >
> > Having a new syscall is fine with me, although seems a little more instrusive
> > than adding a subcall to sysmips().
> 
>  Actually whole sysmips() looks like a crazy hack, much like ioctl(), but
> even worse (passing a pointer in an integer argument, even if it works...
> yuck!).  And it is weird, to say at least, to have different
> interpretations of the return value -- sometimes it's errno and sometimes
> it's something different.
> 

Agree.  Having dual semantics for the return value is bad.

I was actually suggesting to have a new subcall in sysmips (e.g.,
MIPS_NEW_ATOMIC_SET) and still working within the sysmips() call framework.

Is there any concern as for adding a new syscall?

Jun

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Jun Sun]

"Maciej W. Rozycki" wrote:
> 
> On Tue, 29 May 2001, Jun Sun wrote:
> 
> > I think system V requires _test_and_set() being included in the libsys dynamic
> > library.  Does Linux want to be sysv compatible?  If so, we should removed the
> > inlined _test_and_set().
> 
>  Why should we remove the inlined _test_and_set()?  We do have a number of
> other inlined functions in glibc, e.g. memcpy() and friends in
> <bits/string.h> (not for MIPS, actually, but for other hosts), yet it does
> not make glibc SVR4 incompatible.  Of course we always provide non-inlined
> versions of such functions as well -- check with objdump if unsure.
> 

Hmm, I think to write SYSV compatible code one should not used inlined ABI
calls. Otherwise the binary would bypass libsys and becomes not portable among
SYSV machines.

On the other hand, what other MIPS SYSV platforms are there for us to be
compatible?  IRIX? :-)

>  Note they are *extern* inline.
> 

I don't think "extern" changes the picture here because once the call is
inlined the code will bypass libsys - unless my previous understanding is
wrong.


Jun

Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Wed, 30 May 2001, Jun Sun wrote:

> I don't think "extern" changes the picture here because once the call is
> inlined the code will bypass libsys - unless my previous understanding is
> wrong.

 Yes, it does.  If you insist on not inlining it -- you can do it.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: Surprise! (Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Maciej W. Rozycki]

On Wed, 30 May 2001, Jun Sun wrote:

> Agree.  Having dual semantics for the return value is bad.
> 
> I was actually suggesting to have a new subcall in sysmips (e.g.,
> MIPS_NEW_ATOMIC_SET) and still working within the sysmips() call framework.
> 
> Is there any concern as for adding a new syscall?

 I'd prefer to avoid sysmips() (as a whole, not only the MIPS_ATOMIC_SET
subcall) for the reasons I've written previously.  There is really no
point in saving five bytes in the syscall tables just to make use of the
existing mess. 

 Note that adding MIPS_NEW_ATOMIC_SET doesn't make sysmips() more
consistent at all. 

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: Surprise! (Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Ralf Baechle]

On Mon, May 28, 2001 at 05:34:26PM +0200, Maciej W. Rozycki wrote:

> > Alright, I rolled my sleeve and digged into IRIX 6.5, and guess what? 
> > sysmips() does NOT have MIPS_ATOMIC_SET (2001) on IRIX!  See the header below.
> 
>  I remember Ralf writing of this being a compatibility call with RISC/OS
> (is it the original OS of MIPS, Inc.?), IIRC.  Ralf: am I right? 

Yes; this function is also implemented for IRIX 5 which we have some
binary compatibility code for.

> > So apparently MIPS_ATOMIC_SET was invented for Linux only, probably just to
> > implement _test_and_set().  (It would be interesting to see how IRIX
> > implement _test_and_set() on MIPS I machines.  However, the machine I
> > have access uses ll/sc instructions).

I don't recall seeing a _test_and_set() in the IRIX API/ABI.

>  Does IRIX actually run on anything below ISA II?

Well, it ran.  A loong time ago.  IRIX dropped support for MIPS I in IRIX 6.
That makes sense as IRIX 6 is only running on R4000 and better, that is
MIPS III CPUs.

  Ralf

Re: Surprise! (Re: MIPS_ATOMIC_SET again (Re: newest kernel

[Jun Sun]

Ralf Baechle wrote:
> 
> On Mon, May 28, 2001 at 05:34:26PM +0200, Maciej W. Rozycki wrote:
> 
> > > Alright, I rolled my sleeve and digged into IRIX 6.5, and guess what?
> > > sysmips() does NOT have MIPS_ATOMIC_SET (2001) on IRIX!  See the header below.
> >
> >  I remember Ralf writing of this being a compatibility call with RISC/OS
> > (is it the original OS of MIPS, Inc.?), IIRC.  Ralf: am I right?
> 
> Yes; this function is also implemented for IRIX 5 which we have some
> binary compatibility code for.
> 

Is anybody at all still running IRIX binary on Linux?  My impression the
compatibility is broken a while back.

> > > So apparently MIPS_ATOMIC_SET was invented for Linux only, probably just to
> > > implement _test_and_set().  (It would be interesting to see how IRIX
> > > implement _test_and_set() on MIPS I machines.  However, the machine I
> > > have access uses ll/sc instructions).
> 
> I don't recall seeing a _test_and_set() in the IRIX API/ABI.
>

It is part of SYSV MIPS supplement spec.  IRIX 6.5 has it.
 
Jun