Float / Double issues

Float / Double issues

[Tim Hockin]

[-- Attachment #1: Type: text/plain, Size: 346 bytes --]

Hey gang - I have what seems to be two seperate issues on Mips/Linux
(cobalt boxes).

1) Programs using doubles with pthreads get corrupted data in the
doubles.
2) Floats and va_arg crash with an FPE

below are two snippets that show these, at least on our boxes.  If
someone could confirm or deny their existence on any other platform?


Tim




[-- Attachment #2: double.c --]
[-- Type: text/plain, Size: 976 bytes --]

#include <stdio.h>
#include <pthread.h>
#include <stdarg.h>

void doit(int, double, unsigned int);
void vait(unsigned int, ...);
void *ptfn(void *);
void doit2(int, int);
void vait2(int, ...);
void *ptfn2(void *);

int main() 
{
	pthread_t pt1;
	double d1 = 1.234; 

	printf("Without threads: \n");
	ptfn((void *)&d1);
	printf("\n");
	fflush(stdout);

	printf("With threads: \n");
	pthread_create(&pt1, NULL, ptfn, &d1);
	printf("\n");
	fflush(stdout);

	pthread_exit(0);
	return 0;
}

void *ptfn(void *x)
{
	char ar[16];
	unsigned int decimals = 5;
	double d = *(double *)x;

	doit(1, d, decimals);
	vait(decimals, d);
}

void doit(int x, double d, unsigned int ui)
{
	char buff[331];

	sprintf(buff,"double: %.*f",ui,d);
	printf("%s\n", buff);
}

void vait(unsigned int ui, ...)
{
	double d=-10.0;
	va_list va;

	va_start(va, ui);

	printf("double before va_arg: %f\n", d);
	d = va_arg(va, double);
	printf("double after va_arg: %f\n", d);

	va_end(va);

	doit(1, d, ui);
}

[-- Attachment #3: float.c --]
[-- Type: text/plain, Size: 744 bytes --]

#include <stdio.h>
#include <stdarg.h>

void doit(int, float, unsigned int);
void vait(unsigned int, ...);
void *ptfn(void *);

int main() 
{
	float d1 = 1.234; 

	printf("Without threads: \n");
	ptfn((void *)&d1);
	printf("\n");
	fflush(stdout);

	return 0;
}

void *ptfn(void *x)
{
	char ar[16];
	unsigned int decimals = 5;
	float d = *(float *)x;

	doit(1, d, decimals);
	vait(decimals, d);
}

void doit(int x, float d, unsigned int ui)
{
	char buff[331];

	sprintf(buff,"float: %.*f",ui,d);
	printf("%s\n", buff);
}

void vait(unsigned int ui, ...)
{
	float d=-10.0;
	va_list va;

	va_start(va, ui);

	printf("float before va_arg: %f\n", d);
	d = va_arg(va, float);
	printf("float after va_arg: %f\n", d);

	va_end(va);

	doit(1, d, ui);
}

Re: Float / Double issues

[Ralf Baechle]

On Mon, Jul 12, 1999 at 08:09:41PM -0700, Tim Hockin wrote:

> 2) Floats and va_arg crash with an FPE
> 
> below are two snippets that show these, at least on our boxes.  If
> someone could confirm or deny their existence on any other platform?

The float / va_arg bug is a pilot error.  See ANSI C draft chapter 6.5.2.2,
paragraph 6 and7.  In short the problem is that float parameters will be
promoted to double.  You therefore have to to change the call of va_arg
in your float.c to va_arg(va, float).

Obvious, heh ;-)

Btw, SGI's compiler also doesn't attempt to get va_arg(..., float) right.

  Ralf

Re: Float / Double issues

[Ralf Baechle]

On Mon, Jul 12, 1999 at 08:09:41PM -0700, Tim Hockin wrote:

> Hey gang - I have what seems to be two seperate issues on Mips/Linux
> (cobalt boxes).
> 
> 1) Programs using doubles with pthreads get corrupted data in the
> doubles.

That one is funny.  It was/is a longstanding libc bug originally reported
by Dong Liu.  I actually thought it'd be fixed but now thanks to your
report I see it's still broken.  So the multithreaded variant of your
double.c doesn't work at all on our libc.

The bug is in glibc/sysdeps/unix/sysv/linux/mips/clone.S; I suppose
Cobalt's libc which I last worked on late October is still using my
old clone.S from that time or somebody else there came up with a funky
new bug completly on it's own - which might explain why double.c cannot
even successfully create the threads.

More later ...

  Ralf

Re: Float / Double issues

[Ralf Baechle]

[-- Attachment #1: Type: text/plain, Size: 1178 bytes --]

On Wed, Jul 14, 1999 at 11:53:46PM +0200, Ralf Baechle wrote:

> > Hey gang - I have what seems to be two seperate issues on Mips/Linux
> > (cobalt boxes).
> > 
> > 1) Programs using doubles with pthreads get corrupted data in the
> > doubles.
> 
> That one is funny.  It was/is a longstanding libc bug originally reported
> by Dong Liu.  I actually thought it'd be fixed but now thanks to your
> report I see it's still broken.  So the multithreaded variant of your
> double.c doesn't work at all on our libc.
> 
> The bug is in glibc/sysdeps/unix/sysv/linux/mips/clone.S; I suppose
> Cobalt's libc which I last worked on late October is still using my
> old clone.S from that time or somebody else there came up with a funky
> new bug completly on it's own - which might explain why double.c cannot
> even successfully create the threads.
> 
> More later ...

It is later :-)

Attached are fixed versions of clone.S.  I also attach Dong's old test
program.  While now threads are working dong.c still fails for a large
number of threads, that is > 1000 or so with a SIGSEGV.  This of course
doesn't solve your problem yet, Tim - but now we've got the same problem ...

  Ralf

[-- Attachment #2: clone.S --]
[-- Type: text/plain, Size: 2478 bytes --]

/* Copyright (C) 1996 Free Software Foundation, Inc.
   This file is part of the GNU C Library.
   Contributed by Ralf Baechle <ralf@gnu.ai.mit.edu>, 1996.

   The GNU C Library is free software; you can redistribute it and/or
   modify it under the terms of the GNU Library General Public License as
   published by the Free Software Foundation; either version 2 of the
   License, or (at your option) any later version.

   The GNU C Library is distributed in the hope that it will be useful,
   but WITHOUT ANY WARRANTY; without even the implied warranty of
   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
   Library General Public License for more details.

   You should have received a copy of the GNU Library General Public
   License along with the GNU C Library; see the file COPYING.LIB.  If not,
   write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
   Boston, MA 02111-1307, USA.  */

/* clone() is even more special than fork() as it mucks with stacks
   and invokes a function in the right context after its all over.  */

#include <sys/asm.h>
#include <asm/unistd.h>
#include <sysdep.h>
#define _ERRNO_H	1
#include <errnos.h>

/* int clone(int (*fn)(void *arg), void *child_stack, int flags, void *arg) */

	.text
NESTED(__clone,4*SZREG,sp)
	.set	noreorder
	.cpload	$25
	.set	reorder
	subu	sp, 32
	.cprestore 16

#ifdef PROF
	.set	noat
	move	$1, ra
	jal	_mcount
	.set	at
#endif

	/* Sanity check arguments.  */
	li	v0, EINVAL
	beqz	a0, error	/* no NULL function pointers */
	beqz	a1, error	/* no NULL stack pointers */

	subu	a1, 32			# reserve argument save space
	sw	a0, 0(a1)		# save function pointer
	sw	a3, 4(a1)		# save argument pointer

	move	a0, a2			# Do the system call
	li	v0, __NR_clone
	syscall

	bnez	a3, error
	beqz	v0, __thread_start

	addiu	sp, 32
	ret

	/* Something bad happened -- no child created */
error:
	addiu	sp, 32
#ifdef __PIC__
	la	t9, __syscall_error
	jr	t9
#else
	j	__syscall_error
#endif
	END(__clone)

/* Load up the arguments to the function.  Put this block of code in
   its own function so that we can terminate the stack trace with our
   debug info.  */

ENTRY(__thread_start)
	/* cp is already loaded.  */
	.cprestore 16
	lw	t9, 0(sp)		# func ptr
	lw	a0, 4(sp)		# arg ptr

	/* Call the user's function */
	jalr		t9

	/* Call _exit rather than doing it inline for breakpoint purposes */
	move		a0, v0
	jal		_exit
	/* Unreached  */
	END(__thread_start)

weak_alias(__clone, clone)

[-- Attachment #3: dliu.c --]
[-- Type: text/plain, Size: 1023 bytes --]

#include <pthread.h>
#include <stdio.h>

void* new_thread(void* arg)
{
	int i;
	printf("Thread[%s] stack at %x\n",arg, &i);
	for (i = 0; i< 4; i++) {
	    printf("Thread[%s] %d\n", arg, i);
	    sched_yield();
	}
	return(NULL);
}

#define NUM_OF_THREAD 10000
main(int argc, char **argv)

{
        int num = NUM_OF_THREAD;
	pthread_t thread[NUM_OF_THREAD];
	
	char *args[NUM_OF_THREAD];
	int i;
	int last;
	void *status;

	if (argc > 1)
	    num = atoi(argv[1]);
	if (num>NUM_OF_THREAD)
	    num = NUM_OF_THREAD;
	printf("Original thread stack at %x\n", &i);

	for (i = 0 ; i < num; i++) {
	    args[i] = (char *)malloc(80);
	    sprintf(args[i], "%04d", i);
	    if (pthread_create(&thread[i],
			       NULL,
			       new_thread, (void *)args[i])) {
		printf("Error: creating new thread[%d]\n", i);
		break;
	    }
	}

	last = i;

	for (i = 0 ; i < last; i++) {
	    pthread_join(thread[i], &status);
	    printf("thread[%d] return status' address %p\n",i, status);
	}
	printf("%d threads created\n", last);
	exit(0);
}

Re: Float / Double issues - solved, perhaps

[Tim Hockin]

Ralf Baechle wrote:

> On Mon, Jul 12, 1999 at 08:09:41PM -0700, Tim Hockin wrote:
>
> > Hey gang - I have what seems to be two seperate issues on Mips/Linux
> > (cobalt boxes).
> >
> > 1) Programs using doubles with pthreads get corrupted data in the
> > doubles.
>
> That one is funny.  It was/is a longstanding libc bug originally reported
> by Dong Liu.  I actually thought it'd be fixed but now thanks to your
> report I see it's still broken.  So the multithreaded variant of your
> double.c doesn't work at all on our libc.
>
> The bug is in glibc/sysdeps/unix/sysv/linux/mips/clone.S; I suppose
> Cobalt's libc which I last worked on late October is still using my
> old clone.S from that time or somebody else there came up with a funky
> new bug completly on it's own - which might explain why double.c cannot
> even successfully create the threads.

We have the fix, I think.  Not clone.S, but in
glibc-2.0.7/linuxthreads/internals.h line 86 we need to tell gcc that this
struct (_pthread_descr_struct - which defines the offset of a thread's stack
from a well-aligned, malloc() returned addr) be padded to be aligned
correctly on an 8byte boundary:

-};
+} __attribute__ ((aligned(__alignof__(double))));

It's working for us now.  Please confirm if it works for you folks, too.  A
bug report has been filed for glibc, though I doubt if anything official
goes on for 2.0.x, now. Hopefully 2.1.x will fold this in.  A thread's stack
really should always be well aligned :)  Funny, the doubles were actually
calculated right, but stdio functions use va_arg - which expected double's
to be aligned, and weren't.  The work got done, but it couldn't tell us!

As for the va_arg "bug" - I realized it was pilot error shortly after
sending it out.  I just needed to look at something that was NOT this double
bug for a while. :)  So the new question is who wants to do thread support
in gdb !!

Tim

Re: Float / Double issues - solved, perhaps

[Ralf Baechle]

On Wed, Jul 14, 1999 at 06:18:29PM -0700, Tim Hockin wrote:

> We have the fix, I think.  Not clone.S, but in
> glibc-2.0.7/linuxthreads/internals.h line 86 we need to tell gcc that this
> struct (_pthread_descr_struct - which defines the offset of a thread's stack
> from a well-aligned, malloc() returned addr) be padded to be aligned
> correctly on an 8byte boundary:
> 
> -};
> +} __attribute__ ((aligned(__alignof__(double))));
> 
> It's working for us now.  Please confirm if it works for you folks, too.  A
> bug report has been filed for glibc, though I doubt if anything official
> goes on for 2.0.x, now. Hopefully 2.1.x will fold this in.  A thread's stack
> really should always be well aligned :)  Funny, the doubles were actually
> calculated right,

It's dangerous though.  If the code would have attempted to use a MIPS II
ldc1 / sdc1 instruction to access the floating point number, the code
would have been killed by arch/mips/kernel/unaligned.c.

> but stdio functions use va_arg - which expected double's
> to be aligned, and weren't.  The work got done, but it couldn't tell us!

The MIPS ABI wants a 8 byte alignment for the stack and even 16 byte for
ABI64.  Now we know what can happen if not :-)

Thanks for your fix,

  Ralf