Big GCC bug!!! [Was: Re: Do not misuse Coverity please]

[Kyle Moffett <mrmacman_g4@mac.com>]

On Mar 30, 2005, at 14:14, Paulo Marques wrote:
> Just a minor nitpick, though: wouldn't it be possible for an
> application to catch the SIGSEGV and let the code proceed,
> making invalid the assumption made by gcc?

Uhh, it's even worse than that.  Have a look at the following code:
> #include <stdio.h>
> #include <stdlib.h>
> #include <string.h>
> #include <errno.h>
> #include <sys/types.h>
> #include <sys/mman.h>
>
> struct test {
>         int code;
> };
> int test_check_first(struct test *a) {
>         int ret;
>         if (!a) return -1;
>         ret = a->code;
>         return ret;
> }
> int test_check_last(struct test *a) {
>         int ret;
>         ret = a->code;
>         if (!a) return -1;
>         return ret;
> }
>
> int main() {
>         int i;
>         struct test *nullmem = mmap(NULL, 4096, PROT_READ|PROT_WRITE,
>                         MAP_ANON|MAP_FIXED|MAP_PRIVATE, -1, 0);
>         if (nullmem == MAP_FAILED) {
>                 fprintf(stderr,"mmap: %s\n",strerror(errno));
>                 exit(1);
>         }
>         for (i = 0; i < 2; i++) {
>                 nullmem[i].code = i;
>                 printf("nullmem[%d].code = %d\n",i,i);
>                 printf("test_check_first(&nullmem[%d]) = %d\n",i,
>                         test_check_first(&nullmem[i]));
>                 printf("test_check_last(&nullmem[%d]) = %d\n",i,
>                         test_check_last(&nullmem[i]));
>         }
>         munmap(nullmem,4096);
>         exit(0);
> }

Without optimization:
> king:~# gcc -o mmapnull mmapnull.c
> king:~# ./mmapnull
> nullmem[0].code = 0
> test_check_first(&nullmem[0]) = -1
> test_check_last(&nullmem[0]) = -1
> nullmem[1].code = 1
> test_check_first(&nullmem[1]) = 1
> test_check_last(&nullmem[1]) = 1

With optimization:
> king:~# gcc -O2 -o mmapnull mmapnull.c
> king:~# ./mmapnull
> nullmem[0].code = 0
> test_check_first(&nullmem[0]) = -1
> test_check_last(&nullmem[0]) = 0
                         BUG ==> ^^^
> nullmem[1].code = 1
> test_check_first(&nullmem[1]) = 1
> test_check_last(&nullmem[1]) = 1

This is on multiple platforms, including PPC Linux, X86 Linux, and
PPC Mac OS X.  All exhibit the exact same behavior and output.  I
think I'll probably go report a GCC bug now :-D

Dereferencing null pointers is relied upon by a number of various
emulators and such, and is "platform-defined" in the standard, so
since Linux allows mmap at NULL, GCC shouldn't optimize that case
any differently.

Cheers,
Kyle Moffett

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a18 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$
L++++(+++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+
PGP+++ t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$ r  
!y?(-)
------END GEEK CODE BLOCK------