Re: [Qemu-devel] and now bus error for i386 guest

[Shaddy Baddah <shaddy.baddah@shaddybaddah.name>]

Hi,

Blue Swirl wrote:
> On 12/4/07, Shaddy Baddah <shaddy.baddah@shaddybaddah.name> wrote:
>> HI,
>>
>> Blue Swirl wrote:
>>> On 11/14/07, Shaddy Baddah <shaddy.baddah@shaddybaddah.name> wrote:
>>>> Hi again,
>>>>
>>>> After further culling the target list, I was able to install qemu on my
>>>> sun4u host.
>>>>
>>>> However, running it, I get a bus error. See below for details:
>>>>
>>>> $ /opt/qemu-cvs/bin/qemu -m 128 -cdrom
>>>> ~/KNOPPIX_V5.1.1CD-2007-01-04-EN.iso -vnc :1
>>>> Bus error
>>> For some reason, Sparc Linux host is not working. I think Sparc Solaris is OK.
>> Yes, I do recall that I was able to get this working on Sparc Solaris.
>> Anyway, when I get more time, I will have a better stab at debugging
>> this. I got a start on this today, and here is a bit of cut and paste
>> that may or may not be insightful. The address for env1 looks questionable:
>>
>> shaddy@localhost:~/qemu-cvs/qemu-build$ gdb --args ./i386-softmmu/qemu
>> -hda ../../KNOPPIX_V5.1.1CD-2007-01-04-EN.iso -L ../qemu/pc-bios
>> GNU gdb 6.6.90.20070912-debian
>> Copyright (C) 2007 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later
>> <http://gnu.org/licenses/gpl.html>
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
>> and "show warranty" for details.
>> This GDB was configured as "sparc-linux-gnu"...
>> Using host libthread_db library "/lib/libthread_db.so.1".
>> (gdb) run
>> Starting program: /home/shaddy/qemu-cvs/qemu-build/i386-softmmu/qemu
>> -hda ../../KNOPPIX_V5.1.1CD-2007-01-04-EN.iso -L ../qemu/pc-bios
>> [Thread debugging using libthread_db enabled]
>> [New Thread 0xf7f7b550 (LWP 9363)]
>>
>> Program received signal SIGBUS, Bus error.
>> [Switching to Thread 0xf7f7b550 (LWP 9363)]
>> cpu_x86_exec (env1=0x2) at /home/shaddy/qemu-cvs/qemu/cpu-exec.c:307
>> 307                 if (env->exception_index >= 0) {
>> (gdb) info threads
>> * 1 Thread 0xf7f7b550 (LWP 9363)  cpu_x86_exec (env1=0x2)
>>      at /home/shaddy/qemu-cvs/qemu/cpu-exec.c:307
>> (gdb)
>
> My guess is that Linux glibc overwrites global registers at some
> point, like in signal handling or setjmp. The generated code looks OK
> and it shouldn't be different from what Solaris version would
> generate. I've been thinking of different register design (using
> locals or outs) but then the op helpers would need to use different
> mechanism to access T0/T1/T2. Compiling Qemu against uClibc could also
> show something.
As I will detail later, this problem looks to me to be at least 
partially caused by a bug in the compiler optimizer.

>> PS: couldn't this also be debugged from within qemu VM running Debian
>> Sparc, for those that don't have access to a real machine? I got a start
>> on this today as well (because I don't always have access to the Sun
>> Ultra box). Seems stable, and I was able to get a start on a compile,
>> that is currently still going.
>
> Nice idea! Do you mean full system emulator or user mode?
I meant full system emulator mode. I was able to successfully compile 
inside the guest Sparc(32) Debian Lenny system. However, I couldn't 
really run qemu successfully inside the guest system. I was getting a 
hang, and I traced that down to a call to glibc's timer_create function, 
in the dynticks_start_timer function:

      printf("timer_create1\n");
    if (timer_create(CLOCK_REALTIME, &ev, &host_timer)) {
      printf("timer_create\n");
        perror("timer_create");

        /* disable dynticks */
        fprintf(stderr, "Dynamic Ticks disabled\n");

        return -1;
    }
      printf("timer_create2\n");

the bt shows three threads, and two of which are stuck in some type of 
pthread suspend functions, and on stuck in a poll function. I don't know 
much about the clocks, so I just randomly tried to select clocks via 
-clock option. I tried -clock unix, and got a similar bus error. But all 
this precludes what I am about to write with regards to my discoveries 
on the compiler optimizer.

Back on the real Sun Ultra box, I did a bit of debug. I found the 
following bit of failed code, as described in a capture of my gdb session:

$ gdb --args ./i386-softmmu/qemu -cdrom 
../../KNOPPIX_V5.1.1CD-2007-01-04-EN.iso -L ../qemu/pc-bios
GNU gdb 6.6.90.20070912-debian
Copyright (C) 2007 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "sparc-linux-gnu"...
Using host libthread_db library "/lib/libthread_db.so.1".
(gdb) break vl.c:7362
Breakpoint 1 at 0x1e958: file /home/shaddy/qemu-cvs/qemu/vl.c, line 7362.
(gdb) display /i $pc
(gdb) run
Starting program: /home/shaddy/qemu-cvs/qemu-build/i386-softmmu/qemu 
-cdrom ../../KNOPPIX_V5.1.1CD-2007-01-04-EN.iso -L ../qemu/pc-bios
[Thread debugging using libthread_db enabled]
[New Thread 0xf7f77550 (LWP 13579)]
[Switching to Thread 0xf7f77550 (LWP 13579)]

Breakpoint 1, main (argc=1430528, argv=0x15d400) at 
/home/shaddy/qemu-cvs/qemu/vl.c:7362
7362                    env = next_cpu;
1: x/i $pc
0x1e958 <main+13992>:   ld  [ %l6 + 0x8c ], %l1
(gdb) print /x next_cpu
$1 = 0x1cd1388
(gdb) print /x env
$2 = 0x322e3100
(gdb) stepi
7366                    ret = cpu_exec(env);
1: x/i $pc
0x1e95c <main+13996>:   call  0xa90b4 <cpu_x86_exec>
0x1e960 <main+14000>:   mov  %l1, %o0
(gdb) print /x env
$3 = 0x322e3100
(gdb)

as you can see, the assignment is not being effected correctly. So I 
tried again, removing optimization by removing -O2 from the CFLAGS, and 
again debugging:

$ gdb --args ./i386-softmmu/qemu -cdrom 
../../KNOPPIX_V5.1.1CD-2007-01-04-EN.iso -L ../qemu/pc-bios
GNU gdb 6.6.90.20070912-debian
Copyright (C) 2007 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "sparc-linux-gnu"...
Using host libthread_db library "/lib/libthread_db.so.1".
(gdb) break vl.c:7362
Breakpoint 1 at 0x240a4: file /home/shaddy/qemu-cvs/qemu/vl.c, line 7362.
(gdb) display /i $pc
(gdb) run
Starting program: /home/shaddy/qemu-cvs/qemu-build/i386-softmmu/qemu 
-cdrom ../../KNOPPIX_V5.1.1CD-2007-01-04-EN.iso -L ../qemu/pc-bios
[Thread debugging using libthread_db enabled]
[New Thread 0xf7f2f550 (LWP 14336)]
[Switching to Thread 0xf7f2f550 (LWP 14336)]

Breakpoint 1, main_loop () at /home/shaddy/qemu-cvs/qemu/vl.c:7362
7362                    env = next_cpu;
1: x/i $pc
0x240a4 <main_loop+152>:        sethi  %hi(0x258800), %g4
(gdb) print /x next_cpu
$1 = 0x1dcd460
(gdb) print /x env
$2 = 0x258888
(gdb) stepi
0x000240a8      7362                    env = next_cpu;
1: x/i $pc
0x240a8 <main_loop+156>:        or  %g4, 0x4c, %g4      ! 0x25884c 
<next_cpu>
(gdb)
0x000240ac      7362                    env = next_cpu;
1: x/i $pc
0x240ac <main_loop+160>:        ld  [ %g4 ], %g4
(gdb)
0x000240b0      7362                    env = next_cpu;
1: x/i $pc
0x240b0 <main_loop+164>:        st  %g4, [ %fp + -20 ]
(gdb)
7366                    ret = cpu_exec(env);
1: x/i $pc
0x240b4 <main_loop+168>:        ld  [ %fp + -20 ], %o0
(gdb) print /x env
$3 = 0x1dcd460
(gdb) stepi
0x000240b8      7366                    ret = cpu_exec(env);
1: x/i $pc
0x240b8 <main_loop+172>:        call  0x14fa64 <cpu_x86_exec>
0x240bc <main_loop+176>:        nop
(gdb) stepi
0x000240bc      7366                    ret = cpu_exec(env);
1: x/i $pc
0x240bc <main_loop+176>:        nop
(gdb) stepi
cpu_x86_exec (env1=0x1de6530) at /home/shaddy/qemu-cvs/qemu/cpu-exec.c:244
244     {
1: x/i $pc
0x14fa64 <cpu_x86_exec>:        add  %sp, -264, %sp
(gdb)

Now you can see that the assignment is occurring successfully. But for 
reasons that I cannot understand, but suspect to be a compiler bug, the 
transference of the value of env as a parameter to cpu_exec is not 
occurring correctly. My limited knowledge leads me to believe that the 
value for env is being stored at the wrong location in the stack. Am I 
right?

I don't know if it has any significance in this, but during the build, I 
did see the following suspicious warnings:

In file included from /home/shaddy/qemu-cvs/qemu/target-i386/op.c:22:
/home/shaddy/qemu-cvs/qemu/target-i386/exec.h:46: warning: 
call-clobbered register used for global register variable
/home/shaddy/qemu-cvs/qemu/target-i386/exec.h:47: warning: 
call-clobbered register used for global register variable
/home/shaddy/qemu-cvs/qemu/target-i386/exec.h:48: warning: 
call-clobbered register used for global register variable

Summing up, am I being curtailed by a GCC 3.4 bug, under sparc?

HTH,
Shaddy