[Qemu-devel] [patch] gcc4 host support

[Qemu-devel] [patch] gcc4 host support

[Paul Brook]

[-- Attachment #1: Type: text/plain, Size: 2355 bytes --]

The attached patch adds support for gcc4 x86 and x86_64 hosts.

The main problem with gcc4 is that we can no longer force gcc to place the 
return from each function at the end of the function.

My solution is to search the function for the "ret" instruction and replace 
them with a jmp to the next block of code. On RISC targets this would be 
easy. On x86/x86_64 it's significantly harder because instructions are 
variable length. This creates two problems:

- Identify the return instruction:  Dyngen traces through the code following 
any branches. When generating a "ret" to exit the translation block, or a 
"jmp" which will be patched at runtime we generate a privileged instruction 
instead. dyngen recognises these and replaces them up with the correct value. 
For simplicity we still require a single ret instruction. This is easy to 
achieve with the existing FORCE_RET markers.

- Replacing the ret with a jmp: If the ret is not the last instruction we need 
to replace it with a jmp to the next op. Unfortunately a jmp instruction is 2 
or 5 bytes, whereas a ret is just one byte long. To do the replacement we 
need to move some of the surrounding code out of the way. I've made the 
FORCE_RET macro insert 4 bytes of nops. This guarantees that we always have 4 
bytes we can move without having to redirect any jmps. In almost all cases 
dyngen can strip these nop instructions, so they never make it into the 
generated code.

The ppc target code used it's own RETURN macro. I've replaced this with the 
standard FORCE_RET macro (to get the necessary nops), and changed dyngen so 
that it can insert nops after each op for debugging purposes.

I've successfully booted the nbench floppy undef i386-sofmmu guest on 
i686-linux, x86_64-linux and windows hosts with this patch, and verified that 
arm-user emulation still works. I also compiled op.c with -freorder-block, 
and it still worked.

On x86-64 the gcc4 compiled qemu runs a few percent faster than with the 
redhat 3.2.3 system compiler. 

On x86 I've had to hack round other bugs (gcc doesn't like doing 64-bit 
arithmetic with only three 32-bit registers), so is noticably slower. These 
hacks also slow down gcc3.3 by a similar amount.

Depending on the optimization options used it's also necessary to add 
FORCE_RET markers to more op.c. I'll submit those separately.

Paul

[-- Attachment #2: patch.qemu_force_ret --]
[-- Type: text/x-diff, Size: 12232 bytes --]

Index: target-i386/op.c
===================================================================
RCS file: /cvsroot/qemu/qemu/target-i386/op.c,v
retrieving revision 1.37
diff -u -p -r1.37 op.c
--- target-i386/op.c	26 Apr 2005 20:38:17 -0000	1.37
+++ target-i386/op.c	9 May 2005 01:33:04 -0000
@@ -1008,6 +1008,7 @@ void OPPROTO op_aaa(void)
     }
     EAX = (EAX & ~0xffff) | al | (ah << 8);
     CC_SRC = eflags;
+    FORCE_RET();
 }
 
 void OPPROTO op_aas(void)
@@ -1032,6 +1033,7 @@ void OPPROTO op_aas(void)
     }
     EAX = (EAX & ~0xffff) | al | (ah << 8);
     CC_SRC = eflags;
+    FORCE_RET();
 }
 
 void OPPROTO op_daa(void)
@@ -1059,6 +1061,7 @@ void OPPROTO op_daa(void)
     eflags |= parity_table[al]; /* pf */
     eflags |= (al & 0x80); /* sf */
     CC_SRC = eflags;
+    FORCE_RET();
 }
 
 void OPPROTO op_das(void)
@@ -1089,6 +1092,7 @@ void OPPROTO op_das(void)
     eflags |= parity_table[al]; /* pf */
     eflags |= (al & 0x80); /* sf */
     CC_SRC = eflags;
+    FORCE_RET();
 }
 
 /* segment handling */
@@ -1608,6 +1612,7 @@ void OPPROTO op_flds_FT0_A0(void)
 #else
     FT0 = ldfl(A0);
 #endif
+    FORCE_RET();
 }
 
 void OPPROTO op_fldl_FT0_A0(void)
@@ -1618,6 +1623,7 @@ void OPPROTO op_fldl_FT0_A0(void)
 #else
     FT0 = ldfq(A0);
 #endif
+    FORCE_RET();
 }
 
 /* helpers are needed to avoid static constant reference. XXX: find a better way */
@@ -1663,6 +1669,7 @@ void OPPROTO op_fild_FT0_A0(void)
 #else
     FT0 = (CPU86_LDouble)ldsw(A0);
 #endif
+    FORCE_RET();
 }
 
 void OPPROTO op_fildl_FT0_A0(void)
@@ -1673,6 +1680,7 @@ void OPPROTO op_fildl_FT0_A0(void)
 #else
     FT0 = (CPU86_LDouble)((int32_t)ldl(A0));
 #endif
+    FORCE_RET();
 }
 
 void OPPROTO op_fildll_FT0_A0(void)
@@ -1683,6 +1691,7 @@ void OPPROTO op_fildll_FT0_A0(void)
 #else
     FT0 = (CPU86_LDouble)((int64_t)ldq(A0));
 #endif
+    FORCE_RET();
 }
 #endif
 
@@ -2229,6 +2238,7 @@ void OPPROTO op_fldcw_A0(void)
 {
     env->fpuc = lduw(A0);
     update_fp_status();
+    FORCE_RET();
 }
 
 void OPPROTO op_fclex(void)
Index: target-i386/ops_mem.h
===================================================================
RCS file: /cvsroot/qemu/qemu/target-i386/ops_mem.h,v
retrieving revision 1.6
diff -u -p -r1.6 ops_mem.h
--- target-i386/ops_mem.h	13 Mar 2005 09:52:09 -0000	1.6
+++ target-i386/ops_mem.h	9 May 2005 01:33:04 -0000
@@ -1,51 +1,61 @@
 void OPPROTO glue(glue(op_ldub, MEMSUFFIX), _T0_A0)(void)
 {
     T0 = glue(ldub, MEMSUFFIX)(A0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_ldsb, MEMSUFFIX), _T0_A0)(void)
 {
     T0 = glue(ldsb, MEMSUFFIX)(A0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_lduw, MEMSUFFIX), _T0_A0)(void)
 {
     T0 = glue(lduw, MEMSUFFIX)(A0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_ldsw, MEMSUFFIX), _T0_A0)(void)
 {
     T0 = glue(ldsw, MEMSUFFIX)(A0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_ldl, MEMSUFFIX), _T0_A0)(void)
 {
     T0 = (uint32_t)glue(ldl, MEMSUFFIX)(A0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_ldub, MEMSUFFIX), _T1_A0)(void)
 {
     T1 = glue(ldub, MEMSUFFIX)(A0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_ldsb, MEMSUFFIX), _T1_A0)(void)
 {
     T1 = glue(ldsb, MEMSUFFIX)(A0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_lduw, MEMSUFFIX), _T1_A0)(void)
 {
     T1 = glue(lduw, MEMSUFFIX)(A0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_ldsw, MEMSUFFIX), _T1_A0)(void)
 {
     T1 = glue(ldsw, MEMSUFFIX)(A0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_ldl, MEMSUFFIX), _T1_A0)(void)
 {
     T1 = glue(ldl, MEMSUFFIX)(A0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_stb, MEMSUFFIX), _T0_A0)(void)
@@ -92,6 +102,7 @@ void OPPROTO glue(glue(op_ldq, MEMSUFFIX
     uint64_t *p;
     p = (uint64_t *)((char *)env + PARAM1);
     *p = glue(ldq, MEMSUFFIX)(A0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_stq, MEMSUFFIX), _env_A0)(void)
@@ -108,6 +119,7 @@ void OPPROTO glue(glue(op_ldo, MEMSUFFIX
     p = (XMMReg *)((char *)env + PARAM1);
     p->XMM_Q(0) = glue(ldq, MEMSUFFIX)(A0);
     p->XMM_Q(1) = glue(ldq, MEMSUFFIX)(A0 + 8);
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_sto, MEMSUFFIX), _env_A0)(void)
@@ -123,21 +135,25 @@ void OPPROTO glue(glue(op_sto, MEMSUFFIX
 void OPPROTO glue(glue(op_ldsl, MEMSUFFIX), _T0_A0)(void)
 {
     T0 = (int32_t)glue(ldl, MEMSUFFIX)(A0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_ldsl, MEMSUFFIX), _T1_A0)(void)
 {
     T1 = (int32_t)glue(ldl, MEMSUFFIX)(A0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_ldq, MEMSUFFIX), _T0_A0)(void)
 {
     T0 = glue(ldq, MEMSUFFIX)(A0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_ldq, MEMSUFFIX), _T1_A0)(void)
 {
     T1 = glue(ldq, MEMSUFFIX)(A0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_stq, MEMSUFFIX), _T0_A0)(void)
Index: target-i386/ops_template_mem.h
===================================================================
RCS file: /cvsroot/qemu/qemu/target-i386/ops_template_mem.h,v
retrieving revision 1.5
diff -u -p -r1.5 ops_template_mem.h
--- target-i386/ops_template_mem.h	3 Mar 2005 01:14:55 -0000	1.5
+++ target-i386/ops_template_mem.h	9 May 2005 01:33:04 -0000
@@ -284,6 +284,7 @@ void OPPROTO glue(glue(op_shld, MEM_SUFF
 #endif
     CC_SRC = tmp;
     CC_DST = T0;
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_shld, MEM_SUFFIX), _T0_T1_ECX_cc)(void)
@@ -326,6 +327,7 @@ void OPPROTO glue(glue(op_shrd, MEM_SUFF
 #endif
     CC_SRC = tmp;
     CC_DST = T0;
+    FORCE_RET();
 }
 
 
@@ -369,6 +371,7 @@ void OPPROTO glue(glue(op_shld, MEM_SUFF
 #endif
     CC_SRC = tmp;
     CC_DST = T0;
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_shld, MEM_SUFFIX), _T0_T1_ECX_cc)(void)
@@ -407,6 +410,7 @@ void OPPROTO glue(glue(op_shrd, MEM_SUFF
 #endif
     CC_SRC = tmp;
     CC_DST = T0;
+    FORCE_RET();
 }
 
 
@@ -445,6 +449,7 @@ void OPPROTO glue(glue(op_adc, MEM_SUFFI
     CC_SRC = T1;
     CC_DST = T0;
     CC_OP = CC_OP_ADDB + SHIFT + cf * 4;
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_sbb, MEM_SUFFIX), _T0_T1_cc)(void)
@@ -458,6 +463,7 @@ void OPPROTO glue(glue(op_sbb, MEM_SUFFI
     CC_SRC = T1;
     CC_DST = T0;
     CC_OP = CC_OP_SUBB + SHIFT + cf * 4;
+    FORCE_RET();
 }
 
 void OPPROTO glue(glue(op_cmpxchg, MEM_SUFFIX), _T0_T1_EAX_cc)(void)
Index: target-ppc/exec.h
===================================================================
RCS file: /cvsroot/qemu/qemu/target-ppc/exec.h,v
retrieving revision 1.10
diff -u -p -r1.10 exec.h
--- target-ppc/exec.h	13 Mar 2005 17:01:22 -0000	1.10
+++ target-ppc/exec.h	9 May 2005 01:33:04 -0000
@@ -33,11 +33,7 @@ register uint32_t T2 asm(AREG3);
 #define FT1 (env->ft1)
 #define FT2 (env->ft2)
 
-#if defined (DEBUG_OP)
-#define RETURN() __asm__ __volatile__("nop");
-#else
-#define RETURN() __asm__ __volatile__("");
-#endif
+#define RETURN() FORCE_RET()
 
 #include "cpu.h"
 #include "exec-all.h"
Index: target-ppc/op.c
===================================================================
RCS file: /cvsroot/qemu/qemu/target-ppc/op.c,v
retrieving revision 1.16
diff -u -p -r1.16 op.c
--- target-ppc/op.c	13 Mar 2005 17:01:22 -0000	1.16
+++ target-ppc/op.c	9 May 2005 01:33:04 -0000
@@ -489,11 +489,13 @@ PPC_OP(test_ctr)
 PPC_OP(test_ctr_true)
 {
     T0 = (regs->ctr != 0 && (T0 & PARAM(1)) != 0);
+    FORCE_RET();
 }
 
 PPC_OP(test_ctr_false)
 {
     T0 = (regs->ctr != 0 && (T0 & PARAM(1)) == 0);
+    FORCE_RET();
 }
 
 PPC_OP(test_ctrz)
@@ -504,11 +506,13 @@ PPC_OP(test_ctrz)
 PPC_OP(test_ctrz_true)
 {
     T0 = (regs->ctr == 0 && (T0 & PARAM(1)) != 0);
+    FORCE_RET();
 }
 
 PPC_OP(test_ctrz_false)
 {
     T0 = (regs->ctr == 0 && (T0 & PARAM(1)) == 0);
+    FORCE_RET();
 }
 
 PPC_OP(test_true)
@@ -1335,9 +1339,10 @@ PPC_OP(fnabs)
 }
 
 /* fneg */
+void do_fneg (void);
 PPC_OP(fneg)
 {
-    FT0 = -FT0;
+    do_fneg();
     RETURN();
 }
 
Index: target-ppc/op_helper.c
===================================================================
RCS file: /cvsroot/qemu/qemu/target-ppc/op_helper.c,v
retrieving revision 1.12
diff -u -p -r1.12 op_helper.c
--- target-ppc/op_helper.c	13 Mar 2005 17:01:22 -0000	1.12
+++ target-ppc/op_helper.c	9 May 2005 01:33:04 -0000
@@ -428,6 +428,11 @@ void do_fnabs (void)
     FT0 = p.d;
 }
 
+void do_fneg (void)
+{
+    FT0 = -FT0;
+}
+
 /* Instruction cache invalidation helper */
 #define ICACHE_LINE_SIZE 32
 
Index: target-sparc/exec.h
===================================================================
RCS file: /cvsroot/qemu/qemu/target-sparc/exec.h,v
retrieving revision 1.10
diff -u -p -r1.10 exec.h
--- target-sparc/exec.h	13 Feb 2005 19:02:42 -0000	1.10
+++ target-sparc/exec.h	9 May 2005 01:33:04 -0000
@@ -34,6 +34,7 @@ void set_cwp(int new_cwp);
 void do_fitos(void);
 void do_fitod(void);
 void do_fabss(void);
+void do_fnegs(void);
 void do_fsqrts(void);
 void do_fsqrtd(void);
 void do_fcmps(void);
Index: target-sparc/op.c
===================================================================
RCS file: /cvsroot/qemu/qemu/target-sparc/op.c,v
retrieving revision 1.15
diff -u -p -r1.15 op.c
--- target-sparc/op.c	13 Mar 2005 09:55:49 -0000	1.15
+++ target-sparc/op.c	9 May 2005 01:33:04 -0000
@@ -871,7 +871,7 @@ void OPPROTO op_flush_T0(void)
 
 void OPPROTO op_fnegs(void)
 {
-    FT0 = -FT1;
+    do_fnegs();
 }
 
 void OPPROTO op_fabss(void)
Index: target-sparc/op_helper.c
===================================================================
RCS file: /cvsroot/qemu/qemu/target-sparc/op_helper.c,v
retrieving revision 1.12
diff -u -p -r1.12 op_helper.c
--- target-sparc/op_helper.c	6 Apr 2005 20:44:48 -0000	1.12
+++ target-sparc/op_helper.c	9 May 2005 01:33:04 -0000
@@ -25,6 +25,11 @@ void do_fabss(void)
     FT0 = float32_abs(FT1);
 }
 
+void do_fnegs(void)
+{
+    FT0 = float32_chs(FT1);
+}
+
 void do_fsqrts(void)
 {
     FT0 = float32_sqrt(FT1, &env->fp_status);
Index: target-sparc/op_mem.h
===================================================================
RCS file: /cvsroot/qemu/qemu/target-sparc/op_mem.h,v
retrieving revision 1.5
diff -u -p -r1.5 op_mem.h
--- target-sparc/op_mem.h	30 Jan 2005 22:39:04 -0000	1.5
+++ target-sparc/op_mem.h	9 May 2005 01:33:04 -0000
@@ -3,12 +3,14 @@
 void OPPROTO glue(glue(op_, name), MEMSUFFIX)(void)                           \
 {                                                                             \
     T1 = glue(qp, MEMSUFFIX)(T0);                                     \
+    FORCE_RET();                                                              \
 }
 
 #define SPARC_ST_OP(name, op)                                                 \
 void OPPROTO glue(glue(op_, name), MEMSUFFIX)(void)                           \
 {                                                                             \
     glue(op, MEMSUFFIX)(T0, T1);                                      \
+    FORCE_RET()                                                               \
 }
 
 SPARC_LD_OP(ld, ldl);
@@ -26,12 +28,14 @@ void OPPROTO glue(op_std, MEMSUFFIX)(voi
 {
     glue(stl, MEMSUFFIX)(T0, T1);
     glue(stl, MEMSUFFIX)((T0 + 4), T2);
+    FORCE_RET();
 }
 
 void OPPROTO glue(op_ldstub, MEMSUFFIX)(void)
 {
     T1 = glue(ldub, MEMSUFFIX)(T0);
     glue(stb, MEMSUFFIX)(T0, 0xff);     /* XXX: Should be Atomically */
+    FORCE_RET();
 }
 
 void OPPROTO glue(op_swap, MEMSUFFIX)(void)
@@ -39,33 +43,39 @@ void OPPROTO glue(op_swap, MEMSUFFIX)(vo
     target_ulong tmp = glue(ldl, MEMSUFFIX)(T0);
     glue(stl, MEMSUFFIX)(T0, T1);       /* XXX: Should be Atomically */
     T1 = tmp;
+    FORCE_RET();
 }
 
 void OPPROTO glue(op_ldd, MEMSUFFIX)(void)
 {
     T1 = glue(ldl, MEMSUFFIX)(T0);
     T0 = glue(ldl, MEMSUFFIX)((T0 + 4));
+    FORCE_RET();
 }
 
 /***                         Floating-point store                          ***/
 void OPPROTO glue(op_stf, MEMSUFFIX) (void)
 {
     glue(stfl, MEMSUFFIX)(T0, FT0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(op_stdf, MEMSUFFIX) (void)
 {
     glue(stfq, MEMSUFFIX)(T0, DT0);
+    FORCE_RET();
 }
 
 /***                         Floating-point load                           ***/
 void OPPROTO glue(op_ldf, MEMSUFFIX) (void)
 {
     FT0 = glue(ldfl, MEMSUFFIX)(T0);
+    FORCE_RET();
 }
 
 void OPPROTO glue(op_lddf, MEMSUFFIX) (void)
 {
     DT0 = glue(ldfq, MEMSUFFIX)(T0);
+    FORCE_RET();
 }
 #undef MEMSUFFIX

Re: [Qemu-devel] [patch] gcc4 host support

[Paul Brook]

[-- Attachment #1: Type: text/plain, Size: 165 bytes --]

On Wednesday 11 May 2005 22:04, Paul Brook wrote:
> The attached patch adds support for gcc4 x86 and x86_64 hosts.

This time with the correct patch attached.

Paul

[-- Attachment #2: patch.qemu_gcc4 --]
[-- Type: text/x-diff, Size: 29221 bytes --]

Index: dyngen-exec.h
===================================================================
RCS file: /cvsroot/qemu/qemu/dyngen-exec.h,v
retrieving revision 1.25
diff -u -p -r1.25 dyngen-exec.h
--- dyngen-exec.h	24 Apr 2005 18:01:56 -0000	1.25
+++ dyngen-exec.h	11 May 2005 20:38:33 -0000
@@ -155,7 +155,12 @@ extern int printf(const char *, ...);
 #endif
 
 /* force GCC to generate only one epilog at the end of the function */
+#if defined(__i386__) || defined(__x86_64__)
+/* Also add 4 bytes of padding so that we can replace the ret with a jmp.  */
+#define FORCE_RET() asm volatile ("nop;nop;nop;nop");
+#else
 #define FORCE_RET() asm volatile ("");
+#endif
 
 #ifndef OPPROTO
 #define OPPROTO
@@ -205,12 +210,19 @@ extern int __op_jmp0, __op_jmp1, __op_jm
 #endif
 
 #ifdef __i386__
-#define EXIT_TB() asm volatile ("ret")
-#define GOTO_LABEL_PARAM(n) asm volatile ("jmp " ASM_NAME(__op_gen_label) #n)
+/* Dyngen will replace hlt instructions with a ret instruction.  Inserting a
+   ret directly would confuse dyngen.  */
+#define EXIT_TB() asm volatile ("hlt")
+/* Dyngen will replace cli with 0x9e (jmp). 
+   We generate the offset manually.  */
+#define GOTO_LABEL_PARAM(n) \
+  asm volatile ("cli;.long " ASM_NAME(__op_gen_label) #n " - 1f;1:")
 #endif
 #ifdef __x86_64__
-#define EXIT_TB() asm volatile ("ret")
-#define GOTO_LABEL_PARAM(n) asm volatile ("jmp " ASM_NAME(__op_gen_label) #n)
+/* The same as i386.  */
+#define EXIT_TB() asm volatile ("hlt")
+#define GOTO_LABEL_PARAM(n) \
+  asm volatile ("cli;.long " ASM_NAME(__op_gen_label) #n " - 1f;1:")
 #endif
 #ifdef __powerpc__
 #define EXIT_TB() asm volatile ("blr")
Index: dyngen.c
===================================================================
RCS file: /cvsroot/qemu/qemu/dyngen.c,v
retrieving revision 1.40
diff -u -p -r1.40 dyngen.c
--- dyngen.c	27 Apr 2005 19:55:58 -0000	1.40
+++ dyngen.c	11 May 2005 20:38:33 -0000
@@ -32,6 +32,8 @@
 
 #include "config-host.h"
 
+//#define DEBUG_OP
+
 /* NOTE: we test CONFIG_WIN32 instead of _WIN32 to enabled cross
    compilation */
 #if defined(CONFIG_WIN32)
@@ -1343,6 +1345,639 @@ int arm_emit_ldr_info(const char *name, 
 #endif
 
 
+#if defined(HOST_I386) || defined(HOST_X86_64)
+
+/* This byte is the first byte of an instruction.  */
+#define FLAG_INSN     (1 << 0)
+/* This byte has been processed as part of an instruction.  */
+#define FLAG_SCANNED  (1 << 1)
+/* This instruction is a return instruction.  Gcc cometimes generates prefix
+   bytes, so may be more than one byte long.  */
+#define FLAG_RET      (1 << 2)
+/* This is either the target of a jump, or the preceeding instruction uses
+   a pc-relative offset.  */
+#define FLAG_TARGET   (1 << 3)
+/* This is a magic instruction that needs fixing up.  */
+#define FLAG_EXIT     (1 << 4)
+#define MAX_EXITS     5
+
+static void
+bad_opcode(const char *name, uint32_t op)
+{
+    error("Unsupported opcode %0*x in %s", (op > 0xff) ? 4 : 2, op, name);
+}
+
+/* Mark len bytes as scanned,  Returns insn_size + len.  Reports an error
+   if these bytes have already been scanned.  */
+static int
+eat_bytes(const char *name, char *flags, int insn, int insn_size, int len)
+{
+    while (len > 0) {
+        /* This should never occur in sane code.  */
+        if (flags[insn + insn_size] & FLAG_SCANNED)
+            error ("Overlapping instructions in %s", name);
+        flags[insn + insn_size] |= FLAG_SCANNED;
+        insn_size++;
+        len--;
+    }
+    return insn_size;
+}
+
+static void
+trace_i386_insn (const char *name, uint8_t *start_p, char *flags, int insn,
+                 int len)
+{
+    uint8_t *ptr;
+    uint8_t op;
+    int modrm;
+    int is_prefix;
+    int op_size;
+    int addr_size;
+    int insn_size;
+    int is_ret;
+    int is_condjmp;
+    int is_jmp;
+    int is_exit;
+    int is_pcrel;
+    int immed;
+    int seen_rexw;
+    int32_t disp;
+
+    ptr = start_p + insn;
+    /* nonzero if this insn has a ModR/M byte.  */
+    modrm = 1;
+    /* The size of the immediate value in this instruction.  */
+    immed = 0;
+    /* The operand size.  */
+    op_size = 4;
+    /* The address size */
+    addr_size = 4;
+    /* The total length of this instruction.  */
+    insn_size = 0;
+    is_prefix = 1;
+    is_ret = 0;
+    is_condjmp = 0;
+    is_jmp = 0;
+    is_exit = 0;
+    seen_rexw = 0;
+    is_pcrel = 0;
+
+    while (is_prefix) {
+        op = ptr[insn_size];
+        insn_size = eat_bytes(name, flags, insn, insn_size, 1);
+        is_prefix = 0;
+        switch (op >> 4) {
+        case 0:
+        case 1:
+        case 2:
+        case 3:
+            if (op == 0x0f) {
+                /* two-byte opcode.  */
+                op = ptr[insn_size];
+                insn_size = eat_bytes(name, flags, insn, insn_size, 1);
+                switch (op >> 4) {
+                case 0:
+                    if ((op & 0xf) > 3)
+                      modrm = 0;
+                    break;
+                case 1: /* vector move or prefetch */
+                case 2: /* various moves and vector compares.  */
+                case 4: /* cmov */
+                case 5: /* vector instructions */
+                case 6:
+                case 13:
+                case 14:
+                case 15:
+                    break;
+                case 7: /* mmx */
+                    if (op & 0x77) /* emms */
+                      modrm = 0;
+                    break;
+                case 3: /* wrmsr, rdtsc, rdmsr, rdpmc, sysenter, sysexit */
+                    modrm = 0;
+                    break;
+                case 8: /* long conditional jump */
+                    is_condjmp = 1;
+                    immed = op_size;
+                    modrm = 0;
+                    break;
+                case 9: /* setcc */
+                    break;
+                case 10:
+                    switch (op & 0x7) {
+                    case 0: /* push fs/gs */
+                    case 1: /* pop fs/gs */
+                    case 2: /* cpuid/rsm */
+                        modrm = 0;
+                        break;
+                    case 4: /* shld/shrd immediate */
+                        immed = 1;
+                        break;
+                    default: /* Normal instructions with a ModR/M byte.  */
+                        break;
+                    }
+                    break;
+                case 11:
+                    switch (op & 0xf) {
+                    case 10: /* bt, bts, btr, btc */
+                        immed = 1;
+                        break;
+                    default:
+                        /* cmpxchg, lss, btr, lfs, lgs, movzx, btc, bsf, bsr
+                           undefined, and movsx */
+                        break;
+                    }
+                    break;
+                case 12:
+                    if (op & 8) {
+                        /* bswap */
+                        modrm = 0;
+                    } else {
+                        switch (op & 0x7) {
+                        case 2:
+                        case 4:
+                        case 5:
+                        case 6:
+                            immed = 1;
+                            break;
+                        default:
+                            break;
+                        }
+                    }
+                    break;
+                }
+            } else if ((op & 0x07) <= 0x3) {
+                /* General arithmentic ax.  */
+            } else if ((op & 0x07) <= 0x5) {
+                /* General arithmetic ax, immediate.  */
+                if (op & 0x01)
+                    immed = op_size;
+                else
+                    immed = 1;
+                modrm = 0;
+            } else if ((op & 0x23) == 0x22) {
+                /* Segment prefix.  */
+                is_prefix = 1;
+            } else {
+                /* Segment register push/pop or DAA/AAA/DAS/AAS.  */
+                modrm = 0;
+            }
+            break;
+
+#if defined(HOST_X86_64)
+        case 4: /* rex prefix.  */
+            is_prefix = 1;
+            /* The address/operand size is actually 64-bit, but the immediate
+               values in the instruction are still 32-bit.  */
+            op_size = 4;
+            addr_size = 4;
+            if (op & 8)
+                seen_rexw = 1;
+            break;
+#else
+        case 4: /* inc/dec register.  */
+#endif
+        case 5: /* push/pop general register.  */
+            modrm = 0;
+            break;
+
+        case 6:
+            switch (op & 0x0f) {
+            case 0: /* pusha */
+            case 1: /* popa */
+                modrm = 0;
+                break;
+            case 2: /* bound */
+            case 3: /* arpl */
+                break;
+            case 4: /* FS */
+            case 5: /* GS */
+                is_prefix = 1;
+                break;
+            case 6: /* opcode size prefix.  */
+                op_size = 2;
+                is_prefix = 1;
+                break;
+            case 7: /* Address size prefix.  */
+                addr_size = 2;
+                is_prefix = 1;
+                break;
+            case 8: /* push immediate */
+            case 10: /* pop immediate */
+                immed = op_size;
+                modrm = 0;
+                break;
+            case 9: /* imul immediate */
+            case 11: /* imul immediate */
+                immed = op_size;
+                break;
+            case 12: /* insb */
+            case 13: /* insw */
+            case 14: /* outsb */
+            case 15: /* outsw */
+                modrm = 0;
+                break;
+            }
+            break;
+
+        case 7: /* Short conditional jump.  */
+            is_condjmp = 1;
+            immed = 1;
+            modrm = 0;
+            break;
+          
+        case 8:
+            if ((op & 0xf) <= 3) {
+                /* arithmetic immediate.  */
+                if ((op & 3) == 1)
+                    immed = op_size;
+                else
+                    immed = 1;
+            }
+            /* else test, xchg, mov, lea or pop general.  */
+            break;
+
+        case 9:
+            /* Various single-byte opcodes with no modrm byte.  */
+            modrm = 0;
+            if (op == 10) {
+                /* Call */
+                immed = 4;
+            }
+            break;
+
+        case 10:
+            switch ((op & 0xe) >> 1) {
+            case 0: /* mov absoliute immediate.  */
+            case 1:
+                if (seen_rexw)
+                    immed = 8;
+                else
+                    immed = addr_size;
+                break;
+            case 4: /* test immediate.  */
+                if (op & 1)
+                    immed = op_size;
+                else
+                    immed = 1;
+                break;
+            default: /* Various string ops.  */
+                break;
+            }
+            modrm = 0;
+            break;
+
+        case 11: /* move immediate to register */
+            if (op & 8) {
+                if (seen_rexw)
+                    immed = 8;
+                else
+                    immed = op_size;
+            } else {
+                immed = 1;
+            }
+            modrm = 0;
+            break;
+
+          case 12:
+            switch (op & 0xf) {
+            case 0: /* shift immediate */
+            case 1:
+                immed = 1;
+                break;
+            case 2: /* ret immediate */
+                immed = 2;
+                modrm = 0;
+                bad_opcode(name, op);
+                break;
+            case 3: /* ret */
+                modrm = 0;
+                is_ret = 1;
+            case 4: /* les */
+            case 5: /* lds */
+                break;
+            case 6: /* mov immediate byte */
+                immed = 1;
+                break;
+            case 7: /* mov immediate */
+                immed = op_size;
+                break;
+            case 8: /* enter */
+                /* TODO: Is this right?  */
+                immed = 3;
+                modrm = 0;
+                break;
+            case 10: /* retf immediate */
+                immed = 2;
+                modrm = 0;
+                bad_opcode(name, op);
+                break;
+            case 13: /* int */
+                immed = 1;
+                modrm = 0;
+                break;
+            case 11: /* retf */
+            case 15: /* iret */
+                modrm = 0;
+                bad_opcode(name, op);
+                break;
+            default: /* leave, int3 or into */
+                modrm = 0;
+                break;
+            }
+            break;
+
+        case 13:
+            if ((op & 0xf) >= 8) {
+                /* Coprocessor escape.  For our purposes this is just a normal
+                   instruction with a ModR/M byte.  */
+            } else if ((op & 0xf) >= 4) {
+                /* AAM, AAD or XLAT */
+                modrm = 0;
+            }
+            /* else shift instruction */
+            break;
+
+        case 14:
+            switch ((op & 0xc) >> 2) {
+            case 0: /* loop or jcxz */
+                is_condjmp = 1;
+                immed = 1;
+                break;
+            case 1: /* in/out immed */
+                immed = 1;
+                break;
+            case 2: /* call or jmp */
+                switch (op & 3) {
+                case 0: /* call */
+                    immed = op_size;
+                    break;
+                case 1: /* long jump */
+                    immed = 4;
+                    is_jmp = 1;
+                    break;
+                case 2: /* far jmp */
+                    bad_opcode(name, op);
+                    break;
+                case 3: /* short jmp */
+                    immed = 1;
+                    is_jmp = 1;
+                    break;
+                }
+                break;
+            case 3: /* in/out register */
+                break;
+            }
+            modrm = 0;
+            break;
+
+        case 15:
+            switch ((op & 0xe) >> 1) {
+            case 0:
+            case 1:
+                is_prefix = 1;
+                break;
+            case 2:
+            case 4:
+            case 5:
+            case 6:
+                modrm = 0;
+                /* Some privileged insns are used as markers.  */
+                switch (op) {
+                case 0xf4: /* hlt: Exit translation block.  */
+                    is_exit = 1;
+                    break;
+                case 0xfa: /* cli: Jump to label.  */
+                    is_exit = 1;
+                    immed = 4;
+                    break;
+                case 0xfb: /* sti: TB patch jump.  */
+                    /* Mark the insn for patching, but continue sscanning.  */
+                    flags[insn] |= FLAG_EXIT;
+                    immed = 4;
+                    break;
+                }
+                break;
+            case 3: /* unary grp3 */
+                if ((ptr[insn_size] & 0x38) == 0) {
+                    if (op == 0xf7)
+                        immed = op_size;
+                    else
+                        immed = 1; /* test immediate */
+                }
+                break;
+            case 7: /* inc/dec grp4/5 */
+                /* TODO: This includes indirect jumps.  We should fail if we
+                   encounter one of these. */
+                break;
+            }
+            break;
+        }
+    }
+
+    if (modrm) {
+        if (addr_size != 4)
+            error("16-bit addressing mode used in %s", name);
+
+        disp = 0;
+        modrm = ptr[insn_size];
+        insn_size = eat_bytes(name, flags, insn, insn_size, 1);
+        modrm &= 0xc7;
+        switch ((modrm & 0xc0) >> 6) {
+        case 0:
+            if (modrm == 5)
+              disp = 4;
+            break;
+        case 1:
+            disp = 1;
+            break;
+        case 2:
+            disp = 4;
+            break;
+        }
+        if ((modrm & 0xc0) != 0xc0 && (modrm & 0x7) == 4) {
+            /* SIB byte */
+            if (modrm == 4 && (ptr[insn_size] & 0x7) == 5) {
+                disp = 4;
+                is_pcrel = 1;
+            }
+            insn_size = eat_bytes(name, flags, insn, insn_size, 1);
+        }
+        insn_size = eat_bytes(name, flags, insn, insn_size, disp);
+    }
+    insn_size = eat_bytes(name, flags, insn, insn_size, immed);
+    if (is_condjmp || is_jmp) {
+        if (immed == 1) {
+            disp = (int8_t)*(ptr + insn_size - 1);
+        } else {
+            disp = (((int32_t)*(ptr + insn_size - 1)) << 24)
+                   | (((int32_t)*(ptr + insn_size - 2)) << 16)
+                   | (((int32_t)*(ptr + insn_size - 3)) << 8)
+                   | *(ptr + insn_size - 4);
+        }
+        disp += insn_size;
+        /* Jumps to external symbols point to the address of the offset
+           before relocation.  */
+        /* ??? These are probably a tailcall.  We could fix them up by
+           replacing them with jmp to EOB + call, but it's easier to just
+           prevent the compiler generating them.  */
+        if (disp == 1)
+            error("Unconditional jump (sibcall?) in %s", name);
+        disp += insn;
+        if (disp < 0 || disp > len)
+            error("Jump outside instruction in %s", name);
+
+        if ((flags[disp] & (FLAG_INSN | FLAG_SCANNED)) == FLAG_SCANNED)
+            error("Overlapping instructions in %s", name);
+
+        flags[disp] |= (FLAG_INSN | FLAG_TARGET);
+        is_pcrel = 1; 
+    }
+    if (is_pcrel) {
+        /* Mark the following insn as a jump target.  This will stop
+           this instruction being moved.  */
+        flags[insn + insn_size] |= FLAG_TARGET;
+    }
+    if (is_ret)
+      flags[insn] |= FLAG_RET;
+
+    if (is_exit)
+      flags[insn] |= FLAG_EXIT;
+
+    if (!(is_jmp || is_ret || is_exit))
+      flags[insn + insn_size] |= FLAG_INSN;
+}
+
+/* Scan a function body.  Returns the position of the return sequence.
+   Sets *patch_bytes to the number of bytes that need to be copied from that
+   location.  If no patching is required (ie. the return is the last insn)
+   *patch_bytes will be set to -1.  *plen is the number of code bytes to copy.
+ */
+static int trace_i386_op(const char * name, uint8_t *start_p, int *plen,
+                         int *patch_bytes, int *exit_addrs)
+{
+    char *flags;
+    int more;
+    int insn;
+    int retpos;
+    int bytes;
+    int num_exits;
+    int len;
+    int last_insn;
+
+    len = *plen;
+    flags = malloc(len + 1);
+    memset(flags, 0, len + 1);
+    flags[0] |= FLAG_INSN;
+    more = 1;
+    while (more) {
+        more = 0;
+        for (insn = 0; insn < len; insn++) {
+            if ((flags[insn] & (FLAG_INSN | FLAG_SCANNED)) == FLAG_INSN) {
+                trace_i386_insn(name, start_p, flags, insn, len);
+                more = 1;
+            }
+        }
+    }
+
+    /* Strip any unused code at the end of the function.  */
+    while (len > 0 && flags[len - 1] == 0)
+      len--;
+
+    retpos = -1;
+    num_exits = 0;
+    last_insn = 0;
+    for (insn = 0; insn < len; insn++) {
+        if (flags[insn] & FLAG_RET) {
+            /* ??? In theory it should be possible to handle multiple return
+               points.  In practice it's not worth the effort.  */
+            if (retpos != -1)
+                error("Multiple return instructions in %s", name);
+            retpos = insn;
+        }
+        if (flags[insn] & FLAG_EXIT) {
+            if (num_exits == MAX_EXITS)
+                error("Too many block exits in %s", name);
+            exit_addrs[num_exits] = insn;
+            num_exits++;
+        }
+        if (flags[insn] & FLAG_INSN)
+            last_insn = insn;
+    }
+
+    exit_addrs[num_exits] = -1;
+    if (retpos == -1) {
+        if (num_exits == 0) {
+            error ("No return instruction found in %s", name);
+        } else {
+            retpos = len;
+            last_insn = len;
+        }
+    }
+    
+    /* If the return instruction is the last instruction we can just 
+       remove it.  */
+    if (retpos == last_insn)
+        *patch_bytes = -1;
+    else
+        *patch_bytes = 0;
+
+    /* Back up over any nop instructions.  */
+    while (retpos > 0
+           && (flags[retpos] & FLAG_TARGET) == 0
+           && (flags[retpos - 1] & FLAG_INSN) != 0
+           && start_p[retpos - 1] == 0x90) {
+        retpos--;
+    }
+
+    if (*patch_bytes == -1) {
+        *plen = retpos;
+        free (flags);
+        return retpos;
+    }
+    *plen = len;
+
+    /* The ret is in the middle of the function.  Find four more bytes that
+       so the ret can be replaced by a jmp. */
+    /* ??? Use a short jump where possible. */
+    bytes = 4;
+    insn = retpos + 1;
+    /* We can clobber everything up to the next jump target.  */
+    while (insn < len && bytes > 0 && (flags[insn] & FLAG_TARGET) == 0) {
+        insn++;
+        bytes--;
+    }
+    if (bytes > 0) {
+        /* ???: Strip out nop blocks.  */
+        /* We can't do the replacement without clobbering anything important.
+           Copy preceeding instructions(s) to give us some space.  */
+        while (retpos > 0) {
+            /* If this byte is the target of a jmp we can't move it.  */
+            if (flags[retpos] & FLAG_TARGET)
+                break;
+
+            (*patch_bytes)++;
+            bytes--;
+            retpos--;
+
+            /* Break out of the loop if we have enough space and this is either 
+               the first byte of an instruction or a pad byte.  */
+            if ((flags[retpos] & (FLAG_INSN | FLAG_SCANNED)) != FLAG_SCANNED
+                && bytes <= 0) {
+                break;
+            }
+        }
+    }
+
+    if (bytes > 0)
+        error("Unable to replace ret with jmp in %s\n", name);
+
+    free(flags);
+    return retpos;
+}
+
+#endif
+
 #define MAX_ARGS 3
 
 /* generate op code */
@@ -1356,6 +1991,11 @@ void gen_code(const char *name, host_ulo
     uint8_t args_present[MAX_ARGS];
     const char *sym_name, *p;
     EXE_RELOC *rel;
+#if defined(HOST_I386) || defined(HOST_X86_64)
+    int patch_bytes;
+    int retpos;
+    int exit_addrs[MAX_EXITS];
+#endif
 
     /* Compute exact size excluding prologue and epilogue instructions.
      * Increment start_offset to skip epilogue instructions, then compute
@@ -1366,33 +2006,12 @@ void gen_code(const char *name, host_ulo
     p_end = p_start + size;
     start_offset = offset;
 #if defined(HOST_I386) || defined(HOST_X86_64)
-#ifdef CONFIG_FORMAT_COFF
-    {
-        uint8_t *p;
-        p = p_end - 1;
-        if (p == p_start)
-            error("empty code for %s", name);
-        while (*p != 0xc3) {
-            p--;
-            if (p <= p_start)
-                error("ret or jmp expected at the end of %s", name);
-        }
-        copy_size = p - p_start;
-    }
-#else
     {
         int len;
         len = p_end - p_start;
-        if (len == 0)
-            error("empty code for %s", name);
-        if (p_end[-1] == 0xc3) {
-            len--;
-        } else {
-            error("ret or jmp expected at the end of %s", name);
-        }
+        retpos = trace_i386_op(name, p_start, &len, &patch_bytes, exit_addrs);
         copy_size = len;
     }
-#endif    
 #elif defined(HOST_PPC)
     {
         uint8_t *p;
@@ -1559,6 +2178,13 @@ void gen_code(const char *name, host_ulo
     }
 
     if (gen_switch == 2) {
+#if defined(HOST_I386) || defined(HOST_X86_64)
+        if (patch_bytes != -1)
+            copy_size += patch_bytes;
+#ifdef DEBUG_OP
+        copy_size += 2;
+#endif
+#endif
         fprintf(outfile, "DEF(%s, %d, %d)\n", name + 3, nb_args, copy_size);
     } else if (gen_switch == 1) {
 
@@ -1761,7 +2387,43 @@ void gen_code(const char *name, host_ulo
 #error unsupport object format
 #endif
                 }
+		}
+                /* Replace the marker instructions with the actual opcodes.  */
+                for (i = 0; exit_addrs[i] != -1; i++) {
+                    int op;
+                    switch (p_start[exit_addrs[i]])
+                      {
+                      case 0xf4: op = 0xc3; break; /* hlt -> ret */
+                      case 0xfa: op = 0xe9; break; /* cli -> jmp */
+                      case 0xfb: op = 0xe9; break; /* sti -> jmp */
+                      default: error("Internal error");
+                      }
+                    fprintf(outfile, 
+                            "    *(uint8_t *)(gen_code_ptr + %d) = 0x%x;\n",
+                            exit_addrs[i], op);
+                }
+                /* Fix up the return instruction.  */
+                if (patch_bytes != -1) {
+                    if (patch_bytes) {
+                        fprintf(outfile, "    memcpy(gen_code_ptr + %d,"
+                                "gen_code_ptr + %d, %d);\n",
+                                copy_size, retpos, patch_bytes);
+                    }
+                    fprintf(outfile,
+                            "    *(uint8_t *)(gen_code_ptr + %d) = 0xe9;\n",
+                            retpos);
+                    fprintf(outfile,
+                            "    *(uint32_t *)(gen_code_ptr + %d) = 0x%x;\n",
+                            retpos + 1, copy_size - (retpos + 5));
+                    
+                    copy_size += patch_bytes;
                 }
+#ifdef DEBUG_OP
+                fprintf(outfile,
+                        "    *(uint16_t *)(gen_code_ptr + %d) = 0x9090;\n",
+                        copy_size);
+                copy_size += 2;
+#endif
             }
 #elif defined(HOST_X86_64)
             {
@@ -1793,6 +2455,42 @@ void gen_code(const char *name, host_ulo
                     }
                 }
                 }
+                /* Replace the marker instructions with the actual opcodes.  */
+                for (i = 0; exit_addrs[i] != -1; i++) {
+                    int op;
+                    switch (p_start[exit_addrs[i]])
+                      {
+                      case 0xf4: op = 0xc3; break; /* hlt -> ret */
+                      case 0xfa: op = 0xe9; break; /* cli -> jmp */
+                      case 0xfb: op = 0xe9; break; /* sti -> jmp */
+                      default: error("Internal error");
+                      }
+                    fprintf(outfile, 
+                            "    *(uint8_t *)(gen_code_ptr + %d) = 0x%x;\n",
+                            exit_addrs[i], op);
+                }
+                /* Fix up the return instruction.  */
+                if (patch_bytes != -1) {
+                    if (patch_bytes) {
+                        fprintf(outfile, "    memcpy(gen_code_ptr + %d,"
+                                "gen_code_ptr + %d, %d);\n",
+                                copy_size, retpos, patch_bytes);
+                    }
+                    fprintf(outfile,
+                            "    *(uint8_t *)(gen_code_ptr + %d) = 0xe9;\n",
+                            retpos);
+                    fprintf(outfile,
+                            "    *(uint32_t *)(gen_code_ptr + %d) = 0x%x;\n",
+                            retpos + 1, copy_size - (retpos + 5));
+                    
+                    copy_size += patch_bytes;
+                }
+#ifdef DEBUG_OP
+                fprintf(outfile,
+                        "    *(uint16_t *)(gen_code_ptr + %d) = 0x9090;\n",
+                        copy_size);
+                copy_size += 2;
+#endif
             }
 #elif defined(HOST_PPC)
             {
Index: exec-all.h
===================================================================
RCS file: /cvsroot/qemu/qemu/exec-all.h,v
retrieving revision 1.31
diff -u -p -r1.31 exec-all.h
--- exec-all.h	17 Apr 2005 18:32:14 -0000	1.31
+++ exec-all.h	11 May 2005 20:38:33 -0000
@@ -335,14 +335,15 @@ do {\
 
 #elif defined(__i386__) && defined(USE_DIRECT_JUMP)
 
-/* we patch the jump instruction directly */
+/* we patch the jump instruction directly.  Use sti in place of the actual
+   jmp instruction so that dyngen can patch in the correct result.  */
 #define GOTO_TB(opname, tbparam, n)\
 do {\
     asm volatile (".section .data\n"\
 		  ASM_NAME(__op_label) #n "." ASM_NAME(opname) ":\n"\
 		  ".long 1f\n"\
 		  ASM_PREVIOUS_SECTION \
-                  "jmp " ASM_NAME(__op_jmp) #n "\n"\
+                  "sti;.long " ASM_NAME(__op_jmp) #n " - 1f\n"\
 		  "1:\n");\
 } while (0)
 
Index: target-ppc/exec.h
===================================================================
RCS file: /cvsroot/qemu/qemu/target-ppc/exec.h,v
retrieving revision 1.10
diff -u -p -r1.10 exec.h
--- target-ppc/exec.h	13 Mar 2005 17:01:22 -0000	1.10
+++ target-ppc/exec.h	11 May 2005 20:38:35 -0000
@@ -33,11 +33,7 @@ register uint32_t T2 asm(AREG3);
 #define FT1 (env->ft1)
 #define FT2 (env->ft2)
 
-#if defined (DEBUG_OP)
-#define RETURN() __asm__ __volatile__("nop");
-#else
-#define RETURN() __asm__ __volatile__("");
-#endif
+#define RETURN() FORCE_RET()
 
 #include "cpu.h"
 #include "exec-all.h"

Re: [Qemu-devel] [patch] gcc4 host support

[Pascal Terjan]

On 5/12/05, Paul Brook <paul@codesourcery.com> wrote:
> On Wednesday 11 May 2005 22:04, Paul Brook wrote:
> > The attached patch adds support for gcc4 x86 and x86_64 hosts.
> 
> This time with the correct patch attached.

Hello, I can't build qemu under gcc4.0.0 with your patch and using -O2.
I get :
/home/pterjan/rpm/BUILD/qemu-0.7.0/target-i386/ops_sse.h: In function
'op_pshufw_mmx':
/home/pterjan/rpm/BUILD/qemu-0.7.0/target-i386/ops_sse.h:574: error:
unable to find a register to spill in class 'GENERAL_REGS'
/home/pterjan/rpm/BUILD/qemu-0.7.0/target-i386/ops_sse.h:574: error:
this is the insn:
(insn:HI 18 17 19 0
/home/pterjan/rpm/BUILD/qemu-0.7.0/target-i386/ops_sse.h:569 (set
(strict_low_part (subreg:HI (reg/v:DI 63 [ r ]) 0))
        (mem/s/j:HI (plus:SI (mult:SI (reg:SI 64)
                    (const_int 2 [0x2]))
                (reg/v/f:SI 59 [ s ])) [0 <variable>._w S2 A16])) 41
{*movstricthi_1} (insn_list:REG_DEP_TRUE 16 (insn_list:REG_DEP_TRUE 12
(insn_list:REG_DEP_TRUE 53 (nil))))
    (expr_list:REG_DEAD (reg:SI 64)
        (nil)))
/home/pterjan/rpm/BUILD/qemu-0.7.0/target-i386/ops_sse.h:574: confused
by earlier errors, bailing out

Re: [Qemu-devel] [patch] gcc4 host support

[Paul Brook]

On Thursday 12 May 2005 23:13, Pascal Terjan wrote:
> On 5/12/05, Paul Brook <paul@codesourcery.com> wrote:
> > On Wednesday 11 May 2005 22:04, Paul Brook wrote:
> > > The attached patch adds support for gcc4 x86 and x86_64 hosts.
> >
> > This time with the correct patch attached.
>
> Hello, I can't build qemu under gcc4.0.0 with your patch and using -O2.
> I get :
> /home/pterjan/rpm/BUILD/qemu-0.7.0/target-i386/ops_sse.h: In function
> 'op_pshufw_mmx':
> /home/pterjan/rpm/BUILD/qemu-0.7.0/target-i386/ops_sse.h:574: error:
> unable to find a register to spill in class 'GENERAL_REGS'
> /home/pterjan/rpm/BUILD/qemu-0.7.0/target-i386/ops_sse.h:574: error:
> this is the insn:
> (insn:HI 18 17 19 0
> /home/pterjan/rpm/BUILD/qemu-0.7.0/target-i386/ops_sse.h:569 (set
> (strict_low_part (subreg:HI (reg/v:DI 63 [ r ]) 0))
>         (mem/s/j:HI (plus:SI (mult:SI (reg:SI 64)
>                     (const_int 2 [0x2]))
>                 (reg/v/f:SI 59 [ s ])) [0 <variable>._w S2 A16])) 41
> {*movstricthi_1} (insn_list:REG_DEP_TRUE 16 (insn_list:REG_DEP_TRUE 12
> (insn_list:REG_DEP_TRUE 53 (nil))))
>     (expr_list:REG_DEAD (reg:SI 64)
>         (nil)))
> /home/pterjan/rpm/BUILD/qemu-0.7.0/target-i386/ops_sse.h:574: confused
> by earlier errors, bailing out

This is a gcc bug. See gcc.gnu.org/PR16185.
Basically gcc doesn't like doing 64-bit arithmetic on a target with only three 
32-bit registers (We use the other 4 registers to hold the guest CPU state).

You can hack around this by not holding the guest cpu state in registers, but 
this incurs a significant speed penalty.

Paul

Re: [Qemu-devel] [patch] gcc4 host support

[Filip Navara]

[-- Attachment #1: Type: text/plain, Size: 321 bytes --]

Paul Brook wrote:

>On Wednesday 11 May 2005 22:04, Paul Brook wrote:
>  
>
>>The attached patch adds support for gcc4 x86 and x86_64 hosts.
>>    
>>
>
>This time with the correct patch attached.
>
>Paul
>  
>
In addition to this patch I also needed the attached patch to get MinGW 
GCC 3.4.1 builds working...

- Filip

[-- Attachment #2: qemu-gcc4-mingw.diff --]
[-- Type: text/x-patch, Size: 1376 bytes --]

--- dyngen.c	Sat May 14 09:48:34 2005
+++ dyngen.c	Sat May 14 09:37:13 2005
@@ -2255,7 +2255,8 @@ void gen_code(const char *name, host_ulo
 
             for(i = 0, sym = symtab; i < nb_syms; i++, sym++) {
                 sym_name = get_sym_name(sym);
-                if (strstart(sym_name, "__op_label", &p)) {
+                if (strstart(sym_name, "__op_label", &p) ||
+                    strstart(sym_name, "_op_label", &p)) {
                     uint8_t *ptr;
                     unsigned long offset;
                     
--- exec-all.h	Sat May 14 09:48:34 2005
+++ exec-all.h	Sat May 14 09:37:04 2005
@@ -349,12 +349,20 @@ do {\
 
 #else
 
+#if __GNUC__ == 3 && __GNUC_MINOR__ < 4
+#define UNUSED __attribute__((unused))
+#elif defined(__GNUC__)
+#define UNUSED __attribute__((used))
+#else
+#define UNUSED
+#endif
+
 /* jump to next block operations (more portable code, does not need
    cache flushing, but slower because of indirect jump) */
 #define GOTO_TB(opname, tbparam, n)\
 do {\
-    static void __attribute__((unused)) *dummy ## n = &&dummy_label ## n;\
-    static void __attribute__((unused)) *__op_label ## n = &&label ## n;\
+    static void UNUSED *dummy ## n = &&dummy_label ## n;\
+    static void UNUSED *__op_label ## n = &&label ## n;\
     goto *(void *)(((TranslationBlock *)tbparam)->tb_next[n]);\
 label ## n: ;\
 dummy_label ## n: ;\

Re: [Qemu-devel] [patch] gcc4 host support

[Paul Brook]

> In addition to this patch I also needed the attached patch to get MinGW
> GCC 3.4.1 builds working...
>
>...
> -                if (strstart(sym_name, "__op_label", &p)) {
> +                if (strstart(sym_name, "__op_label", &p) ||
> +                    strstart(sym_name, "_op_label", &p)) {

I'm not sure this is the right fix. Did I miss an ASM_NAME macro somewhere?

Paul

Re: [Qemu-devel] [patch] gcc4 host support

[Filip Navara]

Paul Brook wrote:

>>In addition to this patch I also needed the attached patch to get MinGW
>>GCC 3.4.1 builds working...
>>
>>...
>>-                if (strstart(sym_name, "__op_label", &p)) {
>>+                if (strstart(sym_name, "__op_label", &p) ||
>>+                    strstart(sym_name, "_op_label", &p)) {
>>    
>>
>I'm not sure this is the right fix. Did I miss an ASM_NAME macro somewhere?
>  
>
It's not generated by assembler code, but from the C version of GOTO_TB 
macro. I suppose it's a bug in the particular version of compiler...

- Filip

Re: [Qemu-devel] [patch] gcc4 host support

[David Woodhouse]

On Wed, 2005-05-11 at 22:04 +0100, Paul Brook wrote:
> My solution is to search the function for the "ret" instruction and replace 
> them with a jmp to the next block of code. On RISC targets this would be 
> easy. 

About this easy, in fact...

--- qemu/dyngen.c.x	2005-05-16 10:30:43.000000000 +0100
+++ qemu/dyngen.c	2005-05-16 10:32:41.000000000 +0100
@@ -1996,6 +1996,9 @@ void gen_code(const char *name, host_ulo
     int retpos;
     int exit_addrs[MAX_EXITS];
 #endif
+#if defined(HOST_PPC)
+    uint8_t *blr_addr = NULL;
+#endif
 
     /* Compute exact size excluding prologue and epilogue instructions.
      * Increment start_offset to skip epilogue instructions, then compute
@@ -2018,9 +2021,23 @@ void gen_code(const char *name, host_ulo
         p = (void *)(p_end - 4);
         if (p == p_start)
             error("empty code for %s", name);
-        if (get32((uint32_t *)p) != 0x4e800020)
-            error("blr expected at the end of %s", name);
-        copy_size = p - p_start;
+        if (get32((uint32_t *)p) == 0x4e800020) {
+		copy_size = p - p_start; /* blr at end */
+	} else {
+	    /* Find the blr and note its address so that we 
+	       can emit code to rewrite it to a branch. */
+	    do {
+		p -= 4;
+
+		if (get32((uint32_t *)p) == 0x4e800020) {
+		    blr_addr = p;
+		    copy_size = p_end - p_start;
+		    break;
+		}
+	    } while (p > p_start);
+	    if (p == p_start)
+		error("blr expected in %s", name);
+	}
     }
 #elif defined(HOST_S390)
     {
@@ -2633,6 +2650,9 @@ void gen_code(const char *name, host_ulo
 #else
 #error unsupport object format
 #endif
+		if (blr_addr)
+		    fprintf(outfile, "    *(uint32_t *)(gen_code_ptr + %d) = 0x48000000 | %d;\n",
+			    blr_addr - p_start, p_end - blr_addr);
             }
 #elif defined(HOST_S390)
             {

-- 
dwmw2

Re: [Qemu-devel] [patch] gcc4 host support

[Paul Brook]

On Monday 16 May 2005 10:41, David Woodhouse wrote:
> On Wed, 2005-05-11 at 22:04 +0100, Paul Brook wrote:
> > My solution is to search the function for the "ret" instruction and
> > replace them with a jmp to the next block of code. On RISC targets this
> > would be easy.
>
> About this easy, in fact...
>...
> +
> +		if (get32((uint32_t *)p) == 0x4e800020) {
> +		    blr_addr = p;
> +		    copy_size = p_end - p_start;
> +		    break;
> +		}

You probably want to scan the whole function to check there aren't multiple 
blr instructions, and throw an error if there are.

Other than that it looks ok to me.

Paul

Re: [Qemu-devel] [patch] gcc4 host support

[Herbert Poetzl]

On Tue, May 17, 2005 at 09:46:30PM +0100, Paul Brook wrote:
> On Monday 16 May 2005 10:41, David Woodhouse wrote:
> > On Wed, 2005-05-11 at 22:04 +0100, Paul Brook wrote:
> > > My solution is to search the function for the "ret" instruction and
> > > replace them with a jmp to the next block of code. On RISC targets this
> > > would be easy.
> >
> > About this easy, in fact...
> >...
> > +
> > +		if (get32((uint32_t *)p) == 0x4e800020) {
> > +		    blr_addr = p;
> > +		    copy_size = p_end - p_start;
> > +		    break;
> > +		}
> 
> You probably want to scan the whole function to check there aren't multiple 
> blr instructions, and throw an error if there are.

hmm, wouldn't it be much easier to separate compiling
from assembling, and do the 'changes' on the assembler
files instead?

just an idea ...

best,
Herbert

> Other than that it looks ok to me.
> 
> Paul
> 
> 
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel

Re: [Qemu-devel] [patch] gcc4 host support

[Paul Brook]

On Wednesday 18 May 2005 11:06, Herbert Poetzl wrote:
> On Tue, May 17, 2005 at 09:46:30PM +0100, Paul Brook wrote:
> > On Monday 16 May 2005 10:41, David Woodhouse wrote:
> > > On Wed, 2005-05-11 at 22:04 +0100, Paul Brook wrote:
> > > > My solution is to search the function for the "ret" instruction and
> > > > replace them with a jmp to the next block of code. On RISC targets
> > > > this would be easy.
> > >
> > > About this easy, in fact...
> > >...
> > > +
> > > +		if (get32((uint32_t *)p) == 0x4e800020) {
> > > +		    blr_addr = p;
> > > +		    copy_size = p_end - p_start;
> > > +		    break;
> > > +		}
> >
> > You probably want to scan the whole function to check there aren't
> > multiple blr instructions, and throw an error if there are.
>
> hmm, wouldn't it be much easier to separate compiling
> from assembling, and do the 'changes' on the assembler
> files instead?

For x86 maybe. For ppc it's easier to just scan the binary code.
You'd have to write an assembly code parser.

Paul

Re: [Qemu-devel] [patch] gcc4 host support

[David Woodhouse]

On Wed, 2005-05-18 at 17:02 +0100, Paul Brook wrote:
> For x86 maybe. For ppc it's easier to just scan the binary code.
> You'd have to write an assembly code parser.

It wouldn't have to know much. I suspect you could do it in awk without
much trouble.

-- 
dwmw2

Re: [Qemu-devel] [patch] gcc4 host support

[John Hogerhuis]

This all feels wrong in general. The changes should either be adding
some facility to gcc which permits QEMU to use it in this way (gcc is
an open source project after all, so it's at least a possibility), or
we should generate the dynamic code generator once with whatever
version of GCC and make that part of the build, perhaps even letting
it diverge (i.e. add enhancements/optimizations directly to the
generated assembly).

-- John.

Re: [Qemu-devel] [patch] gcc4 host support

[Paul Brook]

On Wednesday 18 May 2005 20:29, John Hogerhuis wrote:
> This all feels wrong in general. The changes should either be adding
> some facility to gcc which permits QEMU to use it in this way (gcc is
> an open source project after all, so it's at least a possibility), 

Are you willing to write and maintain the gcc patches necessary to do this?

qemu is pretty much the only think that has this particular set of 
requirelents (the return instruction must be the last instruction in the 
function). The fact is qemu is abusing the output of gcc in ways that were 
never guaranteed to work.

> or we should generate the dynamic code generator once with whatever
> version of GCC and make that part of the build, perhaps even letting
> it diverge (i.e. add enhancements/optimizations directly to the
> generated assembly).

IMHO That's not really an option. The whole point of dyngen is to avoid having 
to write and maintain big chunks of assembly for every single target.

It's been said before that the long-term solution is to [incrementally] remove 
dyngen altogether, and replace it with a had-written code generator.
I've discussed this in a bit more detail with Fabrice, and have an 
almost-working prototype implementation. When I get something that actually 
works I'll post it to the list for comments.

Paul

Re: [Qemu-devel] [patch] gcc4 host support

[David Woodhouse]

On Wed, 2005-05-18 at 21:48 +0100, Paul Brook wrote:
> It's been said before that the long-term solution is to
> [incrementally] remove dyngen altogether, and replace it with a
> had-written code generator. I've discussed this in a bit more detail
> with Fabrice, and have an almost-working prototype implementation.
> When I get something that actually works I'll post it to the list for
> comments.

How feasible would it be to do this as an alternative front end to GCJ's
JIT?

-- 
dwmw2

Re: [Qemu-devel] [patch] gcc4 host support

[Paul Brook]

On Wednesday 18 May 2005 21:55, David Woodhouse wrote:
> On Wed, 2005-05-18 at 21:48 +0100, Paul Brook wrote:
> > It's been said before that the long-term solution is to
> > [incrementally] remove dyngen altogether, and replace it with a
> > had-written code generator. I've discussed this in a bit more detail
> > with Fabrice, and have an almost-working prototype implementation.
> > When I get something that actually works I'll post it to the list for
> > comments.
>
> How feasible would it be to do this as an alternative front end to GCJ's
> JIT?

GCJ doesn't really have a JIT. It has an interpreter, and the ability to 
compile classes with gcj. I'd expect anything using gcc at runtime would be 
too slow to be worthwhile.

Paul

Re: [Qemu-devel] [patch] gcc4 host support

[jeebs]

From: "David Woodhouse"

> On Wed, 2005-05-18 at 21:48 +0100, Paul Brook wrote:
>> It's been said before that the long-term solution is to
>> [incrementally] remove dyngen altogether, and replace it with a
>> had-written code generator. I've discussed this in a bit more detail
>
> How feasible would it be to do this as an alternative front end to GCJ's
> JIT?

I am definetly out of my depth here, but... Speaking of GCJ's JIT...

Sometime back, Ian Rogers here brought up the PearColator project at: 
http://www.binarytranslator.org/

***
I have been working on an open source Java based PowerPC emulator based
around a JVM's optimising compiler. If you have long running server like
workloads then I have found the performance is approaching QEMU fast
whilst having memory supported by a page based system. However, the
system is a lot less sophisticated - booting operating systems and being
a generic emulator is a distant reality. I have created a website at
http://www.binarytranslator.org/ or
http://www.cs.man.ac.uk/apt/projects/jamaica/tools/PearColator/ . I'm
sure some people would be interested in this and may feel like they want
to contribute. I'm happy to oblige and to share with QEMU. Thanks,
***

Looks like he's getting at least some tolerable numbers...

As I said above, I'm out of my depth here.  I just thought it was worth 
bringing this up in case people had forgotten about it.

Re: [Qemu-devel] [patch] gcc4 host support

[Paul Brook]

> I am definetly out of my depth here, but... Speaking of GCJ's JIT...
>
> Sometime back, Ian Rogers here brought up the PearColator project at:
> http://www.binarytranslator.org/
>
> ***
> I have been working on an open source Java based PowerPC emulator based
> around a JVM's optimising compiler. If you have long running server like
> workloads then I have found the performance is approaching QEMU fast
> whilst having memory supported by a page based system. However, the
> system is a lot less sophisticated - booting operating systems and being
> a generic emulator is a distant reality. I have created a website at
> http://www.binarytranslator.org/ or
> http://www.cs.man.ac.uk/apt/projects/jamaica/tools/PearColator/ . I'm
> sure some people would be interested in this and may feel like they want
> to contribute. I'm happy to oblige and to share with QEMU. Thanks,
> ***
>
> Looks like he's getting at least some tolerable numbers...

I know I'd seen something like this before, thanks for reminding me.

There are several issues with PearColator/RVM:

- It's written in java. qemu is written in C, so a lot of porting would be 
required to get anything working.
- The best benchmark results are half the speed of qemu, and ten times slower 
appears to be a more typical result.
- I can't see an any way of doing an incremental transition. My code generator 
coexists with dyngen, allowing a gentle migration away from dyngen.

Paul

Re: [Qemu-devel] [patch] gcc4 host support

[Ian Rogers]

jeebs@yango.us wrote:

>I am definetly out of my depth here, but... Speaking of GCJ's JIT...
>
>Sometime back, Ian Rogers here brought up the PearColator project at: 
>http://www.binarytranslator.org/
>  
>
We're still here and hacking. PearColator and QEMU are quite different 
projects and we rely on the Jikes RVM code generator - however, that is 
quite advanced. We can run on other JVMs using reflection, but the 
performance is so slow to not make that attractive. QEMU is far more 
developed than we are in terms of running things. I appologize for the 
now broken links, our main project page is:
http://www.cs.manchester.ac.uk/apt/projects/jamaica/
We're under the CPL as is the Jikes RVM and we're following their 
typical development cycle of work, publication and then source release. 
Please contact us for more information.

Regards,

Ian Rogers

Re: [Qemu-devel] [patch] gcc4 host support

[Ian Rogers]

Paul Brook wrote:

>I know I'd seen something like this before, thanks for reminding me.
>
>There are several issues with PearColator/RVM:
>
>- It's written in java. qemu is written in C, so a lot of porting would be 
>required to get anything working.
>- The best benchmark results are half the speed of qemu, and ten times slower 
>appears to be a more typical result.
>- I can't see an any way of doing an incremental transition. My code generator 
>coexists with dyngen, allowing a gentle migration away from dyngen.
>  
>
The currently published benchmark results are for PearColator with just 
an optimizing compiler (worst case in Jikes RVM is over 100 compiler 
stages). We also emulate a TLB in much the same way as PearPC, so 
comparing us to QEMU fast isn't fair (we generate in the region of 10 
instructions for loads and stores whereas QEMU fast can generate 1). Our 
best results are for things that sit in dynamic code for a long period 
of time, which isn't too surprising. We are working on equivalent 
results to QEMU fast, but our approach is different to that of QEMU and 
I completely agree that trying to get QEMU/PearColator to co-exist 
wouldn't be that great.

Regards,

Ian Rogers

Re: [Qemu-devel] [patch] gcc4 host support

[Gwenole Beauchesne]

Le mercredi, 18 mai 2005, à 22:48 Europe/Paris, Paul Brook a écrit :

> It's been said before that the long-term solution is to 
> [incrementally] remove
> dyngen altogether, and replace it with a had-written code generator.
> I've discussed this in a bit more detail with Fabrice, and have an
> almost-working prototype implementation. When I get something that 
> actually
> works I'll post it to the list for comments.

Have you considered the VEX library? I have not tried it yet but it 
looks promising. However, since it aims at providing a common IR, it 
can miss certain optimizations related to condition codes (at least as 
ppc guest).

BTW, since dyngen-based JIT is fast enough both at compile time and at 
run time, it could be used to gather stats for a higher optimizing JIT 
(VEX or whatever). e.g. profiling branches in order to optimize hot 
traces, providing hints for indirect branches, etc.

Re: [Qemu-devel] [patch] gcc4 host support

[Paul Brook]

On Thursday 19 May 2005 08:23, Gwenole Beauchesne wrote:
> Le mercredi, 18 mai 2005, à 22:48 Europe/Paris, Paul Brook a écrit :
> > It's been said before that the long-term solution is to
> > [incrementally] remove
> > dyngen altogether, and replace it with a had-written code generator.
> > I've discussed this in a bit more detail with Fabrice, and have an
> > almost-working prototype implementation. When I get something that
> > actually
> > works I'll post it to the list for comments.
>
> Have you considered the VEX library? I have not tried it yet but it
> looks promising. However, since it aims at providing a common IR, it
> can miss certain optimizations related to condition codes (at least as
> ppc guest).

Do you have a URL? Neither google nor freshmeat.net turn up anything useful.

> BTW, since dyngen-based JIT is fast enough both at compile time and at
> run time, it could be used to gather stats for a higher optimizing JIT
> (VEX or whatever). e.g. profiling branches in order to optimize hot
> traces, providing hints for indirect branches, etc.

One of the problems with dyngen is it is very fragile, see all the problems 
trying to make it work with gcc4. My initial goal with a JIT is to replace 
dyngen.

The big advantage of dyngen is that it is very specialised to qemu. Even 
though it's about the dumbest JIT implementation you can think of, it gets 
remarkably good results. Compare it with the Jikes RVM based ppc emulator 
mentioned recently. The Jikes RVM has ~100 optimizations passes compared to 
dyngen which has 2 or 3. However the end result is still no better than 
dyngen because most of that optimization effort is used to remove abstraction 
that dygen doesn't introduce in the first place.

It would be nice if we could use some sort of portable JIT library, however I 
think in reality a few qemu specific hacks(most of which we already use with 
dyngen) and a relatively dumb JIT are going to perform better.

Paul

Re: [Qemu-devel] [patch] gcc4 host support

[McMullan, Jason]

[-- Attachment #1: Type: text/plain, Size: 500 bytes --]

On Thu, 2005-05-19 at 09:23 +0200, Gwenole Beauchesne wrote:
> Have you considered the VEX library? I have not tried it yet but it 
> looks promising. However, since it aims at providing a common IR, it 
> can miss certain optimizations related to condition codes (at least as 
> ppc guest).

'VEX' is a poorly googlable term:

http://valgrind.org/devel/cvs_svn.html

svn co svn://svn.valgrind.org/vex/trunk vex


-- 
Jason McMullan <jason.mcmullan@timesys.com>
TimeSys Corporation


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [Qemu-devel] [patch] gcc4 host support

[Gwenole Beauchesne]

On Thu, 19 May 2005, Paul Brook wrote:

> Do you have a URL? Neither google nor freshmeat.net turn up anything useful.

Sorry, here it is:
<http://svn.valgrind.org/vex/trunk/> (used for valgrind 3.0)
<http://www.valgrind.org/devel/cvs_svn.html>

> It would be nice if we could use some sort of portable JIT library, however I 
> think in reality a few qemu specific hacks(most of which we already use with 
> dyngen) and a relatively dumb JIT are going to perform better.

What about using GNU lightning as an intermediate solution? It has a 
RISC-style ISA with up to 6 registers exposed. It's "dumb" (no 
optimization) but it's good at generating code. I think it matches most 
the "synthetic instruction" approach of dyngen.

Re: [Qemu-devel] [patch] gcc4 host support

[Paul Brook]

On Thursday 19 May 2005 15:07, Gwenole Beauchesne wrote:
> On Thu, 19 May 2005, Paul Brook wrote:
> > Do you have a URL? Neither google nor freshmeat.net turn up anything
> > useful.
>
> Sorry, here it is:
> <http://svn.valgrind.org/vex/trunk/> (used for valgrind 3.0)
> <http://www.valgrind.org/devel/cvs_svn.html>

This looks interesting, the problem is how to get it to cooperate with the 
dyngen code. We hold some guest CPU state (or at least intermediate state) in 
host registers.

I guess it depends wether you're willing to throw out dyngen and implement 
everything in one go.  I'm currently trying an incremental approach (as 
suggested by Fabrice), in which case I think a custom JIT is pretty much the 
only option.

> > It would be nice if we could use some sort of portable JIT library,
> > however I think in reality a few qemu specific hacks(most of which we
> > already use with dyngen) and a relatively dumb JIT are going to perform
> > better.
>
> What about using GNU lightning as an intermediate solution? It has a
> RISC-style ISA with up to 6 registers exposed. It's "dumb" (no
> optimization) but it's good at generating code. I think it matches most
> the "synthetic instruction" approach of dyngen.

I don't think GNU lightning really gains us much. We'd still have to do the 
hard bits (optimization, register allocation, assigning stack slots, etc). 
The actual native code generation (which is what lightning gives you) is only 
a few hundred lines of code.

Paul

Re: [Qemu-devel] [patch] gcc4 host support

[Ian Rogers]

You may also want to look at Dawson Engler's VCODE. Like GNU Lightning 
it's not been touched in a while.

Ian

Gwenole Beauchesne wrote:

>On Thu, 19 May 2005, Paul Brook wrote:
>
>  
>
>>Do you have a URL? Neither google nor freshmeat.net turn up anything useful.
>>    
>>
>
>Sorry, here it is:
><http://svn.valgrind.org/vex/trunk/> (used for valgrind 3.0)
><http://www.valgrind.org/devel/cvs_svn.html>
>
>  
>
>>It would be nice if we could use some sort of portable JIT library, however I 
>>think in reality a few qemu specific hacks(most of which we already use with 
>>dyngen) and a relatively dumb JIT are going to perform better.
>>    
>>
>
>What about using GNU lightning as an intermediate solution? It has a 
>RISC-style ISA with up to 6 registers exposed. It's "dumb" (no 
>optimization) but it's good at generating code. I think it matches most 
>the "synthetic instruction" approach of dyngen.
>
>
>_______________________________________________
>Qemu-devel mailing list
>Qemu-devel@nongnu.org
>http://lists.nongnu.org/mailman/listinfo/qemu-devel
>  
>

Re: [Qemu-devel] [patch] gcc4 host support

[Thomas Steffen]

On 5/19/05, Paul Brook <paul@codesourcery.com> wrote:

> I don't think GNU lightning really gains us much. We'd still have to do the
> hard bits (optimization, register allocation, assigning stack slots, etc).
> The actual native code generation (which is what lightning gives you) is only
> a few hundred lines of code.

I have the feeling that will apply to most alternatives. The problem
is to turn the C code into machine code, right? That is a difficult
problem, which is why a C compiler is a complex piece of software.
Using a different tool is not going to turn it into a simple problem.

So I think gcc is still the best bet. If the issue with the return
point is the only problem, there should be several possible solution
(with different performance impact).

Thomas

Re: [Qemu-devel] [patch] gcc4 host support

[Paul Brook]

On Thursday 19 May 2005 19:14, Thomas Steffen wrote:
> On 5/19/05, Paul Brook <paul@codesourcery.com> wrote:
> > I don't think GNU lightning really gains us much. We'd still have to do
> > the hard bits (optimization, register allocation, assigning stack slots,
> > etc). The actual native code generation (which is what lightning gives
> > you) is only a few hundred lines of code.
>
> I have the feeling that will apply to most alternatives. The problem
> is to turn the C code into machine code, right? That is a difficult
> problem, which is why a C compiler is a complex piece of software.
> Using a different tool is not going to turn it into a simple problem.

No. The problem is to turn machine code into (a different form of) machine 
code. A lot of the complexity in a compiler is involved with with turning the 
high-level language constructs into simple low-level machine operations.

With qemu we're just translating from one simple form to another, so I'd argue 
that all you really need is a clever way of papering over the differences 
between the host and the guest.

What we have now (dyngen) is basically just an assembler. It maps qemu micro 
ops directly into blocks host code. The only reason dyngen uses gcc is to 
avoid having to hand write host encodings for all the ops. 

Paul

Re: [Qemu-devel] [patch] gcc4 host support

[Tim Walker]

[-- Attachment #1: Type: text/plain, Size: 1607 bytes --]

Paul Brook wrote:

>On Thursday 19 May 2005 19:14, Thomas Steffen wrote:
>  
>
>>On 5/19/05, Paul Brook <paul@codesourcery.com> wrote:
>>    
>>
>>>I don't think GNU lightning really gains us much. We'd still have to do
>>>the hard bits (optimization, register allocation, assigning stack slots,
>>>etc). The actual native code generation (which is what lightning gives
>>>you) is only a few hundred lines of code.
>>>      
>>>
>>I have the feeling that will apply to most alternatives. The problem
>>is to turn the C code into machine code, right? That is a difficult
>>problem, which is why a C compiler is a complex piece of software.
>>Using a different tool is not going to turn it into a simple problem.
>>    
>>
>
>No. The problem is to turn machine code into (a different form of) machine 
>code. A lot of the complexity in a compiler is involved with with turning the 
>high-level language constructs into simple low-level machine operations.
>
>With qemu we're just translating from one simple form to another, so I'd argue 
>that all you really need is a clever way of papering over the differences 
>between the host and the guest.
>
>What we have now (dyngen) is basically just an assembler. It maps qemu micro 
>ops directly into blocks host code. The only reason dyngen uses gcc is to 
>avoid having to hand write host encodings for all the ops. 
>
>Paul
>
>
>_______________________________________________
>Qemu-devel mailing list
>Qemu-devel@nongnu.org
>http://lists.nongnu.org/mailman/listinfo/qemu-devel
>
>  
>
Pardon me for butting in, is any of the host encoding hand written?

Tim

[-- Attachment #2: Type: text/html, Size: 2319 bytes --]

Re: [Qemu-devel] [patch] gcc4 host support

[Paul Brook]

On Thursday 19 May 2005 20:38, Tim Walker wrote:
> Pardon me for butting in, is any of the host encoding hand written?

There is some x86 assembly for the tlb lookup, but everything else is written 
in C.

Paul

Re: [Qemu-devel] [patch] gcc4 host support

[Thomas Steffen]

On 5/19/05, Paul Brook <paul@codesourcery.com> wrote:
> No. The problem is to turn machine code into (a different form of) machine
> code. A lot of the complexity in a compiler is involved with with turning the
> high-level language constructs into simple low-level machine operations.

I see your point. I did write a Z80 emulator on an early x86 once. The
flags where extremely close, and most commands have a direct
correspondency. You just have to decide on a register mapping, and you
can start. I wrote short assembler sequences for each command, very
much like the targets in qemu. But this is a special case: mapping one
architecture on a similar architecture.

Qemu is special an that it avoid both the problem in "papering over
the differences", and it avoids the combinatorial explosion of n
targets on m hosts. And it does this exactly because it uses C to
express machine commands, and not some other machine language. I think
you cannot take this away without changing the very nature of qemu.

The reason I care about this is that qemu has achived a lot more than
all other similar open source projects together. Look at bochs, or
plex86 or valgrind: they are nowhere near the performance of qemu, and
they only support x86 targets. So there must be something very
ingenious about the design of qemu, and I think it is the combination
of gcc and dyngen.

I certainly welcome every possible improvement, but I want to stress
how good qemu alread is.

> With qemu we're just translating from one simple form to another, so I'd argue
> that all you really need is a clever way of papering over the differences
> between the host and the guest.

So many projects have failed in this direction that I am tempted to
assume that this is a flawed approach. Apart from kqemu and VMware,
there is not one convincing solution even for the supposedly trivial
x86 on x86 case.

> What we have now (dyngen) is basically just an assembler. It maps qemu micro
> ops directly into blocks host code. The only reason dyngen uses gcc is to
> avoid having to hand write host encodings for all the ops.

It as also because C avoids the n by m problem. 

Thomas

Re: [Qemu-devel] [patch] gcc4 host support

[John Hogerhuis]

On 5/19/05, Thomas Steffen <steffen.list.account@gmail.com> wrote:

> It as also because C avoids the n by m problem.
> 

My point is that you can have your cake and eat it too.

For any given architecture gcc can generate the first approximation by
way of dyngen or dyngen + gcc enhancements.

But once you are past that point gcc for a given arch, gcc isn't
really going to help you any more. So bring on the hand
coding/optimizations. There's no doubt the hand coded stuff would be
at least as good as gcc output. And that's because you can always make
use of what gcc did if you can't think of anything better.

There's really nothing magic going on we're in any danger of losing.
QEMU generates a dynamic code generator dynamically. Clever but lucky
hack, in that it really has no business working (yet it does, but the
lucky part is why it is going to continue to break).

Short of collusion from the gcc folks, I'm saying that it is no less
than the Halting Problem that is staring us in the face here. But no
more than that either. Practically speaking, we can probably go on
like this indefinitely, it just seems unnecessary to me.

The thing that separates QEMU from other projects like Bochs in my
mind is that it dynamically generates code and it has other clever
bits like samba and slirp integration. A high level view is that QEMU
puts a high priority on usability compared to these other projects.

Re: [Qemu-devel] [patch] gcc4 host support

[Thomas Steffen]

On 5/20/05, John Hogerhuis <jhoger@gmail.com> wrote:

> But once you are past that point gcc for a given arch, gcc isn't
> really going to help you any more. So bring on the hand
> coding/optimizations. 

And for popular combinations of architectures, such as x86 or PPC on
each other, I absolute agree. As in any other routine, you can
probably squeeze out a few cycles if you try hard enough.

> And that's because you can always make
> use of what gcc did if you can't think of anything better.

So the magic stays with us :-). You need some more complexity in the
build process, but I agree that it is a logical improvement.

Thomas

Re: [Qemu-devel] [patch] gcc4 host support

[Paul Brook]

On Thursday 19 May 2005 23:25, John Hogerhuis wrote:
> But once you are past that point gcc for a given arch, gcc isn't
> really going to help you any more. So bring on the hand
> coding/optimizations. There's no doubt the hand coded stuff would be
> at least as good as gcc output. And that's because you can always make
> use of what gcc did if you can't think of anything better.

While I agree with most of what you say, you seem to have misunderstood what I 
mean by a hand written code generator. I mean teaching qemu how to generate 
host code directly, not just rewriting bits of op.c in assembly.

Paul

[Qemu-devel] Fedora 4 + GCC4 + Qemu WAS: [patch] gcc4 host support

[Darryl Dixon]

Hi,

    Fedora Core 4 with gcc4 seems to have some trouble compiling qemu,
even with the below patch attached.  EG:
[...snip many signedness errors...]

gcc -Wall -O2 -g -fno-strict-aliasing -fomit-frame-pointer
-mpreferred-stack-boundary=2 -falign-functions=0 -fno-gcse
-fno-reorder-blocks -fno-optimize-sibling-calls -I.
-I/opt/src/qemu-snapshot-2005-06-15_23/target-i386
-I/opt/src/qemu-snapshot-2005-06-15_23 -D_GNU_SOURCE
-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
-I/opt/src/qemu-snapshot-2005-06-15_23/fpu
-I/opt/src/qemu-snapshot-2005-06-15_23/slirp -c -o
op.o /opt/src/qemu-snapshot-2005-06-15_23/target-i386/op.c
/opt/src/qemu-snapshot-2005-06-15_23/target-i386/ops_template_mem.h: In
function ‘op_cmpxchgb_kernel_T0_T1_EAX_cc’:
/opt/src/qemu-snapshot-2005-06-15_23/softmmu_header.h:184: error: can't
find a register in class ‘GENERAL_REGS’ while reloading ‘asm’
make[1]: *** [op.o] Error 1
make[1]: Leaving directory
`/opt/src/qemu-snapshot-2005-06-15_23/i386-softmmu'
make: *** [all] Error 1

[dixond@unixadmindazfc2 qemu-snapshot-2005-06-15_23]$ gcc --version
gcc (GCC) 4.0.0 20050519 (Red Hat 4.0.0-8)

Cheers,
D



On Thu, 2005-05-12 at 18:00 +0100, Paul Brook wrote:
> On Wednesday 11 May 2005 22:04, Paul Brook wrote:
> > The attached patch adds support for gcc4 x86 and x86_64 hosts.
> 
> This time with the correct patch attached.
> 
> Paul
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel
-- 
Darryl Dixon <esrever_otua@pythonhacker.is-a-geek.net>

Re: [Qemu-devel] Fedora 4 + GCC4 + Qemu WAS: [patch] gcc4 host support

[Paul Brook]

On Friday 17 June 2005 05:30, Darryl Dixon wrote:
> Hi,
>
>     Fedora Core 4 with gcc4 seems to have some trouble compiling qemu,
> even with the below patch attached.  EG:
> [...snip many signedness errors...]

Please read the FAQ. http://www.qemu.org/faq.html

"QEMU does not compile. Why ?

It is likely that you are using GCC 4.x: it is currently not supported by 
QEMU. You must use GCC 3.x."

Paul

Re: [Qemu-devel] Fedora 4 + GCC4 + Qemu WAS: [patch] gcc4 host support

[Paul Brook]

On Friday 17 June 2005 13:58, Darryl Dixon wrote:
> Oh,
>
>     Sorry, obviously I badly misunderstood the nature of the patch that
> you were posting; I assumed it was to enable Qemu to be compiled with
> GCC4.  That it can't is a pretty big downer; it basically means Qemu is
> unusable on FC4 :(  Ouch.  C'est la vie :)

Ah, I see what you mean. I did say in my original post
"On x86 I've had to hack round other bugs..."
Specifically http://gcc.gnu.org/PR16185

Those hacks are way too ugly for me to want to make them public.

You can get an older version of gcc (3.2.3) on FC4 by installing the 
compat-gcc packages.

Paul

Re: [Qemu-devel] Fedora 4 + GCC4 + Qemu WAS: [patch] gcc4 host support

[David Woodhouse]

On Fri, 2005-06-17 at 14:02 +0100, Paul Brook wrote:
> On Friday 17 June 2005 13:58, Darryl Dixon wrote:
> >     Sorry, obviously I badly misunderstood the nature of the patch that
> > you were posting; I assumed it was to enable Qemu to be compiled with
> > GCC4.  That it can't is a pretty big downer; it basically means Qemu is
> > unusable on FC4 :(  Ouch.  C'est la vie :)

Works fine on FC4 PPC :)

> Ah, I see what you mean. I did say in my original post
> "On x86 I've had to hack round other bugs..."
> Specifically http://gcc.gnu.org/PR16185

Is that also in Red Hat bugzilla?

-- 
dwmw2

Re: [Qemu-devel] Fedora 4 + GCC4 + Qemu WAS: [patch] gcc4 host support

[Darryl Dixon]

Aha!  Ok, thanks for that.  I have compiled Qemu OK now on FC4 with
gcc3.2.3 and compiled the accelerator module with gcc4.0 OK.  The two
seem to work together fine so all is good :)

D


On Fri, 2005-06-17 at 14:02 +0100, Paul Brook wrote:
> On Friday 17 June 2005 13:58, Darryl Dixon wrote:
> > Oh,
> >
> >     Sorry, obviously I badly misunderstood the nature of the patch that
> > you were posting; I assumed it was to enable Qemu to be compiled with
> > GCC4.  That it can't is a pretty big downer; it basically means Qemu is
> > unusable on FC4 :(  Ouch.  C'est la vie :)
> 
> Ah, I see what you mean. I did say in my original post
> "On x86 I've had to hack round other bugs..."
> Specifically http://gcc.gnu.org/PR16185
> 
> Those hacks are way too ugly for me to want to make them public.
> 
> You can get an older version of gcc (3.2.3) on FC4 by installing the 
> compat-gcc packages.
> 
> Paul
-- 
Darryl Dixon <esrever_otua@pythonhacker.is-a-geek.net>