[Qemu-devel] x86: FP binary format and overlapping with MMX

[Jan Kiszka <jan.kiszka@siemens.com>]

Hi,

while refactoring and enhancing the x86 part of gdbstub, I came across
an issue in the way CPUX86State.fpregs is stored: According to Intel
specs, floating point register bits 0..63 map on MMX register bits
0..63. But this union

    union {
#ifdef USE_X86LDOUBLE
        CPU86_LDouble d __attribute__((aligned(16)));
#else
        CPU86_LDouble d;
#endif
        MMXReg mmx;
    } fpregs[8];

only ensures this if
 a) the host is little endian (due to sizeof(d) > sizeof(mmx)) and
 b) CPU86_LDouble is actually laid out as on real x86.

How a) can be violated is obvious, but what about b)?

It seems that qemu is using different binary representations, depending
on the FPU emulation mode (native vs. soft). On x86 emulating x86, this
is no issue, as the double extended float type (80 bits) is natively
available and used. But when going to other host archs, possibly
defining CPU86_LDouble as float128, correctness is lost, specifically
/wrt saving/restoring the cruft to/from memory where the guest can
fiddle with individual bits. Moreover, that's where I came from,
byte-swapping for gdb becomes impossible: should I swap MMX or should I
convert/swap FP representation?

Thoughts on this welcome!
Jan

-- 
Siemens AG, Corporate Technology, CT SE 2 ES-OS
Corporate Competence Center Embedded Linux