[PATCH] target/i386: Do not switch to MMX mode for CVTPI2PS/CVTPI2PD with memory operand

[Simon Scherer <scherer.simon89@gmail.com>]

Per the Intel SDM, the CVTPI2PS and CVTPI2PD instructions
cause a transition from x87 FPU to MMX technology operation (the x87
top-of-stack pointer is set to 0 and the tag word is set to all
valid). CVTPI2PD's own description scopes this to the "xmm, mm"
operand form only, explicitly excluding "xmm, m64". But CVTPI2PS's
description states the transition unconditionally, without the same
operand-form distinction.

Testing on real hardware shows CVTPI2PS actually behaves identically
to CVTPI2PD despite the SDM wording: neither instruction performs the
state transition when the source is a memory operand, only when it is
an actual MMX register. This matches a similar Valgrind bug report
and fix, see https://bugs.kde.org/show_bug.cgi?id=357059.

gen_CVTPI2Px() currently calls gen_helper_enter_mmx() unconditionally,
regardless of the source operand's form. Only call it when the source
operand does not have an effective address, i.e. is a real MMX
register.

Resolves: https://gitlab.com/qemu-project/qemu/-/work_items/4394
Signed-off-by: Simon Scherer <scherer.simon89@gmail.com>
---
 target/i386/tcg/emit.c.inc | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/target/i386/tcg/emit.c.inc b/target/i386/tcg/emit.c.inc
index c83ab80940..72c08beb2a 100644
--- a/target/i386/tcg/emit.c.inc
+++ b/target/i386/tcg/emit.c.inc
@@ -1919,7 +1919,15 @@ static void gen_CRC32(DisasContext *s, X86DecodedInsn *decode)
 
 static void gen_CVTPI2Px(DisasContext *s, X86DecodedInsn *decode)
 {
-    gen_helper_enter_mmx(tcg_env);
+    /*
+     * The x87->MMX transition only happens when the source is an MMX
+     * register, not for the memory-operand form. Confirmed on real
+     * hardware, contradicting the SDM's CVTPI2PS text. See
+     * https://bugs.kde.org/show_bug.cgi?id=357059.
+     */
+    if (!decode->op[2].has_ea) {
+        gen_helper_enter_mmx(tcg_env);
+    }
     if (s->prefix & PREFIX_DATA) {
         gen_helper_cvtpi2pd(tcg_env, OP_PTR0, OP_PTR2);
     } else {
-- 
2.53.0