[Qemu-devel] [PATCH] linux-user: Fix ioctl cmd type mismatch on 64-bit targets

[Qemu-devel] [PATCH] linux-user: Fix ioctl cmd type mismatch on 64-bit targets

[Ed Swierk]

linux-user passes the cmd argument of the ioctl syscall as a signed long,
but compares it to an unsigned int when iterating through the ioctl_entries
list.  When the cmd is a large value like 0x80047476 (TARGET_TIOCSWINSZ on
mips64) it gets sign-extended to 0xffffffff80047476, causing the comparison
to fail and resulting in lots of spurious "Unsupported ioctl" errors.
Changing the target_cmd field in the ioctl_entries list to a signed int
causes those values to be sign-extended as well during the comparison.

Signed-off-by: Ed Swierk <eswierk@skyportsystems.com>
---
 linux-user/syscall.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index aaac6a2..d636c81 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -3278,7 +3278,7 @@ typedef abi_long do_ioctl_fn(const IOCTLEntry *ie, uint8_t *buf_temp,
                              int fd, abi_long cmd, abi_long arg);
 
 struct IOCTLEntry {
-    unsigned int target_cmd;
+    int target_cmd;
     unsigned int host_cmd;
     const char *name;
     int access;
-- 
1.9.1