From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1IcbOe-0007tt-GV for qemu-devel@nongnu.org; Tue, 02 Oct 2007 02:38:20 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1IcbOc-0007tW-SX for qemu-devel@nongnu.org; Tue, 02 Oct 2007 02:38:19 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1IcbOc-0007tT-M8 for qemu-devel@nongnu.org; Tue, 02 Oct 2007 02:38:18 -0400 Received: from owa.c2.net ([207.235.78.2] helo=email.c2.net) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1IcbOc-0006oN-GN for qemu-devel@nongnu.org; Tue, 02 Oct 2007 02:38:18 -0400 From: Thayne Harbaugh Content-Type: text/plain Date: Tue, 02 Oct 2007 00:31:01 -0600 Message-Id: <1191306661.5200.53.camel@phantasm.home.enterpriseandprosperity.com> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [Qemu-devel] [PATCH] mixed types in target_mremap() return incorrect value Reply-To: thayne@c2.net, qemu-devel@nongnu.org List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org There are mixed types (long vs target_ulong) in linux-user/mmap.c:target_mremap() and consequently the wrong value is returned (automatic type casting fails) when the host long is larger than the guest long and -1 is returned for an error condition. Consider the initial lines of target_mremap() where a failure is tested: long target_mremap(target_ulong old_addr, target_ulong old_size, target_ulong new_size, unsigned long flags, target_ulong new_addr) { int prot; /* XXX: use 5 args syscall */ new_addr = (long)mremap(g2h(old_addr), old_size, new_size, flags); if (new_addr == -1) return new_addr; Note that signed long is the return type for target_mremap() while target_ulong is used for new_addr and the return value of mremap(). When the host long is larger than the target long the following can happen on mremap() failure: -1 is stored in new_addr which is unsigned. The test for new_addr == -1 is true because the -1 is cast to target_ulong. When new_addr is returned, however, new_addr is cast to long, which is larger than target_ulong for the case of i386 target on x86_64 guest and the -1 is now a positive number that can be fully represented as a positive number even though the return value is signed. This patch is one way to ensure that -1 is always returned for an error condition: Index: qemu/linux-user/mmap.c =================================================================== --- qemu.orig/linux-user/mmap.c 2007-10-02 00:27:36.000000000 -0600 +++ qemu/linux-user/mmap.c 2007-10-02 00:31:41.000000000 -0600 @@ -395,7 +395,7 @@ /* XXX: use 5 args syscall */ new_addr = (long)mremap(g2h(old_addr), old_size, new_size, flags); if (new_addr == -1) - return new_addr; + return -1; new_addr = h2g(new_addr); prot = page_get_flags(old_addr); page_set_flags(old_addr, old_addr + old_size, 0);