From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Jp3QN-0000I1-IV for qemu-devel@nongnu.org; Thu, 24 Apr 2008 11:31:51 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Jp3QL-0000GL-F1 for qemu-devel@nongnu.org; Thu, 24 Apr 2008 11:31:50 -0400 Received: from [199.232.76.173] (port=56654 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Jp3QL-0000GA-4c for qemu-devel@nongnu.org; Thu, 24 Apr 2008 11:31:49 -0400 Received: from tim.rpsys.net ([194.106.48.114]) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1Jp3QK-0008CE-JQ for qemu-devel@nongnu.org; Thu, 24 Apr 2008 11:31:49 -0400 Received: from localhost (localhost [127.0.0.1]) by tim.rpsys.net (8.13.6/8.13.8) with ESMTP id m3OFVdFF000578 for ; Thu, 24 Apr 2008 16:31:39 +0100 Received: from tim.rpsys.net ([127.0.0.1]) by localhost (tim.rpsys.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 00391-05 for ; Thu, 24 Apr 2008 16:31:34 +0100 (BST) Received: from [192.168.1.3] (dax.rpnet.com [192.168.1.3]) (authenticated bits=0) by tim.rpsys.net (8.13.6/8.13.8) with ESMTP id m3OFVV2v000559 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 24 Apr 2008 16:31:31 +0100 From: Richard Purdie Content-Type: multipart/mixed; boundary="=-/+NGdES8tWkEINfpxach" Date: Thu, 24 Apr 2008 16:31:32 +0100 Message-Id: <1209051092.5108.60.camel@dax.rpnet.com> Mime-Version: 1.0 Subject: [Qemu-devel] [PATCH] Fix brk syscall error handling and mmap issues Reply-To: qemu-devel@nongnu.org List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org --=-/+NGdES8tWkEINfpxach Content-Type: text/plain Content-Transfer-Encoding: 7bit When running certain memory heavy programs under QEMU ARM user emulation I've noticed a QEMU segfault. The segfault is occurring just after an mmap call on the host system. The reason is that the target application tries to enlarge the data segment over the top of the memory address where the qemu binary is loaded using a brk syscall. The brk syscall is translated into an mmap call which unmaps the qemu binary from memory after which it unsurprisingly segfaults. Initially I tested this by changing the load address of the QEMU binary but that fix is less than ideal since its still possible to conflict. I've attached a better which where the mmap code checks the call doesn't touch "reserved" pages and returns an error if it does instead of calling mmap. brk syscalls usually occur from within malloc within glibc. If they fail, malloc should start using mmap to request memory instead. The above error should therefore not be fatal and the application should just stop making brk calls and use mmap but this isn't what was observed, it continued to segfault somewhere else in the program. I found that the error return values for the brk syscall were incorrect and were causing this secondary segfault. In case of an error, the brk syscall should return the original brk value, not any kind of error code. The attached patch fixes both these issues. Cheers, Richard --=-/+NGdES8tWkEINfpxach Content-Disposition: attachment; filename=fix_brk.patch Content-Type: text/x-patch; name=fix_brk.patch; charset=UTF-8 Content-Transfer-Encoding: 7bit Index: linux-user/syscall.c =================================================================== --- linux-user/syscall.c (revision 16) +++ linux-user/syscall.c (working copy) @@ -441,7 +441,7 @@ if (!new_brk) return target_brk; if (new_brk < target_original_brk) - return -TARGET_ENOMEM; + return target_brk; brk_page = HOST_PAGE_ALIGN(target_brk); @@ -456,12 +456,11 @@ mapped_addr = get_errno(target_mmap(brk_page, new_alloc_size, PROT_READ|PROT_WRITE, MAP_ANON|MAP_FIXED|MAP_PRIVATE, 0, 0)); - if (is_error(mapped_addr)) { - return mapped_addr; - } else { + + if (!is_error(mapped_addr)) target_brk = new_brk; - return target_brk; - } + + return target_brk; } static inline abi_long copy_from_user_fdset(fd_set *fds, Index: linux-user/mmap.c =================================================================== --- linux-user/mmap.c (revision 16) +++ linux-user/mmap.c (working copy) @@ -260,6 +259,9 @@ host_start += offset - host_offset; start = h2g(host_start); } else { + int flg; + target_ulong addr; + if (start & ~TARGET_PAGE_MASK) { errno = EINVAL; return -1; @@ -267,6 +269,14 @@ end = start + len; real_end = HOST_PAGE_ALIGN(end); + for(addr = real_start; addr < real_end; addr += TARGET_PAGE_SIZE) { + flg = page_get_flags(addr); + if( flg & PAGE_RESERVED ) { + errno = ENXIO; + return -1; + } + } + /* worst case: we cannot map the file because the offset is not aligned, so we read it */ if (!(flags & MAP_ANONYMOUS) && --=-/+NGdES8tWkEINfpxach--