From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45008) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bVJQb-0003Xc-Tw for qemu-devel@nongnu.org; Thu, 04 Aug 2016 10:15:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bVJQZ-0000TZ-TM for qemu-devel@nongnu.org; Thu, 04 Aug 2016 10:15:16 -0400 Received: from mail-lf0-x22e.google.com ([2a00:1450:4010:c07::22e]:34855) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bVJQZ-0000TM-LN for qemu-devel@nongnu.org; Thu, 04 Aug 2016 10:15:15 -0400 Received: by mail-lf0-x22e.google.com with SMTP id f93so183268808lfi.2 for ; Thu, 04 Aug 2016 07:15:15 -0700 (PDT) From: riku.voipio@linaro.org Date: Thu, 4 Aug 2016 17:15:05 +0300 Message-Id: In-Reply-To: References: Subject: [Qemu-devel] [PULL 5/5] linux-user: Handle brk() attempts with very large sizes List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org From: Peter Maydell In do_brk(), we were inadvertently truncating the size of a requested brk() from the guest by putting it into an 'int' variable. This meant that we would incorrectly report success back to the guest rather than a failed allocation, typically resulting in the guest then segfaulting. Use abi_ulong instead. This fixes a crash in the '31370.cc' test in the gcc libstdc++ test suite (the test case starts by trying to allocate a very large size and reduces the size until the allocation succeeds). Signed-off-by: Peter Maydell Signed-off-by: Riku Voipio --- linux-user/syscall.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/linux-user/syscall.c b/linux-user/syscall.c index df6f2a9..833f853 100644 --- a/linux-user/syscall.c +++ b/linux-user/syscall.c @@ -839,7 +839,7 @@ void target_set_brk(abi_ulong new_brk) abi_long do_brk(abi_ulong new_brk) { abi_long mapped_addr; - int new_alloc_size; + abi_ulong new_alloc_size; DEBUGF_BRK("do_brk(" TARGET_ABI_FMT_lx ") -> ", new_brk); -- 2.1.4