From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52458) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YV4Qx-00086q-V5 for qemu-devel@nongnu.org; Mon, 09 Mar 2015 16:37:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YV4Qr-0005XI-Eh for qemu-devel@nongnu.org; Mon, 09 Mar 2015 16:37:51 -0400 Received: from e06smtp16.uk.ibm.com ([195.75.94.112]:41966) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YV4Qr-0005Wy-4N for qemu-devel@nongnu.org; Mon, 09 Mar 2015 16:37:45 -0400 Received: from /spool/local by e06smtp16.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Mar 2015 20:37:42 -0000 Received: from b06cxnps3075.portsmouth.uk.ibm.com (d06relay10.portsmouth.uk.ibm.com [9.149.109.195]) by d06dlp01.portsmouth.uk.ibm.com (Postfix) with ESMTP id 49F1817D8059 for ; Mon, 9 Mar 2015 20:38:01 +0000 (GMT) Received: from d06av11.portsmouth.uk.ibm.com (d06av11.portsmouth.uk.ibm.com [9.149.37.252]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id t29Kbd2N459050 for ; Mon, 9 Mar 2015 20:37:39 GMT Received: from d06av11.portsmouth.uk.ibm.com (localhost [127.0.0.1]) by d06av11.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id t29KbdGh015574 for ; Mon, 9 Mar 2015 14:37:39 -0600 Message-ID: <54FE0492.5080305@de.ibm.com> Date: Mon, 09 Mar 2015 21:37:38 +0100 From: Christian Borntraeger MIME-Version: 1.0 References: <54EEE795.3050501@de.ibm.com> <20150306172324.GF2431@stefanha-thinkpad.redhat.com> In-Reply-To: <20150306172324.GF2431@stefanha-thinkpad.redhat.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] qemu crash in coroutine bdrv_co_do_rw List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: Paolo Bonzini , qemu-devel , Stefan Hajnoczi Am 06.03.2015 um 18:23 schrieb Stefan Hajnoczi: > On Thu, Feb 26, 2015 at 10:29:57AM +0100, Christian Borntraeger wrote: >> is this some know issue? Under heavy load with lots of dataplane devices I sometimes get a segfault in the bdrc_co_do_rw routine: >> >> #0 bdrv_co_do_rw (opaque=0x0) at /home/cborntra/REPOS/qemu/block.c:4791 >> 4791 if (!acb->is_write) { >> (gdb) bt >> #0 bdrv_co_do_rw (opaque=0x0) at /home/cborntra/REPOS/qemu/block.c:4791 >> #1 0x00000000801aeb78 in coroutine_trampoline (i0=, i1=-725099072) at /home/cborntra/REPOS/qemu/coroutine-ucontext.c:80 >> #2 0x000003fffbe1cca2 in __makecontext_ret () from /lib64/libc.so.6 >> Backtrace stopped: previous frame identical to this frame (corrupt stack?) >> (gdb) up >> #1 0x00000000801aeb78 in coroutine_trampoline (i0=, i1=-725099072) at /home/cborntra/REPOS/qemu/coroutine-ucontext.c:80 >> 80 co->entry(co->entry_arg); >> (gdb) print *co >> $1 = {entry = 0x801a3c28 , entry_arg = 0x0, caller = 0x3ffe2fff788, pool_next = {sle_next = 0x3ffd2287990}, co_queue_wakeup = {tqh_first = 0x0, >> tqh_last = 0x3ffd4c7dde0}, co_queue_next = {tqe_next = 0x0, tqe_prev = 0x0}} >> >> As you can see enty_arg is 0, causing the problem. Do you have any quick idea before I start debugging? > > No, I haven't seen this bug before. Are you running qemu.git/master? > > Have you tried disabling the coroutine pool (freelist)? > > Stefan > I was able to increase the likelyhood of hitting this (more vCPUs, less guests). bisect thinks that this makes this shaky: 4d68e86bb10159099da0798f74e7512955f15eec is the first bad commit commit 4d68e86bb10159099da0798f74e7512955f15eec Author: Paolo Bonzini Date: Tue Dec 2 12:05:48 2014 +0100 coroutine: rewrite pool to avoid mutex Christian