From mboxrd@z Thu Jan 1 00:00:00 1970 From: Peter Lieven Subject: Re: [Qemu-devel] Qemu-KVM 0.12.3 and Multipath -> Assertion Date: Tue, 04 May 2010 13:38:56 +0200 Message-ID: <4BE00750.6040804@dlh.net> References: <4BDF3F94.1080608@dlh.net> <4BDFDC44.9030808@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: kvm@vger.kernel.org, qemu-devel@nongnu.org To: Kevin Wolf Return-path: Received: from zion.dlh.net ([91.198.192.1]:47859 "EHLO mail.dlh.net" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1758258Ab0EDLji (ORCPT ); Tue, 4 May 2010 07:39:38 -0400 In-Reply-To: <4BDFDC44.9030808@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: hi kevin, i set a breakpint at bmdma_active_if. the first 2 breaks encountered when the last path in the multipath failed, but the assertion was not true. when i kicked one path back in the breakpoint was reached again, this time leading to an assert. the stacktrace is from the point shortly before. hope this helps. br, peter -- (gdb) b bmdma_active_if Breakpoint 2 at 0x43f2e0: file /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h, line 507. (gdb) c Continuing. [Switching to Thread 0x7f7b3300d950 (LWP 21171)] Breakpoint 2, bmdma_active_if (bmdma=0xe31fd8) at /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507 507 assert(bmdma->unit != (uint8_t)-1); (gdb) c Continuing. Breakpoint 2, bmdma_active_if (bmdma=0xe31fd8) at /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507 507 assert(bmdma->unit != (uint8_t)-1); (gdb) c Continuing. Breakpoint 2, bmdma_active_if (bmdma=0xe31fd8) at /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507 507 assert(bmdma->unit != (uint8_t)-1); (gdb) bt full #0 bmdma_active_if (bmdma=0xe31fd8) at /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507 __PRETTY_FUNCTION__ = "bmdma_active_if" #1 0x000000000043f6ba in ide_read_dma_cb (opaque=0xe31fd8, ret=0) at /usr/src/qemu-kvm-0.12.3/hw/ide/core.c:554 bm = (BMDMAState *) 0xe31fd8 s = (IDEState *) 0xe17940 n = 0 sector_num = 0 #2 0x000000000058730c in dma_bdrv_cb (opaque=0xe17940, ret=0) at /usr/src/qemu-kvm-0.12.3/dma-helpers.c:94 dbs = (DMAAIOCB *) 0xe17940 cur_addr = 0 cur_len = 0 mem = (void *) 0x0 #3 0x000000000049e510 in qemu_laio_process_completion (s=0xe119c0, laiocb=0xe179c0) at linux-aio.c:68 ret = 0 #4 0x000000000049e611 in qemu_laio_enqueue_completed (s=0xe119c0, laiocb=0xe179c0) at linux-aio.c:107 No locals. #5 0x000000000049e787 in qemu_laio_completion_cb (opaque=0xe119c0) at linux-aio.c:144 iocb = (struct iocb *) 0xe179f0 laiocb = (struct qemu_laiocb *) 0xe179c0 val = 1 ret = 8 nevents = 1 i = 0 events = {{data = 0x0, obj = 0xe179f0, res = 4096, res2 = 0}, {data = 0x0, obj = 0x0, res = 0, res2 = 0} , {data = 0x0, obj = 0x0, res = 0, res2 = 4365191}, {data = 0x429abf, obj = 0x7f7b3300c410, res = 4614129721674825936, res2 = 14777248}, {data = 0x3000000018, obj = 0x7f7b3300c4c0, res = 140167113393152, res2 = 47259417504}, {data = 0xe17740, obj = 0xa3300c4e0, res = 140167113393184, res2 = 0}, {data = 0xe17740, obj = 0x0, res = 0, res2 = 17}, {data = 0x7f7b3300ccf0, obj = 0x92, res = 32, res2 = 168}, {data = 0x7f7b33797a00, obj = 0x801000, res = 0, res2 = 140167141433408}, {data = 0x7f7b34496e00, obj = 0x7f7b33797a00, res = 140167113393392, res2 = 8392704}, {data = 0x0, obj = 0x7f7b34aca040, res = 140167134932480, res2 = 140167118209654}, {data = 0x7f7b3300d950, obj = 0x42603d, res = 0, res2 = 42949672960}, {data = 0x7f7b3300c510, obj = 0xe17ba0, res = 14776128, res2 = 43805361568}, {data = 0x7f7b3300ced0, obj = 0x42797e, res = 0, res2 = 14777248}, { data = 0x174, obj = 0x0, res = 373, res2 = 0}, {data = 0x176, obj = 0x0, res = 3221225601, res2 = 0}, {data = 0x4008ae89c0000083, obj = 0x0, res = 209379655938, res2 = 0}, { data = 0x7f7bc0000084, obj = 0x0, res = 3221225602, res2 = 0}, {data = 0x7f7b00000012, obj = 0x0, res = 17, res2 = 0}, {data = 0x0, obj = 0x11, res = 140167113395840, res2 = 146}, {data = 0x20, obj = 0xa8, res = 140167121304064, res2 = 8392704}, {data = 0x0, obj = 0x7f7b34aca040, res = 140167134932480, res2 = 140167121304064}, { data = 0x7f7b3300c680, obj = 0x801000, res = 0, res2 = 140167141433408}, {data = 0x7f7b34496e00, obj = 0x7f7b334a4276, res = 140167113398608, res2 = 4350013}, {data = 0x0, obj = 0xa00000000, res = 140167113393824, res2 = 14777248}, {data = 0xe2c010, obj = 0xa3300c730, res = 140167113396320, res2 = 4356478}, {data = 0x0, obj = 0xe17ba0, res = 372, res2 = 0}, {data = 0x175, obj = 0x0, res = 374, res2 = 0}, {data = 0xc0000081, obj = 0x0, res = 3221225603, res2 = 0}, {data = 0xc0000102, obj = 0x0, res = 3221225604, res2 = 0}, {data = 0xc0000082, obj = 0x0, res = 18, res2 = 0}, {data = 0x11, obj = 0x0, res = 0, res2 = 0}, {data = 0x0, obj = 0x0, res = 0, res2 = 0}, { data = 0x0, obj = 0x0, res = 0, res2 = 0}, {data = 0x0, obj = 0x0, res = 0, res2 = 0}, {data = 0x0, obj = 0x0, res = 0, res2 = 0}, {data = 0x0, obj = 0x0, res = 0, res2 = 140167139245116}, {data = 0x0, obj = 0x7f7b34abe118, res = 9, res2 = 13}, {data = 0x25bf5fc6, obj = 0x7f7b348b40f0, res = 140167117719264, res2 = 6}, { data = 0x96fd7f, obj = 0x7f7b3300c850, res = 140167113394680, res2 = 140167117724520}, {data = 0x0, obj = 0x7f7b34abe168, res = 140167141388288, res2 = 4206037}, { data = 0x7f7b3343a210, obj = 0x402058, res = 21474836480, res2 = 4294968102}, {data = 0x0, obj = 0x7f7b34ac8358, res = 140167113394736, res2 = 140167113394680}, { data = 0x25bf5fc6, obj = 0x7f7b3300c9e0, res = 0, res2 = 140167139246910}, {data = 0x0, obj = 0x7f7b34abe168, res = 5, res2 = 140167139245116}, {data = 0x1, obj = 0x7f7b34abe120, res = 10, res2 = 13}, {data = 0x9fd7b9dd, obj = 0x7f7b348b40f0, res = 140167139205904, res2 = 140166257704989}, {data = 0x27f5ee7, obj = 0x7f7b3300c950, res = 140167113394936, res2 = 140167139205920}, {data = 0x0, obj = 0x7f7b34abe510, res = 140167141434664, res2 = 140167134874710}, { data = 0x7f7b348aa5a8, obj = 0x7f7b34486e30, res = 21474836480, res2 = 4294967319}, {data = 0x7f7b3428427c, obj = 0x7f7b34ac7cc8, res = 140167113394992, res2 = 140167113394936}, {data = 0x9fd7b9dd, obj = 0x7f7b3300cae0, res = 0, res2 = 140167139246910}, {data = 0x0, obj = 0x7f7b34abe510, res = 140166257704965, res2 = 0}, { data = 0x500000001, obj = 0xffffffff, res = 0, res2 = 8627912}, {data = 0x801000, obj = 0x100000000, res = 140167141385488, res2 = 140167141424328}, {data = 0x7f7b3300cb60, obj = 0x7f7b34ac7970, res = 140167134874710, res2 = 0}, {data = 0x5, obj = 0x0, res = 140167117743864, res2 = 140167113398608}, {data = 0x7f7b3300cb00, obj = 0x7f7b348bf592, res = 14665264, res2 = 90}, {data = 0x2, obj = 0x7f7b33508255, res = 140167113398608, res2 = 140167113394944}, {data = 0x801000, obj = 0x0, res = 140167141433408, res2 = 4966659}, {data = 0x50, obj = 0x23300cb2f, res = 140167139206472, res2 = 140167141434664}, {data = 0x98, obj = 0xffffffff, res = 140167113395072, res2 = 2191368}, { data = 0x801000, obj = 0x3, res = 140167141433408, res2 = 140167139245116}, {data = 0x7f7b34486000, obj = 0x7f7b34abe0e8, res = 3, res2 = 13}, {data = 0xa896c0a2, obj = 0x7f7b348b40f0, res = 140167132758616, res2 = 34}, {data = 0x2a25b02, obj = 0x7f7b3300cb90, res = 140167113395512, res2 = 140167132758672}, {data = 0x0, obj = 0x7f7b34abe1b0, res = 140167141396480, res2 = 4204852}, {data = 0x7f7b34284458, obj = 0x400d38, res = 21474836480, res2 = 4294967302}, {data = 0xc008ae67325339e0, ---Type to continue, or q to quit--- obj = 0x7f7b34ac8358, res = 140167113395568, res2 = 140167113395512}, {data = 0xa896c0a2, obj = 0x7f7b3300cd20, res = 0, res2 = 140167139246910}, {data = 0x0, obj = 0x7f7b34abe1b0, res = 140166257704965, res2 = 0}, {data = 0x1, obj = 0x101010101010101, res = 140167113395504, res2 = 14776768}, {data = 0x7f7b3300cd20, obj = 0x13300cd7c, res = 140167141384624, res2 = 140167141426008}, {data = 0x7f7b3300cda0, obj = 0x7f7b34ac8000, res = 4204852, res2 = 4364819}, {data = 0x42994b, obj = 0x7f7b3300ccf0, res = 13837501612500713496, res2 = 14658720}, {data = 0x3000000018, obj = 0x7f7b3300cda0, res = 140167113395424, res2 = 4371109}, { data = 0x7f7b3300cdc0, obj = 0x3300cd4c, res = 64424509441, res2 = 4364819}, {data = 0x42994b, obj = 0x7f7b3300cd50, res = 13837501612500713473, res2 = 14658720}, { data = 0x3000000018, obj = 0x7f7b3300ce00, res = 140167132758816, res2 = 140167141396480}, {data = 0x7f7b3300ce20, obj = 0xffffffff, res = 140167113395580, res2 = 8626296}, { data = 0x801000, obj = 0x0, res = 140167141433408, res2 = 140167134932480}, {data = 0x0, obj = 0x7f7b348b871a, res = 140166257704965, res2 = 0}, {data = 0x7f7b3300cdd0, obj = 0x42b2a5, res = 140167132758816, res2 = 140167113398608}} ts = {tv_sec = 0, tv_nsec = 0} s = (struct qemu_laio_state *) 0xe119c0 #6 0x000000000049e841 in laio_cancel (blockacb=0xe179c0) at linux-aio.c:184 laiocb = (struct qemu_laiocb *) 0xe179c0 event = {data = 0x1, obj = 0x4c7fb1, res = 140167113395792, res2 = 4384262} ret = -22 #7 0x000000000049a29b in bdrv_aio_cancel (acb=0xe179c0) at block.c:1792 No locals. #8 0x000000000058755a in dma_aio_cancel (acb=0xe17940) at /usr/src/qemu-kvm-0.12.3/dma-helpers.c:138 dbs = (DMAAIOCB *) 0xe17940 #9 0x000000000049a29b in bdrv_aio_cancel (acb=0xe17940) at block.c:1792 No locals. #10 0x0000000000444a0c in ide_dma_cancel (bm=0xe31fd8) at /usr/src/qemu-kvm-0.12.3/hw/ide/core.c:2838 No locals. #11 0x0000000000444f39 in bmdma_cmd_writeb (opaque=0xe31fd8, addr=49152, val=8) at /usr/src/qemu-kvm-0.12.3/hw/ide/pci.c:44 bm = (BMDMAState *) 0xe31fd8 #12 0x00000000004c81bc in ioport_write (index=0, address=49152, data=8) at ioport.c:80 func = (IOPortWriteFunc *) 0x444f0c default_func = {0x4c81d0 , 0x4c8225 , 0x4c8282 } #13 0x00000000004c8543 in cpu_outb (addr=49152, val=8 '\b') at ioport.c:198 No locals. #14 0x0000000000429689 in kvm_handle_io (port=49152, data=0x7f7b34ab7000, direction=1, size=1, count=1) at /usr/src/qemu-kvm-0.12.3/kvm-all.c:535 i = 0 ptr = (uint8_t *) 0x7f7b34ab7000 "\b" #15 0x000000000042bac3 in kvm_run (env=0xe17ba0) at /usr/src/qemu-kvm-0.12.3/qemu-kvm.c:964 r = 0 kvm = (kvm_context_t) 0xdfb0d0 run = (struct kvm_run *) 0x7f7b34ab6000 fd = 15 #16 0x000000000042cdda in kvm_cpu_exec (env=0xe17ba0) at /usr/src/qemu-kvm-0.12.3/qemu-kvm.c:1647 r = 0 #17 0x000000000042d564 in kvm_main_loop_cpu (env=0xe17ba0) at /usr/src/qemu-kvm-0.12.3/qemu-kvm.c:1889 run_cpu = 1 #18 0x000000000042d6a5 in ap_main_loop (_env=0xe17ba0) at /usr/src/qemu-kvm-0.12.3/qemu-kvm.c:1939 env = (struct CPUX86State *) 0xe17ba0 signals = {__val = {18446744067267100671, 18446744073709551615 }} data = (struct ioperm_data *) 0x0 #19 0x00007f7b3448d3ba in start_thread () from /lib/libpthread.so.0 No symbol table info available. #20 0x00007f7b3350ffcd in clone () from /lib/libc.so.6 No symbol table info available. #21 0x0000000000000000 in ?? () No symbol table info available. (gdb) c Continuing. kvm: /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507: bmdma_active_if: Assertion `bmdma->unit != (uint8_t)-1' failed. Program received signal SIGABRT, Aborted. 0x00007f7b3345cfb5 in raise () from /lib/libc.so.6 Kevin Wolf wrote: > Am 03.05.2010 23:26, schrieb Peter Lieven: > >> Hi Qemu/KVM Devel Team, >> >> i'm using qemu-kvm 0.12.3 with latest Kernel 2.6.33.3. >> As backend we use open-iSCSI with dm-multipath. >> >> Multipath is configured to queue i/o if no path is available. >> >> If we create a failure on all paths, qemu starts to consume 100% >> CPU due to i/o waits which is ok so far. >> >> 1 odd thing: The Monitor Interface is not responding any more ... >> >> What es a really blocker is that KVM crashes with: >> kvm: /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507: bmdma_active_if: >> Assertion `bmdma->unit != (uint8_t)-1' failed. >> >> after the multipath has reestablisched at least one path. >> > > Can you get a stack backtrace with gdb? > > >> Any ideas? I remember this was working with earlier kernel/kvm/qemu >> versions. >> > > If it works in the same setup with an older qemu version, bisecting > might help. > > Kevin > > From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1O9GTa-00013P-In for qemu-devel@nongnu.org; Tue, 04 May 2010 07:39:46 -0400 Received: from [140.186.70.92] (port=36550 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1O9GTY-00010g-BB for qemu-devel@nongnu.org; Tue, 04 May 2010 07:39:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1O9GTU-0002Qr-Ns for qemu-devel@nongnu.org; Tue, 04 May 2010 07:39:44 -0400 Received: from zion.dlh.net ([91.198.192.1]:57813 helo=mail.dlh.net) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1O9GTU-0002QJ-C1 for qemu-devel@nongnu.org; Tue, 04 May 2010 07:39:40 -0400 Message-ID: <4BE00750.6040804@dlh.net> Date: Tue, 04 May 2010 13:38:56 +0200 From: Peter Lieven MIME-Version: 1.0 Subject: Re: [Qemu-devel] Qemu-KVM 0.12.3 and Multipath -> Assertion References: <4BDF3F94.1080608@dlh.net> <4BDFDC44.9030808@redhat.com> In-Reply-To: <4BDFDC44.9030808@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Kevin Wolf Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org hi kevin, i set a breakpint at bmdma_active_if. the first 2 breaks encountered when the last path in the multipath failed, but the assertion was not true. when i kicked one path back in the breakpoint was reached again, this time leading to an assert. the stacktrace is from the point shortly before. hope this helps. br, peter -- (gdb) b bmdma_active_if Breakpoint 2 at 0x43f2e0: file /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h, line 507. (gdb) c Continuing. [Switching to Thread 0x7f7b3300d950 (LWP 21171)] Breakpoint 2, bmdma_active_if (bmdma=0xe31fd8) at /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507 507 assert(bmdma->unit != (uint8_t)-1); (gdb) c Continuing. Breakpoint 2, bmdma_active_if (bmdma=0xe31fd8) at /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507 507 assert(bmdma->unit != (uint8_t)-1); (gdb) c Continuing. Breakpoint 2, bmdma_active_if (bmdma=0xe31fd8) at /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507 507 assert(bmdma->unit != (uint8_t)-1); (gdb) bt full #0 bmdma_active_if (bmdma=0xe31fd8) at /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507 __PRETTY_FUNCTION__ = "bmdma_active_if" #1 0x000000000043f6ba in ide_read_dma_cb (opaque=0xe31fd8, ret=0) at /usr/src/qemu-kvm-0.12.3/hw/ide/core.c:554 bm = (BMDMAState *) 0xe31fd8 s = (IDEState *) 0xe17940 n = 0 sector_num = 0 #2 0x000000000058730c in dma_bdrv_cb (opaque=0xe17940, ret=0) at /usr/src/qemu-kvm-0.12.3/dma-helpers.c:94 dbs = (DMAAIOCB *) 0xe17940 cur_addr = 0 cur_len = 0 mem = (void *) 0x0 #3 0x000000000049e510 in qemu_laio_process_completion (s=0xe119c0, laiocb=0xe179c0) at linux-aio.c:68 ret = 0 #4 0x000000000049e611 in qemu_laio_enqueue_completed (s=0xe119c0, laiocb=0xe179c0) at linux-aio.c:107 No locals. #5 0x000000000049e787 in qemu_laio_completion_cb (opaque=0xe119c0) at linux-aio.c:144 iocb = (struct iocb *) 0xe179f0 laiocb = (struct qemu_laiocb *) 0xe179c0 val = 1 ret = 8 nevents = 1 i = 0 events = {{data = 0x0, obj = 0xe179f0, res = 4096, res2 = 0}, {data = 0x0, obj = 0x0, res = 0, res2 = 0} , {data = 0x0, obj = 0x0, res = 0, res2 = 4365191}, {data = 0x429abf, obj = 0x7f7b3300c410, res = 4614129721674825936, res2 = 14777248}, {data = 0x3000000018, obj = 0x7f7b3300c4c0, res = 140167113393152, res2 = 47259417504}, {data = 0xe17740, obj = 0xa3300c4e0, res = 140167113393184, res2 = 0}, {data = 0xe17740, obj = 0x0, res = 0, res2 = 17}, {data = 0x7f7b3300ccf0, obj = 0x92, res = 32, res2 = 168}, {data = 0x7f7b33797a00, obj = 0x801000, res = 0, res2 = 140167141433408}, {data = 0x7f7b34496e00, obj = 0x7f7b33797a00, res = 140167113393392, res2 = 8392704}, {data = 0x0, obj = 0x7f7b34aca040, res = 140167134932480, res2 = 140167118209654}, {data = 0x7f7b3300d950, obj = 0x42603d, res = 0, res2 = 42949672960}, {data = 0x7f7b3300c510, obj = 0xe17ba0, res = 14776128, res2 = 43805361568}, {data = 0x7f7b3300ced0, obj = 0x42797e, res = 0, res2 = 14777248}, { data = 0x174, obj = 0x0, res = 373, res2 = 0}, {data = 0x176, obj = 0x0, res = 3221225601, res2 = 0}, {data = 0x4008ae89c0000083, obj = 0x0, res = 209379655938, res2 = 0}, { data = 0x7f7bc0000084, obj = 0x0, res = 3221225602, res2 = 0}, {data = 0x7f7b00000012, obj = 0x0, res = 17, res2 = 0}, {data = 0x0, obj = 0x11, res = 140167113395840, res2 = 146}, {data = 0x20, obj = 0xa8, res = 140167121304064, res2 = 8392704}, {data = 0x0, obj = 0x7f7b34aca040, res = 140167134932480, res2 = 140167121304064}, { data = 0x7f7b3300c680, obj = 0x801000, res = 0, res2 = 140167141433408}, {data = 0x7f7b34496e00, obj = 0x7f7b334a4276, res = 140167113398608, res2 = 4350013}, {data = 0x0, obj = 0xa00000000, res = 140167113393824, res2 = 14777248}, {data = 0xe2c010, obj = 0xa3300c730, res = 140167113396320, res2 = 4356478}, {data = 0x0, obj = 0xe17ba0, res = 372, res2 = 0}, {data = 0x175, obj = 0x0, res = 374, res2 = 0}, {data = 0xc0000081, obj = 0x0, res = 3221225603, res2 = 0}, {data = 0xc0000102, obj = 0x0, res = 3221225604, res2 = 0}, {data = 0xc0000082, obj = 0x0, res = 18, res2 = 0}, {data = 0x11, obj = 0x0, res = 0, res2 = 0}, {data = 0x0, obj = 0x0, res = 0, res2 = 0}, { data = 0x0, obj = 0x0, res = 0, res2 = 0}, {data = 0x0, obj = 0x0, res = 0, res2 = 0}, {data = 0x0, obj = 0x0, res = 0, res2 = 0}, {data = 0x0, obj = 0x0, res = 0, res2 = 140167139245116}, {data = 0x0, obj = 0x7f7b34abe118, res = 9, res2 = 13}, {data = 0x25bf5fc6, obj = 0x7f7b348b40f0, res = 140167117719264, res2 = 6}, { data = 0x96fd7f, obj = 0x7f7b3300c850, res = 140167113394680, res2 = 140167117724520}, {data = 0x0, obj = 0x7f7b34abe168, res = 140167141388288, res2 = 4206037}, { data = 0x7f7b3343a210, obj = 0x402058, res = 21474836480, res2 = 4294968102}, {data = 0x0, obj = 0x7f7b34ac8358, res = 140167113394736, res2 = 140167113394680}, { data = 0x25bf5fc6, obj = 0x7f7b3300c9e0, res = 0, res2 = 140167139246910}, {data = 0x0, obj = 0x7f7b34abe168, res = 5, res2 = 140167139245116}, {data = 0x1, obj = 0x7f7b34abe120, res = 10, res2 = 13}, {data = 0x9fd7b9dd, obj = 0x7f7b348b40f0, res = 140167139205904, res2 = 140166257704989}, {data = 0x27f5ee7, obj = 0x7f7b3300c950, res = 140167113394936, res2 = 140167139205920}, {data = 0x0, obj = 0x7f7b34abe510, res = 140167141434664, res2 = 140167134874710}, { data = 0x7f7b348aa5a8, obj = 0x7f7b34486e30, res = 21474836480, res2 = 4294967319}, {data = 0x7f7b3428427c, obj = 0x7f7b34ac7cc8, res = 140167113394992, res2 = 140167113394936}, {data = 0x9fd7b9dd, obj = 0x7f7b3300cae0, res = 0, res2 = 140167139246910}, {data = 0x0, obj = 0x7f7b34abe510, res = 140166257704965, res2 = 0}, { data = 0x500000001, obj = 0xffffffff, res = 0, res2 = 8627912}, {data = 0x801000, obj = 0x100000000, res = 140167141385488, res2 = 140167141424328}, {data = 0x7f7b3300cb60, obj = 0x7f7b34ac7970, res = 140167134874710, res2 = 0}, {data = 0x5, obj = 0x0, res = 140167117743864, res2 = 140167113398608}, {data = 0x7f7b3300cb00, obj = 0x7f7b348bf592, res = 14665264, res2 = 90}, {data = 0x2, obj = 0x7f7b33508255, res = 140167113398608, res2 = 140167113394944}, {data = 0x801000, obj = 0x0, res = 140167141433408, res2 = 4966659}, {data = 0x50, obj = 0x23300cb2f, res = 140167139206472, res2 = 140167141434664}, {data = 0x98, obj = 0xffffffff, res = 140167113395072, res2 = 2191368}, { data = 0x801000, obj = 0x3, res = 140167141433408, res2 = 140167139245116}, {data = 0x7f7b34486000, obj = 0x7f7b34abe0e8, res = 3, res2 = 13}, {data = 0xa896c0a2, obj = 0x7f7b348b40f0, res = 140167132758616, res2 = 34}, {data = 0x2a25b02, obj = 0x7f7b3300cb90, res = 140167113395512, res2 = 140167132758672}, {data = 0x0, obj = 0x7f7b34abe1b0, res = 140167141396480, res2 = 4204852}, {data = 0x7f7b34284458, obj = 0x400d38, res = 21474836480, res2 = 4294967302}, {data = 0xc008ae67325339e0, ---Type to continue, or q to quit--- obj = 0x7f7b34ac8358, res = 140167113395568, res2 = 140167113395512}, {data = 0xa896c0a2, obj = 0x7f7b3300cd20, res = 0, res2 = 140167139246910}, {data = 0x0, obj = 0x7f7b34abe1b0, res = 140166257704965, res2 = 0}, {data = 0x1, obj = 0x101010101010101, res = 140167113395504, res2 = 14776768}, {data = 0x7f7b3300cd20, obj = 0x13300cd7c, res = 140167141384624, res2 = 140167141426008}, {data = 0x7f7b3300cda0, obj = 0x7f7b34ac8000, res = 4204852, res2 = 4364819}, {data = 0x42994b, obj = 0x7f7b3300ccf0, res = 13837501612500713496, res2 = 14658720}, {data = 0x3000000018, obj = 0x7f7b3300cda0, res = 140167113395424, res2 = 4371109}, { data = 0x7f7b3300cdc0, obj = 0x3300cd4c, res = 64424509441, res2 = 4364819}, {data = 0x42994b, obj = 0x7f7b3300cd50, res = 13837501612500713473, res2 = 14658720}, { data = 0x3000000018, obj = 0x7f7b3300ce00, res = 140167132758816, res2 = 140167141396480}, {data = 0x7f7b3300ce20, obj = 0xffffffff, res = 140167113395580, res2 = 8626296}, { data = 0x801000, obj = 0x0, res = 140167141433408, res2 = 140167134932480}, {data = 0x0, obj = 0x7f7b348b871a, res = 140166257704965, res2 = 0}, {data = 0x7f7b3300cdd0, obj = 0x42b2a5, res = 140167132758816, res2 = 140167113398608}} ts = {tv_sec = 0, tv_nsec = 0} s = (struct qemu_laio_state *) 0xe119c0 #6 0x000000000049e841 in laio_cancel (blockacb=0xe179c0) at linux-aio.c:184 laiocb = (struct qemu_laiocb *) 0xe179c0 event = {data = 0x1, obj = 0x4c7fb1, res = 140167113395792, res2 = 4384262} ret = -22 #7 0x000000000049a29b in bdrv_aio_cancel (acb=0xe179c0) at block.c:1792 No locals. #8 0x000000000058755a in dma_aio_cancel (acb=0xe17940) at /usr/src/qemu-kvm-0.12.3/dma-helpers.c:138 dbs = (DMAAIOCB *) 0xe17940 #9 0x000000000049a29b in bdrv_aio_cancel (acb=0xe17940) at block.c:1792 No locals. #10 0x0000000000444a0c in ide_dma_cancel (bm=0xe31fd8) at /usr/src/qemu-kvm-0.12.3/hw/ide/core.c:2838 No locals. #11 0x0000000000444f39 in bmdma_cmd_writeb (opaque=0xe31fd8, addr=49152, val=8) at /usr/src/qemu-kvm-0.12.3/hw/ide/pci.c:44 bm = (BMDMAState *) 0xe31fd8 #12 0x00000000004c81bc in ioport_write (index=0, address=49152, data=8) at ioport.c:80 func = (IOPortWriteFunc *) 0x444f0c default_func = {0x4c81d0 , 0x4c8225 , 0x4c8282 } #13 0x00000000004c8543 in cpu_outb (addr=49152, val=8 '\b') at ioport.c:198 No locals. #14 0x0000000000429689 in kvm_handle_io (port=49152, data=0x7f7b34ab7000, direction=1, size=1, count=1) at /usr/src/qemu-kvm-0.12.3/kvm-all.c:535 i = 0 ptr = (uint8_t *) 0x7f7b34ab7000 "\b" #15 0x000000000042bac3 in kvm_run (env=0xe17ba0) at /usr/src/qemu-kvm-0.12.3/qemu-kvm.c:964 r = 0 kvm = (kvm_context_t) 0xdfb0d0 run = (struct kvm_run *) 0x7f7b34ab6000 fd = 15 #16 0x000000000042cdda in kvm_cpu_exec (env=0xe17ba0) at /usr/src/qemu-kvm-0.12.3/qemu-kvm.c:1647 r = 0 #17 0x000000000042d564 in kvm_main_loop_cpu (env=0xe17ba0) at /usr/src/qemu-kvm-0.12.3/qemu-kvm.c:1889 run_cpu = 1 #18 0x000000000042d6a5 in ap_main_loop (_env=0xe17ba0) at /usr/src/qemu-kvm-0.12.3/qemu-kvm.c:1939 env = (struct CPUX86State *) 0xe17ba0 signals = {__val = {18446744067267100671, 18446744073709551615 }} data = (struct ioperm_data *) 0x0 #19 0x00007f7b3448d3ba in start_thread () from /lib/libpthread.so.0 No symbol table info available. #20 0x00007f7b3350ffcd in clone () from /lib/libc.so.6 No symbol table info available. #21 0x0000000000000000 in ?? () No symbol table info available. (gdb) c Continuing. kvm: /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507: bmdma_active_if: Assertion `bmdma->unit != (uint8_t)-1' failed. Program received signal SIGABRT, Aborted. 0x00007f7b3345cfb5 in raise () from /lib/libc.so.6 Kevin Wolf wrote: > Am 03.05.2010 23:26, schrieb Peter Lieven: > >> Hi Qemu/KVM Devel Team, >> >> i'm using qemu-kvm 0.12.3 with latest Kernel 2.6.33.3. >> As backend we use open-iSCSI with dm-multipath. >> >> Multipath is configured to queue i/o if no path is available. >> >> If we create a failure on all paths, qemu starts to consume 100% >> CPU due to i/o waits which is ok so far. >> >> 1 odd thing: The Monitor Interface is not responding any more ... >> >> What es a really blocker is that KVM crashes with: >> kvm: /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507: bmdma_active_if: >> Assertion `bmdma->unit != (uint8_t)-1' failed. >> >> after the multipath has reestablisched at least one path. >> > > Can you get a stack backtrace with gdb? > > >> Any ideas? I remember this was working with earlier kernel/kvm/qemu >> versions. >> > > If it works in the same setup with an older qemu version, bisecting > might help. > > Kevin > >