From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46123) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c4R2S-0001Hm-CV for qemu-devel@nongnu.org; Wed, 09 Nov 2016 06:27:37 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c4R2O-0005ET-Cb for qemu-devel@nongnu.org; Wed, 09 Nov 2016 06:27:32 -0500 Received: from mail-wm0-x243.google.com ([2a00:1450:400c:c09::243]:35669) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1c4R2O-0005E5-20 for qemu-devel@nongnu.org; Wed, 09 Nov 2016 06:27:28 -0500 Received: by mail-wm0-x243.google.com with SMTP id a20so17798166wme.2 for ; Wed, 09 Nov 2016 03:27:27 -0800 (PST) Date: Wed, 9 Nov 2016 11:27:24 +0000 From: Stefan Hajnoczi Message-ID: <20161109112724.GC4682@stefanha-x1.localdomain> References: <95e79bc8-4547-b3b1-65b7-f641eb0c92f7@pobox.com> <20161104111419.GG9817@stefanha-x1.localdomain> <20161106180401.GE27308@var.home> <20161107104245.GC5036@stefanha-x1.localdomain> <466003bb-a2c4-bb9b-7b0b-7b2d6dcb16d7@pobox.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="L6iaP+gRLNZHKoI4" Content-Disposition: inline In-Reply-To: <466003bb-a2c4-bb9b-7b0b-7b2d6dcb16d7@pobox.com> Subject: Re: [Qemu-devel] Crashing in tcp_close List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Brian Candler Cc: Samuel Thibault , qemu-devel@nongnu.org, Jan Kiszka --L6iaP+gRLNZHKoI4 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Nov 08, 2016 at 09:22:25PM +0000, Brian Candler wrote: > On 07/11/2016 10:42, Stefan Hajnoczi wrote: > > On Mon, Nov 07, 2016 at 08:42:17AM +0000, Brian Candler wrote: > > > >On 06/11/2016 18:04, Samuel Thibault wrote: > > > > > >Brian, could you run it with > > > > > > > > > > > >export MALLOC_CHECK_=3D2 > > > > > > > > > > > >and also this could be useful: > > > > > > > > > > > >export MALLOC_PERTURB_=3D1234 > > > > > > > > > > > >Also, to rule out the double-free scenario, and try to catch a b= uffer > > > > > >overflow coming from the socket structure itself, I have attache= d a > > > > > >patch which adds some debugging. > > > > > > > >Thanks. I've added the patch, and re-run the stress test. >=20 > Back to the original setup, I can still get dumps. I notice I'm now getti= ng > "malloc_printerr" in the backtrace, but unfortunately I don't get to see = the > actual error message. It would seem that the malloc_check is being done a= nd > finding an issue. I haven't been able to get one in tcp_close again thou= gh > :-( Heap corruption. Valgrind's memcheck tool could be fruitful here: http://valgrind.org/docs/manual/quick-start.html#quick-start.mcrun Stefan > Regards, >=20 > Brian. >=20 > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". > Core was generated by `/usr/local/bin/qemu-system-x86_64 -m 4G -machine > type=3Dpc,accel=3Dkvm -device virt'. > Program terminated with signal SIGABRT, Aborted. > #0 0x00007eff4f3df428 in __GI_raise (sig=3Dsig@entry=3D6) at > ../sysdeps/unix/sysv/linux/raise.c:54 > 54 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. > [Current thread is 1 (Thread 0x7eff50dffa80 (LWP 13616))] > (gdb) bt > #0 0x00007eff4f3df428 in __GI_raise (sig=3Dsig@entry=3D6) at > ../sysdeps/unix/sysv/linux/raise.c:54 > #1 0x00007eff4f3e102a in __GI_abort () at abort.c:89 > #2 0x00007eff4f42bc1f in malloc_printerr (ar_ptr=3D, > ptr=3D, str=3D, > action=3D) at malloc.c:5008 > #3 _int_malloc (av=3Dav@entry=3D0x7eff4f76db20 , > bytes=3Dbytes@entry=3D89) at malloc.c:3384 > #4 0x00007eff4f42c409 in malloc_check (sz=3D88, caller=3D= ) at > hooks.c:295 > #5 0x00007eff50106729 in g_malloc () from > /lib/x86_64-linux-gnu/libglib-2.0.so.0 > #6 0x0000563ca16930cf in qemu_aio_get > (aiocb_info=3Daiocb_info@entry=3D0x563ca1841190 , > bs=3D0x563ca4132f40, cb=3Dcb@entry=3D0x563ca14d85e0 , > opaque=3Dopaque@entry=3D0x563ca5b78910) > at /home/nsrc/qemu-2.7.0/block/io.c:2231 > #7 0x0000563ca1687aa8 in blk_aio_get (opaque=3D0x563ca5b78910, > cb=3D0x563ca14d85e0 , blk=3D0x563ca4132d70, > aiocb_info=3D0x563ca1841190 ) at > /home/nsrc/qemu-2.7.0/block/block-backend.c:1477 > #8 blk_aio_prwv (blk=3D0x563ca4132d70, offset=3D5278244864, bytes=3D4096, > qiov=3D0x563ca5b78968, > co_entry=3Dco_entry@entry=3D0x563ca16872b0 , fla= gs=3D0, > cb=3D0x563ca14d85e0 , > opaque=3D0x563ca5b78910) at > /home/nsrc/qemu-2.7.0/block/block-backend.c:941 > #9 0x0000563ca1687bc0 in blk_aio_pwritev (blk=3D, > offset=3D, qiov=3D, > flags=3D, cb=3D, opaque=3D) > at /home/nsrc/qemu-2.7.0/block/block-backend.c:1054 > #10 0x0000563ca14d8718 in dma_blk_cb (opaque=3D0x563ca5b78910, ret=3D out>) > at /home/nsrc/qemu-2.7.0/dma-helpers.c:167 > #11 0x0000563ca14d8bf8 in dma_blk_io (ctx=3D0x563ca41184a0, > sg=3Dsg@entry=3D0x563ca59a08f0, > offset=3Doffset@entry=3D5278244864, io_func=3Dio_func@entry=3D0x563ca= 15a58e0 > , > io_func_opaque=3Dio_func_opaque@entry=3D0x563ca5c8a350, > cb=3Dcb@entry=3D0x563ca15a7250 , > opaque=3D0x563ca5c8a350, dir=3DDMA_DIRECTION_TO_DEVICE) at > /home/nsrc/qemu-2.7.0/dma-helpers.c:222 > #12 0x0000563ca15a764e in scsi_write_data (req=3D0x563ca5c8a350) at > /home/nsrc/qemu-2.7.0/hw/scsi/scsi-disk.c:540 > #13 0x0000563ca15ac743 in scsi_req_continue (req=3Dreq@entry=3D0x563ca5c8= a350) > at /home/nsrc/qemu-2.7.0/hw/scsi/scsi-bus.c:1680 > #14 0x0000563ca14381a2 in virtio_scsi_handle_cmd_req_submit > (s=3D0x563ca5abc1d0, req=3D) > at /home/nsrc/qemu-2.7.0/hw/scsi/virtio-scsi.c:565 > #15 virtio_scsi_handle_cmd_vq (s=3D0x563ca5abc1d0, vq=3D0x7eff4963f110) > at /home/nsrc/qemu-2.7.0/hw/scsi/virtio-scsi.c:583 > #16 0x0000563ca144a0d6 in virtio_queue_notify_vq (vq=3D0x7eff4963f110) > ---Type to continue, or q to quit--- > at /home/nsrc/qemu-2.7.0/hw/virtio/virtio.c:1113 > #17 0x0000563ca1654965 in aio_dispatch (ctx=3D0x563ca41184a0) at > /home/nsrc/qemu-2.7.0/aio-posix.c:330 > #18 0x0000563ca164a3ae in aio_ctx_dispatch (source=3D, > callback=3D, > user_data=3D) at /home/nsrc/qemu-2.7.0/async.c:234 > #19 0x00007eff501011a7 in g_main_context_dispatch () from > /lib/x86_64-linux-gnu/libglib-2.0.so.0 > #20 0x0000563ca16531db in glib_pollfds_poll () at > /home/nsrc/qemu-2.7.0/main-loop.c:213 > #21 os_host_main_loop_wait (timeout=3D) at > /home/nsrc/qemu-2.7.0/main-loop.c:258 > #22 main_loop_wait (nonblocking=3D) at > /home/nsrc/qemu-2.7.0/main-loop.c:506 > #23 0x0000563ca13be431 in main_loop () at /home/nsrc/qemu-2.7.0/vl.c:1908 > #24 main (argc=3D, argv=3D, envp=3D) > at /home/nsrc/qemu-2.7.0/vl.c:4604 > (gdb) >=20 --L6iaP+gRLNZHKoI4 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJYIwgcAAoJEJykq7OBq3PIC1oH/1iftvyVSL5/sDuLuFBKPneM Ju+wujf8o3SLtgpRET7UXJEUSFB8EXAFeBM38brroVENRxi1dmRgGcDIeXCiAN1w pP9mVUgLQ55Os4SBl0aHNSpqWxgGUPjNnUbloSf235pN8aJAa/oBHgPLvxi88PUV AzpI93We0CFxr0JSy6iS1MkrIhQ5srozVFBehOQshHSGFk/KlhyBRzKiik2Lx2sY 1BPu7TL9/+IBJ3by0P6JpR0Fu+sIAK3GKQhv2hvowJuiAT07NCEZi6ozGlS/Wjr+ l7VgLibD3EKUo7g4fZFD6YxDvDhSrRiEVeA1KGUeyA90D7ktGfgN30+UeiVft9w= =Kgw4 -----END PGP SIGNATURE----- --L6iaP+gRLNZHKoI4--