Re: [Qemu-devel] Crashing in tcp_close

[Stefan Hajnoczi <stefanha@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 5571 bytes --]

On Tue, Nov 08, 2016 at 09:22:25PM +0000, Brian Candler wrote:
> On 07/11/2016 10:42, Stefan Hajnoczi wrote:
> > On Mon, Nov 07, 2016 at 08:42:17AM +0000, Brian Candler wrote:
> > > >On 06/11/2016 18:04, Samuel Thibault wrote:
> > > > > >Brian, could you run it with
> > > > > >
> > > > > >export MALLOC_CHECK_=2
> > > > > >
> > > > > >and also this could be useful:
> > > > > >
> > > > > >export MALLOC_PERTURB_=1234
> > > > > >
> > > > > >Also, to rule out the double-free scenario, and try to catch a buffer
> > > > > >overflow coming from the socket structure itself, I have attached a
> > > > > >patch which adds some debugging.
> > > >
> > > >Thanks. I've added the patch, and re-run the stress test.
> 
> Back to the original setup, I can still get dumps. I notice I'm now getting
> "malloc_printerr" in the backtrace, but unfortunately I don't get to see the
> actual error message. It would seem that the malloc_check is being done and
> finding an issue.  I haven't been able to get one in tcp_close again though
> :-(

Heap corruption.  Valgrind's memcheck tool could be fruitful here:

http://valgrind.org/docs/manual/quick-start.html#quick-start.mcrun

Stefan

> Regards,
> 
> Brian.
> 
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
> Core was generated by `/usr/local/bin/qemu-system-x86_64 -m 4G -machine
> type=pc,accel=kvm -device virt'.
> Program terminated with signal SIGABRT, Aborted.
> #0  0x00007eff4f3df428 in __GI_raise (sig=sig@entry=6) at
> ../sysdeps/unix/sysv/linux/raise.c:54
> 54    ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
> [Current thread is 1 (Thread 0x7eff50dffa80 (LWP 13616))]
> (gdb) bt
> #0  0x00007eff4f3df428 in __GI_raise (sig=sig@entry=6) at
> ../sysdeps/unix/sysv/linux/raise.c:54
> #1  0x00007eff4f3e102a in __GI_abort () at abort.c:89
> #2  0x00007eff4f42bc1f in malloc_printerr (ar_ptr=<optimised out>,
> ptr=<optimised out>, str=<optimised out>,
>     action=<optimised out>) at malloc.c:5008
> #3  _int_malloc (av=av@entry=0x7eff4f76db20 <main_arena>,
> bytes=bytes@entry=89) at malloc.c:3384
> #4  0x00007eff4f42c409 in malloc_check (sz=88, caller=<optimised out>) at
> hooks.c:295
> #5  0x00007eff50106729 in g_malloc () from
> /lib/x86_64-linux-gnu/libglib-2.0.so.0
> #6  0x0000563ca16930cf in qemu_aio_get
> (aiocb_info=aiocb_info@entry=0x563ca1841190 <blk_aio_em_aiocb_info>,
>     bs=0x563ca4132f40, cb=cb@entry=0x563ca14d85e0 <dma_blk_cb>,
> opaque=opaque@entry=0x563ca5b78910)
>     at /home/nsrc/qemu-2.7.0/block/io.c:2231
> #7  0x0000563ca1687aa8 in blk_aio_get (opaque=0x563ca5b78910,
> cb=0x563ca14d85e0 <dma_blk_cb>, blk=0x563ca4132d70,
>     aiocb_info=0x563ca1841190 <blk_aio_em_aiocb_info>) at
> /home/nsrc/qemu-2.7.0/block/block-backend.c:1477
> #8  blk_aio_prwv (blk=0x563ca4132d70, offset=5278244864, bytes=4096,
> qiov=0x563ca5b78968,
>     co_entry=co_entry@entry=0x563ca16872b0 <blk_aio_write_entry>, flags=0,
> cb=0x563ca14d85e0 <dma_blk_cb>,
>     opaque=0x563ca5b78910) at
> /home/nsrc/qemu-2.7.0/block/block-backend.c:941
> #9  0x0000563ca1687bc0 in blk_aio_pwritev (blk=<optimised out>,
> offset=<optimised out>, qiov=<optimised out>,
>     flags=<optimised out>, cb=<optimised out>, opaque=<optimised out>)
>     at /home/nsrc/qemu-2.7.0/block/block-backend.c:1054
> #10 0x0000563ca14d8718 in dma_blk_cb (opaque=0x563ca5b78910, ret=<optimised
> out>)
>     at /home/nsrc/qemu-2.7.0/dma-helpers.c:167
> #11 0x0000563ca14d8bf8 in dma_blk_io (ctx=0x563ca41184a0,
> sg=sg@entry=0x563ca59a08f0,
>     offset=offset@entry=5278244864, io_func=io_func@entry=0x563ca15a58e0
> <scsi_dma_writev>,
>     io_func_opaque=io_func_opaque@entry=0x563ca5c8a350,
> cb=cb@entry=0x563ca15a7250 <scsi_dma_complete>,
>     opaque=0x563ca5c8a350, dir=DMA_DIRECTION_TO_DEVICE) at
> /home/nsrc/qemu-2.7.0/dma-helpers.c:222
> #12 0x0000563ca15a764e in scsi_write_data (req=0x563ca5c8a350) at
> /home/nsrc/qemu-2.7.0/hw/scsi/scsi-disk.c:540
> #13 0x0000563ca15ac743 in scsi_req_continue (req=req@entry=0x563ca5c8a350)
>     at /home/nsrc/qemu-2.7.0/hw/scsi/scsi-bus.c:1680
> #14 0x0000563ca14381a2 in virtio_scsi_handle_cmd_req_submit
> (s=0x563ca5abc1d0, req=<optimised out>)
>     at /home/nsrc/qemu-2.7.0/hw/scsi/virtio-scsi.c:565
> #15 virtio_scsi_handle_cmd_vq (s=0x563ca5abc1d0, vq=0x7eff4963f110)
>     at /home/nsrc/qemu-2.7.0/hw/scsi/virtio-scsi.c:583
> #16 0x0000563ca144a0d6 in virtio_queue_notify_vq (vq=0x7eff4963f110)
> ---Type <return> to continue, or q <return> to quit---
>     at /home/nsrc/qemu-2.7.0/hw/virtio/virtio.c:1113
> #17 0x0000563ca1654965 in aio_dispatch (ctx=0x563ca41184a0) at
> /home/nsrc/qemu-2.7.0/aio-posix.c:330
> #18 0x0000563ca164a3ae in aio_ctx_dispatch (source=<optimised out>,
> callback=<optimised out>,
>     user_data=<optimised out>) at /home/nsrc/qemu-2.7.0/async.c:234
> #19 0x00007eff501011a7 in g_main_context_dispatch () from
> /lib/x86_64-linux-gnu/libglib-2.0.so.0
> #20 0x0000563ca16531db in glib_pollfds_poll () at
> /home/nsrc/qemu-2.7.0/main-loop.c:213
> #21 os_host_main_loop_wait (timeout=<optimised out>) at
> /home/nsrc/qemu-2.7.0/main-loop.c:258
> #22 main_loop_wait (nonblocking=<optimised out>) at
> /home/nsrc/qemu-2.7.0/main-loop.c:506
> #23 0x0000563ca13be431 in main_loop () at /home/nsrc/qemu-2.7.0/vl.c:1908
> #24 main (argc=<optimised out>, argv=<optimised out>, envp=<optimised out>)
> at /home/nsrc/qemu-2.7.0/vl.c:4604
> (gdb)
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]