Re: [Qemu-devel] MTTCG sync-up call today?

["Alex Bennée" <alex.bennee@linaro.org>]

Paolo Bonzini <pbonzini@redhat.com> writes:

> On 12/01/2016 16:11, Alex Bennée wrote:
>> > Sorry for the late answer, I find some time to take a look at it.
>> >
>> > Seems you were right I fixed the exit issue and it seems it was one of
>> > the problem.
>> > I think we must double check how we use cpu->exit_request as Paolo
>> > removed SIG_IPI to exit the CPU.
>> >
>> > I found one additional issue and it seems booting well right now.
>>
>> The other thing that needs cleaning up is the tcg_current_cpu and
>> current_cpu. I suspect the former should go and the restrictions on the
>> later be loosend so the TLS current_cpu is available to deferred tasks.
>
> Yes, you can make TLS current_cpu always non-NULL for multi-threaded TCG.
>
> tcg_current_cpu definitely should go, it doesn't make sense if you have
> multiple threads.
>
>> The thing I'm currently looking at is what happens when something like a
>> virtio completes in a non-CPU thread.
>
> It should just work.  It will cause a cpu_interrupt under the BQL, and
> that sets cpu->interrupt_request.  The code that modifies
> cpu->interrupt_request in the VCPU thread also runs under the BQL.

Hmm I'm seeing a virtio co-routine kick of an unmap:

qemu-system-arm: /home/alex/lsrc/qemu/qemu.git/translate-all.c:1303: tb_invalidate_phys_range: Assertion `have_tb_lock' failed.

Program received signal SIGABRT, Aborted.
0x00007ffff0ae9cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56	../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007ffff0ae9cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff0aed0d8 in __GI_abort () at abort.c:89
#2  0x00007ffff0ae2b86 in __assert_fail_base (fmt=0x7ffff0c33830 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
    assertion=assertion@entry=0x555555a89665 "have_tb_lock", file=file@entry=0x555555a894f0 "/home/alex/lsrc/qemu/qemu.git/translate-all.c", line=line@entry=1303,
    function=function@entry=0x555555a897f0 <__PRETTY_FUNCTION__.33460> "tb_invalidate_phys_range") at assert.c:92
#3  0x00007ffff0ae2c32 in __GI___assert_fail (assertion=assertion@entry=0x555555a89665 "have_tb_lock",
    file=file@entry=0x555555a894f0 "/home/alex/lsrc/qemu/qemu.git/translate-all.c", line=line@entry=1303,
    function=function@entry=0x555555a897f0 <__PRETTY_FUNCTION__.33460> "tb_invalidate_phys_range") at assert.c:101
#4  0x00005555556e5b06 in tb_invalidate_phys_range (start=start@entry=0, end=end@entry=4096) at /home/alex/lsrc/qemu/qemu.git/translate-all.c:1303
#5  0x00005555556dbe42 in invalidate_and_set_dirty (mr=mr@entry=0x555556571800, addr=0, length=length@entry=4096) at /home/alex/lsrc/qemu/qemu.git/exec.c:2420
#6  0x00005555556e1890 in address_space_unmap (as=as@entry=0x555555ff7000 <address_space_memory>, buffer=<optimised out>, len=<optimised out>,
    is_write=is_write@entry=1, access_len=access_len@entry=4096) at /home/alex/lsrc/qemu/qemu.git/exec.c:2933
#7  0x00005555556e19bf in cpu_physical_memory_unmap (buffer=<optimised out>, len=<optimised out>, is_write=is_write@entry=1, access_len=access_len@entry=4096)
    at /home/alex/lsrc/qemu/qemu.git/exec.c:2962
#8  0x000055555578219c in virtqueue_unmap_sg (elem=elem@entry=0x7ffe782c7cf0, len=len@entry=4097, vq=0x555556e6f020)
    at /home/alex/lsrc/qemu/qemu.git/hw/virtio/virtio.c:257
#9  0x0000555555782ac0 in virtqueue_fill (vq=vq@entry=0x555556e6f020, elem=elem@entry=0x7ffe782c7cf0, len=4097, idx=idx@entry=0)
    at /home/alex/lsrc/qemu/qemu.git/hw/virtio/virtio.c:282
#10 0x0000555555782ccf in virtqueue_push (vq=0x555556e6f020, elem=elem@entry=0x7ffe782c7cf0, len=<optimised out>)
    at /home/alex/lsrc/qemu/qemu.git/hw/virtio/virtio.c:308
#11 0x000055555573451a in virtio_blk_complete_request (req=0x7ffe782c7ce0, status=<optimised out>) at /home/alex/lsrc/qemu/qemu.git/hw/block/virtio-blk.c:58
#12 0x0000555555734a13 in virtio_blk_req_complete (status=0 '\000', req=0x7ffe782c7ce0) at /home/alex/lsrc/qemu/qemu.git/hw/block/virtio-blk.c:64
#13 virtio_blk_rw_complete (opaque=<optimised out>, ret=0) at /home/alex/lsrc/qemu/qemu.git/hw/block/virtio-blk.c:122
---Type <return> to continue, or q <return> to quit---
#14 0x0000555555a2d822 in bdrv_co_complete (acb=0x7ffe780189c0) at block/io.c:2122
#15 0x0000555555a87a7a in coroutine_trampoline (i0=<optimised out>, i1=<optimised out>) at util/coroutine-ucontext.c:80
#16 0x00007ffff0afc8b0 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#17 0x00007fff8f5aa6e0 in ?? ()
#18 0x0000000000000000 in ?? ()

I guess the tb_lock could just be grabbed but there is stuff in that
path that assumes current_cpu is valid so I thought the thing to do was
defer the operation until a "real" vCPU can deal with it.



>
> Paolo


--
Alex Bennée