[Qemu-devel] [Bug] qemu_coroutine_enter abort and report error "Co-routine re-entered recursively"

[Qemu-devel] [Bug] qemu_coroutine_enter abort and report error "Co-routine re-entered recursively"

[Halsey Pian]

[-- Attachment #1: Type: text/plain, Size: 2074 bytes --]

Hi All,

 

I have two threads to write two seperate qcow2 files,  but after a while,  the writing would be aborted in qemu_coroutine_enter, and report error “"Co-routine re-entered recursively” .

 

Qemu should be thread safe, right? It seems that there are some variables is not thread safe? Could you have a chance to look it? Thanks!

 

Call stack:

 

#0 0x7ffff5e18989            __GI_raise(sig=sig@entry=6) (../nptl/sysdeps/unix/sysv/linux/raise.c:56)

#1 0x7ffff5e1a098            __GI_abort() (abort.c:90)

#2 0x7ffff728c034            qemu_coroutine_enter(co=0x7fffe0004800, opaque=0x0) (qemu-coroutine.c:117)

#3 0x7ffff727df39            bdrv_co_io_em_complete(opaque=0x7ffff7fd6ae0, ret=0) (block.c:4847)

#4 0x7ffff7270314            thread_pool_completion_bh(opaque=0x7fffe0006ad0) (thread-pool.c:187)

#5 0x7ffff726f873             aio_bh_poll(ctx=0x7fffe0001d00) (async.c:82)

#6 0x7ffff728340b            aio_dispatch(ctx=0x7fffe0001d00) (aio-posix.c:137)

#7 0x7ffff72837b0            aio_poll(ctx=0x7fffe0001d00, blocking=true) (aio-posix.c:248)

#8 ??     0x00007ffff72795a8 in bdrv_prwv_co (bs=0x7fffdc0021c0, offset=12071639552, qiov=0x7fffe67fa590, is_write=true, flags=(unknown: 0)) (block.c:2703)

#9 ??     0x00007ffff727966a in bdrv_rw_co (bs=0x7fffdc0021c0, sector_num=23577421, buf=0x7fffe4629250 "\234\b\335Ǽ\254\213q\301\366\315=\005oI\301\245=\373\004+2?H\212\025\035+\262\274C;X\301FaP\324\335\061ҝ&Y\316=\347\335\020\365\003goɿ\214\312S=\v2]\373\363C\311\341\334\r5k\346k\204\332\023\264\315陌\230\203J\222u\214\066", nb_sectors=128, is_write=true, flags=(unknown: 0)) (block.c:2726)

#10 0x7ffff7279758          bdrv_write(bs=0x7fffdc0021c0, sector_num=23577421, buf=0x7fffe4629250 "\234\b\335Ǽ\254\213q\301\366\315=\005oI\301\245=\373\004+2?H\212\025\035+\262\274C;X\301FaP\324\335\061ҝ&Y\316=\347\335\020\365\003goɿ\214\312S=\v2]\373\363C\311\341\334\r5k\346k\204\332\023\264\315陌\230\203J\222u\214\066", nb_sectors=128) (block.c:2760)

 

 

Best Regards

Halsey Pian

 


[-- Attachment #2: Type: text/html, Size: 6000 bytes --]

Re: [Qemu-devel] [Bug] qemu_coroutine_enter abort and report error "Co-routine re-entered recursively"

[Halsey Pian]

[-- Attachment #1: Type: text/plain, Size: 2394 bytes --]

 

Qemu version: qemu-2.2.0 release

Platform: x86_64

 

 

From: Halsey Pian [mailto:halsey.pian@gmail.com] 
Sent: 2015年3月6日 15:04
To: qemu-devel@nongnu.org
Cc: halsey.pian@gmail.com
Subject: [Qemu-devel][Bug] qemu_coroutine_enter abort and report error "Co-routine re-entered recursively"

 

Hi All,

 

I have two threads to write two seperate qcow2 files,  but after a while,  the writing would be aborted in qemu_coroutine_enter, and report error “"Co-routine re-entered recursively” .

 

Qemu should be thread safe, right? It seems that there are some variables is not thread safe? Could you have a chance to look it? Thanks!

 

Call stack:

 

#0 0x7ffff5e18989            __GI_raise(sig=sig@entry=6) (../nptl/sysdeps/unix/sysv/linux/raise.c:56)

#1 0x7ffff5e1a098            __GI_abort() (abort.c:90)

#2 0x7ffff728c034            qemu_coroutine_enter(co=0x7fffe0004800, opaque=0x0) (qemu-coroutine.c:117)

#3 0x7ffff727df39            bdrv_co_io_em_complete(opaque=0x7ffff7fd6ae0, ret=0) (block.c:4847)

#4 0x7ffff7270314            thread_pool_completion_bh(opaque=0x7fffe0006ad0) (thread-pool.c:187)

#5 0x7ffff726f873             aio_bh_poll(ctx=0x7fffe0001d00) (async.c:82)

#6 0x7ffff728340b            aio_dispatch(ctx=0x7fffe0001d00) (aio-posix.c:137)

#7 0x7ffff72837b0            aio_poll(ctx=0x7fffe0001d00, blocking=true) (aio-posix.c:248)

#8 ??     0x00007ffff72795a8 in bdrv_prwv_co (bs=0x7fffdc0021c0, offset=12071639552, qiov=0x7fffe67fa590, is_write=true, flags=(unknown: 0)) (block.c:2703)

#9 ??     0x00007ffff727966a in bdrv_rw_co (bs=0x7fffdc0021c0, sector_num=23577421, buf=0x7fffe4629250 "\234\b\335Ǽ\254\213q\301\366\315=\005oI\301\245=\373\004+2?H\212\025\035+\262\274C;X\301FaP\324\335\061ҝ&Y\316=\347\335\020\365\003goɿ\214\312S=\v2]\373\363C\311\341\334\r5k\346k\204\332\023\264\315陌\230\203J\222u\214\066", nb_sectors=128, is_write=true, flags=(unknown: 0)) (block.c:2726)

#10 0x7ffff7279758          bdrv_write(bs=0x7fffdc0021c0, sector_num=23577421, buf=0x7fffe4629250 "\234\b\335Ǽ\254\213q\301\366\315=\005oI\301\245=\373\004+2?H\212\025\035+\262\274C;X\301FaP\324\335\061ҝ&Y\316=\347\335\020\365\003goɿ\214\312S=\v2]\373\363C\311\341\334\r5k\346k\204\332\023\264\315陌\230\203J\222u\214\066", nb_sectors=128) (block.c:2760)

 

 

Best Regards

Halsey Pian

 


[-- Attachment #2: Type: text/html, Size: 8111 bytes --]

Re: [Qemu-devel] [Bug] qemu_coroutine_enter abort and report error "Co-routine re-entered recursively"

[Paolo Bonzini]

On 06/03/2015 08:03, Halsey Pian wrote:
> I have two threads to write two seperate qcow2 files,  but after a
> while,  the writing would be aborted in qemu_coroutine_enter, and report
> error “"Co-routine re-entered recursively” .
> 
> Qemu should be thread safe, right? It seems that there are some
> variables is not thread safe? Could you have a chance to look it? Thanks!

QEMU is thread safe but you need to add explicit locking or use separate
event loops in each thread.  If you want to write from separate thread,
you need to do one of the following:

1) use one AioContext per file, and add an AioContext-based event loop
to each thread (see backends/iothread.c);

2) use one AioContext per file, add it (as a GSource) to a GMainContext
and use a GMainLoop-based event loop to each thread;

3) use aio_context_acquire and aio_context_release around each blk_* or
bdrv_* call.

Paolo

Re: [Qemu-devel] [Bug] qemu_coroutine_enter abort and report error "Co-routine re-entered recursively"

[Halsey Pian]

> -----Original Message-----
> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
> Sent: 2015年3月6日 17:45
> To: Halsey Pian; qemu-devel@nongnu.org
> Subject: Re: [Bug] qemu_coroutine_enter abort and report error "Co-routine re-entered recursively"
> 
> 
> 
> On 06/03/2015 08:03, Halsey Pian wrote:
> > I have two threads to write two seperate qcow2 files,  but after a
> > while,  the writing would be aborted in qemu_coroutine_enter, and
> > report error “"Co-routine re-entered recursively” .
> >
> > Qemu should be thread safe, right? It seems that there are some
> > variables is not thread safe? Could you have a chance to look it? Thanks!
> 
> QEMU is thread safe but you need to add explicit locking or use separate event loops in each thread.  If you want to write from
> separate thread, you need to do one of the following:
> 
> 1) use one AioContext per file, and add an AioContext-based event loop to each thread (see backends/iothread.c);
> 
> 2) use one AioContext per file, add it (as a GSource) to a GMainContext and use a GMainLoop-based event loop to each thread;
> 
> 3) use aio_context_acquire and aio_context_release around each blk_* or
> bdrv_* call.
> 
> Paolo

Hi Paolo,

Thanks for your immediate reply. 

I checked relevant source code, and I'm trying changing AioContext.  Would keep changing based on your comments.  Thank you!

[Halsey]