Re: [Qemu-devel] virtqueue corruption in emulation mode?

[Stefan Hajnoczi <stefanha@gmail.com>]

On Wed, Sep 28, 2011 at 3:23 PM, Sinha, Ani <Ani.Sinha@tellabs.com> wrote:
>
> On Sep 28, 2011, at 1:51 AM, Stefan Hajnoczi wrote:
>
>> On Wed, Sep 28, 2011 at 3:01 AM, Sinha, Ani <Ani.Sinha@tellabs.com> wrote:
>>>
>>> On Sep 27, 2011, at 12:17 AM, Stefan Hajnoczi wrote:
>>>
>>>> On Mon, Sep 26, 2011 at 07:16:56PM -0500, Sinha, Ani wrote:
>>>>> I am using the virtqueue (virtqueue_pop, virtqueue_push etc) in the emulated mode (non-kvm mode) from an IO thread (a separate thread different from main QEMU thread). What I am observing is that the virtqueue memory seems to get corrupt. Either qemu crashes while performing virtqueue_push() (virtqueue_push() -> virtqueue_fill() ->bring_used_idx()->lduw_phys()->qemu_get_ram_ptr()->"bad ram offset") or crashes when the guest accesses a bad memory while using virtqueue. Now this never ever happens when I run QEMU in KVM mode (/dev/kvm present) OR when I use my functions from within the main qemu thread. I am unable to figure out why this is happening. I have looked into my code over and over again and I can't seem to explain this behavior. Can any of you guys give me any inkling?
>>>>
>>>> QEMU is not thread-safe in general.  It uses a big lock to protect most
>>>> of its internal state.
>>>
>>>
>>> I see. So may be I should do something like qemu_set_fd_handler(fd, …) where fd is a pipe and the handler does the virtqueue_push() etc?
>>> Now my question is, is it safe to do elem = virtqueue_pop(vq) from main event loop, then so some work on the elem popped out in an worker thread and then at some later point do a virtqueue_push(vq, elem) from that handler (which is called by main_loop() ->main_loop_wait())?  In other words, the vq reference is being used from the main event loop at two different points from two different functions but not in a contiguous fashion within the same function.
>>
>> Yes but do you need a helper thread?  Most of QEMU is based on
>> qemu_set_fd_handler() and callbacks, including for host network and
>> disk I/O.  If you follow the way QEMU does things it should be
>> easiest.
>
> I need a helper thread to do blocking IO. The device IOCTLS are inherently blocking, unfortunately.

posix-aio-compat.c already implements a threadpool.  It is geared
towards using the QEMU block layer (BlockDriverState) but the pure
ioctl codepath can be used without a BlockDriverState by passing NULL.
 So this could save you some effort, check out paio_submit().

Stefan