[Qemu-devel] Re: [PATCH v5 4/5] Inter-VM shared memory PCI device

[Anthony Liguori <anthony@codemonkey.ws>]

On 05/10/2010 10:28 AM, Avi Kivity wrote:
> On 05/10/2010 06:22 PM, Cam Macdonell wrote:
>>
>>>
>>>> +
>>>> +    /* if the position is -1, then it's shared memory region fd */
>>>> +    if (incoming_posn == -1) {
>>>> +
>>>> +        s->num_eventfds = 0;
>>>> +
>>>> +        if (check_shm_size(s, incoming_fd) == -1) {
>>>> +            exit(-1);
>>>> +        }
>>>> +
>>>> +        /* creating a BAR in qemu_chr callback may be crazy */
>>>> +        create_shared_memory_BAR(s, incoming_fd);
>>>>
>>> It probably is... why can't you create it during initialization?
>> This is for the shared memory server implementation, so the fd for the
>> shared memory has to be received (over the qemu char device) from the
>> server before the BAR can be created via qemu_ram_mmap() which adds
>> the necessary memory
>>
>
>
> We could do the handshake during initialization.  I'm worried that the 
> device will appear without the BAR, and strange things will happen.  
> But the chardev API is probably not geared for passing data during init.
>
> Anthony, any ideas?

Why can't we create the BAR with just normal RAM and then change it to a 
mmap()'d fd after initialization?  This will be behavior would be 
important for live migration as it would let you quickly migrate 
preserving the memory contents without waiting for an external program 
to reconnect.

Regards,

Anthony Lioguori

>> Otherwise, if the BAR is allocated during initialization, I would have
>> to use MAP_FIXED to mmap the memory.  This is what I did before the
>> qemu_ram_mmap() function was added.
>
> What would happen to any data written to the BAR before the the 
> handshake completed?  I think it would disappear.

You don't have to do MAP_FIXED.  You can allocate a ram area and map 
that in when disconnected.  When you connect, you create another ram 
area and memcpy() the previous ram area to the new one.  You then map 
the second ram area in.

 From the guest's perspective, it's totally transparent.  For the 
backend, I'd suggest having an explicit "initialized" ack or something 
so that it knows that the data is now mapped to the guest.

If you're doing just a ring queue in shared memory, it should allow 
disconnect/reconnect during live migration asynchronously to the actual 
qemu live migration.

Regards,

Anthony Liguori

> So it's a good idea to make the initialization process atomic.
>