From mboxrd@z Thu Jan 1 00:00:00 1970 From: Anthony Liguori Subject: Re: [PATCH v5 4/5] Inter-VM shared memory PCI device Date: Mon, 10 May 2010 11:52:13 -0500 Message-ID: <4BE839BD.1060502@codemonkey.ws> References: <1271872408-22842-1-git-send-email-cam@cs.ualberta.ca> <1271872408-22842-2-git-send-email-cam@cs.ualberta.ca> <1271872408-22842-3-git-send-email-cam@cs.ualberta.ca> <1271872408-22842-4-git-send-email-cam@cs.ualberta.ca> <1271872408-22842-5-git-send-email-cam@cs.ualberta.ca> <4BE7F517.5010707@redhat.com> <4BE82623.4000905@redhat.com> <4BE82877.1040408@codemonkey.ws> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Avi Kivity , kvm@vger.kernel.org, qemu-devel@nongnu.org To: Cam Macdonell Return-path: Received: from mail-qy0-f183.google.com ([209.85.221.183]:52550 "EHLO mail-qy0-f183.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755526Ab0EJQwV (ORCPT ); Mon, 10 May 2010 12:52:21 -0400 Received: by qyk13 with SMTP id 13so6435447qyk.1 for ; Mon, 10 May 2010 09:52:20 -0700 (PDT) In-Reply-To: Sender: kvm-owner@vger.kernel.org List-ID: On 05/10/2010 11:20 AM, Cam Macdonell wrote: > On Mon, May 10, 2010 at 9:38 AM, Anthony Liguori wrote: > >> On 05/10/2010 10:28 AM, Avi Kivity wrote: >> >>> On 05/10/2010 06:22 PM, Cam Macdonell wrote: >>> >>>> >>>>> >>>>>> + >>>>>> + /* if the position is -1, then it's shared memory region fd */ >>>>>> + if (incoming_posn == -1) { >>>>>> + >>>>>> + s->num_eventfds = 0; >>>>>> + >>>>>> + if (check_shm_size(s, incoming_fd) == -1) { >>>>>> + exit(-1); >>>>>> + } >>>>>> + >>>>>> + /* creating a BAR in qemu_chr callback may be crazy */ >>>>>> + create_shared_memory_BAR(s, incoming_fd); >>>>>> >>>>>> >>>>> It probably is... why can't you create it during initialization? >>>>> >>>> This is for the shared memory server implementation, so the fd for the >>>> shared memory has to be received (over the qemu char device) from the >>>> server before the BAR can be created via qemu_ram_mmap() which adds >>>> the necessary memory >>>> >>>> >>> >>> We could do the handshake during initialization. I'm worried that the >>> device will appear without the BAR, and strange things will happen. But the >>> chardev API is probably not geared for passing data during init. >>> >>> Anthony, any ideas? >>> >> Why can't we create the BAR with just normal RAM and then change it to a >> mmap()'d fd after initialization? This will be behavior would be important >> for live migration as it would let you quickly migrate preserving the memory >> contents without waiting for an external program to reconnect. >> >> Regards, >> >> Anthony Lioguori >> >> >>>> Otherwise, if the BAR is allocated during initialization, I would have >>>> to use MAP_FIXED to mmap the memory. This is what I did before the >>>> qemu_ram_mmap() function was added. >>>> >>> What would happen to any data written to the BAR before the the handshake >>> completed? I think it would disappear. >>> >> You don't have to do MAP_FIXED. You can allocate a ram area and map that in >> when disconnected. When you connect, you create another ram area and >> memcpy() the previous ram area to the new one. You then map the second ram >> area in. >> > the memcpy() would overwrite the contents of the shared memory each > time a guest joins which would be dangerous. > I think those are reasonable semantics and is really the only way to get guest-transparent reconnect. The later is pretty critical for guest transparent live migration. >> From the guest's perspective, it's totally transparent. For the backend, >> I'd suggest having an explicit "initialized" ack or something so that it >> knows that the data is now mapped to the guest. >> > Yes, I think the ack is the way to go, so the guest has to be aware of > it. Would setting a flag in the driver-specific config space be an > acceptable ack that the shared region is now mapped? > You know it's mapped because it's mapped when the pci map function returns. You don't need the guest to explicitly tell you. Regards, Anthony Liguori > Cam >