From mboxrd@z Thu Jan 1 00:00:00 1970 From: Rusty Russell Subject: Re: Using virtio for inter-VM communication Date: Thu, 12 Jun 2014 11:57:07 +0930 Message-ID: <87r42uq2v8.fsf@rustcorp.com.au> References: <20140610184818.2e490419@nbschild1> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20140610184818.2e490419@nbschild1> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org To: qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org Cc: Henning Schild List-Id: virtualization@lists.linuxfoundation.org Henning Schild writes: > Hi, > > i am working on the jailhouse[1] project and am currently looking at > inter-VM communication. We want to connect guests directly with virtual > consoles based on shared memory. The code complexity in the hypervisor > should be minimal, it should just make the shared memory discoverable > and provide a signaling mechanism. Hi Henning, The virtio assumption was that the host can see all of guest memory. This simplifies things significantly, and makes it efficient. If you don't have this, *someone* needs to do a copy. Usually the guest OS does a bounce buffer into your shared region. Goodbye performance. Or you can play remapping tricks. Goodbye performance again. My preferred model is to have a trusted helper (ie. host) which understands how to copy between virtio rings. The backend guest (to steal Xen vocab) R/O maps the descriptor, avail ring and used rings in the guest. It then asks the trusted helper to do various operation (copy into writable descriptor, copy out of readable descriptor, mark used). The virtio ring itself acts as a grant table. Note: that helper mechanism is completely protocol agnostic. It was also explicitly designed into the virtio mechanism (with its 4k boundaries for data structures and its 'len' field to indicate how much was written into the descriptor). It was also never implemented, and remains a thought experiment. However, implementing it in lguest should be fairly easy. Cheers, Rusty. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Rusty Russell Subject: Re: Using virtio for inter-VM communication Date: Thu, 12 Jun 2014 11:57:07 +0930 Message-ID: <87r42uq2v8.fsf@rustcorp.com.au> References: <20140610184818.2e490419@nbschild1> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: Henning Schild To: Henning Schild , qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org Return-path: In-Reply-To: <20140610184818.2e490419@nbschild1> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org List-Id: kvm.vger.kernel.org Henning Schild writes: > Hi, > > i am working on the jailhouse[1] project and am currently looking at > inter-VM communication. We want to connect guests directly with virtual > consoles based on shared memory. The code complexity in the hypervisor > should be minimal, it should just make the shared memory discoverable > and provide a signaling mechanism. Hi Henning, The virtio assumption was that the host can see all of guest memory. This simplifies things significantly, and makes it efficient. If you don't have this, *someone* needs to do a copy. Usually the guest OS does a bounce buffer into your shared region. Goodbye performance. Or you can play remapping tricks. Goodbye performance again. My preferred model is to have a trusted helper (ie. host) which understands how to copy between virtio rings. The backend guest (to steal Xen vocab) R/O maps the descriptor, avail ring and used rings in the guest. It then asks the trusted helper to do various operation (copy into writable descriptor, copy out of readable descriptor, mark used). The virtio ring itself acts as a grant table. Note: that helper mechanism is completely protocol agnostic. It was also explicitly designed into the virtio mechanism (with its 4k boundaries for data structures and its 'len' field to indicate how much was written into the descriptor). It was also never implemented, and remains a thought experiment. However, implementing it in lguest should be fairly easy. Cheers, Rusty. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32788) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WuvNg-0007OY-13 for qemu-devel@nongnu.org; Wed, 11 Jun 2014 23:08:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WuvNb-0005V5-1h for qemu-devel@nongnu.org; Wed, 11 Jun 2014 23:08:47 -0400 Received: from ozlabs.org ([103.22.144.67]:38104) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WuvNa-0005Uk-MK for qemu-devel@nongnu.org; Wed, 11 Jun 2014 23:08:42 -0400 From: Rusty Russell In-Reply-To: <20140610184818.2e490419@nbschild1> References: <20140610184818.2e490419@nbschild1> Date: Thu, 12 Jun 2014 11:57:07 +0930 Message-ID: <87r42uq2v8.fsf@rustcorp.com.au> MIME-Version: 1.0 Content-Type: text/plain Subject: Re: [Qemu-devel] Using virtio for inter-VM communication List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Henning Schild , qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org Henning Schild writes: > Hi, > > i am working on the jailhouse[1] project and am currently looking at > inter-VM communication. We want to connect guests directly with virtual > consoles based on shared memory. The code complexity in the hypervisor > should be minimal, it should just make the shared memory discoverable > and provide a signaling mechanism. Hi Henning, The virtio assumption was that the host can see all of guest memory. This simplifies things significantly, and makes it efficient. If you don't have this, *someone* needs to do a copy. Usually the guest OS does a bounce buffer into your shared region. Goodbye performance. Or you can play remapping tricks. Goodbye performance again. My preferred model is to have a trusted helper (ie. host) which understands how to copy between virtio rings. The backend guest (to steal Xen vocab) R/O maps the descriptor, avail ring and used rings in the guest. It then asks the trusted helper to do various operation (copy into writable descriptor, copy out of readable descriptor, mark used). The virtio ring itself acts as a grant table. Note: that helper mechanism is completely protocol agnostic. It was also explicitly designed into the virtio mechanism (with its 4k boundaries for data structures and its 'len' field to indicate how much was written into the descriptor). It was also never implemented, and remains a thought experiment. However, implementing it in lguest should be fairly easy. Cheers, Rusty.