From mboxrd@z Thu Jan 1 00:00:00 1970 From: Avi Kivity Subject: Re: [PATCH v3 0/2] Inter-VM shared memory PCI device Date: Thu, 25 Mar 2010 23:10:03 +0200 Message-ID: <4BABD12B.3070909@redhat.com> References: <1269497310-21858-1-git-send-email-cam@cs.ualberta.ca> <4BAB2736.7020202@redhat.com> <8286e4ee1003250950l45cc2883yd4788d20f99ef86c@mail.gmail.com> <4BAB9718.3030808@redhat.com> <8286e4ee1003251035o75fed405j45b60d496afa66b5@mail.gmail.com> <4BABA1F4.3000801@redhat.com> <8286e4ee1003251117o74486dck813a47cee54b2d6d@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: kvm@vger.kernel.org, qemu-devel@nongnu.org To: Cam Macdonell Return-path: Received: from mx1.redhat.com ([209.132.183.28]:6177 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751691Ab0CYVKJ (ORCPT ); Thu, 25 Mar 2010 17:10:09 -0400 In-Reply-To: <8286e4ee1003251117o74486dck813a47cee54b2d6d@mail.gmail.com> Sender: kvm-owner@vger.kernel.org List-ID: On 03/25/2010 08:17 PM, Cam Macdonell wrote: > >>> I had a hunch it was probably considered. That explains why irqfd >>> doesn't have a datamatch field. I guess supporting multiple MSI >>> vectors with one doorbell per guest isn't possible if one 1 bit of >>> information can be communicated. >>> >>> >> Actually you can have one doorbell supporting multiple vectors and guests, >> simply divide the data value into two bit fields, one for the vector and one >> for the guest. A single write gets both values into the host, which can >> then use datamatch to trigger the correct eventfd (which is wired to an >> irqfd in another guest). >> > At 4-bits per guest, a single write is then limited to 8 guests (with > 32-bit registers), we could got to 64-bit. > I meant a unicast doorbell: 16 bits for guest ID, 16 bits for vector number. >> >>> So, ioeventfd/irqfd restricts MSI to 1 vector between guests. Should >>> multi-MSI even be supported then in the non-ioeventfd/irq case? >>> Otherwise ioeventfd/irqfd become more than an implementation detail. >>> >>> >> I lost you. Please re-explain. >> > An irqfd can only trigger a single vector in a guest. Right now I > only have one eventfd per guest. So ioeventfd/irqfd restricts the > current implementation to a single vector that a guest can trigger. > Without irqfd, eventfds can be used like registers a write the number > of the vector they want to trigger, but as you point out it is racy. > You can't use eventfds as registers. The next write will add to the current value. > So, supporting multiple vectors via irqfd requires multiple eventfds > for each guest (one per vector). a total of (# of guests) X (# of > vectors) are required. If we're limited to 8 or 16 guests that's not > too bad, but since the server opens them all we're restricted to 1024, > but that's a pretty high ceiling for this purpose. > I'm sure we can raise the fd ulimit for this. Note, I think qemus need the ulimit raised as well, since an fd passed via SCM_RIGHTS probably counts as an open file. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic.