From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1NuuJi-0007N8-R6 for qemu-devel@nongnu.org; Thu, 25 Mar 2010 17:10:14 -0400 Received: from [140.186.70.92] (port=53934 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1NuuJh-0007LE-3c for qemu-devel@nongnu.org; Thu, 25 Mar 2010 17:10:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1NuuJd-0002UJ-G4 for qemu-devel@nongnu.org; Thu, 25 Mar 2010 17:10:11 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41724) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1NuuJd-0002U6-8I for qemu-devel@nongnu.org; Thu, 25 Mar 2010 17:10:09 -0400 Message-ID: <4BABD12B.3070909@redhat.com> Date: Thu, 25 Mar 2010 23:10:03 +0200 From: Avi Kivity MIME-Version: 1.0 References: <1269497310-21858-1-git-send-email-cam@cs.ualberta.ca> <4BAB2736.7020202@redhat.com> <8286e4ee1003250950l45cc2883yd4788d20f99ef86c@mail.gmail.com> <4BAB9718.3030808@redhat.com> <8286e4ee1003251035o75fed405j45b60d496afa66b5@mail.gmail.com> <4BABA1F4.3000801@redhat.com> <8286e4ee1003251117o74486dck813a47cee54b2d6d@mail.gmail.com> In-Reply-To: <8286e4ee1003251117o74486dck813a47cee54b2d6d@mail.gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: [Qemu-devel] Re: [PATCH v3 0/2] Inter-VM shared memory PCI device List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Cam Macdonell Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org On 03/25/2010 08:17 PM, Cam Macdonell wrote: > >>> I had a hunch it was probably considered. That explains why irqfd >>> doesn't have a datamatch field. I guess supporting multiple MSI >>> vectors with one doorbell per guest isn't possible if one 1 bit of >>> information can be communicated. >>> >>> >> Actually you can have one doorbell supporting multiple vectors and guests, >> simply divide the data value into two bit fields, one for the vector and one >> for the guest. A single write gets both values into the host, which can >> then use datamatch to trigger the correct eventfd (which is wired to an >> irqfd in another guest). >> > At 4-bits per guest, a single write is then limited to 8 guests (with > 32-bit registers), we could got to 64-bit. > I meant a unicast doorbell: 16 bits for guest ID, 16 bits for vector number. >> >>> So, ioeventfd/irqfd restricts MSI to 1 vector between guests. Should >>> multi-MSI even be supported then in the non-ioeventfd/irq case? >>> Otherwise ioeventfd/irqfd become more than an implementation detail. >>> >>> >> I lost you. Please re-explain. >> > An irqfd can only trigger a single vector in a guest. Right now I > only have one eventfd per guest. So ioeventfd/irqfd restricts the > current implementation to a single vector that a guest can trigger. > Without irqfd, eventfds can be used like registers a write the number > of the vector they want to trigger, but as you point out it is racy. > You can't use eventfds as registers. The next write will add to the current value. > So, supporting multiple vectors via irqfd requires multiple eventfds > for each guest (one per vector). a total of (# of guests) X (# of > vectors) are required. If we're limited to 8 or 16 guests that's not > too bad, but since the server opens them all we're restricted to 1024, > but that's a pretty high ceiling for this purpose. > I'm sure we can raise the fd ulimit for this. Note, I think qemus need the ulimit raised as well, since an fd passed via SCM_RIGHTS probably counts as an open file. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic.