From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Nurfy-00082N-1W for qemu-devel@nongnu.org; Thu, 25 Mar 2010 14:21:02 -0400 Received: from [140.186.70.92] (port=42978 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Nurfw-0007xg-2m for qemu-devel@nongnu.org; Thu, 25 Mar 2010 14:21:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1Nurd3-0008Uy-PI for qemu-devel@nongnu.org; Thu, 25 Mar 2010 14:18:03 -0400 Received: from mail-pz0-f194.google.com ([209.85.222.194]:57969) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Nurd3-0008Ul-Jo for qemu-devel@nongnu.org; Thu, 25 Mar 2010 14:18:01 -0400 Received: by pzk32 with SMTP id 32so1928061pzk.4 for ; Thu, 25 Mar 2010 11:17:59 -0700 (PDT) MIME-Version: 1.0 Sender: camm@ualberta.ca In-Reply-To: <4BABA1F4.3000801@redhat.com> References: <1269497310-21858-1-git-send-email-cam@cs.ualberta.ca> <4BAB2736.7020202@redhat.com> <8286e4ee1003250950l45cc2883yd4788d20f99ef86c@mail.gmail.com> <4BAB9718.3030808@redhat.com> <8286e4ee1003251035o75fed405j45b60d496afa66b5@mail.gmail.com> <4BABA1F4.3000801@redhat.com> Date: Thu, 25 Mar 2010 12:17:56 -0600 Message-ID: <8286e4ee1003251117o74486dck813a47cee54b2d6d@mail.gmail.com> From: Cam Macdonell Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: [Qemu-devel] Re: [PATCH v3 0/2] Inter-VM shared memory PCI device List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Avi Kivity Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org On Thu, Mar 25, 2010 at 11:48 AM, Avi Kivity wrote: > On 03/25/2010 07:35 PM, Cam Macdonell wrote: >> >>> Ah, I see. =A0You adjusted for the different behaviours in the driver. >>> >>> Still I recommend dropping the status register: this allows single-msi >>> and >>> PIRQ to behave the same way. =A0Also it is racy, if two guests signal a >>> third, >>> they will overwrite each other's status. >>> >> >> With shared interrupts with PIRQ without a status register how does a >> device know it generated the interrupt? >> > > Right, you need a status register. =A0Just don't add any more information= , > since MSI cannot carry any data. Right. > >>> Eventfd values are a counter, not a register. =A0A read() on the other = side >>> returns the sum of all write()s (or eventfd_signal()s). =A0In the conte= xt >>> of >>> irqfd it just means the number of interrupts we coalesced. >>> >>> Multivalue was considered at one time for a different need and rejected= . >>> =A0Really, to solve the race you need a queue, and that can only be don= e in >>> the shared memory segment using locked instructions. >>> >> >> I had a hunch it was probably considered. =A0That explains why irqfd >> doesn't have a datamatch field. =A0I guess supporting multiple MSI >> vectors with one doorbell per guest isn't possible if one 1 bit of >> information can be communicated. >> > > Actually you can have one doorbell supporting multiple vectors and guests= , > simply divide the data value into two bit fields, one for the vector and = one > for the guest. =A0A single write gets both values into the host, which ca= n > then use datamatch to trigger the correct eventfd (which is wired to an > irqfd in another guest). At 4-bits per guest, a single write is then limited to 8 guests (with 32-bit registers), we could got to 64-bit. > >> So, ioeventfd/irqfd restricts MSI to 1 vector between guests. =A0Should >> multi-MSI even be supported then in the non-ioeventfd/irq case? >> Otherwise ioeventfd/irqfd become more than an implementation detail. >> > > I lost you. =A0Please re-explain. An irqfd can only trigger a single vector in a guest. Right now I only have one eventfd per guest. So ioeventfd/irqfd restricts the current implementation to a single vector that a guest can trigger. Without irqfd, eventfds can be used like registers a write the number of the vector they want to trigger, but as you point out it is racy. So, supporting multiple vectors via irqfd requires multiple eventfds for each guest (one per vector). a total of (# of guests) X (# of vectors) are required. If we're limited to 8 or 16 guests that's not too bad, but since the server opens them all we're restricted to 1024, but that's a pretty high ceiling for this purpose. > > > > -- > Do not meddle in the internals of kernels, for they are subtle and quick = to > panic. > >