From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=58680 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OLCo4-0000By-92 for qemu-devel@nongnu.org; Sun, 06 Jun 2010 06:10:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OLCo2-0003Mi-Ls for qemu-devel@nongnu.org; Sun, 06 Jun 2010 06:10:16 -0400 Received: from fmmailgate03.web.de ([217.72.192.234]:55384) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OLCo2-0003MY-6n for qemu-devel@nongnu.org; Sun, 06 Jun 2010 06:10:14 -0400 Message-ID: <4C0B73FF.5040602@web.de> Date: Sun, 06 Jun 2010 12:10:07 +0200 From: Jan Kiszka MIME-Version: 1.0 Subject: Re: [Qemu-devel] Re: [RFT][PATCH 07/15] qemu_irq: Add IRQ handlers with delivery feedback References: <20100603063456.GM24302@redhat.com> <4C0752CB.9030701@web.de> <20100603070300.GN24302@redhat.com> <20100603070559.GO24302@redhat.com> <4C099471.3060507@web.de> <20100606071536.GA2337@redhat.com> <4C0B5098.8030800@web.de> <20100606074934.GD2337@redhat.com> <4C0B5754.9090707@web.de> <20100606092352.GB15902@redhat.com> In-Reply-To: <20100606092352.GB15902@redhat.com> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig9080ACEE04405BEABF717B87" Sender: jan.kiszka@web.de List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Gleb Natapov Cc: Blue Swirl , qemu-devel@nongnu.org, Juan Quintela This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig9080ACEE04405BEABF717B87 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Gleb Natapov wrote: > On Sun, Jun 06, 2010 at 10:07:48AM +0200, Jan Kiszka wrote: >> Gleb Natapov wrote: >>> On Sun, Jun 06, 2010 at 09:39:04AM +0200, Jan Kiszka wrote: >>>> Gleb Natapov wrote: >>>>> On Sat, Jun 05, 2010 at 02:04:01AM +0200, Jan Kiszka wrote: >>>>>>> I'd like to also support EOI handling. When the guest clears the >>>>>>> interrupt condtion, the EOI callback would be called. This could = occur >>>>>>> much later than the IRQ delivery time. I'm not sure if we need th= e >>>>>>> result code in that case. >>>>>>> >>>>>>> If any intermediate device (IOAPIC?) needs to be informed about e= ither >>>>>>> delivery or EOI also, it could create a proxy message with its >>>>>>> callbacks in place. But we need then a separate opaque field (in >>>>>>> addition to payload) to store the original message. >>>>>>> >>>>>>> struct IRQMsg { >>>>>>> DeviceState *src; >>>>>>> void (*delivery_cb)(IRQMsg *msg, int result); >>>>>>> void (*eoi_cb)(IRQMsg *msg, int result); >>>>>>> void *src_opaque; >>>>>>> void *payload; >>>>>>> }; >>>>>> Extending the lifetime of IRQMsg objects beyond the delivery call = stack >>>>>> means qemu_malloc/free for every delivery. I think it takes a _ver= y_ >>>>>> appealing reason to justify this. But so far I do not see any use = case >>>>>> for eio_cb at all. >>>>>> >>>>> I dislike use of eoi for reinfecting missing interrupts since >>>>> it eliminates use of internal PIC/APIC queue of not yet delivered >>>>> interrupts. PIC and APIC has internal queue that can handle two ele= ments: >>>>> one is delivered, but not yet acked interrupt in isr and another is= >>>>> pending interrupt in irr. Using eoi callback (or ack notifier as it= 's >>>>> called inside kernel) interrupt will be considered coalesced even i= f irr >>>>> is cleared, but no ack was received for previously delivered interr= upt. >>>>> But ack notifiers actually has another use: device assignment. Ther= e is >>>>> a plan to move device assignment from kernel to userspace and for t= hat >>>>> ack notifiers will have to be extended to userspace too. If so we c= an >>>>> use them to do irq decoalescing as well. I doubt they should be par= t >>>>> of IRQMsg though. Why not do what kernel does: have globally regist= ered >>>>> notifier based on irqchip/pin. >>>> I read this twice but I still don't get your plan. Do you like or >>>> dislike using EIO for de-coalescing? And how should these notifiers = work? >>>> >>> That's because I confused myself :) I _dislike_ them to be used, but >>> since device assignment requires ack notifiers anyway may be it is be= tter >>> to introduce one mechanism for device assignmen + de-coalescing inste= ad >>> of introducing two different mechanism. Using ack notifiers should be= >>> easy: RTC registers ack notifier and keep track of delivered interrup= ts. >>> If timer triggers after previews irq was set, but before it was acked= >>> coalesced counter is incremented. In ack notifier callback coalesced >>> counter is checked and if it is not zero new irq is set. >> Ack notifier registrations and event deliveries still need to be route= d. >> Piggy-backing this on IRQ messages may be unavoidable for that reason.= > It is done in the kernel without piggy-backing. As it does not include any IRQ routers in front of the interrupt controller. Maybe it works for x86, but it is no generic solution. Also, periodic timer sources get no information about the fact that their interrupt is masked somewhere along the path to the VCPUs and will possibly replay countless IRQs when the masking ends, no? >=20 >> Anyway, I'm going to post my HPET updates with the infrastructure for >> IRQMsg now. Maybe it's helpful to see the other option in reality. >> > One other think to consider current approach does not always work. > Win2K3-64bit-smp and Win2k8-64bit-smp configure RTC interrupt to be > broadcasted to all cpus, but only boot cpu does time calculation. With > current approach if interrupt is delivered to at least one vcpu > it will not be considered coalesced, but if cpu it was delivered to is > not cpu that does time accounting then clock will drift. That means we would have to fire callbacks per receiving CPU and report its number back. Is there a way to find out if we are running such a guest without an '-enable-win2k[38]-64bit-smp-rtc-drift-fix'? Jan --------------enig9080ACEE04405BEABF717B87 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iEYEARECAAYFAkwLdAIACgkQitSsb3rl5xSb0ACglQ0nTBby2ITT4svs7Zo7UZpe Bz8An1dVTJ3HyMxfN+cYpcia1tBZ+dg9 =EQtm -----END PGP SIGNATURE----- --------------enig9080ACEE04405BEABF717B87--