From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03C0BC00140 for ; Wed, 10 Aug 2022 06:51:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231153AbiHJGvo (ORCPT ); Wed, 10 Aug 2022 02:51:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229475AbiHJGvm (ORCPT ); Wed, 10 Aug 2022 02:51:42 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ADC0171BD7; Tue, 9 Aug 2022 23:51:40 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 54603B81AE6; Wed, 10 Aug 2022 06:51:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0EE0CC433D6; Wed, 10 Aug 2022 06:51:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1660114298; bh=ahGwuAXwFB0kVpDwl1Y5QMjaxY0VAa6qcPz8jo98lIE=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=m1ftmx62OGBQQmdEb6o+nyAu7gFB4JAfHm56Ki/zP1RKC9oYM8J2yq92YFlfE56iZ T0FB389el0kFaEqL7QVnE4LXCA/uKqL6qtdfH3L1l+zgdXvq0E9SvyQ2Pn0GwLbX1R IMmdf7zr3AulKgyCjldeSx8mNd1xqH1c9fzThLRGAGW5WHJ6L1JTbSYGnHEgXsmYoh tc0vcch8qKebPkbsxJnhKxbYplgNBqalAYI0gTwD0jwA8bXmNV6lD/0DL9NIk1onOb M+xlMN1v0QvjDhIl1viQJCYbqblmlKVGYdQWvbdsgbYnIfGK0SMHixU8i5xG8bYv1S K9hVJOj4Jmyag== Received: from ip-185-104-136-29.ptr.icomera.net ([185.104.136.29] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oLfZ0-0024Yo-Rg; Wed, 10 Aug 2022 07:51:35 +0100 Date: Wed, 10 Aug 2022 07:51:24 +0100 Message-ID: <87o7wsbngz.wl-maz@kernel.org> From: Marc Zyngier To: Dmytro Maluka Cc: "Dong, Eddie" , "Christopherson,, Sean" , Paolo Bonzini , "kvm@vger.kernel.org" , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "x86@kernel.org" , "H. Peter Anvin" , "linux-kernel@vger.kernel.org" , Eric Auger , Alex Williamson , "Liu, Rong L" , Zhenyu Wang , Tomasz Nowicki , Grzegorz Jaszczyk , "upstream@semihalf.com" , Dmitry Torokhov Subject: Re: [PATCH v2 0/5] KVM: Fix oneshot interrupts forwarding In-Reply-To: References: <20220805193919.1470653-1-dmy@semihalf.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.104.136.29 X-SA-Exim-Rcpt-To: dmy@semihalf.com, eddie.dong@intel.com, seanjc@google.com, pbonzini@redhat.com, kvm@vger.kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, linux-kernel@vger.kernel.org, eric.auger@redhat.com, alex.williamson@redhat.com, rong.l.liu@intel.com, zhenyuw@linux.intel.com, tn@semihalf.com, jaz@semihalf.com, upstream@semihalf.com, dtor@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 10 Aug 2022 00:30:29 +0100, Dmytro Maluka wrote: > > On 8/9/22 10:01 PM, Dong, Eddie wrote: > > > > > >> -----Original Message----- > >> From: Dmytro Maluka > >> Sent: Tuesday, August 9, 2022 12:24 AM > >> To: Dong, Eddie ; Christopherson,, Sean > >> ; Paolo Bonzini ; > >> kvm@vger.kernel.org > >> Cc: Thomas Gleixner ; Ingo Molnar ; > >> Borislav Petkov ; Dave Hansen ; > >> x86@kernel.org; H. Peter Anvin ; linux- > >> kernel@vger.kernel.org; Eric Auger ; Alex > >> Williamson ; Liu, Rong L ; > >> Zhenyu Wang ; Tomasz Nowicki > >> ; Grzegorz Jaszczyk ; > >> upstream@semihalf.com; Dmitry Torokhov > >> Subject: Re: [PATCH v2 0/5] KVM: Fix oneshot interrupts forwarding > >> > >> On 8/9/22 1:26 AM, Dong, Eddie wrote: > >>>> > >>>> The existing KVM mechanism for forwarding of level-triggered > >>>> interrupts using resample eventfd doesn't work quite correctly in the > >>>> case of interrupts that are handled in a Linux guest as oneshot > >>>> interrupts (IRQF_ONESHOT). Such an interrupt is acked to the device > >>>> in its threaded irq handler, i.e. later than it is acked to the > >>>> interrupt controller (EOI at the end of hardirq), not earlier. The > >>>> existing KVM code doesn't take that into account, which results in > >>>> erroneous extra interrupts in the guest caused by premature re-assert of an > >> unacknowledged IRQ by the host. > >>> > >>> Interesting... How it behaviors in native side? > >> > >> In native it behaves correctly, since Linux masks such a oneshot interrupt at the > >> beginning of hardirq, so that the EOI at the end of hardirq doesn't result in its > >> immediate re-assert, and then unmasks it later, after its threaded irq handler > >> completes. > >> > >> In handle_fasteoi_irq(): > >> > >> if (desc->istate & IRQS_ONESHOT) > >> mask_irq(desc); > >> > >> handle_irq_event(desc); > >> > >> cond_unmask_eoi_irq(desc, chip); > >> > >> > >> and later in unmask_threaded_irq(): > >> > >> unmask_irq(desc); > >> > >> I also mentioned that in patch #3 description: > >> "Linux keeps such interrupt masked until its threaded handler finishes, to > >> prevent the EOI from re-asserting an unacknowledged interrupt. > > > > That makes sense. Can you include the full story in cover letter too? > > Ok, I will. > > > > > > >> However, with KVM + vfio (or whatever is listening on the resamplefd) we don't > >> check that the interrupt is still masked in the guest at the moment of EOI. > >> Resamplefd is notified regardless, so vfio prematurely unmasks the host > >> physical IRQ, thus a new (unwanted) physical interrupt is generated in the host > >> and queued for injection to the guest." Sorry to barge in pretty late in the conversation (just been Cc'd on this), but why shouldn't the resamplefd be notified? If there has been an EOI, a new level must be made visible to the guest interrupt controller, no matter what the state of the interrupt masking is. Whether this new level is actually *presented* to a vCPU is another matter entirely, and is arguably a problem for the interrupt controller emulation. For example on arm64, we expect to be able to read the pending state of an interrupt from the guest irrespective of the masking state of that interrupt. Any change to the interrupt flow should preserve this. Thankfully, we don't have the polarity issue (there is no such thing in the GIC architecture) and we only deal with pending/not-pending. Thanks, M. -- Without deviation from the norm, progress is not possible.